[WIP] Enable v1 signin otp tests - #3066
Conversation
There was a problem hiding this comment.
Pull request overview
This PR primarily re-enables and stabilizes Native Auth E2E sign-in/MFA/JIT scenarios (including OTP-based flows) by removing skips, updating delegate plumbing for MFA auth-methods, and adjusting test infrastructure to better support mail-based OTP retrieval.
Changes:
- Reduced E2E test retry repetitions in the iOS and macOS
.xctestplanfiles. - Updated Native Auth E2E delegate spies and multiple sign-in/MFA/JIT tests to run again (removed
XCTSkips) and align with updated MFA delegate signatures. - Updated the mail-based OTP retrieval helpers (new account creation helper, longer default expectation timeout, and updated test email domain constant).
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| MSAL/test/testplan/MSAL Mac Native Auth E2E Tests.xctestplan | Lowers maximum test repetitions for macOS native auth E2E runs. |
| MSAL/test/testplan/MSAL iOS Native Auth E2E Tests.xctestplan | Lowers maximum test repetitions for iOS native auth E2E runs. |
| MSAL/test/integration/native_auth/end_to_end/sign_in/SignInDelegateSpies.swift | Updates spy delegate signature to capture MFA auth methods. |
| MSAL/test/integration/native_auth/end_to_end/sign_in/MSALNativeAuthSignInUsernameEndToEndTests.swift | Re-enables multiple username/OTP sign-in E2E tests by removing skips and adjusting test data usage. |
| MSAL/test/integration/native_auth/end_to_end/sign_in/MSALNativeAuthSignInUsernameAndPasswordEndToEndTests.swift | Re-enables username+password E2E tests and adjusts multi-sign-in scenarios. |
| MSAL/test/integration/native_auth/end_to_end/otp_code_retriever/MailTMConstants.swift | Updates the domain used when generating random sign-up email addresses for OTP flows. |
| MSAL/test/integration/native_auth/end_to_end/MSALNativeAuthEndToEndBaseTestCase.swift | Adds an email-provider account creation helper, updates OTP retrieval signature, and increases default async expectation timeout. |
| MSAL/test/integration/native_auth/end_to_end/mfa/MSALNativeAuthSignInWithMFAEndToEndTests.swift | Re-enables MFA E2E tests and refactors parts of the MFA challenge submission/claims validation flow. |
| MSAL/test/integration/native_auth/end_to_end/mfa/MSALNativeAuthSignInJITEndToEndTests.swift | Re-enables JIT E2E tests and switches to creating authenticated email-provider accounts for OTP-driven steps. |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 9 out of 9 changed files in this pull request and generated no new comments.
Suppressed comments (3)
MSAL/test/integration/native_auth/end_to_end/mfa/MSALNativeAuthSignInWithMFAEndToEndTests.swift:136
- Issue:
authenticationContextATClaimJsonis missing the closing quote after theauthenticationContextId.
Impact: Thecontainscheck can become a false positive (e.g., a token containing"acrs":["c41"...]would also match"acrs":["c4).
Recommendation: Include the closing quote after the interpolated id (you can still omit the closing bracket if you want to allow additional values).
let authenticationContextId = "c4"
let authenticationContextRequestClaimJson = "{\"access_token\":{\"acrs\":{\"essential\":true,\"value\":\"\(authenticationContextId)\"}}}"
let authenticationContextATClaimJson = "\"acrs\":[\"\(authenticationContextId)"
MSAL/test/integration/native_auth/end_to_end/MSALNativeAuthEndToEndBaseTestCase.swift:147
- Issue: The
passwordparameter is shadowed by a locallet password = ...binding.
Impact: Shadowing makes it harder to reason about which password is being used (call-site override vs. conf.json) and is easy to accidentally break during future edits.
Recommendation: Use a distinct local name (e.g.,providerPassword) and pass that through.
func retrieveCodeFor(email: String, password: String? = nil) async -> String? {
guard let password = password ?? retrieveEmailProviderPassword() else {
XCTFail("email_provider_password not found in conf.json")
return nil
}
MSAL/test/integration/native_auth/end_to_end/mfa/MSALNativeAuthSignInWithMFAEndToEndTests.swift:67
- Issue: The invalid-challenge assertion is commented out, so the test no longer validates what kind of error is produced for an incorrect OTP.
Impact: This reduces the regression signal: the test can pass even if the SDK returns an unrelated error type/message.
Recommendation: IfisInvalidChallengeis currently not reliable in this environment, keep at least a minimal assertion that an actionable error description is present (or update the expectation once the SDK behavior is fixed).
XCTAssertTrue(mfaSubmitWrongChallengeDelegateSpy.onMFASubmitChallengeErrorCalled)
// TODO: we get general error instead of isInvalidChallenge.
// XCTAssertEqual(mfaSubmitWrongChallengeDelegateSpy.error?.isInvalidChallenge, true)
PR Checklist (must be completed before review)
PR Title Format
Required Format:
[Keyword1] [Keyword2]: Descriptionmajor,minor, orpatch(case-insensitive)feature,bugfix,engg, ortests(case-insensitive)Examples:
[MAJOR] [Feature]: new API[minor] [bugfix]: fix crash[PATCH] [tests]: add coverageProposed changes
Describe what this PR is trying to do.
Type of change
Risk
Additional information