feat(cc-widgets): outdial widget e2e tests by eigengravy · Pull Request #640 · webex/widgets

eigengravy · 2026-03-04T19:53:03Z

COMPLETES CAI-7682

This pull request addresses

Adding end-to-end Playwright tests for the outdial call workflow across two agent login modes (Desktop and Extension).

by making the following changes

Added playwright/tests/outdial-call-test.spec.ts — new E2E test suite with three describe blocks covering the full outdial lifecycle (enter number, dial, connect, verify call controls, end call, wrapup) for:
- Desktop mode — auto-connect first leg, no manual accept
- Extension mode — first leg rings on Webex Calling extension, agent must answer before customer is dialed
- Updated playwright/suites/station-login-user-state-tests.spec.ts — wired the new createOutdialCallTests into the existing suite entrypoint
- Updated OutdialCall/ARCHITECTURE.md — documented the post-dial flow by login mode with Mermaid sequence diagrams explaining auto-connect (Desktop) vs manual-answer (Extension) behavior
- Updated playwright/ai-docs/ — minor updates to AGENTS.md and ARCHITECTURE.md to reflect the new outdial test coverage

The following scenarios were tested

yarn test:e2e --project=SET_3 --grep "Outdial"

The GAI Coding Policy And Copyright Annotation Best Practices

aws-amplify-us-east-2 · 2026-03-04T19:59:15Z

This pull request is automatically being deployed by Amplify Hosting (learn more).

Access this pull request here: https://pr-640.d1b38q61t1z947.amplifyapp.com

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6f9a30ba55

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

playwright/tests/outdial-call-test.spec.ts

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5b6edddcdf

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

playwright/tests/outdial-call-test.spec.ts

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d2f5f9592d

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

playwright/test-manager.ts

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 43f98cde15

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

playwright/tests/outdial-call-test.spec.ts

playwright/ai-docs/ARCHITECTURE.md

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 217573a34e

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

playwright/tests/outdial-call-test.spec.ts

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 33bf3bad7c

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

chatgpt-codex-connector · 2026-03-12T05:37:05Z

playwright/suites/dial-number-tests.spec.ts

+import createOutdialCallTests from '../tests/outdial-call-test.spec';

 test.describe('Dial Number Task Control Tests', createDialNumberTaskControlTests);
+test.describe('Outdial Call Tests', createOutdialCallTests);


Split outdial tests from dial-number suite state

Adding createOutdialCallTests to SET_6 makes these tests run after createDialNumberTaskControlTests, but that test factory never performs a full testManager.cleanup() teardown, so the first agent session remains active in Desktop mode before outdial setup starts. In a full --project=SET_6 run, setupForOutdialDesktop then reuses the same ${projectName}_AGENT1_ACCESS_TOKEN for a second station login while the prior session is still alive, which can block the new login and fail the outdial suite before test steps begin.

Useful? React with 👍 / 👎.

Shreyas281299 · 2026-03-12T08:12:31Z

playwright/tests/outdial-call-test.spec.ts

+    test.beforeAll(async ({browser}, testInfo) => {
+      const projectName = testInfo.project.name;
+      testManager = new TestManager(projectName);
+      await testManager.setupForOutdialDesktop(browser);


🔴 High Severity - Test crashes before skip check

The test.skip on line 22 checks for PW_DIAL_NUMBER, but setupForOutdialDesktop in this beforeAll runs before the skip check executes. If DIAL_NUMBER_LOGIN_ACCESS_TOKEN is missing, setupOutdialCustomer() will fail in beforeAll, causing the suite to crash instead of skipping gracefully.

Impact: Suite crashes with unhelpful errors instead of skipping when env vars are missing.

Suggestion: Move the env-var guard into beforeAll itself:

test.beforeAll(async ({browser}, testInfo) => { if (\!process.env.PW_DIAL_NUMBER || \!process.env.DIAL_NUMBER_LOGIN_ACCESS_TOKEN) { test.skip(true, 'Required env vars not set'); return; } const projectName = testInfo.project.name; testManager = new TestManager(projectName); await testManager.setupForOutdialDesktop(browser); });

Moved test.skip to the describe level so it runs before beforeAll. This skips the entire block (including setup) when env vars are missing.

Shreyas281299 · 2026-03-12T08:12:39Z

playwright/tests/outdial-call-test.spec.ts

+      await acceptCustomerCall(testManager.callerPage);
+      await waitForState(testManager.agent1Page, USER_STATES.ENGAGED);
+      await verifyCurrentState(testManager.agent1Page, USER_STATES.ENGAGED);
+      await testManager.agent1Page.waitForTimeout(3000);


🟡 Medium Severity - Hard-coded timeout (flaky test anti-pattern)

This waitForTimeout(3000) is an arbitrary delay that can cause flaky tests. If the system is slow, 3 seconds might not be enough; if it's fast, you're wasting time waiting unnecessarily.

Suggestion: Replace with an explicit wait on UI state:

await expect(endCallButton).toBeEnabled({timeout: 5000}); // or wait for a specific element/state that indicates readiness

Removed the waitForTimeout(3000). The test now proceeds directly after verifying the ENGAGED state. This is consistent with how other test files handle it — they rely on waitForState for the state transition and then act immediately.

Shreyas281299 · 2026-03-12T08:12:46Z

playwright/tests/outdial-call-test.spec.ts

+      await waitForState(testManager.agent1Page, USER_STATES.ENGAGED);
+      await verifyCurrentState(testManager.agent1Page, USER_STATES.ENGAGED);
+      await testManager.agent1Page.waitForTimeout(3000);
+      await testManager.agent1Page.getByTestId('call-control:end-call').first().click({timeout: 5000});


🟡 Medium Severity - Inconsistent call termination pattern

This test uses an inline locator + click to end the call, while the Extension test (line 64) uses the endCallTask() utility. This inconsistency makes maintenance harder and could mask mode-specific bugs.

Suggestion: Use the endCallTask utility for consistency, or create a dedicated endOutdialCall utility in outdialUtils.ts that works for both Desktop and Extension modes.

Switched to using endCallTask for the extension page

Shreyas281299 · 2026-03-12T08:12:53Z

playwright/tests/outdial-call-test.spec.ts

+      await verifyCurrentState(testManager.agent1Page, USER_STATES.ENGAGED);
+      await testManager.agent1Page.waitForTimeout(3000);
+      await testManager.agent1Page.getByTestId('call-control:end-call').first().click({timeout: 5000});
+      await testManager.agent1Page.waitForTimeout(2000);


🟡 Medium Severity - Hard-coded timeout (flaky test anti-pattern)

Another waitForTimeout(2000) before wrapup. The Extension test uses waitForWrapupAfterCallEnd utility instead, which is more deterministic.

Suggestion: Use the same pattern as the Extension test:

await waitForWrapupAfterCallEnd(testManager.agent1Page);

Replaced with waitForWrapupAfterCallEnd which uses expect.poll() with exponential backoff — same pattern as the Extension test.

Shreyas281299 · 2026-03-12T08:13:01Z

playwright/tests/outdial-call-test.spec.ts

+
+    test.beforeAll(async ({browser}, testInfo) => {
+      const projectName = testInfo.project.name;
+      testManager = new TestManager(projectName);


🔴 High Severity - Test crashes before skip check

Same issue as Desktop mode (line 17) - setupForOutdialExtension in beforeAll runs before the test.skip check on line 53. If DIAL_NUMBER_LOGIN_ACCESS_TOKEN is missing, the suite will crash.

Suggestion: Apply the same fix as suggested for line 17.

Same fix as Desktop — test.skip moved to describe level.

Shreyas281299 · 2026-03-12T08:13:08Z

playwright/tests/outdial-call-test.spec.ts

+      await waitForState(testManager.agent1Page, USER_STATES.ENGAGED);
+      await verifyCurrentState(testManager.agent1Page, USER_STATES.ENGAGED);
+      await testManager.agent1Page.waitForTimeout(3000);
+      await endCallTask(testManager.agent1ExtensionPage);


🟡 Medium Severity - Hard-coded timeout (flaky test anti-pattern)

Same issue as Desktop mode line 29 - waitForTimeout(3000) is an arbitrary delay that can cause flaky tests.

Suggestion: Replace with an explicit wait on UI state before ending the call.

Removed the waitForTimeout(3000), same as the Desktop fix

Shreyas281299 · 2026-03-12T08:13:36Z

playwright/test-manager.ts

+  }
+
+  private async setupOutdialCustomer(browser: Browser): Promise<void> {
+    const customerToken = process.env.DIAL_NUMBER_LOGIN_ACCESS_TOKEN ?? '';


🔴 High Severity - Empty token not validated

This line uses process.env.DIAL_NUMBER_LOGIN_ACCESS_TOKEN ?? '' which defaults to an empty string if the env var is undefined. When loginExtension is called with an empty string, it will fail with an unhelpful error message.

Impact: Confusing test failures when env var is not set.

Suggestion: Add validation and throw a clear error:

const customerToken = process.env.DIAL_NUMBER_LOGIN_ACCESS_TOKEN; if (!customerToken) { throw new Error('DIAL_NUMBER_LOGIN_ACCESS_TOKEN is required for outdial tests'); }

The describe-level test.skip now checks for DIAL_NUMBER_LOGIN_ACCESS_TOKEN before the suite runs, so setupOutdialCustomer is never called with a missing token. Leaving the ?? '' to stay consistent with the existing pattern in test-manager.ts (e.g. PW_SANDBOX_PASSWORD uses the same default).

Switched to using

const envTokens = this.getEnvTokens(); const customerToken = envTokens.dialNumberLoginAccessToken; if (!customerToken) { throw new Error('Environment variable DIAL_NUMBER_LOGIN_ACCESS_TOKEN is missing or empty');

Shreyas281299 · 2026-03-12T08:13:44Z

playwright/test-manager.ts

    });
  }

+  async setupForOutdialDesktop(browser: Browser): Promise<void> {


🟡 Medium Severity - No cleanup on setup failure

These methods call this.setup() followed by this.setupOutdialCustomer(). If setupOutdialCustomer fails, the agent setup from this.setup() is already done but won't be cleaned up since the test will fail in beforeAll.

Impact: Resource leaks and potential conflicts in subsequent test runs.

Suggestion: Consider wrapping in try/catch with cleanup, or document that cleanup is the caller's responsibility on setup failure.

No other test in the codebase uses try/catch in beforeAll — the convention is to let failures propagate to the test runner. Added the if (testManager) guard in afterAll to match the pattern used across all other test files (14 instances).

Shreyas281299 · 2026-03-12T08:13:52Z

playwright/Utils/outdialUtils.ts

+export async function acceptCustomerCall(customerPage: Page): Promise<void> {
+  await customerPage.bringToFront();
+  await expect(customerPage.locator('#answer').first()).toBeEnabled({timeout: ACCEPT_TASK_TIMEOUT});
+  await customerPage.waitForTimeout(UI_SETTLE_TIMEOUT);


🟡 Medium Severity - Hard-coded wait in utility function

This waitForTimeout(UI_SETTLE_TIMEOUT) is a hard-coded delay that could be flaky. If this is for UI animation/settling, consider using a more deterministic condition.

Suggestion: Replace with waitForLoadState or wait for a specific UI condition:

await customerPage.waitForLoadState('networkidle'); // or await expect(someElement).toBeStable();

UI_SETTLE_TIMEOUT is consistent with codebase convention — waitForTimeout is used throughout the test suite for UI settling between actions.

Shreyas281299 · 2026-03-12T08:14:01Z

playwright/suites/dial-number-tests.spec.ts

+import createOutdialCallTests from '../tests/outdial-call-test.spec';

 test.describe('Dial Number Task Control Tests', createDialNumberTaskControlTests);
+test.describe('Outdial Call Tests', createOutdialCallTests);


🟡 Medium Severity - Test isolation concern

The outdial tests now share SET_6 with dial-number task control tests. If the dial-number tests don't fully clean up their agent sessions in afterAll, the subsequent outdial setup may conflict with stale session state.

Impact: Potential test conflicts and failures.

Suggestion: Verify that dial-number-task-control-test.spec.ts has proper afterAll cleanup, or consider running outdial tests in a separate suite/set for better isolation.

Not seeing issues with shared SET_6 currently. Both suites have afterAll cleanup that handles session teardown.

Shreyas281299

Review Summary: Found 2 High severity and 7 Medium severity issues. High: env-var validation problems causing unhelpful test failures. Medium: multiple hard-coded timeouts (flaky test anti-patterns), inconsistent test patterns, and test isolation concerns. Please address the High severity issues and consider fixing the Medium severity timeouts to improve test reliability.

eigengravy changed the title ~~OutDial Widget e2e tests~~ feat(cc-widgets): outdial widget e2e tests Mar 5, 2026

eigengravy force-pushed the outdial-e2e-tests branch 2 times, most recently from e694b97 to 6f9a30b Compare March 10, 2026 05:18

eigengravy marked this pull request as ready for review March 10, 2026 05:18

eigengravy requested a review from a team as a code owner March 10, 2026 05:18

chatgpt-codex-connector bot reviewed Mar 10, 2026

View reviewed changes

playwright/tests/outdial-call-test.spec.ts Outdated Show resolved Hide resolved

playwright/tests/outdial-call-test.spec.ts Outdated Show resolved Hide resolved

rsarika added validated Indicates that the PR is ready for actions run_e2e Add this label to run E2E test for meeting and CC widgets labels Mar 11, 2026

eigengravy force-pushed the outdial-e2e-tests branch from 6f9a30b to 5b6eddd Compare March 11, 2026 07:38

chatgpt-codex-connector bot reviewed Mar 11, 2026

View reviewed changes

playwright/tests/outdial-call-test.spec.ts Outdated Show resolved Hide resolved

outdial ui automation

d2f5f95

eigengravy force-pushed the outdial-e2e-tests branch from 5b6eddd to d2f5f95 Compare March 11, 2026 12:18

chatgpt-codex-connector bot reviewed Mar 11, 2026

View reviewed changes

playwright/test-manager.ts Outdated Show resolved Hide resolved

fix: move to dn suite and remove dn login

43f98cd

chatgpt-codex-connector bot reviewed Mar 11, 2026

View reviewed changes

playwright/tests/outdial-call-test.spec.ts Show resolved Hide resolved

playwright/ai-docs/ARCHITECTURE.md Outdated Show resolved Hide resolved

Sarang S added 2 commits March 11, 2026 19:08

fix: remove DN outdial mode, update architecture docs

07c2e67

fix: rename PW_DIAL_NUMBER_PSTN to PW_DIAL_NUMBER env variable

217573a

chatgpt-codex-connector bot reviewed Mar 11, 2026

View reviewed changes

playwright/tests/outdial-call-test.spec.ts Show resolved Hide resolved

fix: handle missing env test fail case

33bf3ba

chatgpt-codex-connector bot reviewed Mar 12, 2026

View reviewed changes

Shreyas281299 reviewed Mar 12, 2026

View reviewed changes

Shreyas281299 requested changes Mar 12, 2026

View reviewed changes

Sarang S added 2 commits March 13, 2026 08:38

fix: address comments

4fdf079

fix: add cleanup to dn tests

f520d73

rsarika approved these changes Mar 13, 2026

View reviewed changes

Conversation

eigengravy commented Mar 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

COMPLETES CAI-7682

This pull request addresses

by making the following changes

The following scenarios were tested

The GAI Coding Policy And Copyright Annotation Best Practices

Uh oh!

aws-amplify-us-east-2 bot commented Mar 4, 2026

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector bot Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

eigengravy commented Mar 4, 2026 •

edited

Loading