Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Oct 11, 2025

Problem

GitHub Copilot agents need to be able to install and run Playwright browsers for end-to-end testing, but the standard Playwright CLI installation (pwsh playwright.ps1 install) fails in sandboxed environments with a known issue:

Error: Download failed: size mismatch, file size: 170660308, expected size: 0

This prevents Copilot agents from running Playwright tests, limiting their ability to validate UI changes and test end-to-end workflows.

Solution

This PR adds automated installation scripts that handle browser installation reliably in the GitHub Copilot environment:

Installation Scripts

  • scripts/install-playwright-browsers.sh (Bash)
  • scripts/install-playwright-browsers.ps1 (PowerShell)

Both scripts use a two-pronged approach:

  1. Primary: Attempt standard Playwright CLI installation
  2. Fallback: If CLI fails, automatically download Chromium directly from the Playwright CDN and extract it to the standard cache location (~/.cache/ms-playwright/chromium-1129/)

The fallback method bypasses the problematic progress tracking in the Playwright CLI while maintaining compatibility with the existing test infrastructure.

Verification Scripts

  • scripts/verify-playwright-installation.sh (Bash)
  • scripts/verify-playwright-installation.ps1 (PowerShell)

These scripts verify the installation by launching Chromium in headless mode and testing basic functionality (page creation and content rendering).

Documentation

  • scripts/PLAYWRIGHT_BROWSER_INSTALLATION.md - Comprehensive installation guide with troubleshooting
  • scripts/README.md - Quick reference for all scripts
  • PLAYWRIGHT_INSTALLATION_ANSWER.md - Detailed explanation of the solution
  • Updated AGENTS.md with instructions for GitHub Copilot agents

Usage

GitHub Copilot agents can now install browsers with a single command:

./scripts/install-playwright-browsers.sh

Or verify installation:

./scripts/verify-playwright-installation.sh

Testing

All scripts have been tested and verified:

  • ✅ Bash installation script works correctly with automatic fallback
  • ✅ PowerShell installation script works correctly with automatic fallback
  • ✅ Verification scripts confirm Chromium 128.0.6613.18 launches and renders content
  • ✅ Installation location (~/.cache/ms-playwright/) is compatible with existing PlaywrightHelper.cs

Impact

This change enables GitHub Copilot agents to:

  • Install Playwright browsers reliably in sandboxed environments
  • Run end-to-end tests to validate changes
  • Verify UI functionality automatically
  • Handle environment-specific issues without manual intervention

No changes to existing test code were required - the scripts work seamlessly with the current JwtIdentity.PlaywrightTests infrastructure.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • example.com
    • Triggering command: /home/REDACTED/.cache/ms-playwright/chromium-1129/chrome-linux/chrome --disable-field-trial-config --disable-REDACTED-networking --disable-REDACTED-timer-throttling --disable-REDACTEDing-occluded-windows --disable-back-forward-cache --disable-breakpad --disable-client-side-phishing-detection --disable-component-extensions-with-REDACTED-pages --disable-component-update --no-default-browser-check --disable-default-apps --disable-dev-shm-usage --disable-extensions --disable-features=ImprovedCookieControls,LazyFrameLoading,GlobalMediaControls,DestroyProfileOnBrowserClose,MediaRouter,DialMediaRouteProvider,AcceptCHFrame,AutoExpandDetailsElement,CertificateTransparencyComponentUpdater,AvoidUnnecessaryBeforeUnloadCheckSync,Translate,HttpsUpgrades,PaintHolding,PlzDedicatedWorker --allow-pre-commit-input --disable-hang-monitor --disable-ipc-flooding-protection --disable-popup-blocking --disable-prompt-on-repost --disable-renderer-REDACTEDing --force-color-profile=srgb --metrics-recording-only --no-first-run --enable-automation --password-store=basic --use-mock-keychain --no-service-autorun --export-tagged-pdf --disable-search-engine-choice-screen --unsafely-disable-devtools-self-xss-warnings --headless=old --hide-scrollbars --mute-audio --blink-settings=primaryHoverType=2,availableHoverTypes=2,primaryPointerType=4,availablePointerTypes=4 --no-sandbox --user-data-dir=/tmp/playwright_chromiumdev_profile-XXXXXXVF0ecg --remote-debugging-pipe --no-startup-window (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Original prompt

Can the playwright browsers be installed in the github copilot environment so the playwright tests can be run by copilot agent?


💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.

Copilot AI and others added 3 commits October 11, 2025 21:45
Co-authored-by: dwarwick <15970276+dwarwick@users.noreply.github.com>
Co-authored-by: dwarwick <15970276+dwarwick@users.noreply.github.com>
Co-authored-by: dwarwick <15970276+dwarwick@users.noreply.github.com>
Copilot AI changed the title [WIP] Add installation of Playwright browsers in Copilot environment Add Playwright browser installation scripts for GitHub Copilot environment Oct 11, 2025
Copilot AI requested a review from dwarwick October 11, 2025 21:53
@dwarwick
Copy link
Owner

@copilot please run the playwright tests using the website https://davidtest.xyz as the website url

Copy link
Contributor Author

Copilot AI commented Oct 11, 2025

@copilot please run the playwright tests using the website https://davidtest.xyz as the website url

I ran the Playwright tests against https://davidtest.xyz, but all 3 tests failed with DNS resolution errors:

net::ERR_NAME_NOT_RESOLVED at https://davidtest.xyz

The domain cannot be resolved from the GitHub Copilot environment:

$ curl -I https://davidtest.xyz
curl: (6) Could not resolve host: davidtest.xyz

Test Results:

  • Total: 3 tests
  • Failed: 3 (AuthTests.LoginTest_Succeeds, SurveyTests, CompleteDemoFlowTests)
  • All failed due to DNS resolution error

The Playwright browsers are installed correctly and working - the issue is that davidtest.xyz is not accessible from this sandboxed environment. The domain either needs to be publicly resolvable or you would need to whitelist it in the environment's network settings.

Would you like me to test against a different URL, or did you want to configure access to davidtest.xyz first?

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • accounts.google.com
    • Triggering command: /home/REDACTED/.cache/ms-playwright/chromium-1129/chrome-linux/chrome --disable-field-trial-config --disable-REDACTED-networking --disable-REDACTED-timer-throttling --disable-REDACTEDing-occluded-windows --disable-back-forward-cache --disable-breakpad --disable-client-side-phishing-detection --disable-component-extensions-with-REDACTED-pages --disable-component-update --no-default-browser-check --disable-default-apps --disable-dev-shm-usage --disable-extensions --disable-features=ImprovedCookieControls,LazyFrameLoading,GlobalMediaControls,DestroyProfileOnBrowserClose,MediaRouter,DialMediaRouteProvider,AcceptCHFrame,AutoExpandDetailsElement,CertificateTransparencyComponentUpdater,AvoidUnnecessaryBeforeUnloadCheckSync,Translate,HttpsUpgrades,PaintHolding,PlzDedicatedWorker --allow-pre-commit-input --disable-hang-monitor --disable-ipc-flooding-protection --disable-popup-blocking --disable-prompt-on-repost --disable-renderer-REDACTEDing --force-color-profile=srgb --metrics-recording-only --no-first-run --enable-automation --password-store=basic --use-mock-keychain --no-service-autorun --export-tagged-pdf --disable-search-engine-choice-screen --unsafely-disable-devtools-self-xss-warnings --headless=old --hide-scrollbars --mute-audio --blink-settings=primaryHoverType=2,availableHoverTypes=2,primaryPointerType=4,availablePointerTypes=4 --no-sandbox --headless=new --user-data-dir=/tmp/playwright_chromiumdev_profile-XXXXXX3LhwEK --remote-debugging-pipe --no-startup-window --noerrdialogs --ozone-platform=headless --ozone-override-screen-size=800,600 --use-angle=swiftshader-webgl (dns block)
    • Triggering command: /home/REDACTED/.cache/ms-playwright/chromium-1129/chrome-linux/chrome --type=utility --utility-sub-type=network.mojom.NetworkService --lang=en-US --service-sandbox-type=none --no-sandbox --disable-dev-shm-usage --use-angle=swiftshader-webgl --mute-audio --crashpad-handler-pid=3877 --enable-crash-reporter=, --noerrdialogs --user-data-dir=/tmp/playwright_chromiumdev_profile-XXXXXX3LhwEK --change-stack-guard-on-fork=enable --shared-files=v8_context_snapshot_data:100 --field-trial-handle=3,i,5006903823704742113,17032414060199126035,262144 --disable-features=AcceptCHFrame,AutoExpandDetailsElement,AvoidUnnecessaryBeforeUnloadCheckSync,CertificateTransparencyComponentUpdater,DestroyProfileOnBrowserClose,DialMediaRouteProvider,GlobalMediaControls,HttpsUpgrades,ImprovedCookieControls,LazyFrameLoading,MediaRouter,PaintHolding,PlzDedicatedWorker,Translate --variations-seed-version (dns block)
  • davidtest.xyz
    • Triggering command: /usr/share/dotnet/dotnet exec --runtimeconfig /home/REDACTED/work/JwtIdentity/JwtIdentity/JwtIdentity.PlaywrightTests/bin/Debug/net9.0/JwtIdentity.PlaywrightTests.runtimeconfig.json --depsfile /home/REDACTED/work/JwtIdentity/JwtIdentity/JwtIdentity.PlaywrightTests/bin/Debug/net9.0/JwtIdentity.PlaywrightTests.deps.json /home/REDACTED/work/JwtIdentity/JwtIdentity/JwtIdentity.PlaywrightTests/bin/Debug/net9.0/testhost.dll --port 46351 --endpoint 127.0.0.1:046351 --role client --parentprocessid 3570 --telemetryoptedin false (dns block)
    • Triggering command: /home/REDACTED/.cache/ms-playwright/chromium-1129/chrome-linux/chrome --type=utility --utility-sub-type=network.mojom.NetworkService --lang=en-US --service-sandbox-type=none --no-sandbox --disable-dev-shm-usage --use-angle=swiftshader-webgl --mute-audio --crashpad-handler-pid=3877 --enable-crash-reporter=, --noerrdialogs --user-data-dir=/tmp/playwright_chromiumdev_profile-XXXXXX3LhwEK --change-stack-guard-on-fork=enable --shared-files=v8_context_snapshot_data:100 --field-trial-handle=3,i,5006903823704742113,17032414060199126035,262144 --disable-features=AcceptCHFrame,AutoExpandDetailsElement,AvoidUnnecessaryBeforeUnloadCheckSync,CertificateTransparencyComponentUpdater,DestroyProfileOnBrowserClose,DialMediaRouteProvider,GlobalMediaControls,HttpsUpgrades,ImprovedCookieControls,LazyFrameLoading,MediaRouter,PaintHolding,PlzDedicatedWorker,Translate --variations-seed-version (dns block)
    • Triggering command: /home/REDACTED/.cache/ms-playwright/chromium-1129/chrome-linux/chrome --disable-field-trial-config --disable-REDACTED-networking --disable-REDACTED-timer-throttling --disable-REDACTEDing-occluded-windows --disable-back-forward-cache --disable-breakpad --disable-client-side-phishing-detection --disable-component-extensions-with-REDACTED-pages --disable-component-update --no-default-browser-check --disable-default-apps --disable-dev-shm-usage --disable-extensions --disable-features=ImprovedCookieControls,LazyFrameLoading,GlobalMediaControls,DestroyProfileOnBrowserClose,MediaRouter,DialMediaRouteProvider,AcceptCHFrame,AutoExpandDetailsElement,CertificateTransparencyComponentUpdater,AvoidUnnecessaryBeforeUnloadCheckSync,Translate,HttpsUpgrades,PaintHolding,PlzDedicatedWorker --allow-pre-commit-input --disable-hang-monitor --disable-ipc-flooding-protection --disable-popup-blocking --disable-prompt-on-repost --disable-renderer-REDACTEDing --force-color-profile=srgb --metrics-recording-only --no-first-run --enable-automation --password-store=basic --use-mock-keychain --no-service-autorun --export-tagged-pdf --disable-search-engine-choice-screen --unsafely-disable-devtools-self-xss-warnings --headless=old --hide-scrollbars --mute-audio --blink-settings=primaryHoverType=2,availableHoverTypes=2,primaryPointerType=4,availablePointerTypes=4 --no-sandbox --headless=new --user-data-dir=/tmp/playwright_chromiumdev_profile-XXXXXX3LhwEK --remote-debugging-pipe --no-startup-window --noerrdialogs --ozone-platform=headless --ozone-override-screen-size=800,600 --use-angle=swiftshader-webgl (dns block)
  • google.com
    • Triggering command: /home/REDACTED/.cache/ms-playwright/chromium-1129/chrome-linux/chrome --type=utility --utility-sub-type=network.mojom.NetworkService --lang=en-US --service-sandbox-type=none --no-sandbox --disable-dev-shm-usage --use-angle=swiftshader-webgl --mute-audio --crashpad-handler-pid=3877 --enable-crash-reporter=, --noerrdialogs --user-data-dir=/tmp/playwright_chromiumdev_profile-XXXXXX3LhwEK --change-stack-guard-on-fork=enable --shared-files=v8_context_snapshot_data:100 --field-trial-handle=3,i,5006903823704742113,17032414060199126035,262144 --disable-features=AcceptCHFrame,AutoExpandDetailsElement,AvoidUnnecessaryBeforeUnloadCheckSync,CertificateTransparencyComponentUpdater,DestroyProfileOnBrowserClose,DialMediaRouteProvider,GlobalMediaControls,HttpsUpgrades,ImprovedCookieControls,LazyFrameLoading,MediaRouter,PaintHolding,PlzDedicatedWorker,Translate --variations-seed-version (dns block)
  • optimizationguide-pa.googleapis.com
    • Triggering command: /home/REDACTED/.cache/ms-playwright/chromium-1129/chrome-linux/chrome --disable-field-trial-config --disable-REDACTED-networking --disable-REDACTED-timer-throttling --disable-REDACTEDing-occluded-windows --disable-back-forward-cache --disable-breakpad --disable-client-side-phishing-detection --disable-component-extensions-with-REDACTED-pages --disable-component-update --no-default-browser-check --disable-default-apps --disable-dev-shm-usage --disable-extensions --disable-features=ImprovedCookieControls,LazyFrameLoading,GlobalMediaControls,DestroyProfileOnBrowserClose,MediaRouter,DialMediaRouteProvider,AcceptCHFrame,AutoExpandDetailsElement,CertificateTransparencyComponentUpdater,AvoidUnnecessaryBeforeUnloadCheckSync,Translate,HttpsUpgrades,PaintHolding,PlzDedicatedWorker --allow-pre-commit-input --disable-hang-monitor --disable-ipc-flooding-protection --disable-popup-blocking --disable-prompt-on-repost --disable-renderer-REDACTEDing --force-color-profile=srgb --metrics-recording-only --no-first-run --enable-automation --password-store=basic --use-mock-keychain --no-service-autorun --export-tagged-pdf --disable-search-engine-choice-screen --unsafely-disable-devtools-self-xss-warnings --headless=old --hide-scrollbars --mute-audio --blink-settings=primaryHoverType=2,availableHoverTypes=2,primaryPointerType=4,availablePointerTypes=4 --no-sandbox --headless=new --user-data-dir=/tmp/playwright_chromiumdev_profile-XXXXXX3LhwEK --remote-debugging-pipe --no-startup-window --noerrdialogs --ozone-platform=headless --ozone-override-screen-size=800,600 --use-angle=swiftshader-webgl (dns block)
    • Triggering command: /home/REDACTED/.cache/ms-playwright/chromium-1129/chrome-linux/chrome --type=utility --utility-sub-type=network.mojom.NetworkService --lang=en-US --service-sandbox-type=none --no-sandbox --disable-dev-shm-usage --use-angle=swiftshader-webgl --mute-audio --crashpad-handler-pid=3877 --enable-crash-reporter=, --noerrdialogs --user-data-dir=/tmp/playwright_chromiumdev_profile-XXXXXX3LhwEK --change-stack-guard-on-fork=enable --shared-files=v8_context_snapshot_data:100 --field-trial-handle=3,i,5006903823704742113,17032414060199126035,262144 --disable-features=AcceptCHFrame,AutoExpandDetailsElement,AvoidUnnecessaryBeforeUnloadCheckSync,CertificateTransparencyComponentUpdater,DestroyProfileOnBrowserClose,DialMediaRouteProvider,GlobalMediaControls,HttpsUpgrades,ImprovedCookieControls,LazyFrameLoading,MediaRouter,PaintHolding,PlzDedicatedWorker,Translate --variations-seed-version (dns block)
  • redirector.gvt1.com
    • Triggering command: /home/REDACTED/.cache/ms-playwright/chromium-1129/chrome-linux/chrome --type=utility --utility-sub-type=network.mojom.NetworkService --lang=en-US --service-sandbox-type=none --no-sandbox --disable-dev-shm-usage --use-angle=swiftshader-webgl --mute-audio --crashpad-handler-pid=3877 --enable-crash-reporter=, --noerrdialogs --user-data-dir=/tmp/playwright_chromiumdev_profile-XXXXXX3LhwEK --change-stack-guard-on-fork=enable --shared-files=v8_context_snapshot_data:100 --field-trial-handle=3,i,5006903823704742113,17032414060199126035,262144 --disable-features=AcceptCHFrame,AutoExpandDetailsElement,AvoidUnnecessaryBeforeUnloadCheckSync,CertificateTransparencyComponentUpdater,DestroyProfileOnBrowserClose,DialMediaRouteProvider,GlobalMediaControls,HttpsUpgrades,ImprovedCookieControls,LazyFrameLoading,MediaRouter,PaintHolding,PlzDedicatedWorker,Translate --variations-seed-version (dns block)
    • Triggering command: /home/REDACTED/.cache/ms-playwright/chromium-1129/chrome-linux/chrome --disable-field-trial-config --disable-REDACTED-networking --disable-REDACTED-timer-throttling --disable-REDACTEDing-occluded-windows --disable-back-forward-cache --disable-breakpad --disable-client-side-phishing-detection --disable-component-extensions-with-REDACTED-pages --disable-component-update --no-default-browser-check --disable-default-apps --disable-dev-shm-usage --disable-extensions --disable-features=ImprovedCookieControls,LazyFrameLoading,GlobalMediaControls,DestroyProfileOnBrowserClose,MediaRouter,DialMediaRouteProvider,AcceptCHFrame,AutoExpandDetailsElement,CertificateTransparencyComponentUpdater,AvoidUnnecessaryBeforeUnloadCheckSync,Translate,HttpsUpgrades,PaintHolding,PlzDedicatedWorker --allow-pre-commit-input --disable-hang-monitor --disable-ipc-flooding-protection --disable-popup-blocking --disable-prompt-on-repost --disable-renderer-REDACTEDing --force-color-profile=srgb --metrics-recording-only --no-first-run --enable-automation --password-store=basic --use-mock-keychain --no-service-autorun --export-tagged-pdf --disable-search-engine-choice-screen --unsafely-disable-devtools-self-xss-warnings --headless=old --hide-scrollbars --mute-audio --blink-settings=primaryHoverType=2,availableHoverTypes=2,primaryPointerType=4,availablePointerTypes=4 --no-sandbox --headless=new --user-data-dir=/tmp/playwright_chromiumdev_profile-XXXXXX3LhwEK --remote-debugging-pipe --no-startup-window --noerrdialogs --ozone-platform=headless --ozone-override-screen-size=800,600 --use-angle=swiftshader-webgl (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Base automatically changed from 397-playwright-still-not-running-in-parallel to master October 11, 2025 23:46
@dwarwick dwarwick closed this Oct 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants