Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .claude/skills/playwright-e2e/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,13 @@ Shared code (helpers, locators, configuration) for tests.
- **Default org in workspace** — pass `orgAlias: '…'` (e.g. `MINIMAL_ORG_ALIAS` / `DREAMHOUSE_ORG_ALIAS`) so `.sfdx/config.json` gets `target-org`. Omit `orgAlias` or use `undefined` for **no** `config.json` (no org).
- **Multi-package directory, no org** — `multiPackageNoOrgDesktopTest` (extend `noOrgDesktopTest`); creates a temp workspace with `sfdx-project.json` listing multiple `packageDirectories` (`force-app`, `extra-pkg`). Use `multiPackageNoOrgTest` from `fixtures/index.ts` in test files.

**VSIX mode** (`useVsix` option):

- `createDesktopTest({ useVsix: true })` — installs built VSIXs into a hash-keyed cache dir (`.vscode-test/ext-<hash>/`) and launches VS Code with `--extensions-dir` instead of `--extensionDevelopmentPath`. Exercises real shipping artifact (bundled `dist/`, `.vscodeignore`, `packageUpdates`).
- Default: `process.env.E2E_FROM_VSIX === '1'` — set in CI to enable without code changes.
- Requires `vscode:package` to have run first (produces `.vsix` in package dir). org-browser `test:desktop` depends on `vscode:package` for this reason.
- Idempotent across parallel workers: atomic rename; second worker skips if cache exists.

## Span files (when debugging traces)

Available local + CI/GHA.
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/apexLogE2E.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ on:
type: string
default: ''
push:
branches-ignore: [ main, develop ]
branches-ignore: [ main, develop, feature/W-21432332 ] # vsix-desktop-e2e rollout
paths-ignore:
- '.claude/**'
- '.cursor/**'
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/apexReplayDebuggerE2E.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ on:
type: string
default: ''
push:
branches-ignore: [ main, develop ]
branches-ignore: [ main, develop, feature/W-21432332 ] # vsix-desktop-e2e rollout
paths-ignore:
- '.claude/**'
- '.cursor/**'
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/apexTestingE2E.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ on:
type: string
default: ''
push:
branches-ignore: [ main, develop ]
branches-ignore: [ main, develop, feature/W-21432332 ] # vsix-desktop-e2e rollout
paths-ignore:
- '.claude/**'
- '.cursor/**'
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/coreE2E.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ on:
type: string
default: ''
push:
branches-ignore: [ main, develop ]
branches-ignore: [ main, develop, feature/W-21432332 ] # vsix-desktop-e2e rollout
paths-ignore:
- '.claude/**'
- '.cursor/**'
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/metadataE2E.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ on:
type: string
default: ''
push:
branches-ignore: [ main, develop ]
branches-ignore: [ main, develop, feature/W-21432332 ] # vsix-desktop-e2e rollout
paths-ignore:
- '.claude/**'
- '.cursor/**'
Expand Down
3 changes: 3 additions & 0 deletions .github/workflows/orgBrowserE2E.yml
Original file line number Diff line number Diff line change
Expand Up @@ -174,6 +174,7 @@ jobs:
PLAYWRIGHT_VSCODE_VERSION: ${{ github.event_name != 'push' &&
inputs.vscode_version != '' && inputs.vscode_version ||
vars.PLAYWRIGHT_VSCODE_VERSION }}
E2E_FROM_VSIX: 1
steps:
- name: Checkout repository
uses: actions/checkout@v4
Expand Down Expand Up @@ -267,6 +268,7 @@ jobs:
DREAMHOUSE_ORG_ALIAS: orgBrowserDreamhouseTestOrg
CI: 1
VSCODE_DESKTOP: 1
E2E_FROM_VSIX: 1
run: ${{ matrix.test_runner_prefix }}npm run test:desktop -w salesforcedx-vscode-org-browser -- --reporter=html

- name: Retry failed tests (sequential)
Expand All @@ -276,6 +278,7 @@ jobs:
DREAMHOUSE_ORG_ALIAS: orgBrowserDreamhouseTestOrg
CI: 1
VSCODE_DESKTOP: 1
E2E_FROM_VSIX: 1
PLAYWRIGHT_WORKERS: 1
run: ${{ matrix.test_runner_prefix }}npm run test:desktop -w salesforcedx-vscode-org-browser -- --last-failed --reporter=html

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/orgE2E.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ on:
type: string
default: ''
push:
branches-ignore: [ main, develop ]
branches-ignore: [ main, develop, feature/W-21432332 ] # vsix-desktop-e2e rollout
paths-ignore:
- '.claude/**'
- '.cursor/**'
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/playwrightVscodeExtE2E.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ on:
type: string
default: ''
push:
branches-ignore: [main, develop]
branches-ignore: [main, develop, feature/W-21432332] # vsix-desktop-e2e rollout
paths-ignore:
- '.claude/**'
- '.cursor/**'
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/servicesE2E.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ on:
workflow_dispatch:
workflow_call:
push:
branches-ignore: [ main, develop ]
branches-ignore: [ main, develop, feature/W-21432332 ] # vsix-desktop-e2e rollout
paths-ignore:
- '.claude/**'
- '.cursor/**'
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/snippetsE2E.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ on:
type: string
default: ''
push:
branches-ignore: [ main, develop ]
branches-ignore: [ main, develop, feature/W-21432332 ] # vsix-desktop-e2e rollout
paths-ignore:
- '.claude/**'
- '.cursor/**'
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/soqlE2E.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ on:
type: string
default: ''
push:
branches-ignore: [ main, develop ]
branches-ignore: [ main, develop, feature/W-21432332 ] # vsix-desktop-e2e rollout
paths-ignore:
- '.claude/**'
- '.cursor/**'
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/testCommitExceptMain.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
name: Commit Workflow
on:
push:
branches-ignore: [main, develop]
branches-ignore: [main, develop, feature/W-21432332] # vsix-desktop-e2e rollout
paths-ignore:
- '.claude/**'
- '.cursor/**'
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/visualforceE2E.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ on:
type: string
default: ''
push:
branches-ignore: [ main, develop ]
branches-ignore: [ main, develop, feature/W-21432332 ] # vsix-desktop-e2e rollout
paths-ignore:
- '.claude/**'
- '.cursor/**'
Expand Down
3 changes: 3 additions & 0 deletions packages/playwright-vscode-ext/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,9 @@
"test:desktop": {
"env": {
"VSCODE_DESKTOP": "1",
"E2E_FROM_VSIX": {
"external": true
},
"PLAYWRIGHT_WORKERS": {
"external": true
}
Expand Down
170 changes: 144 additions & 26 deletions packages/playwright-vscode-ext/src/fixtures/createDesktopTest.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,10 @@
// This is Node.js test infrastructure, not extension code
import type { WorkerFixtures, TestFixtures } from './desktopFixtureTypes';
import { test as base, _electron as electron } from '@playwright/test';
import { downloadAndUnzipVSCode } from '@vscode/test-electron';
import type { ChildProcess } from 'node:child_process';
import { downloadAndUnzipVSCode, resolveCliPathFromVSCodeExecutablePath } from '@vscode/test-electron';
import { spawnSync, type ChildProcess } from 'node:child_process';
import * as crypto from 'node:crypto';
import { existsSync, readdirSync } from 'node:fs';
import * as fs from 'node:fs/promises';
import * as path from 'node:path';

Expand Down Expand Up @@ -51,6 +53,89 @@ type CreateDesktopTestOptions = {
disableOtherExtensions?: boolean;
/** Optional user settings to write to User/settings.json (e.g. to reduce GitHub/Git prompts). */
userSettings?: Record<string, unknown>;
/**
* When true, install VSIXs and launch VS Code with --extensions-dir instead of --extensionDevelopmentPath.
* Exercises the real shipping artifact (bundled dist/, packageUpdates, .vscodeignore).
* Defaults to `process.env.E2E_FROM_VSIX === '1'` — set that env var in CI to enable without code changes.
*/
useVsix?: boolean;
};

/**
* Resolve *.vsix files for the given package directories (relative to repo root packages/).
* Fails loudly if a package has 0 or >1 VSIX (wireit should produce exactly one).
*/
const resolveVsixPaths = (repoRoot: string, packageDirs: string[]): string[] =>
packageDirs.map(dir => {
const pkgDir = path.join(repoRoot, 'packages', dir);
const vsixFiles = existsSync(pkgDir) ? readdirSync(pkgDir).filter(f => f.endsWith('.vsix')) : [];
if (vsixFiles.length !== 1) {
throw new Error(
`Expected exactly 1 VSIX in packages/${dir}/ but found ${vsixFiles.length}: [${vsixFiles.join(', ')}]. ` +
`Run 'npm run vscode:package -w ${dir}' first.`
);
}
return path.join(pkgDir, vsixFiles[0]);
});

/**
* Compute a cache key from the combined sha256 of all VSIX files.
* Uses file sizes+mtimes as a fast proxy — avoids reading multi-MB VSIX bytes.
*/
const computeVsixCacheKey = async (vsixPaths: string[]): Promise<string> => {
const hash = crypto.createHash('sha256');
for (const p of vsixPaths) {
const stat = await fs.stat(p);
hash.update(`${p}:${stat.size}:${stat.mtimeMs}`);
}
return hash.digest('hex').slice(0, 16);
};

/**
* Install VSIXs into a hash-keyed cache dir under <repoRoot>/.vscode-test/ext-<hash>/.
* Each worker gets its own per-pid tmp dir to avoid concurrent install conflicts.
* Atomic rename to final cache dir; second worker to finish sees the dir exists and cleans up.
* Idempotent: skips if <dir>/extensions.json already exists (first worker won).
*/
const installVsixsToCache = async (
cacheDir: string,
vsixPaths: string[],
vscodeExecutable: string
): Promise<void> => {
const extensionsJson = path.join(cacheDir, 'extensions.json');
if (existsSync(extensionsJson)) {
return; // already installed — fast path
}

// Per-pid tmp dir: avoids concurrent workers writing to the same directory
const tmpDir = `${cacheDir}.tmp.${process.pid}`;
await fs.rm(tmpDir, { recursive: true, force: true });
await fs.mkdir(tmpDir, { recursive: true });

const cli = resolveCliPathFromVSCodeExecutablePath(vscodeExecutable);
const result = spawnSync(
cli,
[
'--extensions-dir',
tmpDir,
'--user-data-dir',
path.join(tmpDir, '.ud'),
...vsixPaths.flatMap(p => ['--install-extension', p])
],
{ stdio: 'inherit', shell: process.platform === 'win32' }
);

if (result.status !== 0) {
await fs.rm(tmpDir, { recursive: true, force: true });
throw new Error(`VSIX install failed (exit ${result.status}). VSIXs: ${vsixPaths.join(', ')}`);
}

// Atomic rename: first worker wins; others clean up their own tmp and use the winner's dir
try {
await fs.rename(tmpDir, cacheDir);
} catch {
await fs.rm(tmpDir, { recursive: true, force: true });
}
};

/** Creates a Playwright test instance configured for desktop Electron testing with services extension */
Expand All @@ -64,6 +149,12 @@ export const createDesktopTest = (options: CreateDesktopTestOptions) => {
userSettings
} = options;

const useVsix = options.useVsix ?? process.env.E2E_FROM_VSIX === '1';

// Current package name = the packages/<dir> that owns this fixture file.
// fixturesDir is e.g. <repoRoot>/packages/salesforcedx-vscode-org-browser/test/playwright/fixtures
const packageDir = path.basename(path.resolve(fixturesDir, '..', '..', '..'));

const test = base.extend<TestFixtures, WorkerFixtures>({
// Download VS Code once per worker (cached at repo root .vscode-test/)
vscodeExecutable: [
Expand All @@ -77,14 +168,32 @@ export const createDesktopTest = (options: CreateDesktopTestOptions) => {
{ scope: 'worker' }
],

// Install VSIXs once per worker into a hash-keyed cache dir (VSIX mode only)
installedExtensionsDir: [
async ({ vscodeExecutable }, use): Promise<void> => {
if (!useVsix) {
await use(undefined);
return;
}
const repoRoot = resolveRepoRoot(fixturesDir);
const allDirs = [packageDir, 'salesforcedx-vscode-services', ...additionalExtensionDirs];
const vsixPaths = resolveVsixPaths(repoRoot, allDirs);
const cacheKey = await computeVsixCacheKey(vsixPaths);
const cacheDir = path.join(repoRoot, '.vscode-test', `ext-${cacheKey}`);
await installVsixsToCache(cacheDir, vsixPaths, vscodeExecutable);
await use(cacheDir);
},
{ scope: 'worker' }
],

// Create workspace directory (shared with electronApp so tests can access path)
workspaceDir: async ({}, use): Promise<void> => {
const dir = emptyWorkspace ? await createEmptyTestWorkspace() : await createTestWorkspace(orgAlias);
await use(dir);
},

// Launch fresh Electron instance per test
electronApp: async ({ vscodeExecutable, workspaceDir }, use): Promise<void> => {
electronApp: async ({ vscodeExecutable, workspaceDir, installedExtensionsDir }, use): Promise<void> => {
// Use subdirectory of workspace for user data (keeps everything isolated and together)
const userDataDir = path.join(workspaceDir, '.vscode-test-user-data');
await fs.mkdir(userDataDir, { recursive: true });
Expand Down Expand Up @@ -113,24 +222,11 @@ export const createDesktopTest = (options: CreateDesktopTestOptions) => {
await fs.mkdir(userSettingsDir, { recursive: true });
await fs.writeFile(path.join(userSettingsDir, 'settings.json'), JSON.stringify(effectiveUserSettings, null, 2));
}
const extensionsDir = path.join(workspaceDir, '.vscode-test-extensions');
await fs.mkdir(extensionsDir, { recursive: true });

const packageRoot = path.resolve(fixturesDir, '..', '..', '..');

// Collect all extension paths: current extension + services + any additional

const videosDir = path.join(packageRoot, 'test-results', 'videos');
await fs.mkdir(videosDir, { recursive: true });

const extensionArgs = [
// Extension path is the package root (contains package.json and bundled dist/index.js)
packageRoot,
...additionalExtensionDirs
.concat(['salesforcedx-vscode-services'])
.map(dir => path.resolve(packageRoot, '..', dir))
].map(p => `--extensionDevelopmentPath=${p}`);

// Explicitly disable built-in GitHub/Copilot/Chat extensions that can trigger
// the GitHub OAuth browser tab on startup. `--disable-extensions` only disables
// user-installed extensions; built-ins must be disabled individually.
Expand All @@ -142,16 +238,38 @@ export const createDesktopTest = (options: CreateDesktopTestOptions) => {
'GitHub.copilot-chat'
].map(id => `--disable-extension=${id}`);

const launchArgs = [
`--user-data-dir=${userDataDir}`,
`--extensions-dir=${extensionsDir}`,
...extensionArgs,
...(disableOtherExtensions ? ['--disable-extensions'] : []),
...disabledBuiltins,
'--disable-workspace-trust',
'--no-sandbox',
workspaceDir
];
let launchArgs: string[];
if (useVsix) {
// VSIX mode: extensions installed into hash-keyed cache dir; no dev path needed
launchArgs = [
`--user-data-dir=${userDataDir}`,
`--extensions-dir=${installedExtensionsDir}`,
...disabledBuiltins,
'--disable-workspace-trust',
'--no-sandbox',
workspaceDir
];
} else {
const extensionsDir = path.join(workspaceDir, '.vscode-test-extensions');
await fs.mkdir(extensionsDir, { recursive: true });
const extensionArgs = [
// Extension path is the package root (contains package.json and bundled dist/index.js)
packageRoot,
...additionalExtensionDirs
.concat(['salesforcedx-vscode-services'])
.map(dir => path.resolve(packageRoot, '..', dir))
].map(p => `--extensionDevelopmentPath=${p}`);
launchArgs = [
`--user-data-dir=${userDataDir}`,
`--extensions-dir=${extensionsDir}`,
...extensionArgs,
...(disableOtherExtensions ? ['--disable-extensions'] : []),
...disabledBuiltins,
'--disable-workspace-trust',
'--no-sandbox',
workspaceDir
];
}

const electronApp = await electron.launch({
executablePath: vscodeExecutable,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ import type { ElectronApplication, Page } from '@playwright/test';
/** Worker-scoped fixtures (shared across tests in same worker) */
export type WorkerFixtures = {
vscodeExecutable: string;
/** Resolved extensions dir: VSIX-install cache path (VSIX mode) or undefined (dev-path mode). */
installedExtensionsDir: string | undefined;
};

/** Test-scoped fixtures (fresh for each test) */
Expand Down
3 changes: 3 additions & 0 deletions packages/salesforcedx-vscode-apex-log/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -213,6 +213,9 @@
"command": "playwright test --config=test/playwright/playwright.config.desktop.ts",
"env": {
"VSCODE_DESKTOP": "1",
"E2E_FROM_VSIX": {
"external": true
},
"PLAYWRIGHT_WORKERS": {
"external": true
}
Expand Down
Loading
Loading