Skip to content

Commit 70ae3d0

Browse files
authored
feat: add agent-friendly test output commands (#75)
Closes #72 Add test:agent, lint:agent, and typecheck:agent scripts that produce minimal, machine-readable output optimized for AI coding agents. Changes: - Add test:agent scripts using Mocha's min reporter (only failures + summary) - Add lint:agent scripts using ESLint's --quiet flag - Add typecheck:agent scripts using tsc --pretty false - Add runSilent() helper using oclif's captureOutput to suppress command output - Update stubParse helpers to set log-level: silent by default - Add global test setup to set SFCC_LOG_LEVEL=silent - Add logger destination option for custom output streams in tests - Update tests to use runSilent() for non-JSON mode tests - Remove unused operations/sites module (dead code) - Update AGENTS.md with agent-friendly command section - Update testing skill documentation with output silencing patterns
1 parent 2e231be commit 70ae3d0

34 files changed

+374
-188
lines changed

.claude/skills/testing/SKILL.md

Lines changed: 99 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,19 @@ This skill covers project-specific testing patterns for the B2C CLI project.
1818

1919
## Running Tests
2020

21+
For coding agents (minimal output - only failures shown):
22+
23+
```bash
24+
# Run tests - only failures + summary
25+
pnpm run test:agent
26+
27+
# Run tests for specific package
28+
pnpm --filter @salesforce/b2c-tooling-sdk run test:agent
29+
pnpm --filter @salesforce/b2c-cli run test:agent
30+
```
31+
32+
For debugging (full output with coverage):
33+
2134
```bash
2235
# Run all tests with coverage
2336
pnpm run test
@@ -307,6 +320,59 @@ const client = new WebDavClient(TEST_HOST, mockAuth);
307320
const customAuth = new MockAuthStrategy('custom-token');
308321
```
309322

323+
## Silencing Test Output
324+
325+
Commands may produce console output (tables, formatted displays) even in tests. Use these helpers to keep test output clean.
326+
327+
### Using runSilent for Output Capture
328+
329+
The `runSilent` helper uses oclif's `captureOutput` to suppress stdout/stderr:
330+
331+
```typescript
332+
import { runSilent } from '../../helpers/test-setup.js';
333+
334+
it('returns data in non-JSON mode', async () => {
335+
const command = new MyCommand([], {} as any);
336+
// ... setup ...
337+
338+
// Silences any console output from the command
339+
const result = await runSilent(() => command.run());
340+
341+
expect(result.data).to.exist;
342+
});
343+
```
344+
345+
Use `runSilent` when:
346+
- Testing non-JSON output modes (tables, formatted displays)
347+
- The test doesn't need to verify console output content
348+
- You want clean test output with only pass/fail summary
349+
350+
### When Output Verification is Needed
351+
352+
If you need to verify console output, stub `ux.stdout` directly:
353+
354+
```typescript
355+
import { ux } from '@oclif/core';
356+
357+
it('prints table in non-JSON mode', async () => {
358+
const stdoutStub = sinon.stub(ux, 'stdout');
359+
360+
await command.run();
361+
362+
expect(stdoutStub.called).to.be.true;
363+
});
364+
```
365+
366+
### stubParse Sets Silent Logging
367+
368+
The `stubParse` helper automatically sets `'log-level': 'silent'` to reduce pino logger output:
369+
370+
```typescript
371+
// stubParse includes silent log level by default
372+
stubParse(command, {server: 'test.demandware.net'});
373+
// Equivalent to: {server: 'test.demandware.net', 'log-level': 'silent'}
374+
```
375+
310376
## Command Test Guidelines
311377

312378
Command tests should focus on **command-specific logic**, not trivial flag verification.
@@ -445,15 +511,43 @@ pnpm run test
445511
open coverage/index.html
446512
```
447513

514+
## Test Helpers Reference
515+
516+
### CLI Package (`packages/b2c-cli/test/helpers/`)
517+
518+
| Helper | Purpose |
519+
|--------|---------|
520+
| `runSilent(fn)` | Capture and suppress stdout/stderr from command execution |
521+
| `stubParse(command, flags, args)` | Stub oclif's parse method with flags (includes silent log level) |
522+
| `createTestCommand(CommandClass, config, flags, args)` | Create command instance with stubbed parse |
523+
| `createIsolatedConfigHooks()` | Mocha hooks for config isolation |
524+
| `createIsolatedEnvHooks()` | Mocha hooks for env var isolation |
525+
526+
### SDK Package (`packages/b2c-tooling-sdk/test/helpers/`)
527+
528+
| Helper | Purpose |
529+
|--------|---------|
530+
| `MockAuthStrategy` | Mock authentication for API clients |
531+
| `stubParse(command, flags, args)` | Stub oclif's parse method (includes silent log level) |
532+
| `createNullStream()` | Create a writable stream that discards output |
533+
| `CapturingStream` | Writable stream that captures output for assertions |
534+
535+
### SDK Test Utils (exported from package)
536+
537+
```typescript
538+
import { isolateConfig, restoreConfig } from '@salesforce/b2c-tooling-sdk/test-utils';
539+
```
540+
448541
## Writing Tests Checklist
449542

450543
1. Create test file in `test/` mirroring source structure
451544
2. Use `.test.ts` suffix
452545
3. Import from package names, not relative paths
453546
4. Set up MSW server for HTTP tests (avoid fake timers)
454547
5. Use `isolateConfig()`/`restoreConfig()` for config-dependent tests
455-
6. Use `pollInterval` option for polling operations
456-
7. Use MockAuthStrategy for authenticated clients
457-
8. Test both success and error paths
458-
9. Focus on command-specific logic, not trivial delegation
459-
10. Run tests: `pnpm --filter <package> run test`
548+
6. Use `runSilent()` for commands that produce console output
549+
7. Use `pollInterval` option for polling operations
550+
8. Use MockAuthStrategy for authenticated clients
551+
9. Test both success and error paths
552+
10. Focus on command-specific logic, not trivial delegation
553+
11. Run tests: `pnpm --filter <package> run test`

AGENTS.md

Lines changed: 32 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -18,13 +18,41 @@ pnpm run build
1818
pnpm --filter @salesforce/b2c-cli run build
1919
pnpm --filter @salesforce/b2c-tooling-sdk run build
2020

21-
# Run tests (includes linting)
22-
pnpm run test
23-
2421
# Dev mode for CLI (uses source files directly)
2522
pnpm --filter @salesforce/b2c-cli run dev
2623
# or using convenience script
2724
./cli
25+
```
26+
27+
## Commands for Coding Agents
28+
29+
These commands produce condensed output optimized for AI coding agents:
30+
31+
```bash
32+
# Run tests (minimal output - only failures + summary)
33+
pnpm run test:agent
34+
35+
# Run tests for specific package
36+
pnpm --filter @salesforce/b2c-cli run test:agent
37+
pnpm --filter @salesforce/b2c-tooling-sdk run test:agent
38+
39+
# Lint (errors only, no warnings)
40+
pnpm run lint:agent
41+
42+
# Type-check (single-line errors, no color)
43+
pnpm run typecheck:agent
44+
45+
# Format check (lists only files needing formatting)
46+
pnpm run -r format:check
47+
```
48+
49+
## Verbose Commands (Debugging/CI)
50+
51+
Use these for detailed output during debugging or in CI pipelines:
52+
53+
```bash
54+
# Run tests with full output and coverage
55+
pnpm run test
2856

2957
# Run tests for specific package
3058
pnpm --filter @salesforce/b2c-cli run test
@@ -33,7 +61,7 @@ pnpm --filter @salesforce/b2c-tooling-sdk run test
3361
# Format code with prettier
3462
pnpm run -r format
3563

36-
# Lint only (without tests)
64+
# Lint with full output
3765
pnpm run -r lint
3866
```
3967

package.json

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,9 +7,12 @@
77
"start": "pnpm --filter @salesforce/b2c-cli run dev",
88
"test": "pnpm -r test",
99
"test:unit": "pnpm -r run test:unit",
10+
"test:agent": "pnpm -r run test:agent",
1011
"coverage": "pnpm -r run coverage",
1112
"format": "pnpm -r run format",
1213
"lint": "pnpm -r run lint",
14+
"lint:agent": "pnpm -r run lint:agent",
15+
"typecheck:agent": "pnpm -r run typecheck:agent",
1316
"build": "pnpm -r run build",
1417
"docs:api": "typedoc",
1518
"docs:dev": "pnpm run docs:api && vitepress dev docs",

packages/b2c-cli/package.json

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -253,6 +253,8 @@
253253
"scripts": {
254254
"build": "shx rm -rf dist && tsc -p tsconfig.build.json",
255255
"lint": "eslint",
256+
"lint:agent": "eslint --quiet",
257+
"typecheck:agent": "tsc --noEmit --pretty false",
256258
"format": "prettier --write src",
257259
"format:check": "prettier --check src",
258260
"postpack": "shx rm -f oclif.manifest.json",
@@ -262,6 +264,7 @@
262264
"test": "c8 env OCLIF_TEST_ROOT=. mocha --forbid-only --exclude \"test/functional/e2e/**\" \"test/**/*.test.ts\"",
263265
"test:ci": "c8 env OCLIF_TEST_ROOT=. mocha --forbid-only --exclude \"test/functional/e2e/**\" --reporter json --reporter-option output=test-results.json \"test/**/*.test.ts\"",
264266
"test:unit": "env OCLIF_TEST_ROOT=. mocha --forbid-only --exclude \"test/functional/e2e/**\" \"test/**/*.test.ts\"",
267+
"test:agent": "env OCLIF_TEST_ROOT=. mocha --forbid-only --reporter min --exclude \"test/functional/e2e/**\" \"test/**/*.test.ts\"",
265268
"test:e2e": "env OCLIF_TEST_ROOT=. mocha --forbid-only --reporter json --reporter-option output=test-results.json \"test/functional/e2e/**/*.test.ts\"",
266269
"coverage": "c8 report",
267270
"version": "oclif readme && git add README.md",

packages/b2c-cli/test/commands/_test/index.test.ts

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,9 @@ import {runCommand} from '@oclif/test';
77
import {expect} from 'chai';
88

99
describe('_test', () => {
10-
it('runs the smoke test command without errors', async () => {
10+
// Skip in automated tests - this is a debug command that intentionally produces log output
11+
// Run manually with: ./cli _test
12+
it.skip('runs the smoke test command without errors', async () => {
1113
const {error} = await runCommand('_test');
1214
expect(error).to.be.undefined;
1315
});

packages/b2c-cli/test/commands/docs/search.test.ts

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ import {expect} from 'chai';
99
import {afterEach, beforeEach} from 'mocha';
1010
import sinon from 'sinon';
1111
import DocsSearch from '../../../src/commands/docs/search.js';
12-
import {createIsolatedConfigHooks, createTestCommand} from '../../helpers/test-setup.js';
12+
import {createIsolatedConfigHooks, createTestCommand, runSilent} from '../../helpers/test-setup.js';
1313

1414
describe('docs search', () => {
1515
const hooks = createIsolatedConfigHooks();
@@ -43,7 +43,7 @@ describe('docs search', () => {
4343
const listStub = sinon.stub().returns([{id: 'a', title: 'A', filePath: 'a.md'}]);
4444
command.operations = {...command.operations, listDocs: listStub};
4545

46-
const result = await command.run();
46+
const result = await runSilent(() => command.run());
4747

4848
expect(result.entries).to.have.length(1);
4949
});
@@ -69,7 +69,7 @@ describe('docs search', () => {
6969
const searchStub = sinon.stub().returns([{entry: {id: 'a', title: 'A', filePath: 'a.md'}, score: 0.1}]);
7070
command.operations = {...command.operations, searchDocs: searchStub};
7171

72-
const result = await command.run();
72+
const result = await runSilent(() => command.run());
7373

7474
expect(result.results).to.have.length(1);
7575
});

packages/b2c-cli/test/commands/ecdn/security/get.test.ts

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ import {expect} from 'chai';
77
import {afterEach, beforeEach} from 'mocha';
88
import sinon from 'sinon';
99
import EcdnSecurityGet from '../../../../src/commands/ecdn/security/get.js';
10-
import {createIsolatedConfigHooks, createTestCommand} from '../../../helpers/test-setup.js';
10+
import {createIsolatedConfigHooks, createTestCommand, runSilent} from '../../../helpers/test-setup.js';
1111

1212
/**
1313
* Unit tests for eCDN security get command CLI logic.
@@ -98,7 +98,7 @@ describe('ecdn security get', () => {
9898
}),
9999
});
100100

101-
const result = await command.run();
101+
const result = await runSilent(() => command.run());
102102

103103
expect(result.settings.securityLevel).to.equal('high');
104104
expect(result.settings.wafEnabled).to.be.true;

packages/b2c-cli/test/commands/ecdn/zones/list.test.ts

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ import {expect} from 'chai';
77
import {afterEach, beforeEach} from 'mocha';
88
import sinon from 'sinon';
99
import EcdnZonesList from '../../../../src/commands/ecdn/zones/list.js';
10-
import {createIsolatedConfigHooks, createTestCommand} from '../../../helpers/test-setup.js';
10+
import {createIsolatedConfigHooks, createTestCommand, runSilent} from '../../../helpers/test-setup.js';
1111

1212
/**
1313
* Unit tests for eCDN zones list command CLI logic.
@@ -129,7 +129,7 @@ describe('ecdn zones list', () => {
129129
}),
130130
});
131131

132-
const result = await command.run();
132+
const result = await runSilent(() => command.run());
133133

134134
expect(result).to.have.property('total', 1);
135135
expect(result.zones).to.have.lengthOf(1);

packages/b2c-cli/test/commands/ods/create.test.ts

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ import {expect} from 'chai';
88
import sinon from 'sinon';
99
import OdsCreate from '../../../src/commands/ods/create.js';
1010
import {isolateConfig, restoreConfig} from '@salesforce/b2c-tooling-sdk/test-utils';
11+
import {runSilent} from '../../helpers/test-setup.js';
1112

1213
function stubCommandConfigAndLogger(command: any, sandboxApiHost = 'admin.dx.test.com'): void {
1314
Object.defineProperty(command, 'config', {
@@ -199,7 +200,7 @@ describe('ods create', () => {
199200
}),
200201
});
201202

202-
const result = await command.run();
203+
const result = await runSilent(() => command.run());
203204

204205
expect(result.id).to.equal('sb-123');
205206
});
@@ -257,7 +258,7 @@ describe('ods create', () => {
257258
},
258259
});
259260

260-
await command.run();
261+
await runSilent(() => command.run());
261262

262263
expect(requestBody.settings).to.be.undefined;
263264
});

packages/b2c-cli/test/commands/ods/get.test.ts

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ import sinon from 'sinon';
99

1010
import OdsGet from '../../../src/commands/ods/get.js';
1111
import {isolateConfig, restoreConfig} from '@salesforce/b2c-tooling-sdk/test-utils';
12+
import {runSilent} from '../../helpers/test-setup.js';
1213

1314
function stubCommandConfigAndLogger(command: any, sandboxApiHost = 'admin.dx.test.com'): void {
1415
Object.defineProperty(command, 'config', {
@@ -134,7 +135,7 @@ describe('ods get', () => {
134135
}),
135136
});
136137

137-
const result = await command.run();
138+
const result = await runSilent(() => command.run());
138139

139140
// Command returns the sandbox data regardless of JSON mode
140141
expect(result.id).to.equal('sandbox-123');

0 commit comments

Comments
 (0)