Skip to content

SKILL.md

Latest commit

 

History

History
121 lines (84 loc) · 4.25 KB

SKILL.md

File metadata and controls

121 lines (84 loc) · 4.25 KB
name pytest-vcr
description Record, rewrite, and debug VCR cassettes for HTTP recordings. Use when running tests with --record-mode, verifying cassette playback, or inspecting request/response bodies in YAML cassettes.
allowed-tools Bash(uv run pytest *), Bash(uv run python .agents/skills/pytest-vcr/parse_cassette.py *), Bash(.agents/skills/pytest-vcr/run-vertex-tests.sh *), Bash(source .env && uv run pytest *), Bash(git diff *)

Pytest VCR Workflow

Use this skill when recording or re-recording VCR cassettes for tests, or when debugging cassette contents.

Prerequisites

  • Verify .env exists: test -f .env && echo 'ok' || echo 'missing'
  • Missing API keys will cause clear test errors at runtime

Important flags

  • --record-mode=rewrite : Record cassettes (works for both new and existing)
  • --lf : Run only the last failed tests
  • -vv : Verbose output
  • --tb=line : Short traceback output
  • -k="" : Run tests matching the given substring expression

Recording Cassettes

Step 1: Record cassettes

source .env && uv run pytest path/to/test.py::test_function_name -v --tb=line --record-mode=rewrite

Multiple tests can be specified:

source .env && uv run pytest path/to/test.py::test_one path/to/test.py::test_two -v --tb=line --record-mode=rewrite

Step 2: Verify recordings

Run the same tests WITHOUT --record-mode to verify cassettes play back correctly:

source .env && uv run pytest path/to/test.py::test_function_name -vv --tb=line

Step 3: Review snapshots

If tests use snapshot() assertions:

  • The test run in Step 2 auto-fills snapshot content
  • Review the generated snapshot files to ensure they match expected output
  • You only review - don't manually write snapshot contents
  • Snapshots capture what the test actually produced, additional to explicit assertions

Parsing Cassettes

Parse VCR cassette YAML files to inspect request/response bodies without dealing with raw YAML.

Usage

uv run python .agents/skills/pytest-vcr/parse_cassette.py <cassette_path> [--interaction N]

Examples

# Parse all interactions in a cassette
uv run python .agents/skills/pytest-vcr/parse_cassette.py tests/models/cassettes/test_foo/test_bar.yaml

# Parse only interaction 1 (0-indexed)
uv run python .agents/skills/pytest-vcr/parse_cassette.py tests/models/cassettes/test_foo/test_bar.yaml --interaction 1

Output

For each interaction, shows:

  • Request: method, URI, parsed body (truncated base64)
  • Response: status code, parsed body (truncated base64)

Base64 strings longer than 100 chars are truncated for readability.

Vertex AI Tests

Vertex tests use the skip_unless_vertex fixture from tests/conftest.py — they only run in CI or when ENABLE_VERTEX=1 is set. ENABLE_VERTEX=1 is only needed when recording/rewriting cassettes locally; during playback, cassettes replay without live auth. Add skip_unless_vertex: None as a parameter to any new vertex test.

Vertex auth works two ways:

  • GOOGLE_APPLICATION_CREDENTIALS: set this env var to a service account JSON path — no gcloud needed
  • gcloud: the script auto-detects project and checks auth via gcloud

Use the provided script:

# Record Vertex cassettes
.agents/skills/pytest-vcr/run-vertex-tests.sh tests/path/to/test.py -v --tb=line --record-mode=rewrite

# Verify playback
.agents/skills/pytest-vcr/run-vertex-tests.sh tests/path/to/test.py -vv --tb=line

If using gcloud and auth fails:

gcloud auth application-default login
gcloud config set project <your-project-id>

Full Workflow Example

# 1. Record cassette
source .env && uv run pytest tests/models/test_openai.py::test_chat_completion -v --tb=line --record-mode=rewrite

# 2. Verify playback and fill snapshots
source .env && uv run pytest tests/models/test_openai.py::test_chat_completion -vv --tb=line

# 3. Review test code diffs (excludes cassettes)
git diff tests/ -- ':!**/cassettes/**'

# 4. List new/changed cassettes (name only - use parse_cassette.py to inspect)
git diff --name-only tests/ -- '**/cassettes/**'

# 5. Inspect cassette contents if needed
uv run python .agents/skills/pytest-vcr/parse_cassette.py tests/models/cassettes/test_openai/test_chat_completion.yaml