chore(ai): update `tests/AGENTS.md`, add `pytest-vcr` skill and add `ENABLE_VERTEX` flag for better ergonomics by dsfaccini · Pull Request #4211 · pydantic/pydantic-ai

dsfaccini · 2026-02-05T18:07:24Z

Adds testing infrastructure and documentation to help AI coding agents work effectively in this repository.

Pre-Review Checklist

Any AI generated code has been reviewed line-by-line by the human PR author, who stands by it.
No breaking changes in accordance with the version policy.
Linting and type checking pass per make format and make typecheck.
PR title is fit for the release changelog.

Pre-Merge Checklist

New tests for any fix or new behavior, maintaining 100% coverage.
Updated documentation for new features and behaviors, including docstrings for API docs.

Changes Made

tests/AGENTS.md: Testing guidelines covering test file structure, parametrization patterns, the model fixture, VCR cassette workflow, key fixtures (allow_model_requests, env, binary content, SSRF protection), assertion helpers (IsNow, IsStr, IsSameStr, etc.), directory layout, and curated rules extracted from PR review patterns.
.agents/skills/pytest-vcr/SKILL.md: A skill for recording, rewriting, and debugging VCR cassettes, with step-by-step instructions for common workflows and provider-specific auth notes (supporting GOOGLE_APPLICATION_CREDENTIALS, service accounts, and ADC for Vertex).
.agents/skills/pytest-vcr/parse_cassette.py: Helper script to parse and display cassette contents in a readable format.
tests/conftest.py:
- _guard_ssrf_in_vcr_tests: autouse fixture that raises a clear error if a VCR test triggers SSRF validation without the disable_ssrf_protection_for_vcr fixture.
- skip_unless_vertex / updated vertex_provider: allows local Vertex tests when ENABLE_VERTEX=1 is set (previously required CI or manually commenting out a line).

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>

github-actions · 2026-02-05T18:19:23Z

Docs Preview

commit:	`d1129c7`
Preview URL:	https://65e6f484-pydantic-ai-previews.pydantic.workers.dev

.claude/skills/pytest-vcr/SKILL.md

.agents/skills/pytest-vcr/SKILL.md

.claude/skills/pytest-vcr/SKILL.md

tests/AGENTS.md

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

github-actions · 2026-03-07T00:00:18Z

tests/conftest.py

+
+
+@pytest.fixture(autouse=True)
+def _guard_ssrf_in_vcr_tests(request: pytest.FixtureRequest):


@DouweM asked above whether disable_ssrf_protection_for_vcr should just be autouse for all VCR tests. The author's response (explicit opt-in catches accidental URL downloads) makes sense to me, but this hasn't been acknowledged by a maintainer yet — worth confirming the approach before merging.

.agents/skills/pytest-vcr/run-bedrock-tests.sh

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>

…nges # Conflicts: # .gitignore # tests/conftest.py

Remove opinionated Bedrock/Vertex auth scripts and SKILL.md vertex section to reduce PR scope and ease merging. Fix .gitignore to properly un-ignore .agents/skills/ parent directory so whitelisted skills are tracked. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

…c-ai into david-claude-md-changes # Conflicts: # .agents/skills/pytest-vcr/run-vertex-tests.sh

dsfaccini · 2026-03-26T20:29:35Z

@copilot please write a summary of what this PR does

Copilot · 2026-03-26T20:32:05Z

@copilot please write a summary of what this PR does

This PR adds testing infrastructure and documentation to help AI coding agents work effectively in this repo:

tests/AGENTS.md — Testing guidelines covering test file structure, parametrization patterns, VCR cassette workflow, key fixtures (model, allow_model_requests, env, binary content, SSRF protection), assertion helpers, and directory layout. Also includes curated rules extracted from PR review patterns.

.agents/skills/pytest-vcr/SKILL.md — A skill for recording, rewriting, and debugging VCR cassettes, with step-by-step instructions for common workflows and provider-specific auth notes.

.agents/skills/pytest-vcr/parse_cassette.py — Helper script to parse and display cassette contents in a readable format.

tests/conftest.py — Two additions:

_guard_ssrf_in_vcr_tests: autouse fixture that raises a clear error if a VCR test triggers SSRF validation without opting in to disable_ssrf_protection_for_vcr
skip_unless_vertex / updated vertex_provider: allows local Vertex tests when ENABLE_VERTEX=1 is set (previously required CI or commenting out a line)

dsfaccini · 2026-03-26T20:57:44Z

additional to copilot's comments, this PR symlinks .claude/ to .agents/ and adds a CI check so PR's don't accidentally remove the symlink

dsfaccini added 4 commits February 5, 2026 12:22

add bare .env.vertex

9338fd4

env vertex

6dca255

add instructions and skill for testing + template env files

ba8ea32

remove dumb line

bdf9b54

This comment was marked as resolved.

Sign in to view

dsfaccini added chore test-suite Changes or improvements to the test suite better ai improvements to infra that aids ai coding agents in developing pydantic-ai (and with - pydantic-ai) labels Feb 5, 2026

Update .claude/skills/pytest-vcr/run-vertex-tests.sh

bb6f683

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>

This comment was marked as resolved.

Sign in to view

DouweM requested changes Feb 5, 2026

View reviewed changes

Update .claude/skills/pytest-vcr/parse_cassette.py

74c9d47

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>

This comment was marked as resolved.

Sign in to view

dsfaccini added 2 commits February 5, 2026 14:11

address comments

3b94f7f

Merge branch 'main' into david-claude-md-changes

b860143

This comment was marked as resolved.

Sign in to view

Update .env.vertex

1a1dcbc

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>

This comment was marked as resolved.

Sign in to view

dsfaccini and others added 2 commits February 5, 2026 16:26

Update .claude/skills/pytest-vcr/run-vertex-tests.sh

264d06e

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>

Merge branch 'main' into david-claude-md-changes

5ce9469

This comment was marked as resolved.

Sign in to view

claude loves git -C

de77e45

This comment was marked as resolved.

Sign in to view

dsfaccini added 2 commits February 5, 2026 16:38

just export, no .env.vertex

c652b7d

remove tests/CLAUDE.md pattern

ee64b3f

This comment was marked as resolved.

Sign in to view

dsfaccini and others added 2 commits February 5, 2026 17:03

Update .claude/skills/pytest-vcr/parse_cassette.py

ea7b06b

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>

unify vertex fixture

4a84edb

This comment was marked as resolved.

Sign in to view

github-actions bot removed the auto-review label Mar 6, 2026

fix stale path, tighten allowed-tools, remove private method ref

78e8f64

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

dsfaccini added the auto-review label Mar 6, 2026

This comment was marked as resolved.

Sign in to view

github-actions bot reviewed Mar 7, 2026

View reviewed changes

github-actions bot removed the auto-review label Mar 7, 2026

wip: pending dicussion on bedrock approach

1b562c6

dsfaccini commented Mar 10, 2026

View reviewed changes

.agents/skills/pytest-vcr/run-bedrock-tests.sh Outdated Show resolved Hide resolved

fix Cassette import path, remove dev-specific gcloud fallback

4950bcd

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

This comment was marked as resolved.

Sign in to view

dsfaccini and others added 6 commits March 22, 2026 14:32

Update .agents/skills/pytest-vcr/run-bedrock-tests.sh

ab228dd

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>

Merge remote-tracking branch 'upstream/main' into david-claude-md-cha…

bdaeb92

…nges # Conflicts: # .gitignore # tests/conftest.py

Merge branch 'david-claude-md-changes' of github.com:pydantic/pydanti…

5e03208

…c-ai into david-claude-md-changes # Conflicts: # .agents/skills/pytest-vcr/run-vertex-tests.sh

Merge branch 'main' into david-claude-md-changes

1441b09

remove bedrock auth script

45075c4

Copilot started work on behalf of dsfaccini March 26, 2026 20:29 View session

Copilot finished work on behalf of dsfaccini March 26, 2026 20:32

update tests/agents

d1129c7

dsfaccini changed the title ~~chore(ai): add testing instructions and skill~~ chore(ai): update tests/AGENTS.md, add pytest-vcr skill and add ENABLE_VERTEX flag for better ergonomics Mar 27, 2026



		@pytest.fixture(autouse=True)
		def _guard_ssrf_in_vcr_tests(request: pytest.FixtureRequest):

Conversation

dsfaccini commented Feb 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pre-Review Checklist

Pre-Merge Checklist

Changes Made

Uh oh!

This comment was marked as resolved.

Uh oh!

github-actions bot commented Feb 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Docs Preview

Uh oh!

This comment was marked as resolved.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

github-actions bot Mar 7, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

This comment was marked as resolved.

Uh oh!

dsfaccini commented Mar 26, 2026

Uh oh!

Copilot AI commented Mar 26, 2026

Uh oh!

dsfaccini commented Mar 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

dsfaccini commented Feb 5, 2026 •

edited

Loading

github-actions bot commented Feb 5, 2026 •

edited

Loading