Skip to content

Conversation

@cyberthirst
Copy link
Collaborator

@cyberthirst cyberthirst commented May 12, 2025

What I did

How I did it

How to verify it

Commit message

external tools like vyper-hol and ivy need access to vyper's test suite
to verify their implementations against the reference compiler. this
adds `--export <dir>` to pytest which serializes test execution traces
to JSON files mirroring the test directory structure.

each test export includes:
- contract source, annotated AST, compiler settings, ABI, bytecode
- sequence of deployments and calls with inputs/outputs
- environment state (block info, gas, balances)

fixture dependencies are tracked to ensure traces capture the full setup
sequence needed to reproduce each test.

implementation notes:
- fixture visibility is filtered by pytest scope
  (session/module/function) so consumers know what chain state is
  accessible at each point
- fixtures that don't produce traces (no deployments/calls) are omitted
- execution order is preserved in the timeline, not just the dependency
  graph, since replay order matters for chain state

Description for the changelog

Cute Animal Picture

Put a link to a cute animal picture inside the parenthesis-->

@codecov
Copy link

codecov bot commented May 12, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.29%. Comparing base (44f3847) to head (0d61eca).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4638      +/-   ##
==========================================
+ Coverage   93.28%   93.29%   +0.01%     
==========================================
  Files         148      148              
  Lines       20561    20579      +18     
  Branches     3570     3572       +2     
==========================================
+ Hits        19180    19199      +19     
  Misses        924      924              
+ Partials      457      456       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@cyberthirst
Copy link
Collaborator Author

waiting for fix of: paradigmxyz/pyrevm#24

@cyberthirst
Copy link
Collaborator Author

waiting for fix of: paradigmxyz/pyrevm#24

fixed, new release was made - as such i pushed: f3ffd22

@cyberthirst
Copy link
Collaborator Author

sucessfully replayed the latest 10ec93e version in boa and also ivy

@xrchz
Copy link
Contributor

xrchz commented Jan 13, 2026

What do we need to do to get this merged?

@charles-cooper charles-cooper changed the title feat[tool]: export test suite to json feat[tool]: add test export infrastructure Jan 13, 2026
@charles-cooper charles-cooper enabled auto-merge (squash) January 13, 2026 16:46
@charles-cooper charles-cooper merged commit d2ba5c5 into vyperlang:master Jan 13, 2026
160 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants