fix: rediscover built-in generic test macros and reflect them in target/manifest.json under --use-v2-parser - #15925
Conversation
…et/manifest.json under --use-v2-parser
rediscover_adapter_macros() evicted internal-package macros then re-parsed
replacements using only MacroParser over macro_paths. The four built-in
generic tests (test_not_null, test_unique, test_accepted_values,
test_relationships) are {% test %} blocks under tests/generic/, parsed by
GenericTestParser over generic_test_paths instead, so they were evicted and
never restored, leaving them undefined at test-compile time.
Also, target/manifest.json was written by copying Fusion's raw handoff file
before rediscovery ran, so the on-disk artifact still reflected Fusion's
bundled macros rather than the corrected ones actually used to compile.
manifest.json is now written from the corrected in-memory manifest after
rediscovery completes; semantic_manifest.json is unaffected and still copied
as-is.
![codescene-delta-analysis[bot]](https://avatars.githubusercontent.com/in/53921?s=60&v=4){kind=small}
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
There was a problem hiding this comment.
Pull request overview
This PR fixes two Fusion (--use-v2-parser) manifest-loading issues in dbt-core: (1) built-in generic test macros were being evicted during adapter macro rediscovery and not restored, and (2) target/manifest.json could be written from Fusion’s pre-rediscovery handoff artifact rather than the corrected in-memory Manifest, creating an on-disk/compile-time mismatch.
Changes:
- Extend
rediscover_adapter_macros()to also re-parse built-in generic tests viaGenericTestParserovergeneric_test_paths. - Write
target/manifest.jsonfrom the corrected in-memoryManifestafter rediscovery (while continuing to copysemantic_manifest.jsonas-is). - Add/update unit tests to cover restored generic test macros and corrected
manifest.jsonwriting behavior.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
core/dbt/parser/fusion.py |
Writes manifest.json from the corrected in-memory manifest and re-parses generic tests during macro rediscovery. |
tests/unit/parser/test_fusion.py |
Updates/extends unit tests to validate corrected manifest writing and generic test macro restoration. |
.changes/unreleased/Fixes-20260812-133414.yaml |
Changelog entry for corrected target/manifest.json writing behavior. |
.changes/unreleased/Fixes-20260812-130820.yaml |
Changelog entry for restoring built-in generic test macros after Fusion manifest load. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## 1.latest #15925 +/- ##
============================================
- Coverage 91.52% 91.51% -0.01%
============================================
Files 222 222
Lines 28259 28268 +9
============================================
+ Hits 25864 25870 +6
- Misses 2395 2398 +3
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
The Fixes-20260812-130820.yaml changelog entry had leftover XML-like tags appended after the valid YAML content, breaking the check-yaml pre-commit hook in CI.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
There was a problem hiding this comment.
Gates Failed
(1 file with Complex Method)
Our agent can fix these. Install it.
Gates Passed
Reason for failure
| Enforce advisory code health rules | Violations | Code Health Impact | |
|---|---|---|---|
| fusion.py | 1 advisory rule | 8.10 → 8.04 | Suppress |
Quality Gate Profile: Clean Code Collective
Install CodeScene MCP: safeguard and uplift AI-generated code. Catch issues early with our IDE extension and CLI tool.
Summary
Fixes two related bugs in the
--use-v2-parser(Fusion) manifest-loading path, both surfacing as the built-in generic tests (not_null,unique,accepted_values,relationships) failing to compile.Fixes #15914
rediscover_adapter_macros()(added in fix: re-parse adapter macros from installed package after loading Fusion manifest #15529) evicts internal-package macros embedded by Fusion, then re-parses replacements from the installed adapter/dbt-core packages using onlyMacroParserovermacro_paths. The four built-in generic tests are{% test %}blocks undertests/generic/, parsed by a different class (GenericTestParser) over a different path list (generic_test_paths). They were evicted but never restored, leaving them permanently missing from the compile-time macro namespace and producing'test_not_null' is undefined. Fixed by adding an equivalent reparse pass withGenericTestParserovergeneric_test_paths.target/manifest.jsonwas written by copying Fusion's raw handoff file beforerediscover_adapter_macrosran, so the on-disk artifact still reflected Fusion's bundled (pre-correction) macros rather than what was actually used to compile — a discrepancy an external consumer ofmanifest.jsonwould otherwise never catch. Fixed by writingmanifest.jsonfrom the corrected in-memory manifest after rediscovery completes;semantic_manifest.jsonis unaffected by macro rediscovery and is still copied as-is.Added
TestRediscoverAdapterMacros::test_restores_evicted_generic_test_macrosand updatedTestParseWithFusion::test_write_json_writes_corrected_manifest_to_target_dirto cover both fixes. Manually reproduced end-to-end with a minimal duckdb project against bothdbt-core-experimental-parser2.0.0a1 and 2.0.0b1: the built-in generic tests went fromERROR: 'test_not_null' is undefinedtoPASS, and the manifest.json fix was confirmed by injecting a marker into the installed adapter'stests/generic/builtin.sqland observing it now appears in the writtentarget/manifest.jsonpost-rediscovery.