fix: detect ref() calls nested in Jinja macro arguments

[eitsupi]

Fixes #91

Problem

ref() calls passed as arguments to macros that minijinja cannot resolve (package macros such as shared_macros.import_cte(...) or dbt_utils.star(from=ref(...))) were not detected, producing incomplete lineage.

Two layers both dropped them:

1. When rendering hits an unknown macro it fails, and the partial extraction (including refs already recorded from the macro arguments) was discarded.
2. The regex fallback only matched ref() as a whole {{ ref('...') }} expression, so calls nested in macro arguments or {% set %} statements never matched.

Fix

• Keep refs/sources recorded before a render failure instead of discarding them. minijinja evaluates call arguments before resolving the callee, so {{ unknown_macro(ref('a')) }} records ref('a') even though rendering fails. This also recovers dynamically built refs like ref('model_' ~ var('env')) that no regex can find.
• Relax the regex fallback to find ref()/source() anywhere inside {{ ... }} and {% ... %} blocks, then merge it with the partial jinja result (deduplicated). {# #} comments and {% raw %} sections are excluded, text outside jinja blocks (e.g. SQL comments) is not scanned, and matches starting inside string literals are skipped so {{ log("see ref('x')") }} does not create a phantom edge.

Verification

• 18 new unit tests covering the issue example, package/unknown macros, {% set %}, refs after the failure point, dynamic refs, string-literal and {% raw %} false-positive guards
• End-to-end: a project using the exact import_cte pattern from the issue now shows all three upstream models in dlin graph

🤖 Generated with Claude Code

[Copilot]

Pull request overview

This PR fixes incomplete dbt lineage extraction when ref()/source() calls appear inside macro arguments (especially namespaced/package macros that minijinja can’t resolve). It does this by preserving partial Jinja extraction results on render failure and enhancing the regex fallback to detect calls anywhere inside Jinja blocks.

Changes:

• Preserve refs/sources/config recorded during a failed minijinja render and merge with regex results instead of discarding partial extraction.
• Update regex fallback to scan only inside {{ ... }} / {% ... %} blocks (excluding {# #} and {% raw %}) and skip matches inside string literals.
• Add unit tests covering nested macro args, {% set %} statements, raw blocks, and string-literal false-positive guards.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File	Description
crates/dlin-core/src/parser/sql.rs	Improves regex fallback extraction (block-scoped scanning, raw/comment stripping, string-literal guarding) and merges regex results with partial Jinja extraction when rendering fails.
crates/dlin-core/src/parser/jinja.rs	Changes Jinja extraction to return a structured outcome (partial extraction + completion flag) and makes merge logic reusable for combining partial + fallback results.

Comments suppressed due to low confidence (1)

crates/dlin-core/src/parser/sql.rs:320

• extract_config_regex strips only {# ... #} comments, but not {% raw %}...{% endraw %} blocks. Since {% raw %} content is not evaluated by Jinja, a {{ config(...) }} inside a raw block would be incorrectly treated as real config when extract_all falls back to regex merging on render failures. Using the same inert-Jinja stripping as ref/source extraction avoids this false positive.

    extract_all(sql, macro_prefix).config
}

/// Regex fallback for extracting config() settings
fn extract_config_regex(sql: &str) -> SqlConfig {

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.