Skip to content

Rules aggregate by resource_name across packages, causing false positives #572

@habroptilus

Description

@habroptilus

Describe the bug

When a dbt project contains multiple packages that define resources with the same resource_name, some dbt_project_evaluator rules appear to aggregate across packages and produce false positives.

From reading the package code, it looks like several DAG-based rules operate on resource_name rather than a package-scoped identifier. As a result, resources from different packages that happen to share the same name can be treated as the same node.

Examples of affected behavior

I confirmed this with examples shaped like the following:

  • Package A has a staging model stg_shared_name that depends on an internal base model.
  • Package B has another staging model with the same name stg_shared_name, but this one depends directly on a source.

In this setup, Direct Join to Source appears to report stg_shared_name even though the Package A model does not directly depend on a source.

I also confirmed a similar pattern for Source Fanout:

  • Package A has a source shared_source.table_x and a model stg_metrics
  • Package B has a source with the same name shared_source.table_x and a model with the same name stg_metrics

Each package individually has a 1:1 source-to-model relationship, but Source Fanout appears to aggregate them together and report a false positive.

I suspect Models with Too Many Joins and possibly other rules that aggregate by resource_name may be affected as well, although I have only directly verified the two cases above.

Expected behavior

Resources defined in different packages should be treated as distinct graph nodes, even if they share the same resource_name.

Actual behavior

Some rules appear to aggregate across packages based on resource_name, which causes false positives when identical names exist in multiple packages.

Steps to reproduce

A minimal reproduction would look like this:

  1. Create two dbt packages in the same project.
  2. In both packages, define resources with the same model name and/or source name.
  3. Make their lineage different across packages.
  4. Run dbt_project_evaluator.

Why this seems to happen

From reading the implementation, it looks like graph/resource models derive and propagate resource_name, and some downstream rules group or join on that value. If package identity is not included in the effective key, same-named resources across packages can collide.

Impact

This makes it difficult to rely on evaluator output in monorepos or multi-package dbt projects where naming conventions are reused across packages.

Workaround

As a local workaround, we filtered specific rules after evaluation_results, but that only helps for some false positives and does not solve cases where aggregated metrics (for example join counts) are already incorrect upstream.

Environment

  • dbt version: 1.11.7
  • adapter: BigQuery
  • dbt_project_evaluator version: current release in use

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions