Passing through dbt model names or tags to isolate dry run loops

[KieranMul16]

Hey,

I know in your README you say:

It is best practice to compile the entire dbt project when supplying a manifest for dry run.

And, I get the reasons why.

But I think there are valid use cases where a dbt PR only requires a small number of models to be dry run. If it was possible to pass through model names then this could be used for a quicker and snappier pre-commit hook which would be amazing!

I have just started to use this package and I think it is great but it is slow for large dbt projects.

[ccharlesgb]

Hey @KieranMul16, do you have an example of what you'd want the pre-commit hook to look like? Would you use one of the modified state selectors on dbt?

It's an interesting use case to do a quick dry run outside of CICD.

Not sure how large your dbt project is but you could also try bumping up the --threads for the dry run command. You do get diminished returns but if your project is really large you might see a decent speed up.

[KieranMul16]

So, as an example, I currently use a local hook to run all staged files through SQLFluff (installed in the same repo and virtual environment).

repos:
  - repo: local
    hooks:
      - id: lint-tests
        name: Lint all staged files
        entry: pipenv run sqlfluff lint --dialect bigquery
        pass_filenames: true
        language: system
        fail_fast: false
        verbose: true

We do a full pass in CI itself but it just avoids any wastage of time waiting for a full run in CI to tell me I was an idiot 😆 .

For this I would want it to behave very similarly and pass the staged model files into dbt-dry-run using a regex to isolate .SQL files only. That would tell me a failing in CI is due to a more complex cause and not a SQL syntax error that I missed once my dbt has compiled.

Does that make sense?