Added ability to exclude files based on subfolder name

[vvvito]

Resolves #81.

Adding an additional argument called model_path_exclusion_filter which is a list of strings that holds glob-style patterns to parse through table original_file_path in order to exclude tables from the doc compute function.

Example

Here is my dbt folder structure

📁models 
    📁dim
    - dim_core_dates.sql
    - dim_core_forms.sql
    - dim_core_patients.sql
        📁stg
        - dim_stg_dates.sql
        - dim_stg_forms.sql
        - dim_stg_patients.sql
      

Executing the following doc compute command:

# baseline
dbt-coverage compute doc --cov-format markdown --model-path-filter models/dim

Returns the following:

Coverage report (doc)

Model	Columns Covered	%
dim.dim_core_dates	0/72	0.0%
dim.dim_core_forms	13/15	86.7%
dim.dim_core_patients	25/39	64.1%
dim_stg.stg_core_dates	0/72	0.0%
dim_stg.stg_core_forms	0/12	0.0%
dim_stg.stg_core_patients	0/40	0.0%
Total	38/250	15.2%

All the models that are are prefixed with a 'stg' are staging models, and should not be included in the coverage compute.

Note

Total coverage is 15.2% due to the inflated denominator.

Instead, we can now use the --model-path-exclusion-filter to remove models in the denominator count

# compute doc coverage for models found in the `dim` folder but exclude models found in the `stg` sub-folder
dbt-coverage compute doc --cov-format markdown --model-path-exclusion-filter */stg --model-path-filter models/dim

Returns the following:

Coverage report (doc)

Model	Columns Covered	%
dim.dim_core_dates	0/72	0.0%
dim.dim_core_forms	13/15	86.7%
dim.dim_core_patients	25/39	64.1%
Total	38/126	30.1%

Note

The coverage is now 30.1%, as we filtered out models we don't want to include in the denominator.

How to use

Pass in as many --model-path-exclusion-filter arguments you wish to exclude any number of subfolders from being computed

Example:

# This will exclude tables found under a subfolder called 'bronze' or 'silver' any level deep, or a 'stg' subfolder, found within a 'mock' subfolder, any level deep.
dbt-coverage compute doc --model-path-exclusion-filter */bronze --model-path-exclusion-filter */silver --model-path-exclusion-filter */mock/stg

[sweco]

@vvvito, my apologies for the delay. I missed this and only spotted it now.

I've unified the exclusion logic with the inclusion logic - both now use startswith, going to merge now.

[followingell]

@sweco With the above message, are you saying you combined / aligned this PR's and my PR's functionality? 😃

[sweco]

Yes @followingell, that's what I meant. 😊