Fix: convert kwargs to cli params, add aliases #12029
Open
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Resolves #12028
Problem
On programatic invocations, i.e. using dbtRunner.invoke(), the runner fails when e.g. a profiles_dir is specified. This appears to be because, when this is provided as a keyword argument (e.g. dbtRunner.invoke(["build"], profiles_dir='./my_profiles') the kwargs are only passed to the context after creation, meaning any validation on creating the context fails. And this is exactly what happens, because the profiles_dir option checks for type=click.Path(exists=True).
Why is this annoying? Because everytime you run this in a Docker container (or e.g. through an orchestrator) it will fall back to the default directory for this option (~/username/.dbt) which it of course doesn't find.
Solution
Params can not be added to the Click context before it is created, which means they have to be added as string arguments.
flags.py
already provides functionality for this with thecommand_params()
function. What we can do is take the kwargs through this function, and combine them with provided args so that params can be set before the context creation and validation happens.Checklist