Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix/dynamic dag args #314

Merged
merged 7 commits into from
Mar 13, 2025
Merged

Fix/dynamic dag args #314

merged 7 commits into from
Mar 13, 2025

Conversation

anayeaye
Copy link
Contributor

@anayeaye anayeaye commented Mar 12, 2025

What

Some code updates for clarity (not across the full codebase, just in discovery and vector DAGs)

  • Don't ignore a missing bucket arg in the event config. Event.get('bucket') should fail if bucket is not provided so use event['bucket']--this will emit a useful error
  • require named parameters

**dag_args
) as dag:
start = DummyOperator(task_id="Start", dag=dag)
end = DummyOperator(task_id="End", trigger_rule=TriggerRule.ONE_SUCCESS, dag=dag)
discover = discover_from_s3_task()
discover = discover_from_s3_task(event)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Making the params explicit here made it easier to see why the scheduled DAG was failing without an event but the default user-triggered DAG was working (because the run config is submitted when starting a manual run)

@anayeaye anayeaye merged commit c53a790 into dev Mar 13, 2025
3 checks passed
@anayeaye anayeaye deleted the fix/dynamic-dag-args branch March 13, 2025 15:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants