How to isolate Prefect background task workers by environment (dev/staging/prod) using the serve() method #19054
Unanswered
mclean-connor
asked this question in
Q&A
Replies: 1 comment
-
when you say "different environments", what do you actually mean? if you're using cloud that could mean separate workspaces, or if you're in open source, separate servers. you could also be tagging deployments with if your environments correspond to profiles as we recommend, then you can do |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
I'm running into an issue with task workers picking up tasks across different environments. Here's an example setup:
Task definition:
Serving the task:
The problem:
I have separate environments (dev, staging, prod), and when I serve the task worker in one environment (e.g., staging), it sometimes picks up tasks submitted from other environments (e.g., dev). The worker subscribes to example_task globally, without any differentiation (at least configuration that I could find).
What I've tried:
I attempted to dynamically set the task name to differentiate by environment, but that doesn't seem to work as expected.
Question:
Is there a recommended way to differentiate tasks of same function and ensure task workers only process specific versions of the task when using
prefect.task_worker.serve()
?Any guidance would be appreciated!
Beta Was this translation helpful? Give feedback.
All reactions