Question/Idea: Long running(streaming) flow to speed up start time

[Pballer]

My setup:
I have a localhost deployment of Prefect(v3.3.7) using Postgres(v16.4). I serve one flow using my_flow.serve()

A driver process is listening for events and then invoking the flow: run_deployment(name="my_flow", ...)
Each process is in it's own docker container using docker compose.

Issue:
The flow can take as much as 10-15 seconds before it actually runs.
~10 seconds "Scheduled"
~10 seconds "Pending"

Question:
Is there a way to speed this up?

Idea:
Have a long running/streaming flow.
Meaning, my_flow could poll indefinitely for events, and then submit tasks of work.
This way the flow is already in a running state. Not sure if this is an anti-pattern in Prefect.

@flow
def my_flow(task_runner=ThreadPoolTaskRunner(max_workers=3)):
    """Streaming flow that continuously checks for work."""
    while True:
        work = check_for_work()
        if work:
            my_task.submit(work) 

[zzstoatzz]

hi @Pballer

Issue:
The flow can take as much as 10-15 seconds before it actually runs.
~10 seconds "Scheduled"
~10 seconds "Pending"

Is there a way to speed this up?

this is a major reason we added background tasks (advanced example)!

the big difference is that task workers are websocket clients, so you don't have to wait for the poll interval to find the scheduled runs, task workers will pick up published runs almost immediately. like normal, you can call tasks or flows from these tasks that you're serving. so its a way to have stuff run quickly on static infra instead of having a polling worker that submits a request for infra to spin up

[Pballer]

This works perfectly for my static infra deployment! Thank you!

Our main prod deployment is polling worker ECS Fargate based. Are you aware of strategies for speeding up this type of deployment? At lot of it is Fargate setting up resources on AWS' side.

[zzstoatzz]

I don't have immediate thoughts outside of adhering to general container best practices (multi-stage builds etc) to avoid extra startup time, avoiding ad-hoc installation at runtime EXTRA_PIP_PACKAGES etc, and exploring capacity provider configuration options in line with your workload