How to avoid pulling from S3 when the flow code hasn't changed

[anilpixel]

I couldn't find any examples in the official documentation, thank you everyone

[zzstoatzz]

hi @anilpixel - can you add some more detail on your use case?

the pull steps run for each flow run, and running git_clone would be unavoidable in the case of a containerized runtime (ie container spins up fresh for a deployment, the code does not exist on the container)

if you have a static machine where your worker runs, this could be possible, as you might be able to write a custom pull step that checks some hash (in git you could check a SHA but in s3 its not immediately obvious what the right way to check for changes is) to decide whether to pull it down again. but there's likely still some challenges there, like where does the code live on disk such that the worker can submit it correctly?

[anilpixel]

Yes, I am using multiple static machines, and I use Prefect to manage my RPA code. Can I determine if an update is needed through the deployment version?

[zzstoatzz]

i would recommend adding a deployment version which contains your git SHA. that way you could determine whether your code changed simply by looking at the deployment version

[anilpixel]

My idea is to use the RemoteStorage class in a custom pull step to fetch the code, but I'm not sure if it's possible to determine if there's a new version based on the deployment version.

[aadimurthy]

@anilpixel Can you check this? Why do we need to pull, and if we have the option, what if the flow code can be built inside Docker. #18977