Prefect Worker Error on Windows: OSError when Running Deployed Jobs

[srujanreddyj]

Bug summary

When using a Prefect worker installed on Windows for deployed jobs, an error occurs during the flow run process. The specific error is an OSError with the message "Invalid argument" occurring during file I/O operations.

The steps I did are:

1. Installed Prefect in Windows
2. Started a work pool and worker inside the windows server. (daemonized it as well)
3. From Prefect cloud, when I try to use this work pool, I am running into the error.

+ Exception Group Traceback (most recent call last):
| File "C:\Program Files\venv\lib\site-packages\prefect\workers\base.py", line 889, in submitrun_and_capture_errors
| result = await self.run(
| File "C:\Program Files\venv\lib\site-packages\prefect\workers\process.py", line 270, in run
| process = await run_process(
| File "C:\Program Files\venv\lib\site-packages\prefect\utilities\processutils.py", line 273, in run_process
| await consume_process_output(
| File "C:\Program Files\venv\lib\site-packages\prefect\utilities\processutils.py", line 287, in consume_process_output
| async with anyio.create_task_group() as tg:
| File "C:\Program Files\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 680, in aexit
| raise BaseExceptionGroup(
| exceptiongroup.ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
+-+---------------- 1 ----------------
| Traceback (most recent call last):
| File "C:\Program Files\venv\lib\site-packages\prefect\utilities\processutils.py", line 314, in stream_text
| await sink.write(item)
| File "C:\Program Files\venv\lib\site-packages\anyio\_core\_fileio.py", line 109, in write
| return await to_thread.run_sync(self._fp.write, b)
| File "C:\Program Files\venv\lib\site-packages\anyio\to_thread.py", line 56, in run_sync
| return await get_async_backend().run_sync_in_worker_thread(
| File "C:\Program Files\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 2177, in run_sync_in_worker_thread
| return await future
| File "C:\Program Files\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 859, in run
| result = context.run(func, *args)
| OSError: [Errno 22] Invalid argument
+------------------------------------

• The code works fine and executes as intended when I use in windows.

if __name__ == "__main__":
    etl_process()

Version info (prefect version output)

Version:             3.0.2
API version:         0.8.4
Python version:      3.10.11
Git commit:          c846de02
Built:               Fri, Sep 13, 2024 10:48 AM
OS/Arch:             win32/AMD64
Profile:             ephemeral
Server type:         cloud
Pydantic version:    2.9.1

Additional context

No response

[desertaxle]

Thanks for the bug report @srujanreddyj! Which worker type are you using? Knowing will help me reproduce and find a solution for this issue.

[srujanreddyj]

@desertaxle , I am using Process worker type. Try to executee flow runs in subprocesses on the local machine.