-
Notifications
You must be signed in to change notification settings - Fork 2.5k
Description
Problem
When attempting to run the Luigi daemon (luigid) in the background using the --background flag, it crashes with an OSError: [Errno 88] Socket operation on non-socket. This appears to be an issue with how the underlying python-daemon library interacts with sockets in Python 3.13.
The command runs perfectly fine without the --background parameter, and the exact same --background command works without any issues on Python 3.12.
Steps to Reproduce
Set up an environment using Python 3.13.12.
Install Luigi.
Run the following command:
Bash
python --version && luigid --background --logdir ./
Expected behavior
The Luigi daemon should start in the background without raising an exception, matching the behavior seen in Python 3.12.
Actual behavior
The process crashes during daemonization with the following traceback:
Python
Python 3.13.12
Traceback (most recent call last):
File "/mnt/c/pdrive/repos/luigiprobe/.venv/bin/luigid", line 10, in
sys.exit(luigid())
~~~~~~^^
File "/mnt/c/pdrive/repos/luigiprobe/.venv/lib/python3.13/site-packages/luigi/cmdline.py", line 34, in luigid
luigi.process.daemonize(
~~~~~~~~~~~~~~~~~~~~~~~^
luigi.server.run, api_port=opts.port, address=opts.address, pidfile=opts.pidfile, logdir=opts.logdir, unix_socket=opts.unix_socket
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/mnt/c/pdrive/repos/luigiprobe/.venv/lib/python3.13/site-packages/luigi/process.py", line 94, in daemonize
ctx = daemon.DaemonContext(
stdout=stdout_proxy,
...<2 lines>...
initgroups=False,
)
File "/mnt/c/pdrive/repos/luigiprobe/.venv/lib/python3.13/site-packages/daemon/daemon.py", line 280, in init
detach_process = is_detach_process_context_required()
File "/mnt/c/pdrive/repos/luigiprobe/.venv/lib/python3.13/site-packages/daemon/daemon.py", line 801, in is_detach_process_context_required
if is_process_started_by_init() or is_process_started_by_superserver():
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
File "/mnt/c/pdrive/repos/luigiprobe/.venv/lib/python3.13/site-packages/daemon/daemon.py", line 778, in is_process_started_by_superserver
if is_socket(stdin_fd):
~~~~~~~~~^^^^^^^^^^
File "/mnt/c/pdrive/repos/luigiprobe/.venv/lib/python3.13/site-packages/daemon/daemon.py", line 743, in is_socket
file_socket = socket.fromfd(fd, socket.AF_INET, socket.SOCK_RAW)
File "/home/py/conda/lib/python3.13/socket.py", line 549, in fromfd
return socket(family, type, proto, nfd)
File "/home/py/conda/lib/python3.13/socket.py", line 233, in init
_socket.socket.init(self, family, type, proto, fileno)
~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: [Errno 88] Socket operation on non-socket
Environment details:
Python version: 3.13.12 (Fails) / 3.12 (Works)
Luigi version: [Insert your luigi version here, e.g., 3.6.0]
OS: Linux / WSL (Debian/Ubuntu based on paths)
Additional Context
Because the traceback points heavily to daemon/daemon.py, it seems likely this is a compatibility issue between python-daemon (which Luigi uses for the --background flag) and the newly updated socket module in Python 3.13. For now, running luigid in the foreground (without --background) acts as a temporary workaround.