Skip to content

OSError: [Errno 88] Socket operation on non-socket when running luigid --background on Python 3.13 #3407

@s0larplexus

Description

@s0larplexus

Problem

When attempting to run the Luigi daemon (luigid) in the background using the --background flag, it crashes with an OSError: [Errno 88] Socket operation on non-socket. This appears to be an issue with how the underlying python-daemon library interacts with sockets in Python 3.13.

The command runs perfectly fine without the --background parameter, and the exact same --background command works without any issues on Python 3.12.

Steps to Reproduce

Set up an environment using Python 3.13.12.

Install Luigi.

Run the following command:
Bash

python --version && luigid --background --logdir ./

Expected behavior

The Luigi daemon should start in the background without raising an exception, matching the behavior seen in Python 3.12.

Actual behavior

The process crashes during daemonization with the following traceback:
Python

Python 3.13.12
Traceback (most recent call last):
File "/mnt/c/pdrive/repos/luigiprobe/.venv/bin/luigid", line 10, in
sys.exit(luigid())
~~~~~~^^
File "/mnt/c/pdrive/repos/luigiprobe/.venv/lib/python3.13/site-packages/luigi/cmdline.py", line 34, in luigid
luigi.process.daemonize(
~~~~~~~~~~~~~~~~~~~~~~~^
luigi.server.run, api_port=opts.port, address=opts.address, pidfile=opts.pidfile, logdir=opts.logdir, unix_socket=opts.unix_socket
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/mnt/c/pdrive/repos/luigiprobe/.venv/lib/python3.13/site-packages/luigi/process.py", line 94, in daemonize
ctx = daemon.DaemonContext(
stdout=stdout_proxy,
...<2 lines>...
initgroups=False,
)
File "/mnt/c/pdrive/repos/luigiprobe/.venv/lib/python3.13/site-packages/daemon/daemon.py", line 280, in init
detach_process = is_detach_process_context_required()
File "/mnt/c/pdrive/repos/luigiprobe/.venv/lib/python3.13/site-packages/daemon/daemon.py", line 801, in is_detach_process_context_required
if is_process_started_by_init() or is_process_started_by_superserver():
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
File "/mnt/c/pdrive/repos/luigiprobe/.venv/lib/python3.13/site-packages/daemon/daemon.py", line 778, in is_process_started_by_superserver
if is_socket(stdin_fd):
~~~~~~~~~^^^^^^^^^^
File "/mnt/c/pdrive/repos/luigiprobe/.venv/lib/python3.13/site-packages/daemon/daemon.py", line 743, in is_socket
file_socket = socket.fromfd(fd, socket.AF_INET, socket.SOCK_RAW)
File "/home/py/conda/lib/python3.13/socket.py", line 549, in fromfd
return socket(family, type, proto, nfd)
File "/home/py/conda/lib/python3.13/socket.py", line 233, in init
_socket.socket.init(self, family, type, proto, fileno)
~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: [Errno 88] Socket operation on non-socket

Environment details:

Python version: 3.13.12 (Fails) / 3.12 (Works)

Luigi version: [Insert your luigi version here, e.g., 3.6.0]

OS: Linux / WSL (Debian/Ubuntu based on paths)

Additional Context

Because the traceback points heavily to daemon/daemon.py, it seems likely this is a compatibility issue between python-daemon (which Luigi uses for the --background flag) and the newly updated socket module in Python 3.13. For now, running luigid in the foreground (without --background) acts as a temporary workaround.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions