Skip to content

Unable to set spooler-processes using environment variables #531

@jhomer-hscl

Description

@jhomer-hscl

With the following in my ini file

...
spooler-frequency = $(UWSGI_SPOOLER_FREQUENCY_SECONDS)
spooler-import = /opt/resumable_upload/spooler.py
spooler-max-tasks = $(UWSGI_SPOOLER_MAX_TASKS)
spooler-processes = $(UWSGI_SPOOLER_PROCESSES)

spooler = /var/spool/uwsgi/retry

Running uWSGI in a container with the following set in Dockerfile:

...
ENV UWSGI_SPOOLER_FREQUENCY_SECONDS=10
ENV UWSGI_SPOOLER_MAX_TASKS=1000
ENV UWSGI_SPOOLER_PROCESSES=5
...

Logs:

$ docker run --name import-app --rm -it  -p 8443:8443 -e UWSGI_SPOOLER_MAX_TASKS=5 -e UWSGI_SPOOLER_PROCESSES=5 $IMAGE
Checking for script in /prestart.sh
Running script /prestart.sh
/usr/lib/python3.6/site-packages/supervisor/options.py:475: UserWarning: Supervisord is running as root and it is searching for its configuration file in default locations (including its current working directory); you probably want to specify a "-c" argument specifying an absolute path to a configuration file for improved security.
  'Supervisord is running as root and it is searching '
2024-06-03 11:19:44,222 CRIT Supervisor is running as root.  Privileges were not dropped because no user is specified in the config file.  If you intend to run as root, you can set user=root in the config file to avoid this message.
2024-06-03 11:19:44,224 INFO supervisord started with pid 1
2024-06-03 11:19:45,227 INFO spawned: 'quit_on_failure' with pid 20
2024-06-03 11:19:45,229 INFO spawned: 'nginx' with pid 21
2024-06-03 11:19:45,231 INFO spawned: 'uwsgi' with pid 22
2024-06-03 11:19:45,235 INFO success: nginx entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
2024/06/03 11:19:45 [notice] 21#0: using the "epoll" event method
2024/06/03 11:19:45 [notice] 21#0: nginx/1.18.0
2024/06/03 11:19:45 [notice] 21#0: built by gcc 8.5.0 20210514 (Red Hat 8.5.0-10) (GCC)
2024/06/03 11:19:45 [notice] 21#0: OS: Linux 4.18.0-513.11.1.el8_9.0.1.x86_64
2024/06/03 11:19:45 [notice] 21#0: getrlimit(RLIMIT_NOFILE): 1048576:1048576
2024/06/03 11:19:45 [notice] 21#0: start worker processes
2024/06/03 11:19:45 [notice] 21#0: start worker process 23
2024/06/03 11:19:45 [notice] 21#0: start worker process 24
[uWSGI] getting INI configuration from /opt/resumable_upload/resumable_file_sender.ini
*** Starting uWSGI 2.0.25.1 (64bit) on [Mon Jun  3 11:19:45 2024] ***
compiled with version: 8.5.0 20210514 (Red Hat 8.5.0-22) on 03 June 2024 07:43:56
os: Linux-4.18.0-513.11.1.el8_9.0.1.x86_64 #1 SMP Sun Feb 11 10:42:18 UTC 2024
nodename: 3eb4e11d1c0c
machine: x86_64
clock source: unix
detected number of CPU cores: 2
current working directory: /tmp
detected binary path: /usr/local/bin/uwsgi
!!! no internal routing support, rebuild with pcre support !!!
setuid() to 997
chdir() to /opt/resumable_upload
your memory page size is 4096 bytes
detected max file descriptor number: 1048576
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
*** Cache "health" initialized: 0MB (key: 2136 bytes, keys: 6408 bytes, data: 196608 bytes, bitmap: 0 bytes) preallocated ***
uwsgi socket 0 bound to UNIX address /tmp/resumable_file_sender.sock fd 4
Python version: 3.11.7 (main, Jan 26 2024, 20:24:17) [GCC 8.5.0 20210514 (Red Hat 8.5.0-21)]
Python main interpreter initialized at 0x7f406289a8f8
python threads support enabled
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 437424 bytes (427 KB) for 5 cores
*** Operational MODE: preforking ***
[uwsgi-signal] signum 0 registered (wid: 0 modifier1: 0 target: mule)
[uwsgi-signal] signum 1 registered (wid: 0 modifier1: 0 target: mule)
WSGI app 0 (mountpoint='') ready in 0 seconds on interpreter 0x7f406289a8f8 pid: 22 (default app)
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 22)
spawned the uWSGI spooler on dir /var/spool/uwsgi/retry with pid 25
spawned uWSGI worker 1 (pid: 26, cores: 1)
spawned uWSGI worker 2 (pid: 27, cores: 1)
spawned uWSGI worker 3 (pid: 28, cores: 1)
spawned uWSGI worker 4 (pid: 29, cores: 1)
spawned uWSGI worker 5 (pid: 30, cores: 1)
cache sweeper thread enabled
*** Stats server enabled on 127.0.0.1:1717 fd: 30 ***
spawned uWSGI mule 1 (pid: 32)
2024-06-03 11:19:46,652 INFO success: quit_on_failure entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-06-03 11:19:55,662 INFO success: uwsgi entered RUNNING state, process has stayed up for > than 10 seconds (startsecs)

Only 1 spooler process (pid 25) is spawned.

I've checked multiple time that the config lines up and also that the same approach works as expected for other config, i.e. UWSGI_SPOOLER_MAX_TASKS --> spooler-max-tasks is correct and it does.

The documentation suggests it should.

Is this some special case? or broken?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions