System down #2756

WirthmU · 2026-03-04T14:09:49Z

WirthmU
Mar 4, 2026

I did a few optimizations with the swap settings. I am not sure if this is the cause but now I am no longer able to access my cameras website. I get this here:

Proxy Error
The proxy server received an invalid response from an upstream server.
The proxy server could not handle the request
Reason: Error reading from remote server

Indi-allsky service is like this:

(indi-allsky) ashqc2@allsky2:~/indi-allsky $ systemctl --user status indi-allsky
● indi-allsky.service - Indi AllSky Service
     Loaded: loaded (/home/ashqc2/.config/systemd/user/indi-allsky.service; disabled; preset: enabled)
     Active: active (running) since Wed 2026-03-04 15:03:09 CET; 3min 37s ago
TriggeredBy: ● indi-allsky.timer
   Main PID: 3897 (python3)
      Tasks: 12 (limit: 3927)
        CPU: 51.953s
     CGroup: /user.slice/user-1000.slice/user@1000.service/app.slice/indi-allsky.service
             ├─3897 /home/ashqc2/indi-allsky/virtualenv/indi-allsky/bin/python3 allsky.py --log syslog run
             ├─3905 /home/ashqc2/indi-allsky/virtualenv/indi-allsky/bin/python3 allsky.py --log syslog run
             ├─3907 /home/ashqc2/indi-allsky/virtualenv/indi-allsky/bin/python3 allsky.py --log syslog run
             ├─3908 /home/ashqc2/indi-allsky/virtualenv/indi-allsky/bin/python3 allsky.py --log syslog run
             └─3909 /home/ashqc2/indi-allsky/virtualenv/indi-allsky/bin/python3 allsky.py --log syslog run

Mar 04 15:06:43 allsky2 python3[3907]: [INFO] Image-1-3907/MainThread image.processImage() [555]: Overriding Red balance: 3.757034
Mar 04 15:06:43 allsky2 python3[3907]: [INFO] Image-1-3907/MainThread image.processImage() [556]: Overriding Blue balance: 1.504066
Mar 04 15:06:43 allsky2 python3[3907]: [INFO] Image-1-3907/MainThread processing._calculate_8bit_adu() [1298]: ADU average: 1336.5 (83)
Mar 04 15:06:44 allsky2 python3[3907]: [INFO] Image-1-3907/MainThread image.processImage() [742]: Image processed in 1.3730 s
Mar 04 15:06:44 allsky2 python3[3907]: [INFO] Image-1-3907/MainThread image.write_img() [1647]: Compressed image file size: 960.03 KB
Mar 04 15:06:44 allsky2 python3[3907]: [INFO] Image-1-3907/MainThread image.write_img() [1676]: Daytime capture is disabled
Mar 04 15:06:45 allsky2 python3[3905]: [INFO] Capture-1-3905/MainThread capture.saferun() [378]: Camera last ready: 2.5s
Mar 04 15:06:45 allsky2 python3[3905]: [INFO] Capture-1-3905/MainThread capture.saferun() [379]: Exposure state: READY
Mar 04 15:06:45 allsky2 python3[3905]: [INFO] Capture-1-3905/MainThread capture.detectNight() [2028]: Sun alt: 27.8, Moon alt: -39.6, phase 98.6%
Mar 04 15:06:46 allsky2 python3[3909]: [INFO] Sensor-1-3909/MainThread sensor.saferun() [204]: Dew Point: 5.3, Frost Point: 3.4, Heat Index: 28.8

Any help to this?

aaronwmorris · 2026-03-04T15:18:36Z

aaronwmorris
Mar 4, 2026
Maintainer

The web service is gunicorn-indi-allsky. Try restarting that.

sudo systemctl --user restart gunicorn-indi-allsky

The log will be at /var/log/indi-allsky/webapp-indi-allsky.log

1 reply

WirthmU Mar 4, 2026
Author

Everything looks fine:
This is from indi-allsky.log:

2026-03-04T17:32:20.660691+01:00 allsky2 [WARNING] Image-1-892/MainThread image.recalculate_exposure() [2304]: New calculated exposure: 0.002255s (-0.000992) @ gain 1.00 (+0.00) bin 1
2026-03-04T17:32:21.601586+01:00 allsky2 [INFO] Image-1-892/MainThread image.processImage() [742]: Image processed in 1.3339 s
2026-03-04T17:32:21.894147+01:00 allsky2 [INFO] Image-1-892/MainThread image.write_img() [1647]: Compressed image file size: 943.55 KB
2026-03-04T17:32:21.897113+01:00 allsky2 [INFO] Image-1-892/MainThread image.write_img() [1676]: Daytime capture is disabled
2026-03-04T17:32:24.894398+01:00 allsky2 [INFO] Capture-4-932/MainThread capture.saferun() [378]: Camera last ready: 4.6s
2026-03-04T17:32:24.894705+01:00 allsky2 [INFO] Capture-4-932/MainThread capture.saferun() [379]: Exposure state: READY
2026-03-04T17:32:24.894870+01:00 allsky2 [INFO] Capture-4-932/MainThread capture.detectNight() [2028]: Sun alt: 7.0, Moon alt: -21.8, phase 98.4%
2026-03-04T17:32:31.869824+01:00 allsky2 [INFO] Sensor-1-894/MainThread sensor.saferun() [204]: Dew Point: 4.3, Frost Point: 2.5, Heat Index: 30.7
2026-03-04T17:32:32.761412+01:00 allsky2 [INFO] Capture-4-932/MainThread capture.shoot() [2229]: Taking 0.00225469s exposure (gain 1.00 / bin 1)
2026-03-04T17:32:32.762849+01:00 allsky2 [INFO] Capture-4-932/MainThread libcamera.setCcdExposure() [304]: image command: rpicam-still --immediate --nopreview --camera 0 --raw --denoise off --gain 1.00 --shutter 2254 --metadata /tmp/tmpurco5ac6.json --metadata-format json --awb auto --output /tmp/tmpxtabbxlu.dng
2026-03-04T17:32:32.765237+01:00 allsky2 [INFO] Capture-4-932/MainThread capture.saferun() [809]: Total time since last exposure 15.0073 s
2026-03-04T17:32:35.271379+01:00 allsky2 [INFO] Capture-4-932/MainThread libcamera._processMetadata() [422]: libcamera reported gain: 1.00/1.01
2026-03-04T17:32:35.271649+01:00 allsky2 [INFO] Capture-4-932/MainThread capture.saferun() [622]: Exposure received in 2.5099s (+2.5077s)
2026-03-04T17:32:35.515034+01:00 allsky2 [INFO] Image-1-892/MainThread processing._add() [734]: Image data - size: 23.7 MB, bits: 16, cfa: BGGR, offsets: X 0, Y 0, row order: na
2026-03-04T17:32:35.522478+01:00 allsky2 [INFO] Image-1-892/MainThread processing.detectBitDepth() [4420]: Image max value: 4095
2026-03-04T17:32:35.522714+01:00 allsky2 [INFO] Image-1-892/MainThread processing.detectBitDepth() [4438]: Detected bit depth: 12
2026-03-04T17:32:35.522906+01:00 allsky2 [INFO] Image-1-892/MainThread processing._apply_calibration() [1006]: Searching for dark frame: gain 1.00, exposure >= 0.0, temp >= 35.0c
2026-03-04T17:32:35.527972+01:00 allsky2 [WARNING] Image-1-892/MainThread processing._apply_calibration() [1045]: Dark not found: ccd1 16bit 0.0022547s gain 1.00 bin 1 35.00c
2026-03-04T17:32:35.528228+01:00 allsky2 [INFO] Image-1-892/MainThread processing._calibrate() [933]: Black level (scaled): 256
2026-03-04T17:32:35.601765+01:00 allsky2 [INFO] Image-1-892/MainThread sqm.jSqm() [63]: Raw jSQM: 1177.64, Weighted jSQM: 10951.20
2026-03-04T17:32:35.620200+01:00 allsky2 [INFO] Image-1-892/MainThread image.processImage() [459]: Image: 4056 x 3040
2026-03-04T17:32:35.621207+01:00 allsky2 [INFO] Image-1-892/MainThread image.processImage() [555]: Overriding Red balance: 3.757034
2026-03-04T17:32:35.621309+01:00 allsky2 [INFO] Image-1-892/MainThread image.processImage() [556]: Overriding Blue balance: 1.504066
2026-03-04T17:32:35.647903+01:00 allsky2 [INFO] Image-1-892/MainThread processing._calculate_8bit_adu() [1298]: ADU average: 1200.6 (75)
2026-03-04T17:32:35.674664+01:00 allsky2 [WARNING] Image-1-892/MainThread image.recalculate_exposure() [2153]: Found target value for exposure
2026-03-04T17:32:35.674800+01:00 allsky2 [INFO] Image-1-892/MainThread image.write_mask_base_img() [1515]: Generating new mask base
2026-03-04T17:32:36.023884+01:00 allsky2 [INFO] Capture-4-932/MainThread capture.saferun() [378]: Camera last ready: 0.8s
2026-03-04T17:32:36.024131+01:00 allsky2 [INFO] Capture-4-932/MainThread capture.saferun() [379]: Exposure state: READY
2026-03-04T17:32:36.024236+01:00 allsky2 [INFO] Capture-4-932/MainThread capture.detectNight() [2028]: Sun alt: 7.0, Moon alt: -21.7, phase 98.4%
2026-03-04T17:32:38.696626+01:00 allsky2 [INFO] Image-1-892/MainThread image.processImage() [742]: Image processed in 3.4221 s
2026-03-04T17:32:38.987083+01:00 allsky2 [INFO] Image-1-892/MainThread image.write_img() [1647]: Compressed image file size: 860.21 KB
2026-03-04T17:32:38.989614+01:00 allsky2 [INFO] Image-1-892/MainThread image.write_img() [1676]: Daytime capture is disabled
2026-03-04T17:32:46.873741+01:00 allsky2 [INFO] Sensor-1-894/MainThread sensor.saferun() [204]: Dew Point: 4.3, Frost Point: 2.5, Heat Index: 30.7

With this:

(indi-allsky) ashqc2@allsky2:~/indi-allsky $ systemctl --user status indi-allsky
● indi-allsky.service - Indi AllSky Service
     Loaded: loaded (/home/ashqc2/.config/systemd/user/indi-allsky.service; enabled; preset: enabled)
     Active: active (running) since Wed 2026-03-04 17:30:49 CET; 11min ago
TriggeredBy: ● indi-allsky.timer
   Main PID: 811 (python3)
      Tasks: 20 (limit: 3927)
        CPU: 2min 16.520s
     CGroup: /user.slice/user-1000.slice/user@1000.service/app.slice/indi-allsky.service
             ├─ 811 /home/ashqc2/indi-allsky/virtualenv/indi-allsky/bin/python3 allsky.py --log syslog run
             ├─ 892 /home/ashqc2/indi-allsky/virtualenv/indi-allsky/bin/python3 allsky.py --log syslog run
             ├─ 893 /home/ashqc2/indi-allsky/virtualenv/indi-allsky/bin/python3 allsky.py --log syslog run
             ├─ 894 /home/ashqc2/indi-allsky/virtualenv/indi-allsky/bin/python3 allsky.py --log syslog run
             ├─ 932 /home/ashqc2/indi-allsky/virtualenv/indi-allsky/bin/python3 allsky.py --log syslog run
             └─1457 rpicam-still --immediate --nopreview --camera 0 --raw --denoise off --gain 1.00 --shutter 2448 --metadata /tmp/tmpdxdio2nm.json --metadata-format json --awb auto --output /tmp/tmp1hsz1m>

Mar 04 17:41:52 allsky2 python3[932]: [INFO] Capture-4-932/MainThread capture.saferun() [378]: Camera last ready: 2.0s
Mar 04 17:41:52 allsky2 python3[932]: [INFO] Capture-4-932/MainThread capture.saferun() [379]: Exposure state: READY
Mar 04 17:41:52 allsky2 python3[932]: [INFO] Capture-4-932/MainThread capture.detectNight() [2028]: Sun alt: 5.5, Moon alt: -20.3, phase 98.3%
Mar 04 17:42:03 allsky2 python3[932]: [INFO] Capture-4-932/MainThread capture.shoot() [2229]: Taking 0.00244845s exposure (gain 1.00 / bin 1)
Mar 04 17:42:03 allsky2 python3[932]: [INFO] Capture-4-932/MainThread libcamera.setCcdExposure() [304]: image command: rpicam-still --immediate --nopreview --camera 0 --raw --denoise off --gain 1.00 --shut>
Mar 04 17:42:03 allsky2 python3[932]: [INFO] Capture-4-932/MainThread capture.saferun() [809]: Total time since last exposure 15.0138 s
Mar 04 17:42:03 allsky2 python3[894]: [INFO] Sensor-1-894/MainThread sensor.saferun() [204]: Dew Point: 4.1, Frost Point: 2.4, Heat Index: 31.0
Mar 04 17:42:04 allsky2 python3[932]: [INFO] Capture-4-932/MainThread capture.saferun() [378]: Camera last ready: 13.2s
Mar 04 17:42:04 allsky2 python3[932]: [INFO] Capture-4-932/MainThread capture.saferun() [379]: Exposure state: BUSY
Mar 04 17:42:04 allsky2 python3[932]: [INFO] Capture-4-932/MainThread capture.detectNight() [2028]: Sun alt: 5.5, Moon alt: -20.3, phase 98.3%

And this webapp-indi-allsky.log

2026-03-04T14:21:55.990934+01:00 allsky2 gunicorn.indi_allsky.wsgi.error: [1922] Listening at: unix:/var/lib/indi-allsky/gunicorn-indi-allsky.sock (1922)
2026-03-04T14:21:55.991319+01:00 allsky2 gunicorn.indi_allsky.wsgi.error: [1922] Using worker: gthread
2026-03-04T14:21:56.030888+01:00 allsky2 gunicorn.indi_allsky.wsgi.error: [1922] Control socket listening at /home/ashqc2/indi-allsky/gunicorn.ctl
2026-03-04T16:12:07.490128+01:00 allsky2 gunicorn.indi_allsky.wsgi.error: [3766] Starting gunicorn 25.1.0
2026-03-04T16:12:07.491561+01:00 allsky2 gunicorn.indi_allsky.wsgi.error: [3766] Listening at: unix:/var/lib/indi-allsky/gunicorn-indi-allsky.sock (3766)
2026-03-04T16:12:07.492022+01:00 allsky2 gunicorn.indi_allsky.wsgi.error: [3766] Using worker: gthread
2026-03-04T16:12:07.532411+01:00 allsky2 gunicorn.indi_allsky.wsgi.error: [3766] Control socket listening at /home/ashqc2/indi-allsky/gunicorn.ctl
2026-03-04T17:28:30.336557+01:00 allsky2 gunicorn.indi_allsky.wsgi.error: [3766] Handling signal: term
2026-03-04T17:28:35.962046+01:00 allsky2 gunicorn.indi_allsky.wsgi.error: [7470] Starting gunicorn 25.1.0
2026-03-04T17:28:35.963417+01:00 allsky2 gunicorn.indi_allsky.wsgi.error: [7470] Listening at: unix:/var/lib/indi-allsky/gunicorn-indi-allsky.sock (7470)
2026-03-04T17:28:35.963877+01:00 allsky2 gunicorn.indi_allsky.wsgi.error: [7470] Using worker: gthread
2026-03-04T17:28:36.000933+01:00 allsky2 gunicorn.indi_allsky.wsgi.error: [7470] Control socket listening at /home/ashqc2/indi-allsky/gunicorn.ctl
2026-03-04T17:30:50.338762+01:00 allsky2 gunicorn.indi_allsky.wsgi.error: [812] Starting gunicorn 25.1.0
2026-03-04T17:30:50.339992+01:00 allsky2 gunicorn.indi_allsky.wsgi.error: [812] Listening at: unix:/var/lib/indi-allsky/gunicorn-indi-allsky.sock (812)
2026-03-04T17:30:50.340126+01:00 allsky2 gunicorn.indi_allsky.wsgi.error: [812] Using worker: gthread
2026-03-04T17:30:50.433502+01:00 allsky2 gunicorn.indi_allsky.wsgi.error: [812] Control socket listening at /home/ashqc2/indi-allsky/gunicorn.ctl

With this:

(indi-allsky) ashqc2@allsky2:~/indi-allsky $ systemctl --user status gunicorn-indi-allsky
● gunicorn-indi-allsky.service - indi-allsky gunicorn daemon
     Loaded: loaded (/home/ashqc2/.config/systemd/user/gunicorn-indi-allsky.service; enabled; preset: enabled)
     Active: active (running) since Wed 2026-03-04 17:30:50 CET; 10min ago
TriggeredBy: ● gunicorn-indi-allsky.socket
   Main PID: 812 (gunicorn)
     Status: "Gunicorn arbiter booted"
      Tasks: 3 (limit: 3927)
        CPU: 555ms
     CGroup: /user.slice/user-1000.slice/user@1000.service/app.slice/gunicorn-indi-allsky.service
             ├─812 /home/ashqc2/indi-allsky/virtualenv/indi-allsky/bin/python3.11 /home/ashqc2/indi-allsky/virtualenv/indi-allsky/bin/gunicorn --config /etc/indi-allsky/gunicorn.conf.py indi_allsky.wsgi
             └─815 /home/ashqc2/indi-allsky/virtualenv/indi-allsky/bin/python3.11 /home/ashqc2/indi-allsky/virtualenv/indi-allsky/bin/gunicorn --config /etc/indi-allsky/gunicorn.conf.py indi_allsky.wsgi

Mar 04 17:30:49 allsky2 systemd[640]: Starting gunicorn-indi-allsky.service - indi-allsky gunicorn daemon...
Mar 04 17:30:50 allsky2 gunicorn[812]: [2026-03-04 17:30:50 +0100] [812] [INFO] Starting gunicorn 25.1.0
Mar 04 17:30:50 allsky2 gunicorn.indi_allsky.wsgi.error[812]: [812] Starting gunicorn 25.1.0
Mar 04 17:30:50 allsky2 gunicorn[812]: [2026-03-04 17:30:50 +0100] [812] [INFO] Listening at: unix:/var/lib/indi-allsky/gunicorn-indi-allsky.sock (812)
Mar 04 17:30:50 allsky2 gunicorn.indi_allsky.wsgi.error[812]: [812] Listening at: unix:/var/lib/indi-allsky/gunicorn-indi-allsky.sock (812)
Mar 04 17:30:50 allsky2 gunicorn[812]: [2026-03-04 17:30:50 +0100] [812] [INFO] Using worker: gthread
Mar 04 17:30:50 allsky2 gunicorn.indi_allsky.wsgi.error[812]: [812] Using worker: gthread
Mar 04 17:30:50 allsky2 systemd[640]: Started gunicorn-indi-allsky.service - indi-allsky gunicorn daemon.
Mar 04 17:30:50 allsky2 gunicorn[812]: [2026-03-04 17:30:50 +0100] [812] [INFO] Control socket listening at /home/ashqc2/indi-allsky/gunicorn.ctl
Mar 04 17:30:50 allsky2 gunicorn.indi_allsky.wsgi.error[812]: [812] Control socket listening at /home/ashqc2/indi-allsky/gunicorn.ctl

Still no website!

aaronwmorris · 2026-03-04T16:38:49Z

aaronwmorris
Mar 4, 2026
Maintainer

Something may be wrong with the web server configuration. You may need to re-run setup.sh and re-apply the web server config.

1 reply

WirthmU Mar 4, 2026
Author

That worked - its running again.
Thank you for our support!

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

System down #2756

Uh oh!

{{title}}

Uh oh!

Replies: 2 comments 2 replies

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

System down #2756

Uh oh!

WirthmU Mar 4, 2026

Replies: 2 comments · 2 replies

Uh oh!

aaronwmorris Mar 4, 2026 Maintainer

Uh oh!

Uh oh!

WirthmU Mar 4, 2026 Author

Uh oh!

aaronwmorris Mar 4, 2026 Maintainer

Uh oh!

WirthmU Mar 4, 2026 Author

WirthmU
Mar 4, 2026

Replies: 2 comments 2 replies

aaronwmorris
Mar 4, 2026
Maintainer

WirthmU Mar 4, 2026
Author

aaronwmorris
Mar 4, 2026
Maintainer

WirthmU Mar 4, 2026
Author