Images "too blue" / oversaturated during dusk + dawn - but only for a few frames - then back to normal again

[astro-ralph]

Hi all,

I am using Allsky with a RPi HQ camera and noticed a strange behaviour during dusk and dawn.
Here's the problem:

Nighttime routine starts, camera starts taking/saving pictures. Everything works normal, pictures are fine, colors well/normally balanced. Then suddenly (during blue hour) the sensor is "overwhelmed" with blue. But just for a few frames, then everything is back to normal. Same behavior is seen in the morning hours.

This is repeatable, which means it happens every day...

These two frames are taken directly one after the other - camera works fine - normal colors and color balance. Then next frame, suddenly the blue takes over and everything is overwhelmed.

Then, after a few frames, the effect is reversed:

This happens once in the evening, then again sometime in the morning hours.

You can see the effect in action here: https://allsky.ralphlenges.de/videos/allsky-20240520.mp4

It's the same in a cloudy night: https://allsky.ralphlenges.de/videos/allsky-20240612.mp4

The rest of the night is totally fine - this only happens for a few minutes during dusk / dawn.

Here are my settings:

Anyone with a similar behaviour?

Thanks for the support/feedback!

[EricClaeys]

@astro-ralph, never seen this before.
Please do the following:

sudo systemctl stop allsky
sudo truncate -s 0 /var/log/allsky.log
# Set the Debug Level to 3 in the WebUI, then click the "Save changes" button.

After the blue happens again, attach (don't copy/paste) /var/log/allsky.log.

This will give me a better idea of what's causing the problem - Allsky or the camera.

Thanks - Eric

[EricClaeys]

@astro-ralph, I believe the bright blue is simply caused by a brighter image - see the "Mean" value. The higher the number the brighter the image.
The log file will tell me for sure.

[Willi-K]

Hello everyone,
I thought I was the only one with this problem.
I've tried all combinations of settings - to no avail. The "blue images" also appear at night, sometimes less (when it's cloudy) and sometimes more (when it's clear). I would really appreciate some help.
I'm attaching a typical image (normal and blue)
Greetings, Willi

[astro-ralph]

Hi all,

sorry for following up a bit late on this. Behaviour is still there. It happens pretty much every night and there is no clear thing that triggers this.

Here is a clean log from June 16 to June 17.

allsky.log
capture_args.txt
notification_log.txt

The corresponding video is here: https://allsky.ralphlenges.de/videos/allsky-20240616.mp4

You can see the behavior at the beginning and end of the night.

I took a deeper look and tried to find any similarities or patterns. What I can say is that it happens while the camera is still at Gain 1. Never at a higher gain. It always happens when the camera is ramping up the exposure time (but still below 1sec), but not when it is ramping up the gain.

I also do not think that this is a problem with the camera trying the adjust the mean (=higher mean value). In the night of June 16 the camera jumps from mean 19 (normal colors) to mean 25 (blue colors). But then the camera tries to get the mean down again and a bit later the camera is back to around mean 18 while still maintaining the blue color bleed.

The best clue might actually be the histogram. These are two pictures directly one after the other (start of the "blue" phase):

As you can see, the histogram suddenly seems like it is dropping all color, just retaining the blue signal. The red color is missing more or less completely and the green is squashed to the left side of the histogram.

Here's the first portion of the image files animated with the histogram (it takes a few seconds to get going).

Allsky_June16_Histogram.mp4

Hope this helps!

@Willi-K: What camera are you using? Also a RPi HQ?

Thanks again all!

Cheers,
Ralph

[Willi-K]

Yes, I also use a RPi HQ camera.
The blue images are there every night, even in the middle of the night. Very annoying, I hope there is a solution.
At first I thought it was the Raspberry Pi5. But you are using a Pi4. So that can't be the cause. Strangely, everything is OK with the daylight shots.
Here is the link to Allsky https://www.astrofototeam.de/allsky/
In the respective images folder it looks like this.

Greetings,
Willi

[Willi-K]

My blue-heavy images are becoming more and more common. It's worst on clear nights.
Yesterday it was almost completely deep blue.
https://www.astrofototeam.de/allsky/videos/allsky-20240805.mp4

Is there any indication as to why this might be?
Greetings,
Willi

[jfharvey]

Same blue images issue here. I have a RPI 5 and RPI HQ caméra.
some images also have a red tint if you compare to the first three images below.

[Willi-K]

Hello,
not many people seem to be using version 5 of the Raspberry Pi as an Allsky controller. Maybe it really has something to do with that. Astro-Ralph uses a Pi4, though. I'll switch the Allsky to a Pi4 anyway and see what happens.
Greetings Willi

[EricClaeys]

All,
I'm pretty sure this is a libcamera software issue or a camera hardware issue.

May I suggest submitting an Issue on the libcamera GitHub page?
They have always been pretty responsive and have implemented some of our suggestions.

If you submit an Issue, please link to it in this thread so others can add to your Issue, like people have done here.

[babsenfred]

Hallo, same problem here. I’m also using a RPI5 with HQ camera. Really annoying indeed. Is there a solution yet? Can’t find anything in the libcamera page on Github also.

Regards from Belgium

[EricClaeys]

@babsenfred, please see my suggestion from August 7.

[babsenfred]

Hello Eric, made an issue on the Libcamera Github page.
Thanks

[babsenfred]

@EricClaeys , can you provide the info about “rpicam-apps” naushir is asking for?
Thanks

[EricClaeys]

@babsenfred, thanks for creating the Issue. I posted a reply there.

[jfharvey]

I did a couple more tests....It's a really weird issue.

I bought a new HQ camera and I rebuilt a new sd card with the most recent OS with all the updates... and the issue is gone. I have no idea what fixed the issue.

[babsenfred]

Hello @jfharvey , how old was your build when the problem started? Or did you have the problem from the beginning? In my case, my build is 3 weeks old and the problem was from day one. So my SD card is brand new (256 GB). The HQ camera is a lot older but was never used before.

[Willi-K]

As a test, I increased the exposure time from 30 seconds to 60 seconds. Now the problem seems to be almost solved for me too. Only a few images are taken during the night that deviate very, very little from the normal color scheme. But it is no longer disturbing.
For example
https://www.astrofototeam.de/allsky/videos/allsky-20240912.mp4

Visible in the keogram as grey stripes - especially on the right in the cloud area
https://www.astrofototeam.de/allsky/keograms/keogram-20240912.jpg

[babsenfred]

Hello @Willi-K , I use auto exposure. The exposures are 10 to 15 seconds in the night. So did you set the exposure to 60 for both day and night, or only at night?

[ericofpendom]

I have also had this blue screen issue just recently. Mine is a pretty new installation of AllSky and I noticed it for the first time about 4 days ago. I have followed the advice of EricClaeys and changed the log file setup and i will wait and see what happens. 2 days ago I had my night time max exposure set to 40 seconds but changed it back to 60 seconds yesterday after seeing this thread. Last night was very cloudy so didn't see any evidence of the problem. I will keep an eye on it.

[babsenfred]

Hello @ericofpendom, I set the night exposure to manual with a value of 60 seconds. Now I get the ‘bad images’ message, see att.

[ericofpendom]

Hi, I had a repeat of the blue images last night and this morning at transition. I didn't get any messages of bad images though. I shall keep an eye on it over the next few days and I will change my night exposure to manual to see if i get the message. Thanks.

[EricClaeys]

@babsenfred, the "REMOVE_BAD_IMAGES..." message usually mean the image is almost totally black. This often happens at the end of daytime (as specified by your Angle) when it's getting dark and your Daytime Max Exposure is too low.
Is 8:59:05 pm daytime or nighttime?

[EricClaeys]

All,
I'm 99.9% sure the "blue images" problem is due either to hardware or the libcamera software, which Allsky uses to take the pictures. If you are having "blue images", I suggest adding to the rpicam-apps Issue rather than this Allsky Discussion.

The rpicam-apps team doesn't know anything about Allsky, so try not to mention Allsky-specific things. Allsky simply calls either the rpicam-still or libcamera-still command, passing it information like the exposure time to use, white balance, etc.

Look at the timestamp on one of your blue images, then run grep Running /var/log/allsky.log > /tmp/allsky.txt and look in /tmp/allsky.txt for the same timestamp. Copy/paste that line plus a couple lines before and after it into the rpicam-apps Issue, and if you still have a "good" image and the "blue" image, attach those as well.
Plus, let them know what model Pi you have, and when the issue started.