Use Astrospheric API to get cloud info?

[biroron12053]

I have been working on building my new AllSky camera system and I have setup an installation of AllskyTeam. Really like the user modules. They were not available in the old TJ AllSky installation I built 4 or 5 years ago. For the new system, I considered building a cloud sensor and using the cloud module but I can see lots of potential hardware reliability issues with using the IR temperature sensor supported by the cloud module. Has anyone thought about using Astrospheric to get cloud information? An Astrospheric Pro subscription gives access to an API that looks like it provides all the info I could want in JSON format. By any chance, is there already a module that would let me use my Astrospheric Pro subscription API to link my location clould info to my AllSkyTeam setup?

[EricClaeys]

@biroron12053,
@Alex-developer did a great job with the modules and is making the overlay module much easier to use in the next release.
A lot of people are using modules, and several people have written their own.

I will let Alex answer your question.

[biroron12053]

Thanks much. Considering writing my own but thought I would ask if someone might have already done something close.
Still learning how the overlays work and it needs more study before I could even try to write my own.

[chvvkumar]

I made this cloud detection program as I was in a similar situation. I also wanted to detect more than just clouds but also rain, snow, level of clouds etc.

https://github.com/chvvkumar/simpleCloudDetectpriv

[EricClaeys]

@chvvkumar. I get a "404 not found" error when going to the URL

[chvvkumar]

Sorry for the wrong link, that was my private repo to test the software, the right link is here https://github.com/chvvkumar/simpleCloudDetect

[biroron12053]

@chvvkumar I originally thought that analyzing an image captured by the Allsky camera was the ideal solution. I just had no idea how to do that. Therefore, I considered using Astrospheric which already has done the analysis. Didn't catch that their API was limited to so few calls per day. I'll take a closer look at your solution. Thanks!

[Alex-developer]

I have been working on building my new AllSky camera system and I have setup an installation of AllskyTeam. Really like the user modules. They were not available in the old TJ AllSky installation I built 4 or 5 years ago. For the new system, I considered building a cloud sensor and using the cloud module but I can see lots of potential hardware reliability issues with using the IR temperature sensor supported by the cloud module. Has anyone thought about using Astrospheric to get cloud information? An Astrospheric Pro subscription gives access to an API that looks like it provides all the info I could want in JSON format. By any chance, is there already a module that would let me use my Astrospheric Pro subscription API to link my location clould info to my AllSkyTeam setup?

I have had a quick look at the API and it seems very restrictive. For the subscription you get 100 credits and each call to the api to get cloud cover will cost 5 credits. Its also appears to be a prediction rather than actual measurements.

I spent a long time tinkering with various sensors to try to determine cloud cover with varying degrees of success. The best success was with the 90614, I mount it horizontally and reflect the sky off of a 45 degree surface to help protect the sensor. It still found it way too sensitive to ambient changes.

The best option I have found for accurate detection is AI. There are a few options available here

1. The allsky AI module - found in the extra modules. This requires an internet connection
2. I am working on a local version, there is branch in the extra modules for it. This does not require an internet connection
3. Third party code like @chvvkumar has mentioned

I am planning to reach out to a few of the people that have developed AI based systems to try and see if we can get something that works for everyone

Having said all of that if you do want to use he Astropheric API then I would be happy to put a simple module together for you, I would just require a sample of the API data and some help from you in testing it.

As Eric mentioned the module and overlay system is going thru a major update at the moment to make it all much easier to use - watch tis space :-)

Thanks

Alex

[biroron12053]

I still have a lot to learn about how AllSkyTeam modules and overlays work. Hearing about the next release, I will continue to study the system but may not do much in actual programming until the next release is available. As part of my learning, in the last 24 hrs. using the Weather Underground module as a model I cobbled together a very simple test (probably doing things the wrong way) to grab some info from Astrospheric and show a few numbers in an overlay. The test gave me some confidence that I could make it work if I spent more time on it. I knew that Astrospheric currently only gives 100 "credits" per day. I assumed that I could get 100 calls per day. That might be minimally usable (1 call every 15 minutes). Didn't originally catch that they charge 5 credits per call. That would not work for me. I also have looked at some of the IR sensor cloud detection solutions that are described online and I was not happy with the issues with rain, snow, and protecting the sensor from the environment. I have enough trouble with keeping snow, ice, rain, and dirt off of my AllSky camera dome. Just for fun I may assemble an IR cloud sensor system anyway using a narrow field of view IR temperature sensor and a fused quartz window to protect the sensor. It probably wouldn't be for a permanent installation. I am wondering about my local sky. From my observations during many years of using my telescopes, it seems it is almost NEVER completely clear. There is always wisps of clouds. I'd like to make some measurements to see just how bad it is. I have only glanced at the solution from @chvvkumar but it looks very interesting and is closer to what I had originally thought would be ideal, which is actually analyzing the image from the Allsky camera to determine cloud cover, etc. I rely heavily on my Allsky camera to tell me when the sky is clear. The Allsky image shows every few minutes as part of the regular rotation of images on my security system monitors inside the house. The virtualsky overlay is also essential when I am using my telescopes to know when targets are above the trees (regardless of season and growth of the trees) and to help see holes in the clouds that I might observe through. Thanks for all the help with understanding what my options are. I wouldn't be surprised if you guys come up with a better solution than I can even think of.

[Alex-developer]

I still have a lot to learn about how AllSkyTeam modules and overlays work. Hearing about the next release, I will continue to study the system but may not do much in actual programming until the next release is available. As part of my learning, in the last 24 hrs. using the Weather Underground module as a model I cobbled together a very simple test (probably doing things the wrong way) to grab some info from Astrospheric and show a few numbers in an overlay. The test gave me some confidence that I could make it work if I spent more time on it. I knew that Astrospheric currently only gives 100 "credits" per day. I assumed that I could get 100 calls per day. That might be minimally usable (1 call every 15 minutes). Didn't originally catch that they charge 5 credits per call. That would not work for me. I also have looked at some of the IR sensor cloud detection solutions that are described online and I was not happy with the issues with rain, snow, and protecting the sensor from the environment. I have enough trouble with keeping snow, ice, rain, and dirt off of my AllSky camera dome. Just for fun I may assemble an IR cloud sensor system anyway using a narrow field of view IR temperature sensor and a fused quartz window to protect the sensor. It probably wouldn't be for a permanent installation. I am wondering about my local sky. From my observations during many years of using my telescopes, it seems it is almost NEVER completely clear. There is always wisps of clouds. I'd like to make some measurements to see just how bad it is. I have only glanced at the solution from @chvvkumar but it looks very interesting and is closer to what I had originally thought would be ideal, which is actually analyzing the image from the Allsky camera to determine cloud cover, etc. I rely heavily on my Allsky camera to tell me when the sky is clear. The Allsky image shows every few minutes as part of the regular rotation of images on my security system monitors inside the house. The virtualsky overlay is also essential when I am using my telescopes to know when targets are above the trees (regardless of season and growth of the trees) and to help see holes in the clouds that I might observe through. Thanks for all the help with understanding what my options are. I wouldn't be surprised if you guys come up with a better solution than I can even think of.

Glad you got some data :-)

I presume you have found that if you write a properly formatted json file to the extra data folder you can use the data i nthe overlay

Alex

[chvvkumar]

I have been working on building my new AllSky camera system and I have setup an installation of AllskyTeam. Really like the user modules. They were not available in the old TJ AllSky installation I built 4 or 5 years ago. For the new system, I considered building a cloud sensor and using the cloud module but I can see lots of potential hardware reliability issues with using the IR temperature sensor supported by the cloud module. Has anyone thought about using Astrospheric to get cloud information? An Astrospheric Pro subscription gives access to an API that looks like it provides all the info I could want in JSON format. By any chance, is there already a module that would let me use my Astrospheric Pro subscription API to link my location clould info to my AllSkyTeam setup?

I have had a quick look at the API and it seems very restrictive. For the subscription you get 100 credits and each call to the api to get cloud cover will cost 5 credits. Its also appears to be a prediction rather than actual measurements.

I spent a long time tinkering with various sensors to try to determine cloud cover with varying degrees of success. The best success was with the 90614, I mount it horizontally and reflect the sky off of a 45 degree surface to help protect the sensor. It still found it way too sensitive to ambient changes.

The best option I have found for accurate detection is AI. There are a few options available here

1. The allsky AI module - found in the extra modules. This requires an internet connection
2. I am working on a local version, there is branch in the extra modules for it. This does not require an internet connection
3. Third party code like @chvvkumar has mentioned

I am planning to reach out to a few of the people that have developed AI based systems to try and see if we can get something that works for everyone

Having said all of that if you do want to use he Astropheric API then I would be happy to put a simple module together for you, I would just require a sample of the API data and some help from you in testing it.

As Eric mentioned the module and overlay system is going thru a major update at the moment to make it all much easier to use - watch tis space :-)

Thanks

Alex

Alex,

At this point, my solution is going to be a bit unwieldy to use as a module within Allsky as it uses the tensorflow libraries. They are quite large. I have also tried making my docker image into an ARM images for running the container directly on the Pi but I could not find a suitable tensorflow-cpu ARM library. So as of now the image I have is x86 only.

Also, the container I created is at this point a generic image classifier. As the Allsky specific parts of it (like the model, classification list and image path/url) are common to any image as long as the model has been trained to do so.

Saying that, please feel free to use any piece of code there to incorporate into the module you have in mind for Allsky.

[Alex-developer]

At this point, my solution is going to be a bit unwieldy to use as a module within Allsky as it uses the tensorflow libraries. They are quite large. I have also tried making my docker image into an ARM images for running the container directly on the Pi but I could not find a suitable tensorflow-cpu ARM library. So as of now the image I have is x86 only.

Also, the container I created is at this point a generic image classifier. As the Allsky specific parts of it (like the model, classification list and image path/url) are common to any image as long as the model has been trained to do so.

Saying that, please feel free to use any piece of code there to incorporate into the module you have in mind for Allsky.

Thanks appreciate you letting use use any code, if I do then of course it will be credited

There are a few 'modules' that take a long time to run

• AI Cloud detection
• Meteor Detection - I have just rewritten this using a MUCH better system
• Star counting

I am going to speak to @EricClaeys about some ideas I have for solving this problem

Alex

[EricClaeys]

@biroron12053,
If you come up with a good solution and make it a module, we'll give you an Allsky Team Nobile Prize and all the money we've ever made from working on Allsky!!!!

[biroron12053]

Gee... Thanks. :)