Intake links to radiosondes and Meteor flux observations stopped working

[franslql]

Hi,

Today the intake catalogue suddenly stopped working for radiosondes (as far as I can tell of all sources) and for the meteor surface fluxes. Upon running the following commands:

from intake import open_catalog; cat = open_catalog('https://raw.githubusercontent.com/eurec4a/eurec4a-intake/master/catalog.yml'); flux_obs = cat.Meteor.surface_fluxes.to_dask()

I get the following error:
[Errno -70] NetCDF: DAP server error: b'https://observations.ipsl.fr/thredds/dodsC/EUREC4A/SHIPS/RV-METEOR/surface_fluxes/EUREC4A_Meteor_surface_heat_fluxes_20200115_v1.0.nc

I'm also accessing the JOANNE level 3 dropsonde product, which is still working. It seems like that any link that points to thredds/dodsC/EURECA4/ instead of aeris/eurec4a-data/ seemed to have stopped working.

Thank you,
Frans

[observingClouds]

Thanks @franslql for the report! We run weekly checks ourselves to catch any broken/inaccessible datasets. I just retriggered that workflow to see if we see this issue as well.

From past experience some data stores are less stable than others.

Adding @vincentdouet as he might be able to see if there is an issue on IPSL side

[observingClouds]

The test completed and indeed indicates that all datasets that are being accessed via the IPSL THREDDS server fail.

[franslql]

Hi @observingClouds it seems to be working again, thanks for getting back to me so quickly.

[observingClouds]

Great! Let's cross our fingers that this remains so. Feel free to reopen so the community knows about these issues.

[observingClouds]

It actually still fails @franslql .

@vincentdouet could you give us an update?

[observingClouds]

You could also try to reach out on https://espri.ipsl.fr/contact-us/

[vincentdouet]

Sorry, I've just seen your messages. I restarted the Thredds server because I had an error “IOException sending File /opendap/: Too many open files”. I'm now going to look at why we're getting this error. Are you opening a lot of files?

[observingClouds]

Thanks @vincentdouet! @franslql could you let Vincent know if you are opening a lot of files? If you are currently working with the flux data, I imagine the number of files is quite low (~1 file/day).

[franslql]

Hi @vincentdouet and @observingClouds. Sorry for the late reply, I'm working on Melbourne hours. Currently I'm working with the radiosonde output. From yml it looks like 6 netcdf files. I was using the dropsondes and the fluxes before as well, but I'm done with those now.