Add a walk function for navigating THREDDS catalogue #754
Description
As far a I know this functionality does not exist already but believe it would be a welcome addition :
I often need to find all datasets for multiple subfolders of a thredds catalogue. To do this I resort to using a custom function in my data processing scripts (see simple example below) but ideally this would be built into siphon itself.
from siphon.catalog import TDSCatalog
# walk function
def walk(cat, depth=1):
"""Return a generator walking a THREDDS data catalog for datasets.
Parameters
----------
cat : TDSCatalog
THREDDS catalog.
depth : int
Maximum recursive depth. Setting 0 will return only datasets within the top-level catalog. If None,
depth is set to 1000.
"""
yield from cat.datasets.items()
if depth is None:
depth = 1000
if depth > 0:
for name, ref in cat.catalog_refs.items():
try:
child = ref.follow()
yield from walk(child, depth=depth - 1)
except requests.HTTPError as exc:
LOGGER.exception(exc)
# creat catalogue
cat = TDSCatalog(urlcat)
# access all dataset to 20 subfolders
for dd in (cat, depth=20):
print(dd)