Feature request
It would be really nice to test (and maybe benchmark) edge cases for nested_pandas.read_parquet, for example, reading recursively from an S3 "directory tree." I've found that IPAC's Euclid Q1 catalog is really good for that because it has leaf directories and a lot of smaller files. See, for example, this directory s3://nasa-irsa-euclid-q1/contributed/q1/merged_objects/hats/euclid_q1_merged_objects-hats/dataset/Norder=7/.
Related to and blocked by lincc-frameworks/nested-pandas#393
Before submitting
Please check the following:
Feature request
It would be really nice to test (and maybe benchmark) edge cases for
nested_pandas.read_parquet, for example, reading recursively from an S3 "directory tree." I've found that IPAC's Euclid Q1 catalog is really good for that because it has leaf directories and a lot of smaller files. See, for example, this directorys3://nasa-irsa-euclid-q1/contributed/q1/merged_objects/hats/euclid_q1_merged_objects-hats/dataset/Norder=7/.Related to and blocked by lincc-frameworks/nested-pandas#393
Before submitting
Please check the following: