Validate HTTP Range responses in subset downloads

[jacob1299]

Summary

• Subset downloads now check for HTTP 206 before reading the response body
• Uses stream=True so multi-GB responses are never downloaded into memory on failure
• Raises RuntimeError if Range requests are not honored

Ref #514

Test plan

• New test: test_subset_range_validation.py validates RuntimeError on non-206
• Existing subset tests pass (real S3 returns 206, guard is a no-op)
• Existing RRFS template tests pass

[jllmoe]

Thx again jacob1299!

I am very happy to inform you that I have resolved the issue by sync'ing the installed python packages on my laptop to those installed on my Desktop, now the variable-subset downloading is working as expected on my laptop as well.

Thank you so much again for your help and developing the Herbie package and I apologize for the "false alarm" in this specific regard (i.e. not working on my laptop but working on my desktop.).

The current variable-subset downloading in your Herbie package will definitely significant improve my project's efficiency, it would be even better if it can do spatial subset downloading simultaneously (along with variable-subset).

Have a great day!
John

[blaylockbk]

Nice fix. I can see the need for this check. I'll try to get this merged when I have a chance to test it myself.

[jacob1299]

Thx again jacob1299!

I am very happy to inform you that I have resolved the issue by sync'ing the installed python packages on my laptop to those installed on my Desktop, now the variable-subset downloading is working as expected on my laptop as well.

Thank you so much again for your help and developing the Herbie package and I apologize for the "false alarm" in this specific regard (i.e. not working on my laptop but working on my desktop.).

The current variable-subset downloading in your Herbie package will definitely significant improve my project's efficiency, it would be even better if it can do spatial subset downloading simultaneously (along with variable-subset).

Have a great day! John

Glad to hear it's working as expected, and you're very welcome!