DAS-2446 - Allow HOSS to return an unexecuted OPeNDAP URL when requested.#61
DAS-2446 - Allow HOSS to return an unexecuted OPeNDAP URL when requested.#61lyonthefrog merged 19 commits intomainfrom
Conversation
lyonthefrog
force-pushed
the
DAS-2446
branch
from
20bedcf to
071b808
Compare
lyonthefrog
force-pushed
the
DAS-2446
branch
from
071b808 to
7994835
Compare
|
The tests in the notebook all succeeded after running harmony from DAS-2444 branch and running reload-services-config after starting up harmony. The regression tests succeeded as well. |
| actual_catalog = items[0].assets['data'].to_dict() | ||
| expected_catalog = { | ||
| 'href': expected_href, | ||
| 'title': expected_title, | ||
| 'type': expected_mimetype, | ||
| 'roles': ['data'], | ||
| } | ||
|
|
||
| # Check all the dictionary values match except for the href value. | ||
| self.assertTrue( | ||
| all( | ||
| actual_catalog[key] == expected_catalog[key] | ||
| for key in actual_catalog | ||
| if key != 'href' | ||
| ) | ||
| ) | ||
|
|
||
| # The href value must be compared separately because it can contain | ||
| # an constraint expression in the case where an unexecuted OPeNDAP URL | ||
| # is requested, where the variable order is not consistent. | ||
| self.assertEqual( | ||
| sorted(actual_catalog['href']), sorted(expected_catalog['href']) | ||
| ) |
There was a problem hiding this comment.
All this code was added to replace the simple dictionary comparison because now it's possible for href to contain the OPeNDAP URL, which contains the constraint expression where the variable order is inconsistent since required_variables is a set. There may be a less wordy way of doing this though, so open to suggestions!
sudha-murthy
left a comment
sudha-murthy
left a comment
There was a problem hiding this comment.
The harmony-regression-tests ran fine. The nsidc soap regression tests may need to be -re-run
flamingbear
left a comment
flamingbear
left a comment
There was a problem hiding this comment.
This looks pretty good so far. I have a couple of questions and some suggestions and a few minor things to clean up I think. Let me know if I'm not clear anywhere.
|
You might consider updating the hoss regression tests with one that asks for an unexecuted URL, then executes it in the notebook just like a full request. |
flamingbear
left a comment
flamingbear
left a comment
There was a problem hiding this comment.
Thanks for addressing all of the feedback. I think now you just have snyk issues to handle. I believe they updated snyk's python version today. There may be cached values somewhere. You are getting both an unsolvable error and a failed to parse the manifest file, which has been a problem in a few repos when trying to address urllib3 vulnerabilities. But this PR has 0 changes to the dependencies. So You can close and reopen and hope and if that fails I would try to fix in a new PR.
3 participants
Description
This PR allows HOSS to accept a new format parameter
application/x-netcdf4;profile=opendap_url, which will trigger HOSS to return an unexecuted OPeNDAP URL instead of a subset file. The URL format matches output currently used in the GES DISC GOSS application. A few variable names were updated to not assume that we're working with a subset output file.This will involve a new Harmony service chain (currently called sds/hoss-url-l3), but it's not yet solidified. We'll likely change the name, and add the new format option to the service record once CMR and EDSC folks make the required changes. The new format parameter needs to be added to the relevant services in Harmony's
services-uat.yml, so there currently exists an in-progress Harmony branch DAS-2444 for the following tests to be run through.Jira Issue ID
DAS-2446
Local Test Steps
Open Docker Desktop and pull this branch. Also pull and checkout the DAS-2444 harmony branch.
Run HOSS unit tests:
Bootstrap Harmony (remember to be in the DAS-2444 branch):
Notebooks
In
http://localhost:3000:Run notebook in ticket (HOSS_output_URLs.ipynb):
harmony-pyRun ticket/regression tests:
PR Acceptance Checklist
CHANGELOG.mdupdated to include high level summary of PR changes.docker/service_version.txtupdated if publishing a release.