fix: staging db_import is using prod url #580
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently, we use a prod url for https_prefix for both staging ingestion and db import. This created the issue of run, dataset, and deposition key photos using prod url prefixes, which would break before datasets were surfaced to prod.
We did this originally because we wanted our generated s3 files (the neuroglancer config being the primary case) to contain only prod url references, and we would make staging work by replacing these prod urls on the fly. We also changed the db import to also default use prod urls, which is a mistake I made in #545. For db_import, we should still default to a staging https_prefix, since the scenario of generated s3 files doesn't occur for db import, because this is just db import and not ingestion / generate files.
As a solution, I think we should go from our current default behavior:
New default behavior:
Note that currently the following datasets are affected (using prod urls for preview of the run, dataset key photos on staging):
10444, 10457, 10459-10469The fixes in this PR was validated to be working by re-db-importing DS-10458, which it now has the correct urls (DS-10458 previously was also an affected dataset but no longer).