Stratigraphy ingestion system#272
Open
davenquinn wants to merge 56 commits intomainfrom
Open
Conversation
- change type of cols.col_name to text - remove NOT NULL from sections.fo_h and sections.lo_h - remove NOT NULL from units.max_thick, units.min_thick - remove NOT NULL from units.fo_h, units.lo_h, remove default - remove null constraint on obsolete fields in "macrostrat"."strat_names" **NOTE:** we are not removing the foreign key on units -> sections for now, as having units depend on sections seems to be the right approach. may need to further update primary keys on - units_sections - strat_names - strat_tree
* integrate-shanan-update: Update schema Works until units_sections Migrated database Added modifications to schema to Macrostrat database Integrate v3 version Prepare for schema update Reformatted python code Move files into macrostrat.column_ingestion module Bring in shanan's ingestion script
* main: Update dependencies Format code and sort imports Bump tileserver and legacy tileserver versions Improve config for local running of tileserver Update image tileserver connection pooling Format code and sort imports added grant permissions to rebuild scripts tables after they are reran
* main: Format code and sort imports Improve the isolation of macrostrat.core module Moved schema management to its own library
…trat/macrostrat into stratigraphy-ingestion * 'stratigraphy-ingestion' of https://github.com/UW-Macrostrat/macrostrat: Improve handling of lithologies
* main: (50 commits) updating schema Update ingest_process_id_eq_source_id.sql update ingest process id == source id changed state enum to a ingest_state reference table Format code and sort imports bulk script final and using for japan ingestion Move helper functions Format code and sort imports added polygons, points,and lines bulk processing Format code and sort imports standardizing scripts for bulk processing Format code and sort imports updated Japan shp file scraper and metadata scraper. Format code and sort imports added lots of helper functions to process maps manually Format code and sort imports Add options to map export command Added a helper function for dumping lithology information mapping name to notes in the spots to checkin workflow Format code and sort imports ...
* test-framework: Removed dependency on osgeo bindings and got tests to compile Apply some optimizations to testing Standardize map config in tests Updated map fixtures Fix maps metadata schema Good progress on map-staging tests Fixed slow test skipping All test database handling works Added transaction-gating for test fixtures Basic tests pass Updated environment loading Updated conftest Updated CLI basic tests Specifiy a read-only user Shift to testcontainers for more test stability
…trat/macrostrat into stratigraphy-ingestion * 'stratigraphy-ingestion' of https://github.com/UW-Macrostrat/macrostrat: Format code and sort imports
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Starting point for column ingestion from spreadsheets
macrostrat.column_ingestionmoduleAs part of this, we did a major rework of the test architecture for the entire repository.