Skip to content

Modernize rebuild scripts #231

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
davenquinn wants to merge 10 commits into
base: main
Choose a base branch
from
Draft

Modernize rebuild scripts #231

davenquinn wants to merge 10 commits into from

Conversation

@davenquinn
Copy link
Member

@davenquinn davenquinn commented Dec 17, 2025 •
edited
Loading

Modernize stratigraphy rebuild scripts

Scripts

  • autocomplete
  • lookup_strat_names
  • lookup_unit_attrs_api
  • lookup_unit_intervals
  • lookup_units
  • pbdb_matches
  • stats
  • strat_name_footprints
  • unit_boundaries

Tasks for each script

  • Convert the database access from MariaDB to PostgreSQL
  • Clean up control flow and use more modern parameter binding
  • Other fixes for usability

Some of these scripts are outdated, so they may not work without some modification. But we should start with direct SQL translation where possible.

Many run lots of SQL and so might benefit from being run/tested against a local database, or on the cluster.

Our new macrostrat.database library should make the SQL a lot terser and easier to read.

Overall process

  • Convert all scripts to PostgreSQL
  • Check for correct output in macrostrat schema and API results (number of rows, output structure)
  • Move the ~finalized scripts out of the v1 schema

@davenquinn davenquinn marked this pull request as draft December 17, 2025 10:39
@davenquinn
Reverted control flow error
e7232ed
@davenquinn
Merge remote-tracking branch 'origin/modernize-rebuild-scripts' into …
478e917 
…modernize-rebuild-scripts

* origin/modernize-rebuild-scripts:
  Format code and sort imports
@davenquinn
Copy link
Member Author

davenquinn commented Dec 17, 2025

In working on the first few scripts, autocomplete and lookup_strat_names, it seems like one of the only major SQL changes needed to migrate from MariaDB to PostgreSQL syntax is a change from the
UPDATE <table> LEFT JOIN <other table> SET ... WHERE ... syntax
to the UPDATE <table> SET ... FROM <other table> WHERE ... <join_condition> syntax. This may cause weirdness about the join condition (i.e., whether rows with NULLS are included) – we'll have to monitor this.

@davenquinn davenquinn mentioned this pull request Dec 22, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@davenquinn davenquinn

@amyfromandi amyfromandi

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@davenquinn @amyfromandi