Skip to content

✨ Add station metadata storage for rwis,other databases#260

Merged
akrherz merged 1 commit intomainfrom
rwis_stations
Feb 26, 2025
Merged

✨ Add station metadata storage for rwis,other databases#260
akrherz merged 1 commit intomainfrom
rwis_stations

Conversation

@akrherz
Copy link
Copy Markdown
Owner

@akrherz akrherz commented Feb 26, 2025

supporting akrherz/iem#1130

@deepsource-io
Copy link
Copy Markdown

deepsource-io bot commented Feb 26, 2025

Here's the code health analysis summary for commits 067da05..9f7c0ce. View details on DeepSource ↗.

Analysis Summary

AnalyzerStatusSummaryLink
DeepSource SQL LogoSQL❌ Failure
❗ 301 occurences introduced
🎯 253 occurences resolved
View Check ↗
DeepSource Python LogoPython✅ SuccessView Check ↗

💡 If you’re a repository administrator, you can configure the quality gates from the settings.

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello @akrherz, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

Summary of Changes

This pull request adds station metadata storage for rwis and other databases. It introduces a new stations table to the rwis and other schemas, and also includes upgrade scripts to create these tables. The stations table contains various metadata fields for stations, including location, network, elevation, and other relevant information. The schema version is also incremented in both rwis.sql and other.sql.

Highlights

  • Schema Changes: A new stations table is added to both the rwis and other schemas to store station metadata.
  • Upgrade Scripts: Upgrade scripts are included to create the stations table in existing databases.
  • PostGIS Extension: The postgis extension is added to both rwis and other schemas.
  • Schema Version Updates: The schema manager version is incremented in both rwis.sql (from 7 to 8) and other.sql (from 15 to 16).

Changelog

Click here to see the changelog
  • init/other.sql
    • Adds postgis extension.
    • Increments schema manager version from 15 to 16.
    • Creates the stations table with various metadata fields.
    • Sets the owner of the stations table to mesonet.
    • Grants select permissions on the stations table to ldm and nobody.
    • Creates unique indexes on the stations table for id, network and iemid.
  • init/rwis.sql
    • Adds postgis extension.
    • Increments schema manager version from 7 to 8.
    • Creates the stations table with various metadata fields.
    • Sets the owner of the stations table to mesonet.
    • Grants select permissions on the stations table to ldm and nobody.
    • Creates unique indexes on the stations table for id, network and iemid.
  • upgrade/other/16.sql
    • Adds postgis extension.
    • Creates the stations table with various metadata fields.
    • Sets the owner of the stations table to mesonet.
    • Grants select permissions on the stations table to ldm and nobody.
    • Creates unique indexes on the stations table for id, network and iemid.
  • upgrade/rwis/8.sql
    • Adds postgis extension.
    • Creates the stations table with various metadata fields.
    • Sets the owner of the stations table to mesonet.
    • Grants select permissions on the stations table to ldm and nobody.
    • Creates unique indexes on the stations table for id, network and iemid.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in issue comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.


A table is born, with columns defined,
To hold the stations, of every kind.
From RWIS data, to purple air,
Metadata stored, with utmost care.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

The pull request introduces station metadata storage for RWIS and other databases. The changes involve creating a stations table in multiple SQL files. The table includes various fields for storing station-related information. The code appears to be well-structured and aligned with the project's goals. However, there are a few areas that could be improved, as highlighted in the review comments.

Summary of Findings

  • Table Definition Duplication: The stations table definition is duplicated across multiple files (init/other.sql, init/rwis.sql, upgrade/other/16.sql, upgrade/rwis/8.sql). This duplication can lead to inconsistencies and maintenance issues in the future. Consider centralizing the table definition and referencing it from other files.
  • Missing Geometry Column in Upgrade Scripts: The stations table in init/other.sql and init/rwis.sql includes a geom column of type geometry(POINT, 4326), but this column is missing in the stations table definitions within the upgrade scripts (upgrade/other/16.sql and upgrade/rwis/8.sql). This discrepancy could lead to issues when upgrading the database schema.
  • Schema Version Increment: The schema version is incremented in both init/other.sql and init/rwis.sql. Ensure that these version increments are coordinated and that the upgrade scripts are executed in the correct order to maintain database consistency.

Assessment

The pull request introduces station metadata storage for RWIS and other databases, which is a valuable addition to the project. The code changes involve creating a stations table in the other.sql, rwis.sql, upgrade/other/16.sql, and upgrade/rwis/8.sql files. The table includes various fields for storing station-related information, such as ID, name, location, and other relevant parameters. Overall, the changes seem well-structured and aligned with the project's goals. However, there are a few areas that could be improved, as highlighted in the review comments. It's recommended to address these comments before merging the pull request. Please ensure that others review and approve this code before merging.

temp24_hour smallint,
precip24_hour smallint,
wigos varchar(64),
geom geometry(POINT, 4326)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The geom column is present here, but missing in the upgrade scripts (upgrade/other/16.sql and upgrade/rwis/8.sql). Ensure that the upgrade scripts also include this column to maintain consistency across the database schema.

temp24_hour smallint,
precip24_hour smallint,
wigos varchar(64),
geom geometry(POINT, 4326)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The geom column is present here, but missing in the upgrade scripts (upgrade/other/16.sql and upgrade/rwis/8.sql). Ensure that the upgrade scripts also include this column to maintain consistency across the database schema.

temp24_hour smallint,
precip24_hour smallint,
wigos varchar(64),
geom geometry(POINT, 4326)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The geom column is missing here, but present in init/other.sql and init/rwis.sql. Ensure that this column is also included in the upgrade script to maintain consistency across the database schema.

Comment on lines +10 to +47
CREATE TABLE stations(
id varchar(64),
synop int,
name varchar(64),
state char(2),
country char(2),
elevation real,
network varchar(20),
online boolean,
params varchar(300),
county varchar(50),
plot_name varchar(64),
climate_site varchar(6),
remote_id int,
nwn_id int,
spri smallint,
wfo varchar(3),
archive_begin date,
archive_end date,
modified timestamp with time zone,
tzname varchar(32),
iemid SERIAL,
metasite boolean,
sigstage_low real,
sigstage_action real,
sigstage_bankfull real,
sigstage_flood real,
sigstage_moderate real,
sigstage_major real,
sigstage_record real,
ugc_county char(6),
ugc_zone char(6),
ncdc81 varchar(11),
ncei91 varchar(11),
temp24_hour smallint,
precip24_hour smallint,
wigos varchar(64),
geom geometry(POINT, 4326)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This stations table definition is duplicated in init/rwis.sql, upgrade/other/16.sql, and upgrade/rwis/8.sql. Consider creating a common SQL file or function to define the table and referencing it from these files to avoid duplication and potential inconsistencies. This will improve maintainability.

Comment on lines +9 to +46
CREATE TABLE stations(
id varchar(64),
synop int,
name varchar(64),
state char(2),
country char(2),
elevation real,
network varchar(20),
online boolean,
params varchar(300),
county varchar(50),
plot_name varchar(64),
climate_site varchar(6),
remote_id int,
nwn_id int,
spri smallint,
wfo varchar(3),
archive_begin date,
archive_end date,
modified timestamp with time zone,
tzname varchar(32),
iemid SERIAL,
metasite boolean,
sigstage_low real,
sigstage_action real,
sigstage_bankfull real,
sigstage_flood real,
sigstage_moderate real,
sigstage_major real,
sigstage_record real,
ugc_county char(6),
ugc_zone char(6),
ncdc81 varchar(11),
ncei91 varchar(11),
temp24_hour smallint,
precip24_hour smallint,
wigos varchar(64),
geom geometry(POINT, 4326)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This stations table definition is duplicated in init/other.sql, upgrade/other/16.sql, and upgrade/rwis/8.sql. Consider creating a common SQL file or function to define the table and referencing it from these files to avoid duplication and potential inconsistencies. This will improve maintainability.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please suggest how to do this

Comment on lines +3 to +40
CREATE TABLE stations(
id varchar(64),
synop int,
name varchar(64),
state char(2),
country char(2),
elevation real,
network varchar(20),
online boolean,
params varchar(300),
county varchar(50),
plot_name varchar(64),
climate_site varchar(6),
remote_id int,
nwn_id int,
spri smallint,
wfo varchar(3),
archive_begin date,
archive_end date,
modified timestamp with time zone,
tzname varchar(32),
iemid SERIAL,
metasite boolean,
sigstage_low real,
sigstage_action real,
sigstage_bankfull real,
sigstage_flood real,
sigstage_moderate real,
sigstage_major real,
sigstage_record real,
ugc_county char(6),
ugc_zone char(6),
ncdc81 varchar(11),
ncei91 varchar(11),
temp24_hour smallint,
precip24_hour smallint,
wigos varchar(64),
geom geometry(POINT, 4326)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This stations table definition is duplicated in init/other.sql, init/rwis.sql, and upgrade/other/16.sql. Consider creating a common SQL file or function to define the table and referencing it from these files to avoid duplication and potential inconsistencies. This will improve maintainability.

@akrherz akrherz merged commit e74bd49 into main Feb 26, 2025
5 of 6 checks passed
@akrherz akrherz deleted the rwis_stations branch February 26, 2025 17:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant