Skip to content

Create core_eia176__yearly_gas_supply#5227

Merged
e-belfer merged 10 commits into
catalyst-cooperative:mainfrom
MeadBarrel:i4711
Jun 9, 2026
Merged

Create core_eia176__yearly_gas_supply#5227
e-belfer merged 10 commits into
catalyst-cooperative:mainfrom
MeadBarrel:i4711

Conversation

@MeadBarrel

@MeadBarrel MeadBarrel commented May 8, 2026

Copy link
Copy Markdown
Contributor

Overview

Closes #4711 .

What problem does this address?

EIA Form 176 Part 4 (Lines 1.0-7.0) contains operator-level natural and
supplemental gas supply data, including production, storage withdrawals, receipts,
supplemental gaseous fuels, and reported total supply for the report state. This
data was available in the extracted EIA-176 inputs but was not yet exposed as a
PUDL core table.

This table exposes the source-reported Part 4 summary fields. It does not include
the granular continuation-line details for Line 3.0 receipts or Line 6.0
supplemental gaseous fuels.

What did you change?

  • Added core_eia176__yearly_gas_supply with one row per
    (operator_id_eia, report_year).
  • Included standardized Mcf fields for natural gas production, synthetic gas
    production, storage withdrawals, state or U.S. border receipts, citygate
    receipts, other receipts, supplemental gaseous fuels, and reported total
    supply.
  • Added transform validation comparing reported Line 7.0 total supply against
    the sum of reported component fields, with a bounded allowance for known
    historical source-data inconsistencies.
  • Added continuation-line validation for Line 3.0 receipts and Line 6.0
    supplemental gaseous fuels.
  • Added table and field metadata for the new output table, including notes about
    continuation-line detail and historical total-supply inconsistencies.
  • Added an Alembic migration for the new table.
  • Added dbt row-count and non-negative value expectations.

Documentation

Make sure to update relevant aspects of the documentation:

  • Update the release notes: reference the PR and related issues.
  • Update relevant Data Source jinja templates (see docs/data_sources/templates).
  • Update relevant table or source description metadata (see src/metadata).
  • Review and update any other aspects of the documentation that might be affected by this PR.

Testing

How did you make sure this worked? How can a reviewer verify this?

To-do list

  • If updating analyses or data processing functions: make sure to update row count expectations in dbt tests.
  • Run pixi run prek-run to run linters and static code analysis checks.
  • Run pixi run pytest-ci locally to ensure that the merge queue will accept your PR.
  • Review the PR yourself and call out any questions or issues you have.
  • For PRs that change the PUDL outputs significantly, run the full ETL locally and then run the data validations using dbt. If you can't run the ETL locally then run the build-deploy-pudl GitHub Action manually and ensure that it succeeds.

@krivard krivard added the community Issues that contributors have volunteered to take on or fostering more community label May 13, 2026
@krivard krivard moved this from New to In progress in Catalyst Megaproject May 13, 2026
@e-belfer e-belfer added new-data Requests for integration of new data. eia176 Issues related to the EIA Form 176 natural gas supply and disposition dataset. labels May 13, 2026
@e-belfer e-belfer self-requested a review May 13, 2026 18:57

@e-belfer e-belfer left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can make this into a nice skinny table by stacking all the columns into a volume_mcf and a categorical supply_type column. That'll require redoing the migration and dropping these new field definitions for the most part. total_supply_mcf can be streamlined to just total.

Image

It looks like total citygate gets split into two components after 2010. No action item here, but we should note it in the table description.

Image

Looking at the log scale, everything else seems pretty reasonable, with above ground withdrawals being understandably somewhat spiky over time.

Comment thread src/pudl/transform/eia176.py Outdated
Comment thread src/pudl/transform/eia176.py Outdated
Comment thread src/pudl/transform/eia176.py
@e-belfer e-belfer marked this pull request as ready for review June 9, 2026 15:43
@e-belfer e-belfer self-requested a review June 9, 2026 16:33

@e-belfer e-belfer left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once #5219 merges, we'll need to update the alembic migrations to refer to the new head. Otherwise this is ready to go!

Edit: #5219 was bounced from the queue, so merging this first.

Comment thread src/pudl/transform/eia176.py
@e-belfer e-belfer added this pull request to the merge queue Jun 9, 2026
@github-merge-queue github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jun 9, 2026
@e-belfer e-belfer enabled auto-merge June 9, 2026 20:11
@e-belfer e-belfer added this pull request to the merge queue Jun 9, 2026
Merged via the queue into catalyst-cooperative:main with commit 95513b8 Jun 9, 2026
9 checks passed
@github-project-automation github-project-automation Bot moved this from In progress to Done in Catalyst Megaproject Jun 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

community Issues that contributors have volunteered to take on or fostering more community eia176 Issues related to the EIA Form 176 natural gas supply and disposition dataset. new-data Requests for integration of new data.

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Create core_eia176__yearly_gas_supply (Part 4)

3 participants