Skip to content

Stop concatenating null dataframes #164

@zaneselvans

Description

@zaneselvans

In the main PUDL ETL we are currently getting tens of thousands of warnings like:

/Users/zane/miniforge3/envs/pudl-dev/lib/python3.11/site-packages/ferc_xbrl_extractor/xbrl.py:169: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation.

We should probably stop attempting to do these null concatenations, lest it break down in the future. Or maybe we can just assign explicit data types? Or suppress the warnings so we only get it 1 time, instead of 17,000 times?

For the moment these warnings are being ignored completely. See #170

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status

    Icebox

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions