Skip to content

to_dataframe behaves differently with Pandas 3.0 #1001

Description

@mataton

The default fill value for sparse operations has changed from 0.0 to NaN in Pandas 3.0. This can affect downstream functions like the call to scikit-bio's multi_replace (which can't have NaNs in the input) in notebook 5 from the tutorials.

Specifically this line:

abundance_data = subset.to_dataframe().T

will now create a dataframe full of mostly NaNs.

There is already a simple solution, which is to just use the dense=True setting:

abundance_data = subset.to_dataframe(dense=True).T

@wasade Not sure if you want to change anything here in terms of the output of to_dataframe(dense=False), but I at least wanted to bring it to your attention. I'm gonna update the tutorial to use the dense=True option.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions