Skip to content

fix: Fix to_pandas() on empty enum Series did not preserve enum dictionary#26610

Open
nameexhaustion wants to merge 6 commits intomainfrom
nxs/empty-enum-to-pandas
Open

fix: Fix to_pandas() on empty enum Series did not preserve enum dictionary#26610
nameexhaustion wants to merge 6 commits intomainfrom
nxs/empty-enum-to-pandas

Conversation

@nameexhaustion
Copy link
Collaborator

@nameexhaustion nameexhaustion commented Feb 18, 2026

print(pl.Series(dtype=pl.Enum(["A"])).to_pandas())

# Before
Series([], Name: , dtype: category
Categories (0, object): [])
# After
Series([], Name: , dtype: category
Categories (1, str): ['A'])

@github-actions github-actions bot added A-interop-pandas Area: interoperability with pandas fix Bug fix python Related to Python Polars rust Related to Rust Polars labels Feb 18, 2026
@codecov
Copy link

codecov bot commented Feb 18, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.39%. Comparing base (83d6a72) to head (13ee338).
⚠️ Report is 5 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #26610      +/-   ##
==========================================
+ Coverage   80.76%   81.39%   +0.63%     
==========================================
  Files        1795     1795              
  Lines      244990   244992       +2     
  Branches     3079     3079              
==========================================
+ Hits       197871   199423    +1552     
+ Misses      46333    44783    -1550     
  Partials      786      786              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

if pa.types.is_dictionary(pa_arr.type):
pa_arr = pa_arr.cast(pa.dictionary(pa.int64(), pa.large_string()))
pa_dtype = pa.dictionary(pa.int64(), pa.large_string())
# Forcibly have at least 1 row as otherwise cast makes
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this cast? Is this a pyarrow cast? Is this not supposed to be fixed upstream? Seems rather hacky all this.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, pyarrow cast, indeed very hacky.

Not sure this would be considered a bug in pyarrow; I don't think they promise to preserve the dictionary across operations - it's similar to our old categoricals in that every chunk has its own local mapping.

I've pushed an update where we instead just dispatch to DataFrame.to_pandas, which does this cast using our own cast kernels in polars-compute.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-interop-pandas Area: interoperability with pandas fix Bug fix python Related to Python Polars rust Related to Rust Polars

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

Comments