Skip to content

BUG: read_csv pyarrow dtype dictionary omitting an index column - #71858

Open
LionStone-1 wants to merge 1 commit into
pandas-dev:mainfrom
LionStone-1:contribpilot/issue-68041-arrow-index-dtype
Open

LionStone-1 wants to merge 1 commit into
pandas-dev:mainfrom
LionStone-1:contribpilot/issue-68041-arrow-index-dtype

Conversation

@LionStone-1

Copy link
Copy Markdown

Closes #68041.

read_csv(engine="pyarrow") raises IndexError when a dtype dictionary omits a named index column. The fix uses the already-resolved column name instead of treating the name as a numeric position.

Added regression coverage for named, positional, and mixed MultiIndex selections, plus a release note. The scalar-dtype case was already fixed upstream and is not changed here.

Validation on the source checkout:

  • New regression before fix: 5 failed, 3 passed; after fix: 8 passed.
  • Index-column tests: 138 passed, 13 skipped, 5 expected failures.
  • Arrow dtype tests: 227 passed, 15 skipped, 18 expected failures.
  • Repository-pinned Ruff lint/format and git diff --check: passed.
  • Independent read-only review reran the 8 regression cases and found no blocking findings.

The full pandas suite, full pre-commit hooks, and GitHub CI have not run locally.

AI assistance: Codex desktop assisted with investigation, code, regression tests, and this description. Local Codex configuration specifies gpt-6.1-sol with reasoning effort low; those are configuration values, not independently recorded per-turn runtime metadata. The contributor personally reviewed the changes and requested this draft, which remains for their editing before review.

  • Tests added and passed.
  • All repository code checks passed (full pre-commit remains unrun).
  • Release note added.
  • AI assistance disclosed; contributor confirmed personal review.

@github-actions github-actions Bot added the Needs Issue Assignment PR whose author is not assigned to a linked issue label Nov 3, 2026
@github-actions

github-actions Bot commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

Thanks for the pull request, @LionStone-1! It's linked to #68041, but that issue isn't assigned to you yet. To make sure two people don't unknowingly work on the same thing, we ask contributors to claim an issue first. You can comment /take on #68041 to claim it, but be aware that certain issues cannot be taken if they are labeled e.g. Needs Triage — see the contributing guide for more details.

This pull request is unlikely to be reviewed until #68041 is assigned to you.

@LionStone-1
LionStone-1 marked this pull request as ready for review November 3, 2026 07:00

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Needs Issue Assignment PR whose author is not assigned to a linked issue

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BUG: read_csv fails to finalize index when engine='pyarrow' and dtype is scalar (or dict without an index specified)

1 participant