Excluding 'create_index' column breaks Contracts dataset (python)

[dreaded369]

Version
Python Package - 0.3.2
Python 3.12.7

Platform
Linux hamish-proliantml150gen9 6.12.4-1-MANJARO #1 SMP PREEMPT_DYNAMIC Mon, 09 Dec 2024 11:58:37 +0000 x86_64 GNU/Linux

Description
Using the Contracts dataset, if create_index is listed a column to exclude then the query will return zero results. Excluding any other column, or all other column, returns the full number of rows.

No excluded columns
exclude_columns=['']

• rows written: 11

All excluded except create_index
exclude_columns=['init_code', 'code', 'n_init_code_bytes', 'n_code_bytes', 'code_hash', 'chain_id', 'init_code_hash'],

• rows written: 11

Just create_index excluded
exclude_columns=['init_code_hash']

• rows written: 0

This error appears in the summary:

error summary
─────────────
(errors in 4 chunks)

• Failed to convert to DataFrame: not found: create_index (4x)

[AuburyEssentian]

This is caused by default_sort() in contracts.rs returning ["block_number", "create_index"]. When create_index is excluded, the DataFrame sort step tries to reference a column that no longer exists, producing the not found: create_index error.

PR #253 fixes this by skipping excluded columns when building the sort list. It should resolve this issue once merged.

[dreaded369]

This is caused by default_sort() in contracts.rs returning ["block_number", "create_index"]. When create_index is excluded, the DataFrame sort step tries to reference a column that no longer exists, producing the not found: create_index error.

PR #253 fixes this by skipping excluded columns when building the sort list. It should resolve this issue once merged.

Awesome. From chats on telegram I figured out this was the reason, and suggested maybe throwing a warning that create_index can't be excluded and ignoring the exclusion would also be acceptable, but not returning it at all saves on processing and storage.