Dominik/pandas3 cow fix issue180#181
Merged
Merged
Conversation
Build the terminal-state identity block with np.eye instead of mutating the read-only DataFrame.values view in place. Under pandas 3 (mandatory Copy-on-Write) the old in-place assignment raised ValueError: assignment destination is read-only. Fixes both the main path and the trans_states==0 edge branch. Compatible with pandas 2 and 3.
_validate_obsm_key(as_df=False) returns a DataFrame.values view that is read-only under pandas 3 (mandatory Copy-on-Write). select_branch_cells mutated it in place (NaN fill), raising ValueError: assignment destination is read-only on the default SAVE_AS_DF=True workflow. Copy the array before the in-place fill. Sole mutating as_df=False caller; the other two (compute_gene_trends, run_low_density_variability) only read. Amends the CoW fix to cover this second site found in the sibling sweep.
anndata >=0.11 removed the AnnData.__init__ 'dtype' argument, breaking cluster_gene_trends on Python 3.12/3.13 (where the newer anndata resolves) with 'TypeError: got an unexpected keyword argument dtype'. Cast the array to float32 before construction instead — behaviour is identical and compatible with all anndata versions and pandas 2 & 3.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix pandas 3 Copy-on-Write compatibility (#180)
Problem
Under pandas 3, Copy-on-Write is mandatory, so
DataFrame.valuesreturns a read-only view andin-place mutation of it raises
ValueError: assignment destination is read-only. On pandas 2 (CoW off bydefault) the same code worked, so the breakage is pandas-3-specific. Two functions are affected:
run_palantir→_differentiation_entropy: builds a terminal-state identity matrix viabp.values[range(n), range(n)] = 1(the site reported in Pandas 3 compatibility:ValueError: assignment destination is read-onlyin_differentiation_entropy#180, plus an identical occurrence in thelen(trans_states) == 0branch).select_branch_cells: NaN-fills the fate-probability array in place. It obtains that array from_validate_obsm_key(..., as_df=False), which returnsDataFrame.valueswhen probabilities are stored asa DataFrame — the default
run_palantiroutput — so this breaks the standard workflow.Fix
np.eye(...)instead of mutating a.valuesview (both occurrences)..copy()of the fate-probability array before the in-place NaN fill inselect_branch_cells.Both are Copy-on-Write-safe and behave identically on pandas 2 and 3. Results are unchanged; the identity
block's dtype becomes
float64(it was concatenated with float probabilities anyway).Compatibility
No version bump; pandas 2 support retained. Verified in clean venvs (Python 3.13):
tests/test_core_run_palantir.py5/5,tests/test_presults_select_branch_cells.py5/5.Changelog
Added an entry under Version 1.4.5 (future) in
README.md.Closes #180