(fix): skip X writing when X is None#2054
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2054 +/- ##
=======================================
Coverage 87.60% 87.60%
=======================================
Files 46 46
Lines 7049 7052 +3
=======================================
+ Hits 6175 6178 +3
Misses 874 874
|
| if "X" in as_dense and isinstance(x, CSMatrix | BaseCompressedSparseDataset): | ||
| write_sparse_as_dense(f, "X", x, dataset_kwargs=dataset_kwargs) | ||
| elif x is None: | ||
| f.pop("X", None) |
There was a problem hiding this comment.
adding this pop branch make Ruff complain about too high complexity, so I extracted _write_x and _write_raw.
The logic should otherwise be the same for raw, and almost the same for X:
- check if X is to be written sparse-to-dense, if yes, do it, else
- check if X is backed and the file path matches, if yes we’re done with X, else
- (new) if X is None, remove
"X"from the HDF5 group if it’s there, else - write X normally to the file
ilan-gold
left a comment
There was a problem hiding this comment.
does it make sense to add a bytes-for-bytes test for the underlying file?
|
minimal AnnData files are byte-identical between 0.11.4 and this branch: $ uvx --with=anndata==0.11.4 python -c 'from anndata import AnnData; adata = AnnData(shape=(10, 20)); adata.write_h5ad("0.11.h5py")'
$ hatch -e hatch-test.stable run python -c 'from anndata import read_h5ad; read_h5ad("0.11.h5py").write_h5ad("0.12.h5py")'
$ md5sum 0.11.h5py 0.12.h5py
a8cdc9539050b6523f529073636733cc 0.11.h5py
a8cdc9539050b6523f529073636733cc 0.12.h5pymaximalist anndata (with X, so regardless of this PR) isn’t: $ uvx --with=anndata[test]==0.11.4 --with=numba>=0.61 python -c 'from anndata.tests.helpers import gen_adata; adata = gen_adata((10, 20)); adata.write_h5ad("0.11.h5py")'
$ hatch -e hatch-test.stable run python -c 'from anndata import read_h5ad; read_h5ad("0.11.h5py").write_h5ad("0.12.h5py")'
$ md5sum 0.11.h5py 0.12.h5py
74cbadb983b65dccd116c2342abdde30 0.11.h5py
a33f2f9244501426ca941a9b3299245e 0.12.h5py |
|
The HDF5 diff test works locally, I’m requesting your review now, and if it fails on CI, I’ll of course install it somehow and I’ll make sure it runs before merging. |
Co-authored-by: Philipp A <flying-sheep@web.de>
0.12.1#2052The removal of the
strstuff is unrelated, I just didn’t want to step over it anymore without need (we don’t support the zarr version anymore that’s mentioned in the comment that addresses thestrconversion)