GH-45229: [Python] Migrate from scipy.spmatrix to scipy.sparray #46423
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
awaiting committer review
rok
force-pushed
the
gh-45229-scipy-sparse-xxx_array-migration
branch
2 times, most recently
from
7d08c0b to
475eecf
Compare
rok
force-pushed
the
gh-45229-scipy-sparse-xxx_array-migration
branch
from
475eecf to
1ead380
Compare
|
@TheNeuralBit does this look ok to you? |
There was a problem hiding this comment.
This generally looks fine to me. It could be nice to document the potentially breaking change, but I don't think we have a way to do that.
Another thing to consider is version compatibility. It looks like the sparse arrays were added in scipy 1.8.0 so that would be a new lower bound. Again I'm not sure if we have a way to document this
That's a good point. I suppose we should check for scipy version when returning from |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Rationale for this change
As per SciPy's migration guide it would be good to change the constructors we use when creating scipy's sparse object. Further we should allow for creating sparse arrow objects from
scipy.sparray(currently onlyscipy.spmatrix)objects and test for it.What changes are included in this PR?
This adjusts
pa.SparseCXXMatrixconstructors andpa.SparseCXXMatrix.to_scipyto achieve goals listed above and adds tests.Are these changes tested?
Yes.
Are there any user-facing changes?
pyarrow.coo_matrix.to_scipyandpyarrow.csr_matrix.to_scipywould now return different types (sparray instead of spmatrix) which would probably be a breaking change (behaviors of the types would be similar).Additionally
pa.SparseCXXMatrix.from_scipywould now takescipy.sparrayandscipy.spmatrixwhich would not be a breaking change.