Adopt the array api by mwcraig · Pull Request #885 · astropy/ccdproc

mwcraig · 2025-03-19T15:17:48Z

This PR makes the necessary changes to adopt the array API. There are still some things to sort out:

Run tests mostly against numpy, but do one run against jax (or some other non-numpy array library)
Check for remaining invocations of numpy in the code.
Check whether changing .data_arr.mask to .data_arr_mask constitutes an API change. If it does (and I think it does) then patch things up so that .data_arr.mask points to .data_arr_mask
Clean up any remaining handling of immutable arrays (array_api_extra makes handling those easy)

Once those are done the next step will be to find an outside reviewer for this...

codecov · 2025-03-20T15:04:20Z

Codecov Report

❌ Patch coverage is 94.10569% with 29 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (main@5aa9219). Learn more about missing BASE report.
⚠️ Report is 134 commits behind head on main.

Files with missing lines	Patch %	Lines
ccdproc/_ccddata_wrapper_for_array_api.py	84.51%	24 Missing ⚠️
ccdproc/core.py	98.25%	3 Missing ⚠️
ccdproc/image_collection.py	92.30%	2 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #885   +/-   ##
=======================================
  Coverage        ?   96.56%           
=======================================
  Files           ?        8           
  Lines           ?     1571           
  Branches        ?        0           
=======================================
  Hits            ?     1517           
  Misses          ?       54           
  Partials        ?        0

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

eteq · 2025-06-24T21:51:22Z

As a quick FYI I ran the test suite in this PR on a machine with a NVidia GPU with CCDPROC_ARRAY_LIBRARY=cupy and got a whole pile of errors (sent the log directly to @mwcraig but can also share snippets here if someone else wants to see them)

nstarman · 2025-06-25T18:23:58Z

 readme = "README.rst"
 license = { text = "BSD-3-Clause" }
-requires-python = ">=3.8"
+requires-python = ">=3.11"


Adopt Spec 0? Has that been adopted?

Good catch!

Wait, misread this -- what is Spectrum 0?

https://scientific-python.org/specs/spec-0000/

So the move here would be to bump it up to 3.12?

3.11 for a few more months. Then 3.12. I was also thinking about adding the badge saying "Spec 0", e.g. you can see in the README in https://github.com/GalacticDynamics/unxt

nstarman · 2025-06-26T22:49:40Z

 readme = "README.rst"
 license = { text = "BSD-3-Clause" }
-requires-python = ">=3.8"
+requires-python = ">=3.11"


https://scientific-python.org/specs/spec-0000/

These are necessary because upstream dependencies (astropy) explicitly check for numpy arrays rather than using the array API.

Properly set uncertainty unit in gain_correct

new_data never has a .data the way the test is written

nstarman · 2025-09-04T19:04:21Z

🎉

mwcraig added this to the 2.5.0 milestone Mar 19, 2025

mwcraig added the enhancement label Mar 19, 2025

mwcraig force-pushed the explore-array-api branch from 85625f0 to 04293b7 Compare March 27, 2025 18:05

mwcraig marked this pull request as ready for review June 24, 2025 14:55

mwcraig changed the title ~~WIP: Adopt the array api~~ Adopt the array api Jun 24, 2025

mwcraig requested a review from Copilot June 24, 2025 16:42

This comment was marked as outdated.

Sign in to view

nstarman reviewed Jun 25, 2025

View reviewed changes

nstarman reviewed Jun 26, 2025

View reviewed changes

Comment thread ccdproc/core.py Outdated

Comment thread pyproject.toml

mwcraig force-pushed the explore-array-api branch 3 times, most recently from 2fa76b7 to 3b2b9fc Compare July 3, 2025 18:28

mwcraig added 15 commits July 5, 2025 10:05

Initial attempt to adopt array API

6dd9166

A couple more changes

d3b6203

Add function for check if something is an array

f0bc6b8

Remove numpy from core

424ebdc

WIP rewrite of a boolean index

7f5050d

WIP testing updates

6a6bfa2

Change more almost_equal to allclose

1d2d171

Use the random number generator from numpy

22bc04d

Remove more in-place array operations

245d641

Set a reasonable tolerance value in float comparisons

55b90be

Continue to use numpy arrays in a few places

23de261

These are necessary because upstream dependencies (astropy) explicitly check for numpy arrays rather than using the array API.

Refactor warning handling for numerical warnings

dffadce

Avoid explicit use of numpy masked arrays

12ff8b0

Rewrite test to not modify array in-place

63f81ba

Use assert_allclose instead of older alternatives

4ae3fc6

mwcraig added 21 commits August 5, 2025 09:38

Use CCDData compatibility wrapper in trim_image

b17566b

Cast number to array namespace

fdec811

Remove another instance of conversion to a numpy array

7531eee

More fixes for flat_correct

c196b93

One more flat fix

f9ad753

Do not use pytest.approx because it forces a numpy conversion

a0f738a

Add cast to array namespace

65ef951

Add cast to array namespace in gain_correct

52d6d45

Fix handling of gain value/unit

9f59fb5

Properly set uncertainty unit in gain_correct

Avoid mask setter in more places

c3ffbdf

Fix typo

9de6a92

Wrap the CCDData object inside transform_image

558ee79

Wrap before cop because copy sets mask

df71b4d

Fix typo

4845ec2

Ensure uncertainty in test uses array namespace

406ded9

Set mask properly in test

b062491

Make error from data, not ccd object

42bb716

Fix several array API issues in lacosmic

c204f4d

Ensure uncertainty is wrapped even if CCDData is already wrapped

ebf9127

Add numpy conversion for now

aedb3b2

Fix error in test

2a877a9

new_data never has a .data the way the test is written

mwcraig force-pushed the explore-array-api branch from 5c81f63 to 2a877a9 Compare August 18, 2025 14:24

mwcraig added 2 commits August 18, 2025 09:42

Cast mask to array API object

40ffbae

Allow CuPy arithmetic operation names in uncertainty propagation

e2a808f

mwcraig force-pushed the explore-array-api branch from 91b1118 to e2a808f Compare August 29, 2025 14:22

mwcraig mentioned this pull request Sep 4, 2025

CuPy issues #909

Open

mwcraig merged commit 22ab5ca into astropy:main Sep 4, 2025
15 checks passed

mwcraig deleted the explore-array-api branch September 4, 2025 18:35

mwcraig mentioned this pull request Apr 13, 2026

Cycle 5: Array API in ccdproc and NDData and work on astrowidgets (Craig) astropy/astropy-project#515

Open

Uh oh!

Conversation

mwcraig commented Mar 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov Bot commented Mar 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

This comment was marked as outdated.

Uh oh!

eteq commented Jun 24, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nstarman Jun 25, 2025

Choose a reason for hiding this comment

Uh oh!

mwcraig Jun 26, 2025

Choose a reason for hiding this comment

Uh oh!

mwcraig Jun 26, 2025

Choose a reason for hiding this comment

Uh oh!

nstarman Jun 26, 2025

Choose a reason for hiding this comment

Uh oh!

mwcraig Jun 26, 2025

Choose a reason for hiding this comment

Uh oh!

nstarman Jun 27, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

nstarman Jun 26, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nstarman commented Sep 4, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

mwcraig commented Mar 19, 2025 •

edited

Loading

codecov Bot commented Mar 20, 2025 •

edited

Loading