Skip to content

Commit c741675

Browse files
steven-murraysteven-murray
authored
Merge pull request #50 from HERA-Team/clean-gpu
Updated vis_gpu API to match vis_cpu
2 parents f6df26e + 7e749d6 commit c741675

44 files changed

Lines changed: 3451 additions & 1484 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.coveragerc‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
[run]
33
branch = True
44
source = vis_cpu
5-
omit = */vis_gpu.py
5+
omit = */cli.py
66

77
[paths]
88
source =
@@ -22,6 +22,9 @@ exclude_lines =
2222
# Don't complain if tests don't hit defensive assertion code:
2323
raise AssertionError
2424
raise NotImplementedError
25+
except ImportError
26+
raise ImportError
27+
HAVE_GPU:
2528

2629
# Don't complain if non-runnable code isn't run:
2730
if 0:

‎.github/workflows/run_notebooks.yaml‎

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
name: Run Notebooks
2+
3+
# Test on all pushes, except when the push is literally just a tag (because we
4+
# tag automatically via CI, and therefore there's no extra code in that push).
5+
# Also, only test on pull requests into master/dev.
6+
on:
7+
push:
8+
tags-ignore:
9+
- 'v*'
10+
pull_request:
11+
branches:
12+
- 'main'
13+
14+
jobs:
15+
notebooks:
16+
name: Running Docs Notebooks
17+
runs-on: ubuntu-latest
18+
defaults:
19+
run:
20+
# Adding -l {0} ensures conda can be found properly in each step
21+
shell: bash -l {0}
22+
steps:
23+
- uses: actions/checkout@main
24+
with:
25+
fetch-depth: 1
26+
27+
- name: Cache conda
28+
uses: actions/cache@v2
29+
env:
30+
# Increase this value to reset cache if ci/test-env.yml has not changed
31+
CACHE_NUMBER: 0
32+
with:
33+
path: ~/conda_pkgs_dir
34+
key:
35+
${{ runner.os }}-conda-${{ env.CACHE_NUMBER }}-${{ matrix.python-version }}-${{ hashFiles('ci/test-env.yml', 'setup.cfg') }}
36+
37+
- name: Setup Miniconda
38+
uses: conda-incubator/setup-miniconda@v2.1.1
39+
with:
40+
# auto-update-conda: true
41+
miniconda-version: "latest"
42+
python-version: '3.10'
43+
environment-file: ci/notebook-env.yml
44+
activate-environment: viscpu
45+
channels: conda-forge,defaults
46+
channel-priority: strict
47+
use-only-tar-bz2: true
48+
49+
- name: Conda Info
50+
run: |
51+
conda info -a
52+
conda list
53+
54+
- name: Install
55+
run: |
56+
echo $(which pip)
57+
pip install .[test]
58+
59+
- name: Install ipykernel
60+
run: python -m ipykernel install --user --name viscpu --display-name "viscpu"
61+
62+
- name: Run Notebooks
63+
run: |
64+
papermill -k viscpu docs/tutorials/vis_cpu_tutorial.ipynb tmp.ipynb

‎.github/workflows/test_suite.yaml‎

Lines changed: 1 addition & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ jobs:
2323
fail-fast: false
2424
matrix:
2525
os: [ubuntu-latest, macos-latest]
26-
python-version: [3.7, 3.8, 3.9]
26+
python-version: [3.8, 3.9, "3.10"]
2727
defaults:
2828
run:
2929
# Adding -l {0} ensures conda can be found properly in each step
@@ -72,11 +72,3 @@ jobs:
7272
- name: Run Tests
7373
run: |
7474
python -m pytest --cov=vis_cpu --cov-config=.coveragerc --cov-report xml:./coverage.xml --durations=25
75-
76-
- name: Upload Coverage
77-
uses: codecov/codecov-action@v1.5.2
78-
if: matrix.os == 'ubuntu-latest' && success()
79-
with:
80-
file: ./coverage.xml # optional
81-
fail_ci_if_error: true # optional (default = false)
82-
verbose: true # optional (default = false)

‎.github/workflows/test_suite_gpu.yaml‎

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
name: GPU Tests
2+
3+
# Test on all pushes, except when the push is literally just a tag (because we
4+
# tag automatically via CI, and therefore there's no extra code in that push).
5+
# Also, only test on pull requests into master/dev.
6+
on:
7+
push:
8+
tags-ignore:
9+
- 'v*'
10+
pull_request:
11+
branches:
12+
- 'main'
13+
14+
jobs:
15+
tests:
16+
env:
17+
ENV_NAME: tests
18+
PYTHON: "3.10"
19+
OS: Ubuntu
20+
name: Self-Hosted Tests (GPU)
21+
runs-on: [self-hosted, gpu]
22+
defaults:
23+
run:
24+
# Adding -l {0} ensures conda can be found properly in each step
25+
shell: bash -l {0}
26+
steps:
27+
- name: Add Home to PATH
28+
run: |
29+
echo "/home/locoadmin/bin" >> $GITHUB_PATH
30+
echo "/usr/local/cuda/bin" >> $GITHUB_PATH
31+
echo "LD_LIBRARY_PATH=/usr/lib/wsl/lib:$LD_LIBRARY_PATH" >> $GITHUB_ENV
32+
33+
- uses: actions/checkout@main
34+
with:
35+
fetch-depth: 0
36+
37+
- name: Cache conda
38+
uses: actions/cache@v2
39+
env:
40+
# Increase this value to reset cache if ci/test-env.yml has not changed
41+
CACHE_NUMBER: 0
42+
with:
43+
path: ~/conda_pkgs_dir
44+
key:
45+
${{ runner.os }}-conda-${{ env.CACHE_NUMBER }}-${{ env.PYTHON }}-${{ hashFiles('ci/test-env.yml', 'setup.cfg') }}
46+
47+
- name: Setup Miniconda
48+
uses: conda-incubator/setup-miniconda@v2.1.1
49+
with:
50+
# auto-update-conda: true
51+
miniconda-version: "latest"
52+
python-version: ${{ env.PYTHON }}
53+
environment-file: ci/test-env.yml
54+
activate-environment: tests
55+
channels: conda-forge,defaults
56+
channel-priority: strict
57+
use-only-tar-bz2: true
58+
59+
- name: Conda Info
60+
run: |
61+
conda info -a
62+
conda list
63+
PYVER=`python -c "import sys; print('{:d}.{:d}'.format(sys.version_info.major, sys.version_info.minor))"`
64+
if [[ $PYVER != $PYTHON ]]; then
65+
exit 1;
66+
fi
67+
68+
- name: Install
69+
run: |
70+
echo $(which pip)
71+
pip install .[test,gpu]
72+
73+
- name: Run Tests
74+
run: |
75+
python -m pytest --cov=vis_cpu --cov-config=.coveragerc --cov-report xml:./coverage.xml --durations=25 --log-cli-level=DEBUG
76+
77+
- uses: codecov/codecov-action@v2
78+
if: success()
79+
with:
80+
fail_ci_if_error: true
81+
verbose: true
82+
file: ./coverage.xml

‎.gitignore‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@
1111
__pycache__/*
1212
.cache/*
1313
.*.swp
14-
*/.ipynb_checkpoints/*
14+
**/.ipynb_checkpoints/*
1515
.DS_Store
1616

1717
# Project files
@@ -45,7 +45,11 @@ docs/_rst/*
4545
docs/_build/*
4646
cover/*
4747
MANIFEST
48+
docs/_autosummary/*
4849

4950
# Per-project virtualenvs
5051
.venv*/
5152
.vscode/
53+
full-stats-*.txt
54+
stats-*.pkl
55+
summary-stats-*.pkl

‎.pre-commit-config.yaml‎

Lines changed: 13 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
1-
exclude: '^docs/conf.py|^tests/data/'
1+
exclude: '^docs/conf.py|^src/vis_cpu/data/'
22

33
repos:
44
- repo: https://github.com/pre-commit/pre-commit-hooks
5-
rev: v4.2.0
5+
rev: v4.3.0
66
hooks:
77
- id: trailing-whitespace
88
- id: check-added-large-files
@@ -33,7 +33,7 @@ repos:
3333
- flake8-print
3434

3535
- repo: https://github.com/psf/black
36-
rev: 22.3.0
36+
rev: 22.6.0
3737
hooks:
3838
- id: black
3939

@@ -47,7 +47,13 @@ repos:
4747
hooks:
4848
- id: isort
4949

50-
- repo: https://github.com/asottile/pyupgrade
51-
rev: v2.32.0
52-
hooks:
53-
- id: pyupgrade
50+
- repo: https://github.com/asottile/pyupgrade
51+
rev: v2.37.1
52+
hooks:
53+
- id: pyupgrade
54+
args: [--py38-plus]
55+
56+
- repo: https://github.com/asottile/setup-cfg-fmt
57+
rev: v1.20.2
58+
hooks:
59+
- id: setup-cfg-fmt

‎CHANGELOG.rst‎

Lines changed: 41 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,14 +2,53 @@
22
Changelog
33
=========
44

5-
dev-version
6-
===========
5+
Version 1.0.0
6+
=============
7+
8+
Version 1.0 is a major update that brings the GPU implementation up to the same API
9+
as the CPU implementation. It also *removes* support for (l,m)-grid beams.
10+
11+
Removed
12+
-------
13+
14+
- Support for ``bm_pix`` and ``use_pixel_beams`` (in both CPU and GPU implementations).
15+
Now, using a ``UVBeam`` object will automatically use interpolation on the gridded
16+
underlying data (which is typically in az/za). This can be done directly using
17+
methods in ``UVBeam``, or via new GPU methods. If you input an ``AnalyticBeam``, the
18+
beam will instead just merely be evaluated.
19+
20+
Added
21+
-----
22+
23+
- Polarization support for GPU implementation.
724

825
Changed
926
-------
1027

1128
- Faster performance if using ``beam_list`` and the frequency is not in the ``freq_array``.
1229
(interpolation done before the loop).
30+
- Factor of ~10x speed-up of ``vis_cpu`` due to changing the final ``einsum`` into a
31+
matrix product.
32+
- **BREAKING CHANGE:** the output from the CPU and GPU implementations has changed
33+
shape: it is now ``(Ntimes, Nfeed, Nfeed, Nant, Nant)`` (and without the feed axes
34+
for non-polarized data).
35+
36+
Internals
37+
---------
38+
39+
- ``vis_cpu`` and ``vis_gpu`` *modules* renamed to ``cpu`` and ``gpu`` respectively, to
40+
avoid some problems with name clashes.
41+
- New more comprehensive tests comparing the GPU and CPU implementations against
42+
each other and against pyuvsim.
43+
- New tests of documentation notebooks to ensure they're up to date.
44+
45+
Documentation
46+
-------------
47+
48+
- Updated sphinx them to Furo.
49+
- More complete Module Reference documentation.
50+
- Updated tutorial to match the new API.
51+
- Added a new "Understanding the Algorithm" page (with math!)
1352

1453
Version 0.4.3
1554
=============

‎ci/notebook-env.yml‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
name: viscpu
2+
channels:
3+
- conda-forge
4+
- defaults
5+
dependencies:
6+
- mpi4py>=3.0 # Required here so that we get
7+
- astropy>=4
8+
- numpy>=1.20
9+
- scipy>=1.6
10+
- pip>=21.1
11+
- pytest>=6.2.4
12+
- pytest-cov>=2.11.1
13+
- matplotlib>=3.3.4
14+
- ipython>=7.22
15+
- h5py>=3.2
16+
- ffmpeg
17+
- pyuvdata # For testing when using UVBeam object
18+
- jupyter
19+
- ipykernel
20+
- papermill
21+
- pip:
22+
- pyuvsim[sim]>=1.2
23+
- pyradiosky>=0.1.1

‎docs/_autosummary/vis_cpu.vis_cpu.rst‎

Lines changed: 0 additions & 6 deletions
This file was deleted.

‎docs/_autosummary/vis_cpu.vis_gpu.rst‎

Lines changed: 0 additions & 6 deletions
This file was deleted.

0 commit comments

Comments
 (0)