Open
Conversation
…or pipeline parallelization / system utilization
Replace sphinx_rtd_theme with furo (default options). Remove all RTD-specific CSS overrides (.wy-* selectors, typography block) and replace with Furo-only customisations. Add furo_layout.js: a lightweight TOC collapse/expand toggle button that persists state in localStorage. When the TOC is hidden, the content column grows by 15em to absorb the freed space; left/right text padding remains unchanged. Build speed improvements: - Re-enable -j auto parallel builds in docs/Makefile - Add html_fast incremental target (preserves doctrees/stubs) - Add build-docs-fast pixi task in pyproject.toml - Set autosummary_generate_overwrite=False to skip re-generating existing stubs on incremental runs Also remove duplicate sphinx.ext.coverage from conf.py extensions and register furo in pyproject.toml and requirements_docs.txt.
Replace conda-based build.commands with structured build.jobs using the officially documented pixi pattern. Install pixi via asdf in create_environment, solve the docs environment in install, then run sphinx-build directly to $READTHEDOCS_OUTPUT in the build step. Separate html and pdf build targets use --frozen to reuse the already installed environment without re-solving. Add build-pdf pixi task in pyproject.toml to mirror the RTD pdf build locally.
- Remove "Indices and tables" section from index.rst - Simplify sidebar title to "Pipeline (version)" by setting html_title and removing newline from project name - Hide Sphinx/Furo attribution in footer via CSS while keeping copyright - Clear footer_icons in Furo theme options
- Replace "demonstration" framing with accurate description now that doc-build refactoring is merged into main - Remove outdated reference to development branch - Acknowledge contributions from MPIfR, NOVA, and UKATC under contract to ESO - Add NOVA reference link (nova-astronomy.nl)
- Set autosectionlabel_prefix_document=True to resolve duplicate section label warnings across documents - Remove redundant local TOC from interface.rst (Furo sidebar suffices) - Add reset-zoom button (↺) to all mermaid diagrams; button appears on hover, resets d3-zoom transform to identity on click - Add supporting CSS for mermaid zoom wrapper and reset button styling
- Create devnotes.rst as a single entry point for all developer notes, reducing sidebar clutter - Remove individual entries and "Indices and tables" section from index.rst; add devnotes under API Reference toctree
…ntext in `apisummary.rst`
… and `index.rst`
Convert releases.rst and timeline.rst to MyST-compatible
Markdown (.md). Both files used RST-only syntax (.. mermaid::,
.. contents::, :field:, .. note::) that is now replaced with
fenced directives, definition lists, and standard Markdown links.
- releases: 6-column list-table reduced to 5 columns by merging
Documentation + Download into a single Resources column; download
link text shortened to "Download (linux/macOS)"; :widths: auto added
- timeline: .. mermaid:: blocks converted to {mermaid}; gitGraph
YAML front-matter replaced with %%{init:…}%% inline config; RST
field lists converted to deflist syntax; .. note:: → {note}
- timeline moved from Basics to Misc. section in index.rst
- Add autosummary base.rst template so generated stubs use short
names (objname) instead of full dotted paths (fullname)
- Reduce global font size to ~13px via :root { font-size: 81.25% }
in custom_theme.css so all rem-based Furo sizes scale uniformly
- Add cli_function_stubs Sphinx extension to generate per-function stub pages and patch module stubs with hidden toctrees, giving CLI functions proper sidebar nesting under their parent module. - Move automodapi directives from inline apisummary.rst into individual files under automodapi_src/ so each module becomes a collapsible sidebar parent with its members nested beneath. - Register cli_function_stubs extension and add _extension/ to sys.path in conf.py. - Simplify section titles in apisummary.rst, replacing inline automodapi blocks with toctree references.
- Group domain/infrastructure and h*.tasks toctrees under a new "Pipeline Modules" section with subsections. - Rename "Full APIs" to "Complete API Reference". - Raise docutils line_length_limit in conf.py setup() to fix the Task Classes inheritance diagram being silently dropped.
Add per-task descriptions, figures, QA scoring notes, workflow details, and WebLog references from the Pipeline User Guide (PLUG) LaTeX source (sections 9-10) into Google-style docstrings of 50 CLI wrapper functions across h/, hif/, hifa/, and hsd/. Add 34 figures referenced by the new docstrings under docs/source/figures/. Set napoleon_use_rtype=False in docs/source/conf.py to suppress redundant return-type rendering.
Convert CalLibrary Overview PDF to `docs/source/callibrary.md` with proper MyST formatting: headings, code blocks, footnotes, internal cross-references, and LaTeX math notation. Add callibrary to the index toctree under Misc.
Replace the wide pipe table in recipes.md with MyST list-table directives grouped by project (ALMA-IF, ALMA-SD, Nobeyama, VLA, VLASS). Extract JIRA URLs into reference-style links. Convert releases.rst to releases.md using MyST list-table syntax. Split the Resources column into separate Documentation and Download columns with full filenames. Add PL2025 ALMA/Cycle-12 release entry. Shorten Pipeline-* prefixes in older version labels.
…o MyST Markdown format
…yST Markdown format - Update Mermaid configuration and syntax in documentation for MyST compatibility - Adjust Mermaid configuration in interface documentation for improved layout
…e-cli-documentation
…patibility - Migrate GitHub Copilot instructions to reflect Python 3.12+ standards. - Update environment configuration to use Python 3.12 in `environment_vlass2026.yml`. - Modify `requirements.txt` to target CASA >= 6.7.1 with Python 3.12+. - Adjust `pyproject.toml` to require Python 3.12 and update classifiers. - Revise Dockerfile instructions in `container.md` for improved clarity. - Update `test-unit-pixi.yml` to remove deprecated Python 3.10 environments. - Correct references in timetracker.py to Python 3.12 documentation. - Add `README.md` for dependencies and environment configuration.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR modernizes the documentation stack and developer tooling around a Python 3.12+ baseline, while substantially improving Sphinx-generated CLI task documentation and the docs site UX (theme + navigation).
Changes:
- Bump project/runtime baseline to Python 3.12+ (and update CASA/dependency pinning accordingly).
- Migrate docs build to Furo + improved autosummary/autodoc generation (custom CLI stub extension, incremental build target, RTD config updates).
- Expand/refresh CLI task docstrings and restructure docs pages (RST → MyST Markdown where helpful, new toctree organization).
Reviewed changes
Copilot reviewed 90 out of 127 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| requirements_docs.txt | Add Furo as a docs dependency. |
| requirements.txt | Update CASA/Python target and simplify Astropy requirement for 3.12+. |
| pyproject.toml | Require Python >=3.12; adjust pixi environments; add docs helper tasks; add Furo to docs env. |
| pipeline/infrastructure/timetracker.py | Update doc link to Python 3.12 shelve docs. |
| pipeline/hsd/cli/hsd_tsysflag.py | Rewrite CLI docstring for clearer user-facing documentation. |
| pipeline/hsd/cli/hsd_skycal.py | Expand CLI docstring with WebLog details/figure. |
| pipeline/hsd/cli/hsd_restoredata.py | Clarify restore workflow and expected products in docstring. |
| pipeline/hsd/cli/hsd_k2jycal.py | Expand docstring with WebLog/QA/figure info. |
| pipeline/hsd/cli/hsd_importdata.py | Expand docstring with QA + pointing plot details. |
| pipeline/hsd/cli/hsd_imaging.py | Expand imaging docstring with WebLog content + QA details. |
| pipeline/hsd/cli/hsd_flagdata.py | Expand flagging docstring with agents and QA details. |
| pipeline/hsd/cli/hsd_exportdata.py | Clarify exported products + update examples. |
| pipeline/hsd/cli/hsd_blflag.py | Expand baseline-flagging docstring + QA notes. |
| pipeline/hsd/cli/hsd_baseline.py | Major docstring rewrite with algorithm + WebLog/QA details. |
| pipeline/hsd/cli/hsd_atmcor.py | Expand ATM correction docstring with models/WebLog/QA notes. |
| pipeline/hsd/cli/hsd_applycal.py | Expand calibration-application docstring with WebLog/QA + figure. |
| pipeline/hifa/tasks/bandpass/almaphcorbandpass.py | Improve parameter documentation wording/clarity. |
| pipeline/hifa/cli/hifa_wvrgcalflag.py | Substantial docstring rewrite explaining workflow + QA. |
| pipeline/hifa/cli/hifa_tsysflagcontamination.py | Expand docstring with algorithm/WebLog/QA + figure. |
| pipeline/hifa/cli/hifa_tsysflag.py | Major docstring rewrite detailing metrics, defaults, and QA. |
| pipeline/hifa/cli/hifa_timegaincal.py | Add WebLog figure + QA description; clean up whitespace. |
| pipeline/hifa/cli/hifa_targetflag.py | Add WebLog example figure + QA note. |
| pipeline/hifa/cli/hifa_spwphaseup.py | Add figures + phase decoherence assessment section + QA notes. |
| pipeline/hifa/cli/hifa_session_refant.py | Expand session refant selection algorithm + QA notes. |
| pipeline/hifa/cli/hifa_restoredata.py | Clarify restore workflow and expected products in docstring. |
| pipeline/hifa/cli/hifa_renorm.py | Major docstring rewrite with algorithm/WebLog/QA + figures. |
| pipeline/hifa/cli/hifa_polcalflag.py | Expand docstring and QA notes. |
| pipeline/hifa/cli/hifa_polcal.py | Expand docstring with WebLog content + QA notes + figure. |
| pipeline/hifa/cli/hifa_lock_refant.py | Clarify behavior and recipe context in docstring. |
| pipeline/hifa/cli/hifa_importdata.py | Expand docstring with WebLog/QA details; wrap long example. |
| pipeline/hifa/cli/hifa_imageprecheck.py | Clarify robust-selection strategy + QA notes. |
| pipeline/hifa/cli/hifa_gfluxscaleflag.py | Rewrite docstring to document calibration+flagging workflow + QA. |
| pipeline/hifa/cli/hifa_gfluxscale.py | Major docstring rewrite with workflow/figures/QA + resolved-cal selection. |
| pipeline/hifa/cli/hifa_fluxcalflag.py | Expand docstring with flagged-line ranges + QA details. |
| pipeline/hifa/cli/hifa_flagtargets.py | Clarify template-based target flagging behavior. |
| pipeline/hifa/cli/hifa_flagdata.py | Expand docstring with WebLog/QA + edge-flagging explanation formatting. |
| pipeline/hifa/cli/hifa_exportdata.py | Clarify exported products + QA + update examples. |
| pipeline/hifa/cli/hifa_diffgaincal.py | Add low-SNR heuristic description + figure + QA notes. |
| pipeline/hifa/cli/hifa_bandpassflag.py | Expand docstring with workflow, figures, and QA notes. |
| pipeline/hifa/cli/hifa_bandpass.py | Expand docstring with phase-up logic, QA metrics, and figures. |
| pipeline/hifa/cli/hifa_antpos.py | Clarify that corrections are applied; add WebLog + QA notes. |
| pipeline/hif/cli/hif_uvcontsub.py | Expand docstring with PL2024 behavior + QA notes; fix example syntax. |
| pipeline/hif/cli/hif_setmodels.py | Expand docstring with flux.csv context + QA scoring. |
| pipeline/hif/cli/hif_selfcal.py | Major docstring rewrite documenting iterative workflow + QA + figure. |
| pipeline/hif/cli/hif_refant.py | Expand docstring with ranking metric + QA scoring. |
| pipeline/hif/cli/hif_rawflagchans.py | Expand docstring with algorithm detail + QA scoring. |
| pipeline/hif/cli/hif_mstransform.py | Clarify output MS naming/content + QA scoring. |
| pipeline/hif/cli/hif_makeimlist.py | Expand docstring with imaging modes + QA scoring. |
| pipeline/hif/cli/hif_makeimages.py | Expand docstring with masking/thresholds/QA + figures. |
| pipeline/hif/cli/hif_lowgainflag.py | Expand docstring with workflow + QA scoring. |
| pipeline/hif/cli/hif_findcont.py | Major docstring rewrite documenting heuristics + QA + figure. |
| pipeline/hif/cli/hif_checkproductsize.py | Expand docstring with mitigation cascade + QA scoring. |
| pipeline/hif/cli/hif_applycal.py | Expand docstring with WebLog/QA details. |
| pipeline/h/cli/h_tsyscal.py | Expand docstring with WebLog info + QA scoring. |
| environment_vlass2026.yml | Bump conda env Python to 3.12. |
| docs/source/timeline.rst | Remove RST version of timeline (migrated to Markdown). |
| docs/source/timeline.md | Add MyST Markdown timeline with Mermaid diagrams. |
| docs/source/interface.rst | Remove RST interface page (migrated to Markdown). |
| docs/source/interface.md | Add MyST Markdown interface page with Mermaid flowchart. |
| docs/source/index.rst | Restructure toctrees (Basics/API/Misc + new devnotes entry). |
| docs/source/devnotes.rst | New developer-notes index page for docs navigation. |
| docs/source/develdocmd/recipes.md | Convert recipe list to MyST list-table and add link refs. |
| docs/source/develdocmd/parallelization.md | Add new developer doc on parallelization backend options. |
| docs/source/dependencies.rst | Remove RST dependencies page (migrated to Markdown). |
| docs/source/dependencies.md | Add MyST Markdown dependencies page with notes/structure. |
| docs/source/conf.py | Switch HTML theme to Furo; add local extension path; configure autosummary + MyST fence directive. |
| docs/source/automodapi_src/pipeline.infrastructure.vdp.rst | New automodapi stub page for pipeline.infrastructure.vdp. |
| docs/source/automodapi_src/pipeline.infrastructure.project.rst | New automodapi stub page for pipeline.infrastructure.project. |
| docs/source/automodapi_src/pipeline.infrastructure.launcher.rst | New automodapi stub page for pipeline.infrastructure.launcher. |
| docs/source/automodapi_src/pipeline.infrastructure.imagelibrary.rst | New automodapi stub page for pipeline.infrastructure.imagelibrary. |
| docs/source/automodapi_src/pipeline.infrastructure.callibrary.rst | New automodapi stub page for pipeline.infrastructure.callibrary. |
| docs/source/automodapi_src/pipeline.hsdn.tasks.rst | New automodapi stub page for pipeline.hsdn.tasks. |
| docs/source/automodapi_src/pipeline.hsd.tasks.rst | New automodapi stub page for pipeline.hsd.tasks. |
| docs/source/automodapi_src/pipeline.hifv.tasks.rst | New automodapi stub page for pipeline.hifv.tasks. |
| docs/source/automodapi_src/pipeline.hifa.tasks.rst | New automodapi stub page for pipeline.hifa.tasks. |
| docs/source/automodapi_src/pipeline.hif.tasks.rst | New automodapi stub page for pipeline.hif.tasks. |
| docs/source/automodapi_src/pipeline.h.tasks.rst | New automodapi stub page for pipeline.h.tasks. |
| docs/source/automodapi_src/pipeline.domain.rst | New automodapi stub page for pipeline.domain. |
| docs/source/apisummary.rst | Reorganize API docs into tasks + modules + full API sections. |
| docs/source/_templates/autosummary/base.rst | Add minimal autosummary base template. |
| docs/source/_static/furo_layout.js | Add TOC collapse toggle + Mermaid reset-zoom button injection. |
| docs/source/_static/custom_theme.css | Add Furo-specific CSS (TOC toggle, Mermaid reset button, font size tweaks). |
| docs/source/_extension/cli_function_stubs.py | New Sphinx extension to generate per-function CLI autosummary stubs + sidebar toctrees. |
| docs/README.rst | Update internal docs README messaging and contributor acknowledgement. |
| docs/Makefile | Enable parallel Sphinx builds; add html_fast incremental target. |
| .readthedocs.yaml | Migrate RTD build to pixi-based env and add PDF artifact output. |
| .github/workflows/test-unit-pixi.yml | Remove Python 3.10 pixi envs from the unit-test matrix. |
| .github/copilot-instructions.md | Update Copilot guidance to Python 3.12+ typing conventions (PEP 695). |
Comment on lines
+68
to
+69
| // We inject a small reset button that calls svg.__zoom.invert() to restore | ||
| // the identity transform. |
There was a problem hiding this comment.
This comment says the reset button calls svg.__zoom.invert(), but the implementation below does not (it manually sets the <g> transform and overwrites svg.__zoom). Please update the comment to match the actual behavior to avoid misleading future maintenance.
Suggested change
| // We inject a small reset button that calls svg.__zoom.invert() to restore | |
| // the identity transform. | |
| // We inject a small reset button that manually restores the SVG's identity | |
| // transform and resets svg.__zoom to an identity zoom state. |
Comment on lines
+24
to
+26
| 2. **derivative** (``flag_derivative``): flag spectra with a high median derivative | ||
| (``fd_max_limit``), targeting ``ringing'' spectra. | ||
| 3. **edgechans** (``flag_edgechans``): flag the edge channels of each spw |
There was a problem hiding this comment.
The inline literal markup is unbalanced here (opens with but closes with ''). This will render incorrectly (and can break Sphinx parsing). Consider usingringing`` or plain quotes instead.
Comment on lines
+106
to
+117
| text = mod_stub.read_text() | ||
| if _TOCTREE_MARKER in text: | ||
| continue # already patched | ||
|
|
||
| toctree_entries = '\n'.join(f' {mod_name}.{n}' for n in func_names) | ||
| patch = ( | ||
| f'\n.. {_TOCTREE_MARKER}\n' | ||
| f'.. toctree::\n' | ||
| f' :hidden:\n\n' | ||
| f'{toctree_entries}\n' | ||
| ) | ||
| mod_stub.write_text(text + patch) |
There was a problem hiding this comment.
The module-stub patching is skipped once the marker is present, so the hidden toctree will never be updated if new CLI functions are added (or removed) after the first build. Consider rewriting/replacing the existing marked toctree block each run (or at least appending any newly discovered functions) so navigation stays in sync with the code.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #27 +/- ##
==========================================
+ Coverage 16.64% 16.91% +0.26%
==========================================
Files 835 836 +1
Lines 108155 108509 +354
Branches 17122 17152 +30
==========================================
+ Hits 18006 18350 +344
- Misses 89768 89772 +4
- Partials 381 387 +6
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
…e-cli-documentation Resolve conflicts: - requirements.txt
…odules; increment minimum CASA revision
Change `__all__` from set to list in `hsd/tasks/common/direction_utils.py`. Sphinx 9.1 requires `__all__` to be a list or tuple, and the set literal caused autosummary to crash with a ValueError during recursive doc generation.
… bdsf dependency for macOS
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces several improvements across documentation, developer tooling, and the Sphinx documentation build system. The main focus is on modernizing Python typing standards, enhancing the developer and user documentation, and improving the Sphinx documentation workflow and theme usability.
Python Standards and Documentation Guidelines:
.github/copilot-instructions.mdto require Python 3.12+ (including PEP 695 type alias and generic syntax), and expanded modernization checks to flag deprecated typing constructs such asTypeVarandTypeAlias. [1] [2]Sphinx Documentation Build System and Workflow:
pixienvironments and updated build steps for both HTML and PDF output. This includes switching to the latest Python, simplifying install steps, and improving reproducibility.html_fastfor faster incremental Sphinx builds during development.Sphinx Theme and User Experience Enhancements:
Documentation Content Updates:
docs/README.rstto clarify project scope, contributors, and documentation structure. [1] [2] [3]Python Standards and Guidelines
Sphinx Build System and CLI Documentation
pixienvironments and Python 3.12+, with improved build/install steps and dropped Python 3.10 support. [1] [2]Sphinx Theme and Usability
Documentation Content