Add 0.10 docs by coretl · Pull Request #700 · bluesky/ophyd-async

coretl · 2024-12-09T16:50:42Z

Rewrite of the documentation and its structure including:

Making sim and epics.demo devices do the same
Placeholder for the same for tango.demo and fastcs.demo devices
New tutorials, how-tos and explanations
Reduce installed dependencies

The first half of #721
Fixes #677
Fixes #671
Fixes #484
Fixes #480
Fixes #479
Fixes #459
Fixes #443 (well enough anyway)
Fixes #386
Fixes #349
Fixes #303
Fixes #246
Fixes #227
Fixes #193
Fixes #622

coretl · 2025-01-22T16:32:44Z

@evalott100 please can you make a new branch off this one, then PR back to the docs branch fixing some of the warnings in https://github.com/bluesky/ophyd-async/actions/runs/12911955696/job/36005625932?

Some of them are just missing docstrings, but there are a bunch that say something like:

docstring of ophyd_async.core._signal.set_and_wait_for_value:1: WARNING: py:class reference target not found: ophyd_async.core._signal_backend.SignalDatatypeT

I think this is because importing ophyd_async.core._signal_backend.SignalDatatypeT into ophyd_async.core loses its doc-comment (doc-comment because it is a TypeVar not a class). I have no idea how to get sphinx to recognise that SignalDatatypeT exists and is of a particular type.

I have tried:

Adding the following to ophyd_async.core.__init__:

"""
.. data:: SignalDatatypeT

    The supported `Signal` datatypes
"""

Doing the same with .. class:: SignalDatatypeT
Doing the same but in ophyd_async.core._signal_backend
Peppering the codebase with from __future__ import annotations

My next steps were going to be:

Add from __future__ import annotations for everything (don't know if you can do this for the whole module)
Use autodoc_type_aliases to map ophyd_async.core._signal_backend.SignalDatatypeT to ophyd_async.core.SignalDatatypeT as defined by one of the methods above
Do some googling

Forks off of #700

evvaaaa · 2025-01-30T12:10:06Z

Will continue tomorrow.

Observations

`TypeVar`s

All broken throughout.

Event model types

It can't reference DataKey in the type hint despite event model being in the intersphinx mapping.

/home/eva/Code/ophyd-async/src/ophyd_async/core/__init__.py:docstring of ophyd_async.core._signal_backend.make_datakey:1: WARNING: 'any' reference target not found: DataKey

TriggerInfo

Can't get pydantic fields working so get the errors

...
/home/eva/Code/ophyd-async/src/ophyd_async/core/_detector.py:docstring of ophyd_async.core._detector.TriggerInfo:1: WARNING: py:obj reference target not found: ophyd_async.core._detector.TriggerInfo.livetime
/home/eva/Code/ophyd-async/src/ophyd_async/core/_detector.py:docstring of ophyd_async.core._detector.TriggerInfo:1: WARNING: py:obj reference target not found: ophyd_async.core._detector.TriggerInfo.frame_timeout
...

DeviceProcessor

/home/eva/Code/ophyd-async/src/ophyd_async/core/__init__.py:docstring of ophyd_async.core._device.init_devices:1: WARNING: py:class reference target not found: ophyd_async.core._device.DeviceProcessor

Sphinx can't find this, not in public api.

Numpy types

Related to TypeVar issues I think. But it's giving this issue on DeviceFiller, no idea why.

...
/home/eva/Code/ophyd-async/src/ophyd_async/core/_device_filler.py:docstring of ophyd_async.core._device_filler.DeviceFiller:1: WARNING: py:class reference target not found: numpy.int8
...

Asyncio types

Can't find asyncio.Task in async status...

/home/eva/Code/ophyd-async/src/ophyd_async/core/_status.py:docstring of ophyd_async.core._status.AsyncStatus:1: WARNING: py:class reference target not found: _asyncio.Task

We should be getting this through the python intersphinx mapping already present. I assume it's because of AS and WAS TypeVar weirdness.

Table

I think this is because of pydantic weirdness in the case of Table.all, and also decorators in the case of validate_array_dtypes.

/home/eva/Code/ophyd-async/src/ophyd_async/core/_table.py:docstring of ophyd_async.core._table.Table:1: WARNING: py:obj reference target not found: ophyd_async.core._table.Table.validate_array_dtypes
/home/eva/Code/ophyd-async/src/ophyd_async/core/_table.py:docstring of ophyd_async.core._table.Table:1: WARNING: py:obj reference target not found: ophyd_async.core._table.Table.all fields
/home/eva/Code/ophyd-async/src/ophyd_async/core/_table.py:docstring of ophyd_async.core._table.Table:1: WARNING: py:obj reference target not found: ophyd_async.core._table.Table.validate_lengths
/home/eva/Code/ophyd-async/src/ophyd_async/core/_table.py:docstring of ophyd_async.core._table.Table:1: WARNING: py:obj reference target not found: ophyd_async.core._table.Table.all fields
/home/eva/Code/ophyd-async/src/ophyd_async/core/_table.py:docstring of ophyd_async.core._table.Table.validate_array_dtypes:1: WARNING: py:obj reference target not found: ophyd_async.core._table.Table.all fields
/home/eva/Code/ophyd-async/src/ophyd_async/core/_table.py:docstring of ophyd_async.core._table.Table.validate_lengths:1: WARNING: py:obj reference target not found: ophyd_async.core._table.Table.all fields

AD Core stuff

...
/scratch/twj43146/Programming/ophyd-async/src/ophyd_async/epics/adcore/_core_io.py:docstring of ophyd_async.epics.adcore._core_io.DetectorState:1:<autosummary>:1: WARNING: py:obj reference target not found: ophyd_async.epics.adcore.NDFileIO
...

To be covered in #754.

Unused `InvalidFlyMotorException`

`InvalidFlyMotorException`` is not actually used anywhere:

ophyd-async/src/ophyd_async/epics/motor.py

Lines 26 to 32 in 10a0de2

    
           class MotorLimitsException(Exception): 
        
               pass 
        
           class InvalidFlyMotorException(Exception): 
        
               pass

Tango

Struggling with the DeviceProxy, AsyncDeviceProxy, and SyncDeviceProxy which define the __doc__ on a partial:

DeviceProxy = partial(get_device_proxy, green_mode=GreenMode.Asyncio)
DeviceProxy.__doc__ = """
    DeviceProxy(self, dev_name, wait=False, timeout=None)
        -> DeviceProxy
    ...
"""

scratch/twj43146/Programming/ophyd-async/src/ophyd_async/tango/core/__init__.py:docstring of ophyd_async.tango.core._signal.tango_signal_w:1: WARNING: py:class reference target not found: tango._tango.DeviceProxy
/scratch/twj43146/Programming/ophyd-async/src/ophyd_async/tango/core/__init__.py:docstring of ophyd_async.tango.core._signal.tango_signal_x:1: WARNING: py:class reference target not found: tango._tango.DeviceProxy
/scratch/twj43146/Programming/ophyd-async/src/ophyd_async/tango/core/_base_device.py:docstring of ophyd_async.tango.core._base_device.TangoDevice:1: WARNING: py:class reference target not found: tango._tango.DeviceProxy

Also having problems with DevVoid, and CommandInfo:

/scratch/twj43146/Programming/ophyd-async/src/ophyd_async/tango/demo/_tango/_servers.py:docstring of ophyd_async.tango.demo._tango._servers.DemoCounter.reset:1: WARNING: py:class reference target not found: DevVoid

/scratch/twj43146/Programming/ophyd-async/src/ophyd_async/tango/core/__init__.py:docstring of ophyd_async.tango.core._tango_transport.get_trl_descriptor:1: WARNING: py:class reference target not found: tango._tango.CommandInfo

Myst cross-reference issues

/scratch/twj43146/Programming/ophyd-async/docs/how-to/choose-right-baseclass.md:33: WARNING: 'myst' cross-reference target not found: 'TangoConnector'
/scratch/twj43146/Programming/ophyd-async/docs/how-to/choose-right-baseclass.md:33: WARNING: 'myst' cross-reference target not found: 'EpicsConnector'
/scratch/twj43146/Programming/ophyd-async/docs/how-to/choose-right-baseclass.md:33: WARNING: 'myst' cross-reference target not found: 'PviConnector'
/scratch/twj43146/Programming/ophyd-async/docs/how-to/implement-ad-detector.md:16: WARNING: 'myst' cross-reference target not found: 'ADBaseIO'
/scratch/twj43146/Programming/ophyd-async/docs/how-to/implement-ad-detector.md:25: WARNING: 'myst' cross-reference target not found: 'ADBaseController'
/scratch/twj43146/Programming/ophyd-async/docs/how-to/implement-ad-detector.md:40: WARNING: 'myst' cross-reference target not found: 'ADWriter'
/scratch/twj43146/Programming/ophyd-async/docs/how-to/implement-ad-detector.md:40: WARNING: 'myst' cross-reference target not found: 'adcore.ADHDFWriter'
/scratch/twj43146/Programming/ophyd-async/docs/how-to/implement-ad-detector.md:44: WARNING: 'myst' cross-reference target not found: 'adcore.NDPluginBase'
/scratch/twj43146/Programming/ophyd-async/docs/tutorials/implementing-devices.md:169: WARNING: 'myst' cross-reference target not found: 'SignalDatatype'

coretl · 2025-02-10T14:40:55Z

@evalott100 please could you give this another review?

I propose to merge this in approximately this state with further work in other tickets:

Docstring format not decided yet: Decide on docstring format #755
AreaDetector modules need to be more uniform: Update all the detectors and ADCore to match ADAravis #754
observe_value docstring Document observe_value better #741
load/save docs Write docs on load/save #758 and Document how to do stage and load as an analogy to ophyd's stage_sigs #674
tutorial bootstrapping installs Update the tutorial docs to not rely on dev install #134

I will write up the last remaining tutorial (detectors) as part of another PR

jwlodek

Found a few typos

docs/explanations/design-goals.md

src/ophyd_async/core/_providers.py

Co-authored-by: Jakub Wlodek <jwlodek@bnl.gov>

evvaaaa

Lots here, changes to the docs are great!

All the following should be regarded as NITs.

docs/how-to/interact-with-signals.md

docs/tutorials/using-devices.md

src/ophyd_async/core/_protocol.py

src/ophyd_async/epics/demo/_stage.py

src/ophyd_async/sim/_point_detector.py

src/ophyd_async/sim/_stage.py

src/ophyd_async/testing/_mock_signal_utils.py

Co-authored-by: Eva Lott <evalott100@gmail.com>

Addressed

* Rename ophyd_async.{epics,tango}.sim -> demo * Fix typo in import linter * Rewrite sim and epics.demo devices to do the same * Fix SignalDatatypeT docs * Localise usage of T to each module * Add docstrings without breaking type checking * Add some more docs * Tidy ADAravis and write how-to areaDetector * Localise use of T in tests * Fix tests * Try to shorten tutorials * Patch out more plotting * Fix tests on windows * Add another article * More docstrings * Switch to autodoc2 and write a lot more docstrings * Fix tests * Add back in pyepics for dev deps * More tweaks * Apply suggestions from code review Co-authored-by: Jakub Wlodek <jwlodek@bnl.gov> * Tweak * Added devices, signals and backends article * Fix links * Add diagram * Update docs/how-to/interact-with-signals.md Co-authored-by: Eva Lott <evalott100@gmail.com> * Review comments --------- Co-authored-by: Eva <evalott100@gmail.com> Co-authored-by: Eva Lott <eva.lott@diamond.ac.uk> Co-authored-by: Jakub Wlodek <jwlodek@bnl.gov>

mrakitin requested a review from jennmald December 9, 2024 16:57

coretl force-pushed the docs branch from d5d5137 to 8ddce1e Compare December 17, 2024 09:14

coretl force-pushed the docs branch 4 times, most recently from a07bd8d to 05bad25 Compare January 13, 2025 16:57

coretl force-pushed the docs branch from 05bad25 to 2da89ea Compare January 17, 2025 16:46

coretl added 3 commits January 20, 2025 16:42

Rename ophyd_async.{epics,tango}.sim -> demo

ab4abd6

Fix typo in import linter

4bbcb0b

Rewrite sim and epics.demo devices to do the same

20b4aa7

coretl force-pushed the docs branch from 2da89ea to 4ea18ef Compare January 20, 2025 16:43

coretl changed the title ~~Add 1.0 docs~~ Add 0.10 docs Jan 20, 2025

coretl force-pushed the docs branch 2 times, most recently from 73cc77e to f68f00c Compare January 21, 2025 16:54

WIP

9563aee

coretl force-pushed the docs branch from f68f00c to 9563aee Compare January 22, 2025 15:51

Merge remote-tracking branch 'origin/main' into docs

f8df64c

evvaaaa pushed a commit that referenced this pull request Jan 28, 2025

WIP: docs with TypeVar in private modules

4a5ca3b

Forks off of #700

coretl and others added 5 commits January 29, 2025 09:11

Fix SignalDatatypeT docs

9045e03

Localise usage of T to each module

e5e9df0

Add docstrings without breaking type checking

e2faf90

Add some more docs

6415270

WIP: added docstrings

bbe4ffa

coretl added 5 commits January 30, 2025 15:39

Tidy ADAravis and write how-to areaDetector

021790c

Localise use of T in tests

f44961e

Fix tests

43dae53

Merge branch 'docs' of github.com:bluesky/ophyd-async into docs

872bd54

Merge remote-tracking branch 'origin/main' into docs

c702f2e

coretl added 6 commits January 31, 2025 15:55

Merge branch 'docs' of github.com:bluesky/ophyd-async into docs

c7230ac

More docstrings

ca0ff7f

Switch to autodoc2 and write a lot more docstrings

f4528d0

Fix tests

8c39c55

Add back in pyepics for dev deps

5995251

More tweaks

898f419

coretl requested a review from evvaaaa February 10, 2025 14:41

coretl marked this pull request as ready for review February 10, 2025 14:41

jwlodek previously requested changes Feb 10, 2025

View reviewed changes

docs/explanations/design-goals.md Show resolved Hide resolved

src/ophyd_async/core/_providers.py Outdated Show resolved Hide resolved

src/ophyd_async/core/_providers.py Outdated Show resolved Hide resolved

Apply suggestions from code review

ac1ee97

Co-authored-by: Jakub Wlodek <jwlodek@bnl.gov>

This was referenced Feb 10, 2025

turn on nitpicking for Sphinx docs build #654

Closed

Investigate if T can be imported per file that uses it instead of via utils #480

Closed

coretl added 2 commits February 10, 2025 16:59

Tweak

1829bb6

Merge branch 'docs' of github.com:bluesky/ophyd-async into docs

92faaaf

coretl mentioned this pull request Feb 10, 2025

Write docs on load/save #758

Closed

coretl and others added 3 commits February 10, 2025 18:57

Added devices, signals and backends article

b61a716

Fix links

7101d66

Add diagram

8466911

evvaaaa reviewed Feb 11, 2025

View reviewed changes

coretl and others added 3 commits February 11, 2025 09:56

Update docs/how-to/interact-with-signals.md

a835eec

Co-authored-by: Eva Lott <evalott100@gmail.com>

Review comments

4e6e4ff

Merge branch 'docs' of github.com:bluesky/ophyd-async into docs

1159f5b

coretl requested a review from evvaaaa February 11, 2025 10:06

evvaaaa approved these changes Feb 11, 2025

View reviewed changes

coretl merged commit 663ad9d into main Feb 11, 2025
21 of 22 checks passed

coretl deleted the docs branch February 11, 2025 10:08

coretl mentioned this pull request Mar 5, 2025

Remove unwanted dependencies #285

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add 0.10 docs#700

Add 0.10 docs#700
coretl merged 43 commits intomainfrom
docs

coretl commented Dec 9, 2024 •

edited

Loading

Uh oh!

coretl commented Jan 22, 2025

Uh oh!

evvaaaa commented Jan 30, 2025 •

edited

Loading

Uh oh!

coretl commented Feb 10, 2025 •

edited

Loading

Uh oh!

jwlodek left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

evvaaaa left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

coretl commented Dec 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coretl commented Jan 22, 2025

Uh oh!

evvaaaa commented Jan 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Observations

TypeVars

Event model types

TriggerInfo

DeviceProcessor

Numpy types

Asyncio types

Table

AD Core stuff

Unused InvalidFlyMotorException

Tango

Myst cross-reference issues

Uh oh!

coretl commented Feb 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jwlodek left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

evvaaaa left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

coretl commented Dec 9, 2024 •

edited

Loading

evvaaaa commented Jan 30, 2025 •

edited

Loading

`TypeVar`s

Unused `InvalidFlyMotorException`

coretl commented Feb 10, 2025 •

edited

Loading