Add CompareStatus, TransitionStatus, AllAndStatus and OrAnyStatus #1265

cappel89 · 2025-09-25T14:29:12Z

Summary

This PR adds 4 new StatusObjects that simplify signal comparison, transitions or combining status objects.

CompareStatus

Status to compare updates from a signal. It supports float & int for a few operations ('==', '!=', '<', '<=', '>', '>='.) and strings for '==', '!='. A list of values can be specified for which an exception should be set on the status.

from ophyd.signal import Signal
from ophyd.status import CompareStatus
sig = Signal(name='sig')
sig.put(0)
status = CompareStatus(signal=sig, value=5, operation=">")
sig.put(1)
status.done # is true

TransitionStatus

Status to define a transition of states that the signal needs to run through. It is possible to specify whether the transition should be strict. A list of values can be specified for which an exception should be set.

from ophyd.signal import Signal
from ophyd.status import TransitionStatus
sig = Signal(name='sig')
sig.put(0)
status = TransitionStatus(signal=sig, transitions=[1, 2, 3], strict=False)
sig.put(1)
sig.put(2)
sig.put(3)
status.done # is true

AndAllStatus

Similar to AndStatus, but a DeviceStatus and takes a list of status objects.

from ophyd import Device
from ophyd.status import AndAllStatus, StatusBase, DeviceStatus

dev = Device(name='dev')
st1 = StatusBase()
st2 = DeviceStatus()

and_st = AndAllStatus(dev, [st1,st2])
st1.set_finished()
st2.set_finished()
and_st.done # is True

OrAnyStatus

Similar to OrStatus, but a DeviceStatus and takes a list of status objects.

from ophyd import Device
from ophyd.status import OrAnyStatus, StatusBase, DeviceStatus

dev = Device(name='dev')
st1 = StatusBase()
st2 = DeviceStatus(dev)

and_st = OrAnyStatus(dev, [st1,st2])
st1.set_finished()
and_st.done # is True

cappel89 · 2025-09-25T15:17:25Z

Hi @ZLLentz,

the third and biggest one with the StatusObjects presented during the workshop. Again, let me know if anything should be adapted.

tacaswell · 2025-09-25T20:11:06Z

ophyd/status.py

+
+    Parameters
+    ----------
+    signal: The device signal to compare.


Please use numpy-style docstrings.

tacaswell · 2025-09-25T20:11:44Z

ophyd/status.py

+        value: float | int | str,
+        *,
+        operation: Literal["==", "!=", "<", "<=", ">", ">="] = "==",
+        raise_exc_value: float | int | str | list[float | int | str] | None = None,


Can we call this failure_value? That is raises an exception is an implementation detail not its purpose.

tacaswell · 2025-09-25T20:13:37Z

ophyd/status.py

+
+    def _compare_callback(self, value, **kwargs) -> bool:
+        """Callback for subscription status"""
+        if value in self._raise_exc_values:


how does this check (which is always ==) interact with greater / less that tests?

Would it make sense to say "The device has failed if the value goes above X"?

We adapted the code, renamed operation to operation_success, and added operation_failure

tacaswell · 2025-09-25T20:13:55Z

ophyd/status.py

+
+    Parameters
+    ----------
+    signal: The device signal to monitor.


same comment as above, please use numpy style docstrings.

tacaswell · 2025-09-25T20:14:54Z

ophyd/status.py

+
+    def __init__(
+        self,
+        signal: Signal,


I remain a bit concerned about adding typing ad-hoc, but compared to the #362 I'm less concerned about bringing in complete typing on a new object.

Typehints here seem to be useful, I would propose to keep them. Please let me know if you disagree.

tacaswell · 2025-09-25T20:16:17Z

ophyd/status.py

+        if isinstance(value, str):
+            if operation not in ("==", "!="):
+                raise ValueError(
+                    f"Invalid operation: {operation} for string comparison. Must be '==' or '!='."
+                )
+        if operation not in ("==", "!=", "<", "<=", ">", ">="):
+            raise ValueError(
+                f"Invalid operation: {operation}. Must be one of '==', '!=', '<', '<=', '>', '>='."
+            )


Would it make sense to also check that the signal has length 1?

Sure, it's true that the assumption is that the signal has length 1.

As this may easily get too complicated with signals returning numpy arrays or lists, we simply safeguard against the scenario and set the exception to the error that results from the comparison.

Can we use signal.describe() on init? Not bullet proof, but I think it may be less messy than trying to do the typechecking later.

"this is too much trouble, let ducktyping fail" is a fine answer.

tacaswell · 2025-09-25T20:16:55Z

ophyd/status.py

+        transitions: list[float | int | str],
+        *,
+        strict: bool = True,
+        raise_states: list[float | int | str] | None = None,


same as above, I think failure_states or similar is a clearer name.

tacaswell · 2025-09-25T20:18:59Z

ophyd/status.py

+        if not isinstance(transitions, list):
+            raise ValueError(
+                f"Transitions must be a list of values. Received: {transitions}"
+            )
+        self._transitions = transitions


Suggested change

if not isinstance(transitions, list):

raise ValueError(

f"Transitions must be a list of values. Received: {transitions}"

)

self._transitions = transitions

self._transitions = tuple(transitions)

If we want to do type checking we should check against Sequence (and the typing should be relaxed) but I think it is better to normalize the input through tuple and let Python handle the type checking for us.

ZLLentz · 2025-09-25T19:52:00Z

ophyd/status.py

+        return self._is_finished()
+
+    def _is_finished(self) -> bool:
+        """Check if the status is finished"""
+        return self._index >= len(self._transitions)


Nitpick: this one-line function seems like it's only used in one place and I'm not sure if it makes the code more readable or not.

ZLLentz · 2025-09-25T20:13:54Z

ophyd/status.py

+                for st in self.status_list:
+                    with st._lock:
+                        if st.done and not st.success:
+                            self.set_exception(st.exception())  # st._exception


I wonder if there's any way to leverage ExceptionGroup here, but maybe not if we must be done and failed when any status has failed.

The issue is that Python 3.8 is not supporting ExceptionGroup yet.

ZLLentz · 2025-09-25T20:15:37Z

ophyd/status.py

+                    combined_exceptions = RuntimeError(
+                        "; ".join(f"{type(exc).__name__}: {exc}" for exc in exceptions)
+                    )


This one can definitely be an ExceptionGroup

same as above

ZLLentz · 2025-09-25T20:17:55Z

ophyd/status.py

+            with st._lock:
+                st.add_callback(inner)
+
+    def set_exception(self, exc):


I notice these two new statuses override set_exception while nothing else does. What's the purpose of the override here?

This is a remnant from trying to set the exception on the sub-statuses. I removed it.

tacaswell · 2025-09-25T20:20:26Z

ophyd/status.py

+            )
+            return False
+        if self._index == 0:
+            if value == self._transitions[0]:


Should this have some checking that the first change we see is to the first state in strict mode?

Hej,
we realized that the usage of strict was not properly documented. We have added a section with examples in the notes section of the docstring. In short, the strict option does not imply that an exception will be raised, it simply decides whether a status resolves in success or not. To mark states for exception, we would expect that failure_states should be used. Please let us know what you think about the section in the docstring.

class TransitionStatus(SubscriptionStatus): """ Status class to monitor transitions of a signal value through a list of specified transitions. The status is finished when all transitions have been observed in order. The keyword argument `strict` determines whether the transitions must occur in strict order or not. The strict option only becomes relevant once the first transition has been observed. If `failure_states` is provided, the status will raise an exception if the signal value matches any of the values in `failure_states`. Parameters ---------- signal: Signal The device signal to monitor. transitions: list A list of values to transition through. strict: bool, optional Whether the transitions must occur in strict order. Defaults to True. failure_states: list, optional A list of values that will raise an exception if encountered. Defaults to None. run: bool, optional Whether to run the status callback on creation or not. Defaults to True. event_type: optional The type of event to trigger on transition. Defaults to None (default sub). timeout: float | None, optional The timeout for the status. Defaults to None (indefinite). settle_time: float, optional The time to wait for the signal to settle before checking transitions. Defaults to 0. Notes ----- The 'strict' option does not raise if transitions are observed which are out of order. It only determines whether a transition is accepted if it is observed from the previous value in the list of transitions to the next value. For example, with strict=True and transitions=[1, 2, 3], the sequence 0 -> 1 -> 2 -> 3 is accepted, but 0 -> 2 -> 1 -> 3 is not and the status will not complete. With strict=False, both sequences are accepted. However, with strict=True, the sequence 0 -> 1 -> 3 -> 1 -> 2 -> 3 is accepted. To raise an exception if an out-of-order transition is observed, use the `failure_states` keyword argument. """

tacaswell · 2025-09-25T20:22:16Z

ophyd/status.py

+
+class AndAllStatus(DeviceStatus):
+    """
+    A status that combines mutiple status objects in a list using logical and.


As above, please use numpydoc style docstrings.

tacaswell · 2025-09-25T20:25:19Z

ophyd/status.py

+                if self.done:
+                    return
+
+                for st in self.status_list:


Why loop through the list? Isn't the status object passed in the most recently changed one and we can just react to that?

tacaswell · 2025-09-25T20:26:17Z

ophyd/status.py

+    A status that combines mutiple status objects in a list using logical and.
+    The status is finished when all status objects in the list are finished.
+    If any status object fails, the combined status will also fail and
+    set the exception from the first failed status on all sub-statuses.


I do not see where the exception is being pushed down to the children, am I missing something?

Sorry, this is a leftover sentence from an initial idea. I will remove this sentence.

Sub-statuses may have their own logic for set_exception/set_finished. As this logic may not necessarily properly check if the status is already done, we risk that they raise in a manner that may be confusing. To avoid this, we dropped the idea again and instead just call set_finished/set_exception on the AndAllStatus.

tacaswell · 2025-09-25T20:32:04Z

ophyd/status.py

+        return item in self.status_list
+
+
+class OrAnyStatus(DeviceStatus):


What is the use case for "or"? . When we added AndStatus + __and__ we could not think of a good example of when it would be useful and chose to leave it out rather than go for the completionist approach.

If we add this, then we should also add __or__ to StatusBase which will in turn bring up my question of what this is adding over st1 or st2 or st3?

I briefly tried the st1 or st2 or st3 syntax, and was positively surprised. I was not aware of it. The difference to AnyOrStatus is that in its current implementation, the AnyOrStatus will only result in an exception if all status objects are failed. I could see that this comes in handy in the future, although we currently do not have an immediate use case in action.

dev = Device(name='dev') st1 = StatusBase() st2 = StatusBase() st3 = StatusBase() any_or_st = AnyOrStatus(dev, [st1,st2,st3]) or_st = st1 or st2 or st3 st1.set_exception(ValueError("Test")) any_or_st.done # False or_st.done # True, success= False st2.set_finished() any_or_st.done # True, success = True or_st.done # True, success = False

As the behavior is different, I am not certain if overwriting __or__ in StatusBase is actually desired. Both scenarios seem to be valid use cases.

If you do not over-ride __or__ python returns the first "true" object or the last object.

In [4]: or_st = st1 or st2 or st3 In [5]: or_st is st1 Out[5]: True In [6]: type(or_st) Out[6]: ophyd.status.StatusBase

so or is not doing what you want!

However, I had something twisted in my head, we can override & and | with __and__ and __or__

This actually makes the case to implement __or__ stronger to me as the default behavior "works", but is not at all the intended meaning.

I see, sorry for not getting this before fully. With &, the AndStatus almost fully serves the purpose which we would like to see. I will take this into account when I split the PR, and take a look at how __or__ could be implemented.

cappel89 · 2025-09-26T17:02:40Z

I'll fix the linter later today.

Note: I might be easier if the whole file complies to black -l 116 *.py, as I can not run the command on the whole file without reformatting significant amounts of code that I do not want to touch!

tacaswell · 2025-09-26T19:19:44Z

Could you split the compare/transition status and the and/or status into separate PRs?

I suspect we will merge the first two faster than the second two.

…rAnyStatus

cappel89 · 2025-09-26T21:03:45Z

Could you split the compare/transition status and the and/or status into separate PRs?

I suspect we will merge the first two faster than the second two.

Done!

tacaswell · 2025-09-29T14:43:40Z

ophyd/status.py

+    def _compare_callback(self, value, **kwargs) -> bool:
+        """Callback for subscription status"""
+        try:
+            if isinstance(value, list):


isinstance checks like this are a bit of a trap as things like tuple will not hit this (and then checking for sequences require checking it is not a string, ...) and it gets messy.

cappel89 force-pushed the add_new_status_objects branch 2 times, most recently from 28b80c9 to 427d0cc Compare September 25, 2025 15:15

cappel89 force-pushed the add_new_status_objects branch from 427d0cc to 5752e6c Compare September 25, 2025 15:22

tacaswell reviewed Sep 25, 2025

View reviewed changes

ZLLentz reviewed Sep 25, 2025

View reviewed changes

tacaswell reviewed Sep 25, 2025

View reviewed changes

cappel89 force-pushed the add_new_status_objects branch 3 times, most recently from 9029ec2 to 7011fe2 Compare September 26, 2025 18:36

cappel89 added 3 commits September 26, 2025 15:46

feat(status): add CompareStatus, TransitionStatus, AllAndStatus and O…

e7a73b9

…rAnyStatus

refactor: cleanup and fix linter comments

7b8f94e

refactor: cleanup, make fixes following PR review

61cb5d0

cappel89 force-pushed the add_new_status_objects branch from 2e4e944 to 6096c1a Compare September 26, 2025 20:48

refactor: remove AndAllStatus and OrAnyStatus

4348dcd

cappel89 force-pushed the add_new_status_objects branch from 6096c1a to 4348dcd Compare September 26, 2025 20:51

cappel89 mentioned this pull request Sep 26, 2025

Add OrStatus, improve error propagation for AndStatus #1266

Open

tacaswell reviewed Sep 29, 2025

View reviewed changes

		return item in self.status_list


		class OrAnyStatus(DeviceStatus):

Add CompareStatus, TransitionStatus, AllAndStatus and OrAnyStatus #1265

Are you sure you want to change the base?

Add CompareStatus, TransitionStatus, AllAndStatus and OrAnyStatus #1265

Uh oh!

Conversation

cappel89 commented Sep 25, 2025

Summary

CompareStatus

TransitionStatus

AndAllStatus

OrAnyStatus

Uh oh!

cappel89 commented Sep 25, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!