feat(api): return location sequences from commands #17435

sfoster1 · 2025-02-05T17:24:39Z

By adding all the locations of a labware used in a command that establishes or changes the position of a labware, we can free the client from having to accumulate all previous such commands to display a single such command.

These locations are shown as location sequences. These are lists of heterogenous unions keyed by kind. The kinds mostly correspond to the kinds of locations that the engine already uses - they're different types because they have the kind key so you can tell them apart when serialized instead of just via python isinstance checks. All these sequences end in either an OnCutoutFixtureLocation which lists a cutout id and possible cutout fixtures; or a NotOnDeckLocation that specifies OFF_DECK or system.

Different robots have different kinds of sequences; on the OT-2, modules are loaded on deck slots, and so a labware on a module would have [OnModule, OnAddressableArea, OnCutoutFixture] while on the Flex, modules are loaded on cutout fixtures and a labware on a module would have [OnAdressableArea, OnModule, OnCutoutFixture] (having the OnModule there lets us have the module ID; it is not technically necessary).

There are some command implementations that are straightforward, and some that are charmingly quirky.

loadLabware, reloadLabware, loadLid all just return the location of the newly added labware. Simple, clean, wondrous
loadLidStack creates a lid stack object and returns its location, and also creates N lid objects and returns all of their locations, which is done in its own array that is guaranteed to have the same ordering as all the lid ids so we can keep the types of the locations consistent across commands and also avoid altering already-existing result elements
moveLabware needs an origin and a destination location, so it has them. In fact, one destination location isn't enough. When we move labware to a trash location, we need to encode both (1) which trash location it was and (2) that it is a trash location. After the command, the position of the labware is OFF_DECK (because it went into the trash), after all. So we get both immediateDestinationLocationSequence (which in this scenario would be the addressable area of the trash) and eventualDestinationLocationSequence (which in this scenario would be off deck). If the two are the same, then the two are the same
the stacker store and retrieve commands have info that isn't really actually useful right now, but it will be soon, so let's carve out some space for them. For now the labware just bounces back and forth between the module/off deck and the module/stacker staging slot.

to come out of draft

ts bindings
stacker store and retrieve
~~[ ] add "what is the old and new state of the locations named in the command" to these values~~
not going to do that ^ in this pr just so we can get it merged

These are like offset location sequences, but they involve instance IDs instead of geometry identifiers. They'll be returned from various commands.

Load labware now tells you the location sequence of the labware it just loaded.

MoveLabware is a little different than the other implementations. It needs an origin and a destination sequence, since it's a move - that's straightforward. But it also needs to drive a distinction between an immediate destination: where is the labware dropped off? and an eventual destination: where does the labware end up? so that the necessary information to figure out what happens when you move a labware to the trash is there. We need to identify the trash, and which trash; but we also need to indicate that the labware is going to get thrown out. So we have three total locations.

TamarZanzouri · 2025-02-06T15:10:27Z

api/src/opentrons/protocol_engine/commands/load_lid_stack.py

+    stackLocationSequence: LabwareLocationSequence | None = Field(
+        None, description="The location sequence for this stack."
+    )
+    locationSequences: List[LabwareLocationSequence] | None = Field(
+        None,
+        description="The location sequences for the lids just loaded into the stack. These are in the same order as labwareIds.",


I am confused between these 2 props. can we maybe clarify it?

if one of them is a "physical adapter" maybe it will clarify if we add it to the desc

TamarZanzouri · 2025-02-06T16:29:51Z

api/src/opentrons/protocol_engine/commands/move_labware.py

+    eventualDestinationLocationSequence: LabwareLocationSequence | None = Field(
+        None,
+        description=(
+            "The full location in which this labware will eventually reside. This will typically be the same as its "
+            "immediate destination, but if this labware is going to the trash then this field will be off deck."
+        ),
+    )
+    immediateDestinationLocationSequence: LabwareLocationSequence | None = Field(


can you explain the diff between these two props?

Yeah, this is for when you move stuff to the trash. We want to know that it's going to the trash, so we can render it; but it doesn't end up in the trash, it ends up off deck, and we want to know that too. So we have both

TamarZanzouri · 2025-02-06T17:30:01Z

api/src/opentrons/protocol_engine/types/location.py

+def labware_location_is_system(
+    location: LabwareLocation,
+) -> TypeGuard[_SystemLocationType]:
+    """Check if a location is the system location."""


might have missed this convo but what is a system location?

SYSTEM is a new location used for uh I forget. It's like off deck but not literally off deck, I think it's for deleted stuff

I think it's where lid stacks go when they become empty - is that right @CaseyBatten?

TamarZanzouri · 2025-02-06T17:32:30Z

api/src/opentrons/protocol_engine/state/geometry.py

+                labware_location.moduleId
+            )
+            try:
+                self._modules.get_flex_stacker_substate(labware_location.moduleId)


are we doing this get just for catching the exception?

TamarZanzouri

overall looks good! had a few questions/clarifications.

These will probably need to change more.

codecov · 2025-02-06T22:20:43Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 24.66%. Comparing base (2b87a7a) to head (1f277a6).
Report is 31 commits behind head on edge.

Additional details and impacted files

@@            Coverage Diff             @@
##             edge   #17435      +/-   ##
==========================================
+ Coverage   17.34%   24.66%   +7.31%     
==========================================
  Files        3128     3271     +143     
  Lines      226114   232920    +6806     
  Branches     6884     9713    +2829     
==========================================
+ Hits        39225    57440   +18215     
+ Misses     186889   175465   -11424     
- Partials        0       15      +15

Flag	Coverage Δ
protocol-designer	`17.42% <ø> (+0.07%)`	⬆️
step-generation	`4.08% <ø> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
shared-data/command/types/setup.ts	`100.00% <ø> (ø)`

... and 541 files with indirect coverage changes

TamarZanzouri

LGTM! thank you for you explanations and your fixes!

github-actions · 2025-02-07T14:51:41Z

A PR has been opened to address analyses snapshot changes. Please review the changes here: #17464

github-actions · 2025-02-07T15:36:42Z

A PR has been opened to address analyses snapshot changes. Please review the changes here: #17464

…snapshots (#17464) This PR was requested on the PR #17435 Co-authored-by: sfoster1 <[email protected]>

github-actions · 2025-02-07T15:51:34Z

A PR has been opened to address analyses snapshot changes. Please review the changes here: #17467

…snapshots (#17467) This PR was requested on the PR #17435 Co-authored-by: sfoster1 <[email protected]>

github-actions · 2025-02-07T16:22:01Z

A PR has been opened to address analyses snapshot changes. Please review the changes here: #17468

github-actions · 2025-02-10T15:16:43Z

A PR has been opened to address analyses snapshot changes. Please review the changes here: #17468

github-actions · 2025-02-10T15:53:28Z

A PR has been opened to address analyses snapshot changes. Please review the changes here: #17468

SyntaxColoring

Thanks! LGTM when tests are passing and comments are resolved to your satisfaction.

SyntaxColoring · 2025-02-10T15:44:29Z

api/src/opentrons/protocol_runner/legacy_command_mapper.py

+                # These legacy json protocols don't get location sequences because
+                # to do so we'd have to go back and look up where the module gets loaded


What are the implications of this? Doesn't this mean...something?...in the app will break when running "old" JSON and PAPI protocols?

All it means is that we have to carry the backwards-compat code in consuming clients until we remove support for those older kinds of protocols. The types establish pretty firmly that these location sequences are not guaranteed to be present for this reason.

SyntaxColoring · 2025-02-10T15:49:30Z

shared-data/command/types/setup.ts

+
+export interface OnLabwareLocationSequenceComponent {
+  kind: 'onLabware'
+  labwareId: string
+  lidId: string | null
+}
+
+export interface OnModuleLocationSequenceComponent {
+  kind: 'onModule'
+  moduleId: string
+}
+
+export interface OnAddressableAreaLocationSequenceComponent {
+  kind: 'onAddressableArea'
+  addressableAreaName: string
+  slotName: string | null
+}
+
+export interface NotOnDeckLocationSequenceCompoennt {
+  kind: 'notOnDeck'
+  logicalLocationName: 'offDeck' | 'systemLocation'
+}
+
+export type LocationSequenceComponent =
+  | OnLabwareLocationSequenceComponent
+  | OnModuleLocationSequenceComponent
+  | OnAddressableAreaLocationSequenceComponent
+  | NotOnDeckLocationSequenceCompoennt
+
+export type LabwareLocationSequence = LocationSequenceComponent[]
+


Does the eventualDestinationLocationSequence/immediateDestinationLocationSequence thing get cleaner if we stop using a shared offDeck/systemLocation constant for trashed labware, and instead have a LocationSequenceComponent with kind: 'inTrash'?

Roughly like:

{ "kind": "inTrash", "addressableAreaName": "gripperWasteChute" }

It would, but I would only want to do that if we added that notion to the engine internal (and labware key external) representation of locations; I really want to keep elements of the location sequence as things that would show up in static representations of the loaded-labware state

SyntaxColoring · 2025-02-10T15:55:05Z

api/tests/opentrons/protocol_engine/state/test_geometry_view.py

+        command=LoadLabware(
+            id="load-labware-1",
+            createdAt=datetime.now(),
+            key="load-labware-1",
+            status=CommandStatus.SUCCEEDED,
+            result=LoadLabwareResult(
+                labwareId="labware-id-1",
+                definition=nice_labware_definition,
+                offsetId=None,
+            ),
+            params=LoadLabwareParams(
+                location=DeckSlotLocation(slotName=DeckSlotName.SLOT_C2),
+                loadName=nice_labware_definition.parameters.loadName,
+                namespace=nice_labware_definition.namespace,
+                version=nice_labware_definition.version,
+            ),
+        ),
+        state_update=StateUpdate(
+            loaded_labware=LoadedLabwareUpdate(
+                labware_id="labware-id-1",
+                definition=nice_labware_definition,
+                offset_id=None,
+                new_location=DeckSlotLocation(slotName=DeckSlotName.SLOT_C2),
+                display_name=None,
+            ),
+            addressable_area_used=AddressableAreaUsedUpdate(addressable_area_name="C2"),
+        ),


If it makes things cleaner, I think all of these command instantiations except for the LoadModule ones can be replaced with dummies like Comment, since the state stores only care about state_update now, not the command. (With the exception of ModuleStore, which still cares about the command because we haven't gotten to it yet.) In the future, I hope to delete command from SucceedCommandAction so all of these tests can be rid of this boilerplate.

api/tests/opentrons/protocol_engine/state/test_geometry_view.py

api/src/opentrons/protocol_engine/types/location.py

github-actions · 2025-02-10T16:33:16Z

A PR has been opened to address analyses snapshot changes. Please review the changes here: #17468

…snapshots (#17468) This PR was requested on the PR #17435 Co-authored-by: sfoster1 <[email protected]>

github-actions · 2025-02-10T19:59:13Z

A PR has been opened to address analyses snapshot changes. Please review the changes here: #17484

github-actions · 2025-02-10T21:28:42Z

A PR has been opened to address analyses snapshot changes. Please review the changes here: #17484

…snapshots (#17484) This PR was requested on the PR #17435 Co-authored-by: sfoster1 <[email protected]>

By adding all the locations of a labware used in a command that establishes or changes the position of a labware, we can free the client from having to accumulate all previous such commands to display a single such command. These locations are shown as location sequences. These are lists of heterogenous unions keyed by `kind`. The kinds mostly correspond to the kinds of locations that the engine already uses - they're different types because they have the `kind` key so you can tell them apart when serialized instead of just via python isinstance checks. All these sequences end in either an `OnCutoutFixtureLocation` which lists a cutout id and possible cutout fixtures; or a `NotOnDeckLocation` that specifies `OFF_DECK` or system. Different robots have different kinds of sequences; on the OT-2, modules are loaded on deck slots, and so a labware on a module would have `[OnModule, OnAddressableArea, OnCutoutFixture]` while on the Flex, modules are loaded on cutout fixtures and a labware on a module would have `[OnAdressableArea, OnModule, OnCutoutFixture]` (having the `OnModule` there lets us have the module ID; it is not technically necessary). There are some command implementations that are straightforward, and some that are charmingly quirky. - `loadLabware`, `reloadLabware`, `loadLid` all just return the location of the newly added labware. Simple, clean, wondrous - `loadLidStack` creates a lid stack object and returns its location, and also creates N lid objects and returns all of _their_ locations, which is done in its own array that is guaranteed to have the same ordering as all the lid ids so we can keep the types of the locations consistent across commands and also avoid altering already-existing result elements - `moveLabware` needs an origin and a destination location, so it has them. In fact, one destination location isn't enough. When we move labware to a trash location, we need to encode both (1) which trash location it was and (2) that it is a trash location. After the command, the position of the labware is `OFF_DECK` (because it went into the trash), after all. So we get both `immediateDestinationLocationSequence` (which in this scenario would be the addressable area of the trash) and `eventualDestinationLocationSequence` (which in this scenario would be off deck). If the two are the same, then the two are the same - the stacker `store` and `retrieve` commands have info that isn't really actually useful right now, but it will be soon, so let's carve out some space for them. For now the labware just bounces back and forth between the module/off deck and the module/stacker staging slot. ## to come out of draft - [x] ts bindings - [x] stacker store and retrieve - ~[ ] add "what is the old and new state of the locations named in the command" to these values~ not going to do that ^ in this pr just so we can get it merged --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: sfoster1 <[email protected]>

sfoster1 added 6 commits February 4, 2025 14:07

refactor(api): add a way to get location sequences

ba90394

These are like offset location sequences, but they involve instance IDs instead of geometry identifiers. They'll be returned from various commands.

feat(api): add location result to loadLabware

56d102c

Load labware now tells you the location sequence of the labware it just loaded.

feat(api): add location sequence to loadLid

8d9d026

feat(api): add location sequence to loadLidStack

2d0d096

feat(api): add locationSequence to reloadLabware

b8cbdde

sfoster1 requested a review from a team as a code owner February 5, 2025 17:24

sfoster1 marked this pull request as draft February 5, 2025 17:24

TamarZanzouri reviewed Feb 6, 2025

View reviewed changes

sfoster1 added 2 commits February 6, 2025 16:53

feat(api): return locations from stacker module commands

59b1e34

These will probably need to change more.

fixup the description for the location field

4744958

sfoster1 marked this pull request as ready for review February 6, 2025 22:02

chore: add ts types

494579d

sfoster1 requested a review from a team as a code owner February 6, 2025 22:16

sfoster1 added the gen-analyses-snapshot-pr Generate a healing PR if the analyses snapshot test fails label Feb 7, 2025

TamarZanzouri approved these changes Feb 7, 2025

View reviewed changes

github-actions bot mentioned this pull request Feb 7, 2025

fix(analyses-snapshot-testing): heal exec-1161-locations-from-commands snapshots #17464

Merged

fixup tests

4868d85

fix(analyses-snapshot-testing): heal exec-1161-locations-from-commands …

50c453f

…snapshots (#17464) This PR was requested on the PR #17435 Co-authored-by: sfoster1 <[email protected]>

github-actions bot mentioned this pull request Feb 7, 2025

fix(analyses-snapshot-testing): heal exec-1161-locations-from-commands snapshots #17467

Merged

fix(analyses-snapshot-testing): heal exec-1161-locations-from-commands …

33a0e58

…snapshots (#17467) This PR was requested on the PR #17435 Co-authored-by: sfoster1 <[email protected]>

github-actions bot mentioned this pull request Feb 7, 2025

fix(analyses-snapshot-testing): heal exec-1161-locations-from-commands snapshots #17468

Merged

take locations down to the cutout

e075d03

fix an ordering dependency mismatch

0bd0350

lint and format

5983cd6

SyntaxColoring approved these changes Feb 10, 2025

View reviewed changes

sfoster1 and others added 3 commits February 10, 2025 14:34

fix some copy pasta

2ba8252

catch up the ts bindings

5ad672d

fix(analyses-snapshot-testing): heal exec-1161-locations-from-commands …

31b9b95

…snapshots (#17468) This PR was requested on the PR #17435 Co-authored-by: sfoster1 <[email protected]>

github-actions bot mentioned this pull request Feb 10, 2025

fix(analyses-snapshot-testing): heal exec-1161-locations-from-commands snapshots #17484

Merged

sfoster1 added 2 commits February 10, 2025 16:02

fix: add lid ids to the uuid replacement in snapshot

6af9b6f

format, though this doesn't seem right

3a85315

fix(analyses-snapshot-testing): heal exec-1161-locations-from-commands …

1f277a6

…snapshots (#17484) This PR was requested on the PR #17435 Co-authored-by: sfoster1 <[email protected]>

SyntaxColoring approved these changes Feb 10, 2025

View reviewed changes

sfoster1 merged commit 48ac4e6 into edge Feb 10, 2025
54 checks passed

sfoster1 deleted the exec-1161-locations-from-commands branch February 10, 2025 22:06

		# These legacy json protocols don't get location sequences because
		# to do so we'd have to go back and look up where the module gets loaded

feat(api): return location sequences from commands #17435

feat(api): return location sequences from commands #17435

Uh oh!

Conversation

sfoster1 commented Feb 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

to come out of draft

Uh oh!

Choose a reason for hiding this comment

Uh oh!

TamarZanzouri Feb 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

TamarZanzouri left a comment

Choose a reason for hiding this comment

Uh oh!

codecov bot commented Feb 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

TamarZanzouri left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Feb 7, 2025

Uh oh!

github-actions bot commented Feb 7, 2025

Uh oh!

github-actions bot commented Feb 7, 2025

Uh oh!

github-actions bot commented Feb 7, 2025

Uh oh!

github-actions bot commented Feb 10, 2025

Uh oh!

github-actions bot commented Feb 10, 2025

Uh oh!

SyntaxColoring left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Feb 10, 2025

Uh oh!

github-actions bot commented Feb 10, 2025

Uh oh!

github-actions bot commented Feb 10, 2025

Uh oh!

Uh oh!

Uh oh!

sfoster1 commented Feb 5, 2025 •

edited

Loading

TamarZanzouri Feb 6, 2025 •

edited

Loading

codecov bot commented Feb 6, 2025 •

edited

Loading

TamarZanzouri left a comment •

edited

Loading