Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(api,robot-server,shared-data): Split LabwareDefinition Pydantic model for labware schemas 2 and 3 #17563

Merged
merged 9 commits into from
Feb 21, 2025

Conversation

SyntaxColoring
Copy link
Contributor

@SyntaxColoring SyntaxColoring commented Feb 20, 2025

Overview

Closes EXEC-1206.

Test Plan and Hands on Testing

Just automated tests.

Changelog

Main changes:

  • Split the LabwareDefinition Pydantic model into LabwareDefinition2 and LabwareDefinition3, for labware schemas 2 and 3.

  • Many minor updates across our backend to account for the above.

  • Try to be mindful about which things in our backend should deal with LabwareDefinition2 | LabwareDefinition3 and which things should now limit themselves to LabwareDefinition2.

    For example, our Python types for JSON protocol schema 8 now accept LabwareDefinition2 | LabwareDefinition3, whereas for schemas ≤7 they continue to only support LabwareDefinition2.

Plus a couple of smaller fixes:

  • Split our the Pydantic WellDefinition model into RectangularWellDefinition and CircularWellDefinition.
  • Fix a bug in the TypedDict-based bindings where geometryDefinitionId was omittable for rectangular wells but not circular ones.

Review requests

Can you think of any other places in our backend that should deliberately limit themselves to LabwareDefinition2, instead of accepting LabwareDefinition2 | LabwareDefinition3?

Risk assessment

Low. This shouldn't be any riskier than what we had before.

@SyntaxColoring SyntaxColoring force-pushed the split_pydantic_labwaredefinition branch from 3e67f83 to d994928 Compare February 20, 2025 20:09
Comment on lines -696 to -725
innerLabwareGeometry={
"welldefinition1111": InnerWellGeometry(
sections=[
CuboidalFrustum(
shape="cuboidal",
topXDimension=7.6,
topYDimension=8.5,
bottomXDimension=5.6,
bottomYDimension=6.5,
topHeight=45,
bottomHeight=20,
),
CuboidalFrustum(
shape="cuboidal",
topXDimension=5.6,
topYDimension=6.5,
bottomXDimension=4.5,
bottomYDimension=4.0,
topHeight=20,
bottomHeight=10,
),
SphericalSegment(
shape="spherical",
radiusOfCurvature=6,
topHeight=10,
bottomHeight=0.0,
),
],
)
},
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test definition had a mixture of schema 2 and 3 stuff, and it was unclear to me which direction to take it. I've deleted the schema 3 stuff because it seemed like it was unused in this file.

Copy link

codecov bot commented Feb 20, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 63.45%. Comparing base (82acdce) to head (87596c7).
Report is 4 commits behind head on edge.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             edge   #17563      +/-   ##
==========================================
- Coverage   63.45%   63.45%   -0.01%     
==========================================
  Files        2840     2840              
  Lines      218427   218412      -15     
  Branches    18123    18123              
==========================================
- Hits       138605   138590      -15     
  Misses      79629    79629              
  Partials      193      193              
Flag Coverage Δ
protocol-designer 18.85% <ø> (ø)
step-generation 4.36% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...pentrons_shared_data/labware/labware_definition.py 59.20% <ø> (-0.96%) ⬇️
...data/python/opentrons_shared_data/labware/types.py 100.00% <ø> (ø)
..._shared_data/protocol/models/protocol_schema_v6.py 100.00% <ø> (ø)
..._shared_data/protocol/models/protocol_schema_v7.py 100.00% <ø> (ø)

f" using nominal fallback value of {nominal_fallback}"
)
log.debug(message, exc_info=e)
return nominal_fallback
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Heads up for this temporary hack that I'm doing for OT-2 tip length calibration.

The ticket I linked, EXEC-1230, currently talks about totally blocking these calibration flows from dealing with labware schema 3. I'm changing my mind on that—I think we have to get them to deal with labware schema 3 correctly somehow—and I'll rewrite EXEC-1230 when I know more about the approach.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you think we have to get them to deal with labware schema 3 correctly? We could instead not ever issue OT-2 tiprack or calibration block definitions in schema 3. We possibly lock out loading tipracks in schema 3 when the robot is an OT-2, for this reason. This means that OT-2 tipracks wouldn't get the benefit of the more-correct geometry loading, but this is an ancient and known problem with OT-2 tipracks that people have been handing for years.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you think we have to get them to deal with labware schema 3 correctly?

Basically because I didn't feel it was acceptable to abandon OT-2 tip racks like that. If our goal is to improve labware positioning, but we fail at standard OT-2 tip racks, then that's a pretty big bummer. They're certainly not edge cases. I think it's worth trying to resolve even if it means us (or me) pushing a little harder.

Locking OT-2 tip racks and calibration blocks to schema 2 sounds like a good escape hatch, if it comes down to it, but I hope it doesn't come down to it.

I can deprioritize this and revisit the question later, but mark it as something we must resolve somehow before v8.4.0—either get it to work with schema 3, or delete the schema 3 defs and lock it to schema 2.

Base automatically changed from more_labware_def_refactors to edge February 20, 2025 20:31
Instead of sharing attributes between schema 2 and 3, share attributes between rectangular and circular wells. In other words, separate the schema versions a little bit more.

Fix a bug where the type of geometryDefinitionId was inconsistent between rectangular and circular wells.
@SyntaxColoring SyntaxColoring force-pushed the split_pydantic_labwaredefinition branch from 523528f to df90055 Compare February 20, 2025 20:43
@SyntaxColoring SyntaxColoring marked this pull request as ready for review February 20, 2025 20:47
@SyntaxColoring SyntaxColoring requested a review from a team as a code owner February 20, 2025 20:47
@SyntaxColoring SyntaxColoring requested a review from a team February 20, 2025 20:47
@SyntaxColoring SyntaxColoring force-pushed the split_pydantic_labwaredefinition branch from df90055 to 9c1f0e4 Compare February 20, 2025 21:20
Resolve conflicts in shared-data/python/opentrons_shared_data/labware/labware_definition.py. Add the new $otSharedSchema property to LabwareDefinition3.
Copy link
Member

@sfoster1 sfoster1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me and OT-2 calibration is the place where I think we should disallow labware schema 3.

To expand on that inline comment, maybe in this PR we should also remove any schema 3 definitions of OT-2 tipracks from shared-data.

# $otSharedSchema deliberately omitted for now. See `"extra": "allow"` in model_config.
schemaVersion: Literal[1, 2, 3]
class LabwareDefinition2(BaseModel):
# todo(mm, 2025-02-18): Is it correct to accept schemaVersion==1 here?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

uh it almost certainly shouldn't be, this code may have started out trying to handle schema 1 labware but i almost guarantee it does not currently do that.

f" using nominal fallback value of {nominal_fallback}"
)
log.debug(message, exc_info=e)
return nominal_fallback
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you think we have to get them to deal with labware schema 3 correctly? We could instead not ever issue OT-2 tiprack or calibration block definitions in schema 3. We possibly lock out loading tipracks in schema 3 when the robot is an OT-2, for this reason. This means that OT-2 tipracks wouldn't get the benefit of the more-correct geometry loading, but this is an ancient and known problem with OT-2 tipracks that people have been handing for years.

@SyntaxColoring SyntaxColoring merged commit 4bb9d72 into edge Feb 21, 2025
74 checks passed
@SyntaxColoring SyntaxColoring deleted the split_pydantic_labwaredefinition branch February 21, 2025 16:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants