Skip to content

Migrate Backend <--> Frontend comms to protobuf over NT #1875

Open
@mcm001

Description

@mcm001

Is your feature request related to a problem? Please describe.

We currently use msgpacked-JSON over websocket to communicate between backend and frontend. The contents of the JSON message we send are undocumented and it leads to fun hacks like casting to a raw HashMap over in the Java backend. By moving to NT4, we're switching our WS backend, which might Just Fix #1827 .

Describe the solution you'd like

PhotonVision hosts a NT server on TCP port 5801, and clients connect to it. Message ICDs are defined in protobuf, which we can use internally for some of our pub-sub system as well.

First pass can be found at https://github.com/mcm001/photonvision/tree/nt-frontend-poc -- this shows using ascope's protobuf decoder and Chris's NT library to decode the result_proto entry.

Image

[NT4] decoded protobuf data:  {"targets":[{"yaw":9.189625323506112,"pitch":-8.29944069188508,"area":0.17127821180555555,"skew":0,"fiducial_id":3,"best_camera_to_target":{"translation":{"x":4.75960040202207,"y":-0.7699917311775007,"z":-0.7032990530303476},"rotation":{"q":{"w":-0.005189341343654008,"x":-0.15540880402521998,"y":0.005860925783958193,"z":-0.9878192263348718}}},"alt_camera_to_target":{"translation":{"x":4.781375162549686,"y":-0.7731560770502773,"z":-0.706930419096861},"rotation":{"q":{"w":-0.13776783268749462,"x":-0.007202707859492868,"y":-0.015690688534553243,"z":-0.9903140651174217}}},"pose_ambiguity":0.8421108368456712,"min_area_rect_corners":[{"x":787.000125467573,"y":499.0000016856238},{"x":826.9501483376376,"y":498.001249575798},{"x":828.000118673052,"y":539.9999983143762},{"x":788.0500958029874,"y":540.998750424202}],"detected_corners":[{"x":788.3407592773436,"y":539.8829345703123},{"x":828.530090332031,"y":540.42041015625},{"x":826.7855224609376,"y":500.2556457519533},{"x":787.0337524414065,"y":499.7416076660156}],"obj_detection_id":-1,"obj_detection_conf":-1},{"yaw":2.3534699038406455,"pitch":-8.323441117088965,"area":0.1736111111111111,"skew":0,"fiducial_id":4,"best_camera_to_target":{"translation":{"x":4.794998491522927,"y":-0.19704289031529953,"z":-0.7019917986480779},"rotation":{"q":{"w":-0.034238700889571864,"x":-0.12181944670936937,"y":0.003464031240264397,"z":-0.9919555102182678}}},"alt_camera_to_target":{"translation":{"x":0,"y":0,"z":0},"rotation":{"q":{"w":1.1102230246251565e-16,"x":0,"y":0,"z":-1}}},"pose_ambiguity":0,"min_area_rect_corners":[{"x":647.0006273137725,"y":536.9750123084702},{"x":648.000002662878,"y":497.0000033135305},{"x":688.9993726862275,"y":498.02498769152976},{"x":687.999997337122,"y":537.9999966864696}],"detected_corners":[{"x":648.8710327148439,"y":537.4219970703125},{"x":688.7018432617185,"y":538.3585205078125},{"x":688.212158203125,"y":498.38018798828136},{"x":648.7171630859377,"y":497.55416870117193}],"obj_detection_id":-1,"obj_detection_conf":-1}],"sequence_id":"33919","capture_timestamp_micros":"4077083074","nt_publish_timestamp_micros":"4077102674","time_since_last_pong_micros":"9223372036854776000"}

Describe alternatives you've considered

  1. Do Nothing: The current thing works, and this is a lot of work. I think that more defined message interfaces will save us time in the long run
  2. Stick with the current protocol, but use a different WS server to side-step WebSocket connection opens, but no data is sent #1827 . WPILib also has a web server, so this is possible. Serving web assets straight from the JAR is quite handy though, and switching web server backends would open us up to more weird bugs like the Jackson bug that required a server restart after changing static IP settings.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions