Description
Is your feature request related to a problem? Please describe.
We currently use msgpacked-JSON over websocket to communicate between backend and frontend. The contents of the JSON message we send are undocumented and it leads to fun hacks like casting to a raw HashMap over in the Java backend. By moving to NT4, we're switching our WS backend, which might Just Fix #1827 .
Describe the solution you'd like
PhotonVision hosts a NT server on TCP port 5801, and clients connect to it. Message ICDs are defined in protobuf, which we can use internally for some of our pub-sub system as well.
First pass can be found at https://github.com/mcm001/photonvision/tree/nt-frontend-poc -- this shows using ascope's protobuf decoder and Chris's NT library to decode the result_proto entry.
[NT4] decoded protobuf data: {"targets":[{"yaw":9.189625323506112,"pitch":-8.29944069188508,"area":0.17127821180555555,"skew":0,"fiducial_id":3,"best_camera_to_target":{"translation":{"x":4.75960040202207,"y":-0.7699917311775007,"z":-0.7032990530303476},"rotation":{"q":{"w":-0.005189341343654008,"x":-0.15540880402521998,"y":0.005860925783958193,"z":-0.9878192263348718}}},"alt_camera_to_target":{"translation":{"x":4.781375162549686,"y":-0.7731560770502773,"z":-0.706930419096861},"rotation":{"q":{"w":-0.13776783268749462,"x":-0.007202707859492868,"y":-0.015690688534553243,"z":-0.9903140651174217}}},"pose_ambiguity":0.8421108368456712,"min_area_rect_corners":[{"x":787.000125467573,"y":499.0000016856238},{"x":826.9501483376376,"y":498.001249575798},{"x":828.000118673052,"y":539.9999983143762},{"x":788.0500958029874,"y":540.998750424202}],"detected_corners":[{"x":788.3407592773436,"y":539.8829345703123},{"x":828.530090332031,"y":540.42041015625},{"x":826.7855224609376,"y":500.2556457519533},{"x":787.0337524414065,"y":499.7416076660156}],"obj_detection_id":-1,"obj_detection_conf":-1},{"yaw":2.3534699038406455,"pitch":-8.323441117088965,"area":0.1736111111111111,"skew":0,"fiducial_id":4,"best_camera_to_target":{"translation":{"x":4.794998491522927,"y":-0.19704289031529953,"z":-0.7019917986480779},"rotation":{"q":{"w":-0.034238700889571864,"x":-0.12181944670936937,"y":0.003464031240264397,"z":-0.9919555102182678}}},"alt_camera_to_target":{"translation":{"x":0,"y":0,"z":0},"rotation":{"q":{"w":1.1102230246251565e-16,"x":0,"y":0,"z":-1}}},"pose_ambiguity":0,"min_area_rect_corners":[{"x":647.0006273137725,"y":536.9750123084702},{"x":648.000002662878,"y":497.0000033135305},{"x":688.9993726862275,"y":498.02498769152976},{"x":687.999997337122,"y":537.9999966864696}],"detected_corners":[{"x":648.8710327148439,"y":537.4219970703125},{"x":688.7018432617185,"y":538.3585205078125},{"x":688.212158203125,"y":498.38018798828136},{"x":648.7171630859377,"y":497.55416870117193}],"obj_detection_id":-1,"obj_detection_conf":-1}],"sequence_id":"33919","capture_timestamp_micros":"4077083074","nt_publish_timestamp_micros":"4077102674","time_since_last_pong_micros":"9223372036854776000"}
Describe alternatives you've considered
- Do Nothing: The current thing works, and this is a lot of work. I think that more defined message interfaces will save us time in the long run
- Stick with the current protocol, but use a different WS server to side-step WebSocket connection opens, but no data is sent #1827 . WPILib also has a web server, so this is possible. Serving web assets straight from the JAR is quite handy though, and switching web server backends would open us up to more weird bugs like the Jackson bug that required a server restart after changing static IP settings.