[BUG] Fails to transmit printer status (on Janus datachannel) when size > 1500 due to large file_metadata

[puterboy]

In __init__.py there is a function status_update_to_client_loop which prints the _print_job_tracker.status every 0.75 or 2 seconds. This in turn calls post_printer_status_to_client which gets the status payload as a json message from the status routine in print_job_tracker.py. The status includes a range of printer status data, including file_metadata.

The status message in turn is transmitted over the Janus datachannel by calling send_msg_to_client in client_conn.py which encodes and compresses the message before sending via the send routine where the size of the message is limited to MAX_PAYLOAD_SIZE=1500 (since this is UDP). If the size is too large, the send FAILS COMPLETELY and an error message is posted to octoprint.log.

Now, the status includes file_metadata which in turn includes all the additions to the file metadata added by various Octoprint plugins.

Some of this metadata -- such as that provided by PrintTimeGenius is quite detailed causing the total compressed message to ALWAYS exceed 1500 bytes.

As a result the send routine ALWAYS fails meaning that no status data is EVER successfully sent.
This results in an error message every 2 seconds in octoprint.log of form:
2024-01-28 17:46:07,625 - octoprint.plugins.obico - ERROR - datachannel payload too big (2029)

This may explain some of the other bugs I am encountering where filament usage is not properly transmitted.

Note that PrintTimeGenius is a pretty popular plugin so I suspect that others are also encountering this problem but since the fail is silent, others have not detected it yet.

I can brute force prevent overflowing by commenting-out the lines in print_job_tracker.py that add self.get_file_metadata(plugin, data) or self._file_metadata_cache to data['status'] but I understand that some of the metadata may actually be useful.

If so, the most robust solution would presumably be to chop the message up into chunks < 1500 bytes and reassemble on the receiving end.

[heathen82]

Same issue here... I use PrintTimeGenius and get the error in the log. Not a big deal, but I like to watch the syslog live through a console. Anyone look into this?

[puterboy]

@kennethjiang - any progress here?
This is a real, verified bug.
I am not a programmer (more just a hacker) but I wouldn't think it would be hard to chop up the messages into chunks < 1500 bytes and reassemble on the receiving end -- pretty much any packet transmission modality works that way...

Alternatively, as a temporary fix, only include metadata tags that Obico recognizes in the stream. This should work since by definition if Obico doesn't recognize the metadata tag, it's not going to be used anyway in the Obico app.

[kennethjiang]

Closed in 48397bb. It's a simple chnage. sorry for taking so long.

[puterboy]

Thanks.
Looked at the commit.
Unless I am missing something, it seems like the only substantive change is that the error severity has been demoted from error to debug
Won't there still be an issue of large packets not being sent at all? (as the send function aborts and returns before sending with or without an error message)

[kennethjiang]

Thanks. Looked at the commit. Unless I am missing something, it seems like the only substantive change is that the error severity has been demoted from error to debug Won't there still be an issue of large packets not being sent at all? (as the send function aborts and returns before sending with or without an error message)

These messages are not mission-critical. It'll only speed up the status update. Without them, the update will just have a 30-60s delay.

There is no way around it if the packet are too big. UDP packets are limited to 1500 bytes

[puterboy]

The current code ends up dropping all the packets leading (as you mention) to a 30-60s delay.
I can think of several better ways...

1. Packetize larger packets into packets < 1500 bytes (ideal but requires more coding)
2. Only include the elements of file_metadata that are actually needed and used by Obico (presumably since Obico doesn't even know about other plugins there is no advantage in including metadata contributed by them
3. Don't include any file_metadata but include other elements of the packet
4. Blindly truncate packets at 1500 bytes (rather than abort)

Note I have been using #3 for quite a while -- as it was blindingly easy to code as I just had to comment out the 2 places where file_metadata is retrieved.

However #2 seems to be perhaps the best balance between functionality and ease of coding.

If you agree with approach #2 and tell me what file_metadata is actually used by Obico, I can take a stab at changing the routine get_file_metadata to only retrieve the needed elements of the files metadata structure for the corresponding file/job.

Or alternatively, I could recode the routine to only include the base Octoprint elements, ignoring ones that are added by optional plugins. Based on the API documentation, it seems like the base elements are as per the following example:

{
      "name": "whistle_v2.gcode",
      "path": "whistle_v2.gcode",
      "type": "machinecode",
      "typePath": ["machinecode", "gcode"],
      "hash": "...",
      "size": 1468987,
      "date": 1378847754,
      "origin": "local",
      "refs": {
        "resource": "http://example.com/api/files/local/whistle_v2.gcode",
        "download": "http://example.com/downloads/files/local/whistle_v2.gcode"
      },
      "gcodeAnalysis": {
        "estimatedPrintTime": 1188,
        "filament": {
          "length": 810,
          "volume": 5.36
        }
      },
      "print": {
        "failure": 4,
        "success": 23,
        "last": {
          "date": 1387144346,
          "success": true
        }
      }
    }

[puterboy]

Here is a patch based on #2 using just the above keys plus 'obico'

--- print_job_tracker.py.orig  2024-05-31 02:42:03.481101518 -0400
+++ /tmp/print_job_tracker.py.new      2024-06-09 14:27:43.556375042 -0400
@@ -133,7 +133,11 @@
             if not origin or not path:
                 return None

-            return plugin._file_manager._storage_managers.get(origin).get_metadata(path) or {}
+            metadata = plugin._file_manager._storage_managers.get(origin).get_metadata(path) or {}
+            #Keep only base metadata keys (see: https://docs.octoprint.org/en/master/api/datamodel.html#file-information) plus 'obico' key
+            keys_keep = ['obico', 'name', 'path', 'type', 'typePath', 'hash', 'size', 'date', 'origin', 'refs', 'gcodeAnalysis', 'prints', 'statistics']
+            metadata_keep = {key : value for key, value in metadata.items() if key in keys_keep}
+            return metadata_keep
         except Exception as e:
             _logger.exception(e)
             return None

If you are OK with this, I will submit as PR

[kennethjiang]

Yes there are a few places where file_metadata is used such as https://github.com/TheSpaghettiDetective/obico-server/blob/0b2da44c8673dc6cebada1f8982941d070e1e4bb/frontend/src/components/printer-control/PrintProgressWidget.vue#L259

[puterboy]

Do you want me to submit the above patch as a PR?
I would assume it would be sufficient to include only the base Octoprint & Obico metadata (per my patch) as surely Obico doesn't rely on other random metadata introduced by other 3rd party add-ons.

[kennethjiang]

Do you want me to submit the above patch as a PR? I would assume it would be sufficient to include only the base Octoprint & Obico metadata (per my patch) as surely Obico doesn't rely on other random metadata introduced by other 3rd party add-ons.

Sure please submit a PR.

[puterboy]

PR submitted
#236