Websocket endpoint

[vshekar]

Checklist

• Add a Changelog entry
• Add the ticket number which this PR closes to the comment section

[danielballan]

Notes on schema:

register (CatalogNodeAdapter.put_data_source):

hset:

• datasource (JSON)
• metadata (JSON)
  • timestamp

upload (CatalogArrayAdapter.write):
hset:

• payload (binary)
• metadata (JSON)
  • timestamp
  • mimetype
  • structure-specific details (JSON)
    • for array: {"block": (0, 0)} or {"block": null}
    • for table: {} ??
    • ...

[danielballan]

Start server, with optional --redis-uri argument. (We will rename this to something more general, like --cache to accommodate other implementations like NATS.)

tiled serve catalog --temp --api-key secret --redis-uri redis://localhost:6379

Connect with the Python client, and create an array.

c = from_uri('http://localhost:8000', api_key='secret')
x = c.write_array([1,2,3], key='x')

Subscribe for updates on this array.

websocat 'ws://localhost:8000/api/v1/stream/x' -H 'Authorization:secret'

Back in the Python client, alter (overwrite, in this case) the array.

x.write(np.array([4,5,7]))

Websocket gets an update!

websocat 'ws://localhost:8000/api/v1/stream/x' -H 'Authorization:secret'
{"sequence":1,"timestamp":"2025-07-08T20:19:32.361533","payload":[4,5,7]}

Above we get JSON but msgpack can be requested:

websocat 'ws://localhost:8000/api/v1/stream/x?envelope_format=msgpack' -H 'Authorization: Apikey'

A whole lot is missing—too much to bother enumerating here—but this shows the basic pieces communicating.

[vshekar]

Couple of considerations to make for the /stream/stop endpoint
As it is currently implemented, the data indicating the stream is done, is in-band
This would pose 2 problems:

1. Do we allow the producer from updating the node after the stream is closed?
2. If we only have in-band encoding of the end-of-stream, data will be lost after redis ttl so a new client will be connected indefinitely
   Out of band encoding of this info should solve both issues. Question is where will it go?

[danielballan]

Agreed. We added a column to the nodes table which I think is the right out-of-band encoded to address these questions.

That will also facilitate (in a future PR...) search queries like, "List the nodes that are streaming."

[danielballan]

TO DO:

• Tests
• At least a start at docs
• Rename interfaces to avoid hard-coding redis in the public API (paving the way for NATS etc.)

[danielballan]

A heartbeat for @kivel: We have removed "redis" from public APIs. The next step toward generalization would be some polymorphic adapter ("adapter" in the general sense, not a "Tiled Adapter") that will enable a common interface to Redis-backed and NATS-backed (etc.) implementations. That might happen in a later PR, but the track is laid.

[danielballan]

Skipping these tests on Windows, for now, would be acceptable IMO.

[danielballan]

This can be done at the module level, I believe. Less work for Future Us when we add more tests or if we add Windows support.

[gwbischof]

Sounds good, updated

[danielballan]

Suggested change

	redis:
	cache:

[danielballan]

Is this config respected? I suspect it might be ignored because I don't see any changes in tiled.config that would parse it and pass it along into the settings.

[gwbischof]

I fixed up this file. But it looks like this file is just for validating config files not something that gets loaded into settings?

[danielballan]

The low coverage number comes in large from some misconfiguration. Merging now, with a fix for that to follow, to hit our pre-announced outage window.