Websocket support

[vstreame]

(Note: This PR is rebased on #12 just to have some test infra in place. We'll need to merge that before this one can land)
(Note 2: Cannot get tests to pass until we get a new version of nerf out that works with HTTP 3.0 APIs. PR: lpil/nerf#3)

What

This is an experimental PR to add Websocket support to the Gleam API. The main change is that when dealing with websockets we more than one "handler". Not only do we need the initial HTTP handler, but we also needs handlers for

• Initializing the websocket connection
• Whenever a new Frame comes in
• Whenever an Erlang message is send to the PID

As well as we need a piece of state to share between all of these handlers.

Please view the tests for the proposed API.

Why

This would unlock Gleam as a language to build "real-time" applications with. Which would be a very neat use-case!

How

This PR is mostly to discuss the proposed API, but here is also how it currently implements that API (which I'm less sure of). Instead of passing a single handler function to init/1 we instead pass down a map with known keys.

• handler - The function that is passed a http/request.{Request} and returns a WSResponse. A WSResponse is either a wrapper over a normal http/response.{Response} or it's an Upgrade(init_state). When Upgrade is returned this is signal to us that the developer wants to upgrade the request to a persistant websocket connection.
• on_ws_init - This handler is used to grab any relevant pids and/or return a frame immediately to the client
• on_ws_frame - Handler called whenever a new WS frame is sent from the client
• on_message - Handler called whenever pid receives as message from Erlang

Each handler is expected to tell us if we want to send any Frames in response as well as what the new state should be updated to.

Links

• Cowboy docs on Websockets: https://ninenines.eu/docs/en/cowboy/2.9/guide/ws_protocol/

[lpil]

Fab work! Really cool stuff.

I think we're going to need a more flexible API for the websocket service. Currently a message of some kind comes in from the websocket client and then the server decides to either reply or not. This will work for many cases, but other times we need to be able to send a message to the client without the client sending anything. For example, notifying the user of a website when they have recieved a new direct message from another user.

To implement this I think we would want to model the service as an actor that accepts messages. This is an issue as I intend to make some large breaking changes in the OTP/Erlang libraries after finishing with this iteration of LSP work, and I want that to have as small an impact on other libraries such as this one.

On the other hand, this is fantastically useful! Even if we cannot today solve all the possible websocket use cases that doesn't mean we don't want this. If we don't have any good ideas I'd suggest we publish this as a second package, and perhaps later merge them together.

I've also got some very minor stylistic comments around naming but that can be discussed later :)

What are your thoughts?

[vstreame]

@lpil Thanks for the feedback! So disclaimer, I'm still pretty unfamiliar with gleam/otp still, but it seems like this API may still work by hooking into the websocket_init/1 handler as described here: https://ninenines.eu/docs/en/cowboy/2.9/guide/ws_handlers/#_post_upgrade_initialization

I would expect that would be the place that one one hook in to register their new self() pid with some sort of event bus / connection tracker.

Then that connection tracker can send normal messages to the pid which will trigger the on_message handler which will will allow us to then trigger a Frame to be sent to the client?

Does that all make sense? Or am I'm misunderstanding how gleam/otp currently works?

[lpil]

You've got the general idea, and that is largely how cowboy websockets work today as I understand. It becomes more complex when you consider the types of messages send to the websocket service process. If there's a handle_message callback in the websocket service abstraction the type of the message must be Dynamic as it would be called for any possible message, and in that case we have no type information. To have typed messages the abstraction would need to be based around gleam_otp's (soon to be substantially modified) channels, as they are how we link type information to Erlang messages.

The API would likely need to look less like a gen_server and more like gleam_otp's actor. How easy it would be to do this within the constraints of cowboy's APIs I'm unsure.

[vstreame]

@lpil Ah, yes totally get it now when it comes to types. I think what I'll do is publish this as a separate experimental package for now then just so I can start playing around with real time apps with Gleam myself. Then once those new OTP APIs exist we can create a better more official API for this package!

[vstreame]

FYI, got tests all working over in the new repository here: https://github.com/vstreame/gleam_cowboy_websockets

[lpil]

Thinking about this more, perhaps it's OK that the typing is not amazing here. If that's a limitation of cowboy it's good motivation to make a Gleam websocket server that is well typed and hopefully even faster than cowboy!

[vstreame]

@lpil I've also spent the last week learning more about gleam/otp and how the actor and process modules work better now. I may be able to take a second crack at this to better use the actor APIs