Client support for encryption

[seliopou]

The lwt and async client implementations currently do not support encryption. Add support for SSL and/or TLS without adopting the approach that conduit did, i.e., build hacks and direct dependencies.

Instead, investigate and, if possible, implement a first-class module interface where different implementations can be passed in when constructing the client. That way, people are free to use the http state machine and runtime while being able to select the encryption library of their own choosing.

Alternatively, if ocaml-tls support is mature enough, then it'd be simpler and preferable to just adopt that library as the only option.

[avsm]

We'll have TLS 1.3 support in ocaml-tls shortly, but I do think that it would be better to have some sort of a first-class module interface here if possible. The reason is that there is a fairly vast design space of TLS options in libssl that some consumer may eventually want, but also that there are non-TLS encryptions options emerging (e.g. Noise). The alternatives are not yet standardised, but a first-class module being plumbed through would at least make it plausible to prototype them against httpaf.

[hannesm]

how would a first-class module interface roughly look like, if I may ask?

[seliopou]

Not sure, but I can imagine two possibilities. One would roughly take over the socket, so that you would have to do read and write calls through the first class module interface. The other would be something like a state machine that just takes in plain text and spits out cypher text (and vice versa). With this interface you would do your own read calls and pass it to the first class module, and the whatever writes it produced you would pass to the socket.

I imagine that the latter wouldn't work for SSL, but may work for TLS.

[hannesm]

@seliopou thanks for your answer. i'm interested in the design space, having seen conduit (which imho tries to solve too many problems at once, and does not deal with server cert verification (neither supports client certificates last time i looked)) and mirage-flow (which is only a read/write/close interface, the connect/accept (i.e. how to construct a flow) needs to be provided externally - which may be a fair deal since construction thereof depends on the flow) as deployed solutions so far, and @dinosaure tuyau as a wip.

it is still not entirely clear to me how yet another solution would look like code-wise -- what the requirements are, esp. thinking about websockets and "authenticating with a tls certificate at a later point in the session" (i.e. when accessing /admin). not clear whether this is actually in scope.

[dinosaure]

Currently, I did a proof-of-concept with Mirage and Httpaf (and others stuffs). It will be a good end-to-end minimal project to replace conduit by tuyau and try to get the right approach where conduit is like an octopus in many parts of the Mirage eco-system.

[dinosaure]

I did a little work to port HTTP/AF with tuyau and support MirageOS and ocaml-tls. The signature is close to the idea given by httpaf-lwt-unix where we start an infinite loop as Lwt_io.establish_server_with_client_address does. The work is available here: https://github.com/dinosaure/paf-le-chien/blob/master/lib/paf.mli

[dinosaure]

Just to come back, paf.0.0.1 was release which has the support of TLS (with ocaml-tls) on the server side. The client side is able to use ocaml-tls (with ca-certs if the user wants) or lwt_ssl as far as an implementation which respects Mirage_flow.S exists.

The project is mostly focused on MirageOS of course but, by definition, it's compatible with UNIX.

[hannesm]

as a note here, we had the need for such a thing -- please take a look at https://github.com/roburio/httpaf-lwt-client if you're interested. Please note that this does not use functors, and only supports the lwt backend. I'm sure we could merge that back into http/af (-lwt-unix) and have an async client in the same style, if you like.

[hannesm]

Instead of a first-class module interface, what is used is a variant [ `Tls of Tls_lwt.Unix.t | `Plain of Lwt_unix.file_descr ] and match in read / write / close.