Yarn provisioner; can ResponseManager's connection handling using HTTP, instead of raw TCP sockets?

[MaicoTimmerman]

In our deployment, we are deploying many different notebook servers inside a Kubernetes cluster, behind a JupyterHub. In the case of kernel startup, the connection info communicated back using a raw socket. I'm wondering if there is a rationale for the ResponseManager requiring a raw level 4 TCP socket?

The fact that it's a raw socket, makes it such that we can not utilize level 7 features, such as proxy passing based on routes. We'd have to proxy based on ports, which complicates many things. Without going into specifics on our cluster, I'm trying to understand is there anything that withholds us from bringing this logic into API call? In those cases, routing would be taken care entirely using the JupyterHub.

edit: Now that I'm thinking about it, it would also simplify the security aspect, as kernels no longer have to deal with encrypting the connection info payload.

[kevin-bates]

Hi @MaicoTimmerman - thank you for opening this issue and I sincerely apologize for the lack of any response (and can't guarantee it will happen again - to be honest).

My knowledge level in this particular area is lacking, but perhaps if you submitted a pull request, I could study what is being done. By all means making GP more accessible and secure is goal we should strive to achieve. Thank you.