-
Couldn't load subscription status.
- Fork 54
Description
Thanks for creating this spark-ui-proxy. In my case, I have a spark standalone cluster with spark master configured with zookeeper to achieve spark master high availability. There are couple of nodes running spark master service in ALIVE (Active) and STANDBY mode.
I had to run this spark-ui-proxy on both the master instances. When I use a apache httpd virtual hosting for load balancing this, requests are not sticking to ALIVE spark master, instead it is going in round robin fashion as both the spark-ui-proxy servers are listen on a port and responding with a status!
To overcome this, I am trying to depend on mod_proxy_hcheck of apache httpd which will disable the balancer member based on the GET calls response. Unfortunately , with this spark-ui-proxy I am getting OK status on both the spark master which again takes me back to round-robin issue between ALIVE and STANDBY master nodes.
Within your code , I see below for health checking, which is returning OK in both the cases:
Add an health checking endpoint.
if self.path in ["/healthz"]:
self.send_response(code=200)
self.send_header("Content-type", "text/plain")
self.end_headers()
self.wfile.write("OK")
return
Is it possible to return ACTIVE or STANDBY responses which will really help in achiving HA with spark-master using this spark-ui-proxy ?
Appreciate any help.