task/WG-371: improve status health check endpoint

[nathanfranklin]

Overview

Adds an authenticated /status/complete endpoint to validate worker, Redis, and DB connectivity from within the worker network. Returns 200 if all components are healthy, 503 if any fail. This will be used in Gatus monitoring. This is primarily to address firewall issues on deployed VMs where workers cannot reach a needed resource (like redis or database). So to catch things like the recentn firewall issue where users cannot reach Redis on the geoapi-services machine

Related Jira tickets

• WG-371

Summary of Changes

• Add /status/complete authenticated endpoint that submits a Celery task checking Redis and DB connectivity from the worker network
• Health check task submitted at max priority to avoid being blocked by queued default tasks. So when high load for our default queue this could also trigger but for the time-being this should help us.

Testing Steps

1. Start geoapi locally (make start)
2. Log in to Hazmapper at http://localhost:8888
3. Navigate to http://localhost:8000/status/complete/ — should return {"overall":"ok","components":{"redis":{"status":"ok","detail":null},"database":{"status":"ok","detail":null}}}

Notes

• This endpoint is intended for use in Gatus monitoring of deployed services. I'll add there next.

[taoteg]

LGTM! More monitoring FTW!