docs: describe the health endpoints the daemon actually serves

API.md and PROJECT_INDEX.md documented GET /health/liveness and
/health/readiness. Neither exists: the daemon serves /health, /healthz,
/ready and /live. Both response shapes were wrong too.

The OpenAPI description of /health did not match the served body either.
It named uptime instead of uptimeSeconds, described checks as booleans
where each is an object, omitted timestamp and system, and documented a
503 that /health never returns -- it always answers 200 and puts the
verdict in the body. All four endpoints are now specified against shared
HealthResponse, HealthCheck and SystemInfo schemas.

Also corrected: the scheduler check is no longer a stub, so it reports
degraded and names the jobs it refused; NewHealthChecker takes the
scheduler and the build's version rather than a hardcoded "1.0.0"; the
checker has a Stop; and waiting on ShutdownChan is not waiting for
shutdown to finish -- that is Done.

durationMs carries nanoseconds, not milliseconds: it serialises a Go
time.Duration. Stated wherever the field appears. Renaming the field
would break every existing consumer, so it stays.

Signed-off-by: Sebastian Mendel <github@sebastianmendel.de>

Author: CybotTM

docs/API.md

@@ -315,34 +315,91 @@ POST /api/scheduler/reload
 
 ## Monitoring
 
+Four endpoints, all unauthenticated:
+
+| Endpoint | Body | Status code |
+|---|---|---|
+| `GET /health` | full report | always `200` |
+| `GET /healthz` | full report (alias) | always `200` |
+| `GET /ready` | full report | `503` when `unhealthy`, else `200` |
+| `GET /live` | `OK` as plain text | always `200` |
+
 ### Health Check
 ```http
-GET /health/liveness
+GET /health
 ```
 
+`/health` answers `200` even while reporting a problem — the verdict is in the
+body, so a monitoring system can read *why*. Point a probe that must fail at
+`/ready` instead.
+
 **Response:**
 ```json
 {
-  "status": "healthy",
-  "timestamp": "2024-01-01T12:00:00Z"
+  "status": "degraded",
+  "timestamp": "2026-08-02T16:55:52Z",
+  "uptimeSeconds": 41.2,
+  "version": "v0.28.1",
+  "checks": {
+    "docker": {
+      "name": "docker",
+      "status": "healthy",
+      "message": "Docker 29.6.2 running with 25 containers",
+      "lastChecked": "2026-08-02T16:55:52Z",
+      "durationMs": 8332586
+    },
+    "scheduler": {
+      "name": "scheduler",
+      "status": "degraded",
+      "message": "1 configured job(s) are not scheduled and will not run: broken",
+      "lastChecked": "2026-08-02T16:55:52Z",
+      "durationMs": 3180
+    },
+    "system": {
+      "name": "system",
+      "status": "healthy",
+      "message": "System resources normal",
+      "lastChecked": "2026-08-02T16:55:52Z",
+      "durationMs": 553638
+    }
+  },
+  "system": {
+    "goVersion": "go1.26.0",
+    "goroutines": 24,
+    "cpus": 8,
+    "memoryAllocBytes": 3512976,
+    "memoryTotalBytes": 12730376,
+    "gcRuns": 1
+  }
 }
 ```
 
+`status` is the worst status among the checks. `version` is the running build,
+or `dev` when built without ldflags. Note that `durationMs` carries
+**nanoseconds** despite its name — it serialises a Go `time.Duration`.
+
+The `scheduler` check reports `degraded` and names every configured job the
+scheduler refused, so a job that never fires — an unparsable schedule, a
+duplicate name — is visible to monitoring rather than only in the startup log.
+
 ### Readiness Check
 ```http
-GET /health/readiness
+GET /ready
 ```
 
-**Response:**
-```json
-{
-  "ready": true,
-  "scheduler": "running",
-  "docker": "connected",
-  "database": "n/a"
-}
+Same body as `/health`. The status code carries the verdict: `503` when the
+overall status is `unhealthy`, `200` otherwise. `degraded` deliberately stays
+`200` — one job with a typo should not take a daemon out of rotation while its
+other jobs keep running.
+
+### Liveness Check
+```http
+GET /live
 ```
 
+Answers `200` with the plain-text body `OK` as long as the process serves HTTP.
+It runs no checks, so it never reports on Docker or the scheduler.
+
 ### Prometheus Metrics
 ```http
 GET /metrics

docs/PROJECT_INDEX.md

@@ -131,8 +131,9 @@ ofelia_circuit_breaker_*       # Circuit breaker states
 ```
 
 ### Health Endpoints
-- `/health/liveness`: Service availability
-- `/health/readiness`: Service readiness
+- `/health`, `/healthz`: Full health report, always 200 — the verdict is in the body
+- `/ready`: Same report, 503 when the overall status is `unhealthy`
+- `/live`: Plain-text `OK` as long as the process serves HTTP, runs no checks
 - `/metrics`: Prometheus metrics endpoint
 
 ## 🔌 API Reference

docs/QUICK_REFERENCE.md

@@ -319,7 +319,10 @@ GET    /api/jobs/removed       # Removed jobs
 
 # Config & Health
 GET    /api/config             # Current configuration
-GET    /health                 # Health check
+GET    /health                 # Full report, always 200 - verdict is in the body
+GET    /healthz                # Alias of /health
+GET    /ready                  # Same report, 503 when unhealthy
+GET    /live                   # Plain-text OK, runs no checks
 GET    /metrics                # Prometheus metrics
 ```
 

docs/TROUBLESHOOTING.md

@@ -438,7 +438,9 @@ SDK provider failed to connect to Docker: pinging docker: context deadline excee
 Docker API version negotiation timed out; continuing with default API version
 ```
 
-`/health` returns non-2xx within ~5 seconds; `/ready` reports `unhealthy` for the `docker` check.
+Within ~5 seconds the `docker` check turns `unhealthy`, which makes the overall
+status `unhealthy` and `/ready` answer **503**. `/health` still answers 200 —
+it always does — so read its body rather than its status code.
 
 `ofelia doctor` reports each Docker call individually (Ping ~5s, `HasImageLocally` ~5s per image).
 

docs/openapi.yaml

@@ -116,53 +116,79 @@ paths:
 
   /health:
     get:
-      summary: Health check endpoint
+      summary: Detailed health report
+      description: >-
+        Always answers 200, including while the report says `degraded` or
+        `unhealthy` — the status is in the body, not the status code, so a
+        monitoring system can read *why* rather than only that something is
+        wrong. Use /ready for a probe that fails the request.
       tags:
         - Monitoring
       security: []
       responses:
         '200':
-          description: Service is healthy
+          description: The current report, whatever its status
           content:
             application/json:
               schema:
-                type: object
-                properties:
-                  status:
-                    type: string
-                    enum: [healthy, degraded, unhealthy]
-                    example: healthy
-                  uptime:
-                    type: number
-                    description: Uptime in seconds
-                    example: 3600
-                  version:
-                    type: string
-                    example: "1.1.0"
-                  checks:
-                    type: object
-                    properties:
-                      docker:
-                        type: boolean
-                        example: true
-                      scheduler:
-                        type: boolean
-                        example: true
+                $ref: '#/components/schemas/HealthResponse'
+
+  /healthz:
+    get:
+      summary: Detailed health report (alias of /health)
+      tags:
+        - Monitoring
+      security: []
+      responses:
+        '200':
+          description: The current report, whatever its status
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/HealthResponse'
+
+  /ready:
+    get:
+      summary: Readiness probe
+      description: >-
+        Same body as /health, but the status code carries the verdict: 503 when
+        the overall status is `unhealthy`, 200 otherwise. `degraded` stays 200
+        on purpose — one job with an unparsable schedule should not take the
+        whole daemon out of rotation while its other jobs keep running.
+      tags:
+        - Monitoring
+      security: []
+      responses:
+        '200':
+          description: Ready — overall status is `healthy` or `degraded`
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/HealthResponse'
         '503':
-          description: Service unhealthy
+          description: Not ready — overall status is `unhealthy`
           content:
             application/json:
               schema:
-                type: object
-                properties:
-                  status:
-                    type: string
-                    example: unhealthy
-                  errors:
-                    type: array
-                    items:
-                      type: string
-                    example: ["Docker connection failed"]
+                $ref: '#/components/schemas/HealthResponse'
+
+  /live:
+    get:
+      summary: Liveness probe
+      description: >-
+        Answers as long as the process serves HTTP at all. It runs no checks,
+        so it never reports on Docker or the scheduler.
+      tags:
+        - Monitoring
+      security: []
+      responses:
+        '200':
+          description: The process is running
+          content:
+            text/plain:
+              schema:
+                type: string
+                example: OK
 
   /jobs:
     get:
@@ -327,6 +353,86 @@ components:
       description: Session cookie authentication
 
   schemas:
+    HealthCheck:
+      type: object
+      description: One individual check within the health report.
+      properties:
+        name:
+          type: string
+          example: scheduler
+        status:
+          type: string
+          enum: [healthy, degraded, unhealthy]
+          example: degraded
+        message:
+          type: string
+          description: Omitted when empty.
+          example: "1 configured job(s) are not scheduled and will not run: broken"
+        lastChecked:
+          type: string
+          format: date-time
+        durationMs:
+          type: integer
+          format: int64
+          description: >-
+            How long the check took, in **nanoseconds** despite the field name —
+            it serialises a Go time.Duration, which marshals as nanoseconds. A
+            local Docker ping reports roughly 8000000 here, i.e. 8 ms.
+          example: 8332586
+      required: [name, status, lastChecked, durationMs]
+
+    HealthResponse:
+      type: object
+      description: >-
+        The body served by /health, /healthz and /ready. `status` is the worst
+        status among `checks`: any unhealthy check makes the whole report
+        unhealthy, otherwise any degraded check makes it degraded.
+      properties:
+        status:
+          type: string
+          enum: [healthy, degraded, unhealthy]
+          example: degraded
+        timestamp:
+          type: string
+          format: date-time
+        uptimeSeconds:
+          type: number
+          example: 3600
+        version:
+          type: string
+          description: >-
+            The running build's version, or `dev` when built without ldflags.
+          example: v0.28.1
+        checks:
+          type: object
+          description: Keyed by check name — `docker`, `scheduler` and `system`.
+          additionalProperties:
+            $ref: '#/components/schemas/HealthCheck'
+        system:
+          $ref: '#/components/schemas/SystemInfo'
+      required: [status, timestamp, uptimeSeconds, version, checks, system]
+
+    SystemInfo:
+      type: object
+      properties:
+        goVersion:
+          type: string
+          example: go1.26.0
+        goroutines:
+          type: integer
+          example: 24
+        cpus:
+          type: integer
+          example: 8
+        memoryAllocBytes:
+          type: integer
+          format: int64
+        memoryTotalBytes:
+          type: integer
+          format: int64
+        gcRuns:
+          type: integer
+
     Job:
       type: object
       properties:

docs/packages/core.md

@@ -130,6 +130,23 @@ type Scheduler struct {
 - `Start()`: Begin scheduling
 - `Stop()`: Graceful shutdown
 - `RunJob()`: Manual job trigger
+- `GetUnschedulableJobs()`: Jobs the scheduler refused, keyed by name with the
+  reason. A job whose schedule will not parse is configured but never runs, so
+  the health check reports it instead of leaving it in a startup log line. A
+  name that later registers clears its own entry, as does removing the job, so
+  a corrected config reloaded at runtime recovers without a restart. The
+  returned map is a copy.
+
+**Shutdown:** `ShutdownManager` runs hooks in priority groups, lowest first,
+each group completing before the next starts.
+
+- `ShutdownChan()` closes when shutdown **starts** — before the first hook runs.
+- `Done()` closes when every group has finished, including the path where a
+  hook overruns the timeout.
+
+Anything that ends the process must wait on `Done()`. Waiting on
+`ShutdownChan()` ends it while the hooks are still running, which leaves every
+group after the first killed in flight.
 
 ### Context