starknet_transaction_prover: /health returns 503 when service is saturated

[avi-starkware]

Adds SaturationMonitor (shared by ProvingRpcServerImpl and
HealthLayer) that tracks whether the concurrency semaphore has been
continuously rejecting proving requests. Once that has held for the
configured window (health_max_saturated_ms, default 10s), /health
returns 503 with an opaque body so load balancers can drain the pod
before in-flight requests start failing.

Co-Authored-By: Claude Opus 4.7 (1M context) noreply@anthropic.com

[cursor]

PR Summary

Medium Risk
Changes load-balancer health behavior for the prover; mis-tuned health_max_saturated_ms could drain pods too early or too late, but scope is limited to health/concurrency signaling with tests.

Overview
Introduces SaturationMonitor shared between the proving RPC handler and HealthLayer: concurrency rejections start a timer; a successful permit acquire clears it. After health_max_saturated_ms (default 10s, CLI/env HEALTH_MAX_SATURATED_MS) of continuous rejections, GET /health returns 503 with an opaque {"status":"unhealthy","reason":"saturated"} body so load balancers can drain the pod; recovery returns 200.

Wiring: main builds the monitor and a configured HealthLayer, passes both into ProvingRpcServerImpl and start_server / TLS (middleware macro now takes an explicit health_layer instead of a default HealthLayer).

^{Reviewed by Cursor Bugbot for commit b321f22. Bugbot is set up for automated code reviews on this repo. Configure here.}

[avi-starkware]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• starknet_transaction_prover: document observability surface in README #14203
• starknet_transaction_prover: origin-level breadcrumbs for validation errors #14172
• starknet_transaction_prover: /health returns 503 when service is saturated #14171 👈 (View in Graphite)
• starknet_transaction_prover: bump prover_panics_total on panic hook fire #14170
• starknet_transaction_prover: HTTP request count + latency + in-flight metrics #14169
• starknet_transaction_prover: proving-job duration + outcome metrics #14168
• starknet_transaction_prover: Prometheus /metrics endpoint with build_info #14167
• starknet_transaction_prover: global panic hook + graceful SIGTERM shutdown #14166
• starknet_transaction_prover,tower_ohttp: OHTTP-unlinkable request-id for decapsulated content #14222
• starknet_transaction_prover: per-request structured log with request-id #14165
• starknet_transaction_prover: unify HTTP middleware stack via macro #14286
• main-v0.14.3

This stack of pull requests is managed by Graphite. Learn more about stacking.

[reviewable-StarkWare]

This change is 

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit d3f1139. Configure here.}