[feature]: add hierarchical tenant/client rate limits, cache quotas, and QoS

[xiaofan-luan]

Problem / motivation

A single tenant or client can consume gateway bandwidth, origin-fill bandwidth, NVMe IOPS, connection slots, and the entire cache capacity. Request-QPS limiting alone does not prevent noisy-neighbor failures: one large scan may use few requests while exhausting bytes and evicting every other tenant's working set.

Limits must apply consistently across replicated gateways/workers and use an authenticated identity, not source IP or a caller-supplied tenant header.

Proposed solution

Add hierarchical QoS and quota enforcement:

cluster -> tenant -> instance/namespace -> client/principal -> operation

Independent dimensions:

• request rate and burst
• response bytes/sec
• concurrent requests/connections
• origin-miss/fill concurrency and bandwidth
• cache capacity/resident bytes
• queued work and queue time

Additional behavior:

• Separate budgets for cache hits and origin misses.
• Tenant-aware admission and eviction.
• Guaranteed minimum plus shared elastic capacity.
• Weighted fair scheduling; foreground reads outrank prewarm/background fills.
• Stable S3-compatible overload responses such as 503 SlowDown.
• Bounded queues and circuit-breaker integration.

Scope

• Trusted principal-to-tenant/client mapping.
• Per-replica versus cluster-wide accounting contract.
• Dynamic configuration, dry-run, and rollback.
• Metrics and management views without object-key cardinality.
• Protection for gateway, worker, NVMe, network, and origin resources.

Non-goals

• Use source IP as the primary client identity.
• Promise perfectly precise global limits at the expense of availability without an explicit design decision.
• Make rate limiting the only cache-admission policy.

Acceptance criteria

• One tenant cannot exceed its configured cache capacity or fill-bandwidth share.
• Large-range traffic is limited by bytes/concurrency, not only QPS.
• Limits remain bounded when gateway/worker replica counts change.
• Cached hits and origin misses have independently configurable budgets.
• Prewarm cannot starve foreground reads.
• Throttled S3 requests return documented retryable errors and are observable by tenant/client/operation/reason.
• Multi-tenant load tests demonstrate configured fairness under concurrent scans, hot reads, cold misses, and origin throttling.

Design alignment

This splits the multi-tenancy, quota, and QoS work identified in roadmap #274 into a concrete resource-control contract.

Related: #274, #440