Skip to content

feat(models): add model health check support#870

Merged
leoguillaume merged 14 commits into
mainfrom
362-model-health-check-support
May 19, 2026
Merged

feat(models): add model health check support#870
leoguillaume merged 14 commits into
mainfrom
362-model-health-check-support

Conversation

@leoguillaume
Copy link
Copy Markdown
Member

@leoguillaume leoguillaume commented May 18, 2026

Overview

Adds per-model health status for routers the caller can access, using Redis provider metrics (normalized latency history + inflight) and a Little’s-law-style overload indicator (green / yellow / red).

  • New GET /health/models endpoint (authenticated) returning [{ "id": "", "status": "green|yellow|red" }]
  • GetHealthModelsUseCase: filters by user access, skips routers without providers, aggregates worst status across providers
  • Metrics refactor: ModelMetricsLogger → ProviderMetricsLogger with RedisProviderMetricsLogger; provider-level metric logging in model clients
  • ProviderRepository.get_all_providers(); Router.has_providers helper

Breaking changes:

  • No breaking changes
  • This PR contains breaking changes (explain below)

Check lists

Review checklist

Before requesting a review, please take a moment to confirm that the following aspects have been considered and addressed. This section helps ensure the PR is ready for review, safe to merge, and deployable. If any items are left unchecked, please add a brief explanation for context.

  • Updated or added documentation
  • Updated or added unit tests
  • Updated or added integration tests
  • No debug logs or commented-out code left
  • No secrets or environment variables committed in clear text
  • Code is linted and formatted using the project pre-commit hooks

If api/sql/models.py has been modified, please confirm that the following steps have been completed:

  • Alembic migration has been generated
  • Alembic migration upgrade has been tested locally
  • Alembic migration downgrade has been tested locally

Deployment checklist

For each of the following items, please confirm if the PR concerns the deployment of the changes:

  • Alembic migration has been generated
  • Configuration file has been modified
  • Environment variables have been modified

If new or updated environment variables are required, please list them here, otherwise delete this part. If other special deployment steps are required, please describe them here, otherwise delete this part.

Additional Notes

Please provide any additional information or context that may be relevant to this PR, otherwise delete this part. This could be any specific areas you would like the reviewers to focus on during their review of this PR (complex logic, risky changes, performance-sensitive code, etc.)

@leoguillaume leoguillaume linked an issue May 18, 2026 that may be closed by this pull request
@leoguillaume leoguillaume changed the title Add model health check support feat(models): add model health check support May 18, 2026
@leoguillaume leoguillaume force-pushed the 362-model-health-check-support branch from 117141a to 312ab52 Compare May 18, 2026 15:31
@leoguillaume leoguillaume marked this pull request as ready for review May 18, 2026 15:32
@leoguillaume leoguillaume self-assigned this May 18, 2026
@leoguillaume leoguillaume merged commit 9981a1c into main May 19, 2026
@leoguillaume leoguillaume deleted the 362-model-health-check-support branch May 19, 2026 15:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Model health check support

1 participant