Skip to content

fix(ui): distinguish AI Runway integration from runtime install status#294

Open
robert-cronin wants to merge 4 commits into
kaito-project:mainfrom
robert-cronin:fix/runtime-installation-status-244
Open

fix(ui): distinguish AI Runway integration from runtime install status#294
robert-cronin wants to merge 4 commits into
kaito-project:mainfrom
robert-cronin:fix/runtime-installation-status-244

Conversation

@robert-cronin

Copy link
Copy Markdown
Member

Description

On Settings → Runtimes, the page didn't distinguish AI Runway's integration with a runtime (the provider shim's heartbeat) from the underlying runtime being installed (operator + CRDs in the cluster). When a user looked at KAITO they couldn't tell whether to click Install or troubleshoot the AI Runway side — a single mixed signal was driving multiple unrelated states.

This PR splits those concepts into two clearly-labelled, independently-evaluated statuses, with plain-language copy aimed at users who aren't familiar with Kubernetes (per AGENTS.md).

AI Prompt (Optional)

🤖 AI Prompt Used
Please work on https://github.com/kaito-project/airunway/issues/244 — the dashboard
runtime installation status should distinguish "AI Runway provider integration
registered" from "underlying runtime installed", with plain-language UI copy and
test coverage for the regression scenario described in the issue.

AI Tool: Claude (GitHub Copilot CLI)

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • 🎨 UI/UX improvement

Related Issues

Fixes #244

Changes Made

Shared types (shared/types/settings.ts, shared/types/installation.ts)

  • Added three optional fields to RuntimeStatus and InstallationStatus:
    • shimRegistered — an InferenceProviderConfig exists for the runtime
    • shimConnectedstatus.ready === true and lastHeartbeat is within 3 minutes (3× the shim's 1-minute interval)
    • shimLastHeartbeat — raw timestamp for UI display
  • All fields are optional, so older backends remain wire-compatible with this frontend.

Backend (backend/src/services/kubernetes.ts, backend/src/routes/installation.ts)

  • Added exported computeShimStatus() helper with a SHIM_HEARTBEAT_FRESHNESS_MS = 3 * 60 * 1000 window. Boundary uses <=; future heartbeats are treated as fresh (defensive against clock skew).
  • Wired into getRuntimesStatus() and /providers/:providerId/status so both surfaces emit the new fields.

Frontend (frontend/src/pages/SettingsPage.tsx)

  • New AI Runway integration row on both the runtime card and the detail panel, rendered only for runtimes that require a CRD (LLM-D / vLLM keep their existing CRD-less flow unchanged).
  • Status reads Connected / Not heartbeating in plain language, with a human-friendly "Xs/m/h ago" age.
  • Detail panel adds a clarifying sentence: "Status of the underlying {Runtime} operator and CRDs in the cluster. Use the install button below to deploy them if missing."
  • The word "shim" never appears in user-facing UI text.

Testing

  • Unit tests pass (bun run test) — 697/697 backend, 121/121 frontend
  • Manual testing performed
  • Tested with a Kubernetes cluster (kind cluster with the AI Runway shim + KAITO toggled in/out via helm upgrade --install / helm uninstall)
airunway-issue-244

New test coverage:

  • backend/src/services/kubernetes-runtime-status.test.ts — +7 unit tests for computeShimStatus (3-min boundary, future heartbeats, missing fields, invalid timestamps) + 2 integration tests in getRuntimesStatus (regression for the user's exact scenario + stale-heartbeat case)
  • backend/src/routes/installation.test.ts — +1 test verifying the installation route exposes the new fields
  • frontend/src/pages/SettingsPage.test.tsx — +2 tests covering the Connected and Not heartbeating rendering paths

Manual repro of the original #244 scenario:

# 1. Install KAITO via the dashboard's Install button, then tear it down:
helm uninstall -n kaito-workspace kaito-workspace

# 2. Freshen the shim heartbeat so it's "connected":
kubectl patch inferenceproviderconfig kaito --subresource status --type=merge \
  -p "{\"status\":{\"lastHeartbeat\":\"$(date -u +%Y-%m-%dT%H:%M:%SZ)\",\"ready\":true}}"

Reload Settings → Runtimes. The KAITO card now shows Not Installed with AI Runway integration: Connected alongside it, and the Install KAITO button is surfaced — the user knows exactly what to do.

All four meaningful state combinations were walked through end-to-end during manual testing:

Integration Underlying runtime Result
✗ Not heartbeating ✗ Not Installed Both red — initial state
✗ Not heartbeating ✅ Installed Mixed — proves they're independent
✓ Connected ✅ Installed Both green — fully healthy
✓ Connected ✗ Not Installed The original #244 bug — now unambiguous

Checklist

  • My code follows the project's style guidelines
  • I have run bun run lint (pre-existing ESLint config issue in the repo prevents the script from running — unrelated to this change)
  • I have added tests that prove my fix works
  • New and existing unit tests pass locally
  • I have updated documentation if needed (no doc changes required — UI-only fix)
  • My changes generate no new warnings

Screenshots

After (original #244 scenario — shim heartbeating, KAITO operator/CRD absent):

The card shows Not Installed with AI Runway integration: Connected clearly separated, and the detail panel offers the Install KAITO button alongside the explanatory copy.

Additional Notes

  • Heartbeat freshness window is 3× the shim's 1-minute heartbeat interval (constant SHIM_HEARTBEAT_FRESHNESS_MS in kubernetes.ts) — easy to tune in one place if the heartbeat interval changes.
  • Backwards-compatible: if an older backend doesn't emit shimRegistered, the integration row simply doesn't render and the page degrades to the previous behaviour.

kaito-project#244)

When a provider integration (e.g. the KAITO shim) is registered but the
underlying runtime operator and CRDs are not installed, the Settings →
Runtimes page only reflected the underlying runtime probe results,
leaving users confused about which side of the integration was actually
healthy.

Surface the integration state independently so the UI can show both at
once:

  * Add 'shimRegistered', 'shimConnected', 'shimLastHeartbeat' to the
    shared RuntimeStatus and InstallationStatus types.
  * Compute shim status in the backend from the InferenceProviderConfig
    'status.ready' and 'status.lastHeartbeat' fields, treating the
    integration as connected only when a heartbeat has arrived within
    3x the controller heartbeat interval (3 minutes).
  * Render an 'AI Runway integration: Connected / Not heartbeating'
    row on each runtime card and in the installation detail panel for
    runtimes that require an upstream operator (LLM-D and vLLM keep
    their existing CRD-less treatment unchanged).
  * Add a short clarifying sentence in the detail panel describing
    that the CRD / Operator rows refer to the underlying runtime.

The reported behaviour (PR kaito-project#175) of accurately showing red X icons and
the Install button when the upstream operator is missing is preserved;
this change makes it explicit why the integration can still report
healthy while the runtime itself is not yet installed.

Tested with backend (697/697) and frontend (121/121) suites; added
unit tests for the new heartbeat-freshness helper, a regression test
for the user's scenario, and frontend tests covering the new card and
detail-panel rows.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@robert-cronin robert-cronin requested a review from a team as a code owner May 26, 2026 01:52
Copilot AI review requested due to automatic review settings May 26, 2026 01:52

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses confusion on Settings → Runtimes by separating the status of the AI Runway provider integration (shim heartbeat) from the status of the underlying runtime installation (operator + CRDs), so users can tell whether to click Install or troubleshoot integration connectivity.

Changes:

  • Added optional shimRegistered, shimConnected, and shimLastHeartbeat fields to runtime and installation status types and API responses.
  • Implemented computeShimStatus() in the backend and wired it into both runtimes status and provider installation status endpoints.
  • Updated the Settings UI to show a distinct “AI Runway integration” status line and added regression tests for the #244 scenario.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
shared/types/settings.ts Extends RuntimeStatus with optional shim integration status fields.
shared/types/installation.ts Extends InstallationStatus with optional shim integration status fields.
backend/src/services/kubernetes.ts Adds computeShimStatus() and includes shim fields in getRuntimesStatus().
backend/src/services/kubernetes-runtime-status.test.ts Adds unit tests for computeShimStatus() and regression/integration tests for getRuntimesStatus().
backend/src/routes/installation.ts Exposes shim fields from /providers/:providerId/status.
backend/src/routes/installation.test.ts Tests that the installation status route includes shim fields.
frontend/src/pages/SettingsPage.tsx Adds “AI Runway integration” status rendering (card + detail panel) and heartbeat age formatting.
frontend/src/pages/SettingsPage.test.tsx Adds UI tests covering Connected and Not heartbeating rendering paths and the #244 regression scenario.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread frontend/src/pages/SettingsPage.tsx
Comment thread frontend/src/pages/SettingsPage.tsx Outdated
@robert-cronin robert-cronin changed the title fix(ui): distinguish AI Runway integration from runtime install statu… fix(ui): distinguish AI Runway integration from runtime install status May 26, 2026
Copilot AI review requested due to automatic review settings June 9, 2026 10:13

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated no new comments.

Signed-off-by: Robbie Cronin <robert.owen.cronin@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Dashboard runtime installation status

2 participants