Add observable model operations UI

[dzarlax]

Summary

• add an Evals admin tab with free-check statuses, eval reports, and workload suite details
• persist explicit checking/running states before model operations call providers
• show progress indicators and details links on model catalog check/eval actions

Tests

• go test -count=1 ./...

Summary by CodeRabbit

Release Notes

• New Features

  • Added Evals page displaying model operations status, checks overview, and evaluation reports with detailed results
  • Added live status tracking for model checks and evaluation runs
  • Added new Evals navigation tab in admin interface

• Improvements

  • Enhanced check and evaluation UI with loading indicators and real-time state feedback
  • Display of operation timestamps, latency metrics, and detailed failure information

[coderabbitai]

Caution

Review failed

An error occurred during the review process. Please try again later.

📝 Walkthrough

Walkthrough

Adds a dedicated /evals admin page backed by expanded modelCheckStatus and modelEvalStatus schemas that track provider/model identifiers, lifecycle timestamps, and per-case results. Handlers gain concurrency control via TryLock, writing intermediate checking/running states before long operations. Normalization helpers backfill legacy cached data on read and write. A new evals.html template, updated layout/models-row partials, route registration, and tests complete the feature.

Changes

Evals Admin Page

Layer / File(s)	Summary
Expanded status schemas and normalization helpers internal/adminapi/model_checks.go, internal/adminapi/model_evals.go	modelCheckStatus gains Provider, ModelID, StartedAt, UpdatedAt fields. modelEvalStatus expands to include provider/model identifiers, suite name, lifecycle timestamps, status text, failure list, and raw results. Normalization helpers (normalizeSingleModelCheck, normalizeModelChecks, normalizeSingleModelEval, normalizeModelEvals) and status constructors (checkingModelCheck, runningModelEval, failedModelEval) are introduced to backfill and derive consistent state on read and write.
Concurrency control and intermediate state persistence in handlers internal/adminapi/handlers.go, internal/adminapi/model_evals.go	handleModelCheck acquires modelEvalMu.TryLock(), returning 429 if busy, then writes a checking status before probing. attachModelChecks skips policy overwrite for checking status. handleModelEval persists a running status inside the lock before executing the suite. runModelEval returns a fully populated modelEvalStatus. New handleEvals handler renders viewEvals with a 5-second timeout.
Route registration and template wiring internal/adminapi/server.go, internal/adminapi/templates.go	/evals route registered to handleEvals. viewEvals constant added and wired to templates/evals.html in the views map.
evals.html template internal/adminapi/templates/evals.html	Full evals view with Model Operations header/badges, Free model checks table (status/timestamps/latency/errors), Eval Reports table (per-case results/tool calls/failures/content previews), and Workload Suite section with expandable case details.
Layout CSS, nav tab, and models-row HTMX state updates internal/adminapi/templates/partials_layout.html, internal/adminapi/templates/partials_models_row.html	Adds running-state CSS, ops-summary/ops-badge classes with ok/warn/running variants, and an Evals nav tab. Check free, Run eval, and Run paid eval buttons gain HTMX loading indicators and disabled states during checking/running. model_eval_status broadens its render condition to CheckedAt OR Status.
Tests internal/adminapi/adminapi_test.go	TestTemplatesParse extended with a viewEvals entry. Three new tests: TestEvalsPageRendersChecksReportsAndSuite (HTML output assertions), TestModelChecksPersistCheckingBeforeProbe (checking→free_verified state machine), TestModelEvalLoadLegacyAndDetailedReports (legacy normalization and detailed result round-trip).

Sequence Diagram(s)

sequenceDiagram
    participant Browser
    participant handleModelCheck
    participant modelEvalMu
    participant saveModelCheck
    participant modelProbe

    Browser->>handleModelCheck: POST /models/check
    handleModelCheck->>modelEvalMu: TryLock()
    alt already locked
        handleModelCheck-->>Browser: 429 another model operation is already running
    else lock acquired
        handleModelCheck->>saveModelCheck: checkingModelCheck(provider, modelID)
        saveModelCheck-->>handleModelCheck: ok
        handleModelCheck->>modelProbe: probe(provider, modelID)
        modelProbe-->>handleModelCheck: latency / error
        handleModelCheck->>saveModelCheck: free_verified status + latency
        handleModelCheck-->>Browser: 200
        handleModelCheck->>modelEvalMu: Unlock()
    end

Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

• Dzarlax-AI/personal_assistant#7: Directly touches handleModelCheck, attachModelChecks, and free-model check-status persistence — the same code paths this PR extends with concurrency control and the checking intermediate state.

Poem

🐇 Hoppity-hop through the eval gate,
The bunny checks models—no time to wait!
A "checking" flag planted before the probe runs,
"Running" persisted while the eval suite spins.
Legacy shapes normalized with a flick,
New badges glow green (or warn—oh that's sick).
🥕 The evals tab blooms in the admin nav!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 6.67% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The pull request title 'Add observable model operations UI' directly and clearly describes the main change: adding user interface components for observing model operations.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch codex/observable-model-ops

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 709a17f525

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Clear persisted running state after request cancellation

After this write succeeds, the only path that replaces running uses ctx derived from r.Context() below. If the browser or proxy cancels the htmx request while the eval is in flight, the final saveModelEval(ctx, ...) runs with a canceled context and PutSetting fails, leaving this model permanently marked running in kv_settings and the UI disables the eval button; the /models/check path has the same pattern for checking. Use a detached bounded context for the final status update or expire stale running/checking records.

Useful? React with 👍 / 👎.