Skip to content

Commit 84ca5a6

Browse files
authored
feat: wire Learning Agent (post-close retrospective + estimator feedback loop) (#183)
* fix(rbac+estimation): repair count_active_by_role GROUP BY and harden every estimate_bud_dates caller The dynamic-role refactor introduced count_active_by_role with a parameterised CASE in both SELECT and GROUP BY. SQLAlchemy compiled each call to _effective_role_case() as a fresh expression, so the two CASEs emitted distinct bind parameters and Postgres rejected the query with "column roles.scope_type must appear in the GROUP BY clause". That GroupingError cascaded through get_role_capacity into estimate_bud_dates and crashed every caller. The original failure was then masked as InFailedSQLTransactionError on the next innocent statement because every "except Exception" around estimation swallowed the error without rolling back, leaving the connection in an aborted state. Two-part fix: 1. Repository: hoist the CASE into a single labelled expression so SELECT and GROUP BY reference the same object. Postgres accepts GROUP BY by output column alias, sidestepping the expression-identity trap. 2. Every estimate_bud_dates caller that ran on a shared session is now wrapped in db.begin_nested() (Postgres SAVEPOINT). A query failure inside the estimator only rolls back its own writes; the outer txn stays alive and prior flushed writes survive. Same session, same connection — no row-lock conflict with the outer's pending writes. Each except now logs with exc_info=True so the real traceback is not lost again. - handle_prd_result: explicit flush of linked-feature inserts before the savepoint so an autoflush inside it cannot scope them to the savepoint and lose them on rollback. - handle_tech_arch_result: existing flush already anchored tech_spec_md + impacted_repos; only the swallow shape changed. - handle_testing_result: moved the trailing flush above the savepoint for the same anchoring reason. - check_all_prs_merged: create_agent_task_for_stage already commits internally, so no extra flush; SAVEPOINT still isolates the webhook's trailing commit from estimator failure. - code_review_override endpoint: outer was already committed earlier in the handler, so estimation safely runs in a fresh AsyncSessionLocal without a row-lock conflict. Inline imports in the touched handlers were hoisted to top of file per the project's import policy. Verified end-to-end against live Postgres on the failing bud: - count_active_by_role / get_role_capacity / estimate_bud_dates all succeed. - Forced SQL error inside db.begin_nested() raises cleanly; outer txn stays valid; subsequent outer query sees the pre-savepoint flush. - Full handler flow (outer flush -> SAVEPOINT estimate incl. LLM call -> outer rollback) completes in 5.7s with no deadlock. - All four triggers (prd_completed, tech_arch_completed, testing_completed, prs_merged) pass. ruff + mypy clean on the four modified files. Signed-off-by: Arun Rajkumar <mickyarunr@gmail.com> * chore(format): apply ruff format on the estimation-resilience edits The PR-181 commit passed `ruff check` locally but not `ruff format --check` — CI's separate format gate flagged three files. Pure whitespace: ruff collapses multi-line calls / logger.warning args that fit on one line. No behavioural change. lint, mypy, and the behaviour-verifying live tests from the original commit still pass. Signed-off-by: Arun Rajkumar <mickyarunr@gmail.com> * fix(settings): wire connectionsLoaded flag so UAT toggle survives refresh The Pinia settings store's emptyState() always populates connections.budStages with default { uatEnabled: true }, so the lazy-fetch guard `if (!settingsStore.connections.budStages) fetchConnections()` never fired on cold page load. On a hard refresh of BUDDetail / BUDBoard / RepoList / SettingsQAAutomation the views used the stale default and ignored the saved-off backend value, making UAT reappear after the user had explicitly disabled it. Add a connectionsLoaded ref to the settings store that flips true only on successful fetchConnections / saveConnections. Switch the four broken guards to gate on !settingsStore.connectionsLoaded. BUDBoard had no fetch call at all on mount; add one. Verified end-to-end: disable UAT in Settings, hard-refresh each view, UAT no longer reappears. Re-enable, all surfaces show it again. Signed-off-by: Arun Rajkumar <mickyarunr@gmail.com> * feat(metrics): add feature_learnings.metrics JSONB column Single versioned envelope to hold the structured per-BUD learning metrics (original_estimated_days, phase_metrics, contributors, parallelism_score) that the Learning Agent writes on close and the Learnings tab + estimator rollup consume. One JSONB column rather than three keeps migrations cheap as the schema evolves. The data is read whole (LLM prompt + tab + rollup), never queried into, so a column-per-shape would just create churn. Migration is autogenerated. Applied locally; head is af5af83d4327. Signed-off-by: Arun Rajkumar <mickyarunr@gmail.com> * feat(metrics): compute and persist per-BUD learning metrics on close Centralizes the post-close FeatureLearning write in a new bud_metrics service hooked into on_bud_closed(). The legacy 3-field writer in feature_lifecycle._record_feature_learning is deleted; the new pipeline also captures per-phase actuals vs original estimates, per-contributor breakdown, and a development-window parallelism score, persisted as a versioned JSONB envelope in feature_learnings.metrics. Trigger: fires only when bud.status == CLOSED, so XP/SP/scan continue to run on PROD as before while the full lifecycle metrics wait for the BUD's final state (manual close or auto-close after all impacted repos ship). Idempotency: FeatureLearningRepository.get_for_bud short-circuits the whole compute when a row already carries a non-null metrics envelope, so PROD->CLOSED double-fire and webhook re-deliveries converge on a single persisted row. Structure stays under the project's ~200-line file cap by splitting into bud_metrics (orchestrator), bud_metrics_phases (timeline-derived per-phase actuals), and bud_metrics_contributors (per-user breakdown plus parallelism). All SQL lives in repositories per the project rule: new FeatureLearningRepository, plus DevActivityLog.list_commit_tuples, BUDEstimateSnapshot.get_earliest_for_bud, and User.get_many_by_ids. A phase-reentry case (e.g. testing rejection bouncing UAT to DEVELOPMENT and back) currently collapses the inner round-trip into the outer window; a phase_reentered_window_widened log line surfaces the signal so the smarter "sum disjoint windows" implementation lands once we see real bouncing data. Velocity-aggregate roll-forward is intentionally deferred to the next commit (this one would not compile against the missing module). Signed-off-by: Arun Rajkumar <mickyarunr@gmail.com> * feat(metrics): add velocity_aggregates rollup table with incremental update One row per (org_id, complexity, phase). Each row carries a rolling 50-sample window of recent actual_days plus precomputed p50/p70/p85, a PERT (a, m, b) triple, and Welford running mean / M2. Updated incrementally from bud_metrics.compute_and_persist after every BUD close — one UPSERT per phase, no scan of feature_learnings on the write path. Idempotent against PROD->CLOSED double-fire and webhook re-deliveries: each bucket stores the set of bud_ids that have already contributed, short-circuiting re-rolls without touching the math. The dedup list rotates in lockstep with the sample window, safe only because the upstream compute_and_persist skips the whole pipeline when the feature_learnings.metrics envelope already exists. Math lives in velocity_aggregate_math (pure functions, easily unit-testable) so the orchestrator stays under the project's ~200-line cap. Percentile uses NIST nearest-rank (ceil(n*pct)-1) so small windows aren't biased upward — int(n*pct) returned p80 on a 5-element window when asked for p70. write_30d_snapshot re-fetches via a tenant-scoped select instead of the raw db.get(VelocityAggregate, id) so a caller holding org A's repo can't accidentally mutate org B's bucket. Migration is autogenerated. The phase column reuses the existing bud_status Postgres enum via postgresql.ENUM(..., create_type=False) per the alembic_enum_create_table gotcha in project memory. velocity_aggregates is not yet read by the estimator — that switch lands in the next commit. Today's write-only deployment is safe: buckets accumulate, the estimator continues to use the legacy proportional split until estimation_context is rewired. Signed-off-by: Arun Rajkumar <mickyarunr@gmail.com> * feat(estimation): read velocity_aggregates with proportional-split fallback get_historical_phase_durations now reads the precomputed velocity_aggregates rollup first — one indexed scan over a tiny table (orgs * 5 complexities * 8 phases ~= 40 rows worst-case). Emits the bucket's sample_window directly so the Monte Carlo loop keeps bootstrapping over real per-BUD durations rather than collapsing to point estimates. Phases with fewer than MIN_SAMPLES_FOR_TRUSTED (=5) per-phase samples fall back to the legacy proportional-split path. The two paths are additive — fresh orgs and rarely-touched phases keep estimate stability while the rollup ramps up. The transition is observable via the historical_phase_durations_loaded log line, which emits a per-phase source label ("aggregates" vs "proportional") so we can see orgs flipping from approximation to real data as their bucket fills. The estimator reads ONLY the rollup + the legacy bucket — never feature_learnings.retrospective_md. Estimate determinism stays decoupled from LLM-generated text quality; only the numerical sample_window contributes. All SQL stays in repositories — the new path goes through VelocityAggregateRepository.list_for_complexity_range; the legacy path was already routed through BUDRepository. Tests: the existing 6 historical-phase tests gain a two-call mock helper (rollup query first, legacy query second) and new tests cover (a) the rollup-takes-precedence path, (b) the short-bucket top-up from legacy, (c) empty-both fallback to LLM-only. 14 estimation_ context tests + 43 estimation engine/llm tests pass. Signed-off-by: Arun Rajkumar <mickyarunr@gmail.com> * feat(agents): register learning task for closed status with cross-BUD context Wires the post-close Learning Agent end-to-end: - New build_learning_prompt in app/agents/prompts/learning_prompt.py reads the just-persisted FeatureLearning.metrics envelope and pre-fetches the top-3 most semantically similar prior retrospectives via cosine over feature_learnings.embedding. Both are injected inline as JSON so the agent has structured metrics and cross-BUD trend context without an MCP round-trip. - New handle_learning_result in agent_result_handlers strips the explanatory star-Insight blocks (per the claude_subprocess_isolation memory), embeds the first 2000 chars for future similarity lookups, persists onto feature_learnings.retrospective_md via set_retrospective, records a LEARNING_RECORDED timeline event, and publishes a bud:{id}:activity message so the BUD detail tab can refresh live. - bud_agent_handler registers "closed" in PROMPT_BUILDERS and RESULT_HANDLERS so the existing JOB_BUD_AGENT dispatch picks up the new task type without any handler-loop changes. - bud_stage_seeder gains a "closed" mapping to the technical-writer skill / learning agent_type. Empty output_section bypasses the content-exists guard in create_agent_task_for_stage since the recap goes into feature_learnings, not a column on bud_documents. - skill_mapping.BUD_STAGE_AGENT_TYPE adds CLOSED -> LEARNING so the per-BUD override and org-default skill resolution paths work. - on_bud_closed spawns create_agent_task_for_stage(bud, "closed") ONLY when should_auto_generate_phase(bud.auto_generate_phases, "closed") is true. Default off keeps the External-LLM contract intact — orgs that bring their own AI tooling won't see us spawn LLM work on close unless they explicitly enable it. - BUDTimelineEventType.LEARNING_RECORDED enum entry plus the agent_skills row already seeded by skill_loader (learning -> technical-writer per AGENT_SKILL_MAP) complete the table-side picture. 24 agent-trigger + estimation tests + 5 agent-skill-map-alignment tests pass. mypy clean across 488 files. Signed-off-by: Arun Rajkumar <mickyarunr@gmail.com> * feat(skills): rewrite technical-writer skill for retrospective workflow Skill body replaced wholesale. The frontmatter is unchanged (name=Technical Writer, tools=Read/Write/Glob/Grep, mcp_tools= get_bud_context, timeout_seconds=600, model=sonnet), so no claude_guard audit is needed — only the prompt narrative changes. New body teaches the agent to author the post-close retrospective that build_learning_prompt asks for: - Hard output contract: markdown only, six fixed section headings (Summary / Estimate vs Actual / Phase Drift / Velocity Notes / Parallel Work Effect / Recommendations), no preamble, no JSON fences. Persisted verbatim to feature_learnings.retrospective_md and rendered on the Learnings tab. - Section-by-section guidance grounded on the structured metrics envelope (phase_metrics, contributors, parallelism_score, original_estimated_days) the caller injects. Every recommendation is required to be specific, actionable, and tied to the BUD's pattern — bans platitudes and out-of-control actions. - Cross-BUD context handling: use the prior_recaps list to spot trends across similar BUDs (e.g. "design phase has dragged on the last 3"), cite by BUD number, never copy-paste prior recs. - Velocity-notes framing is explicitly anti-shaming: surface concentration risk and TODO/commit asymmetry at the system level, never as personal criticism. No code changes — skill files are loaded at runtime by skill_loader.load_skill, so this lands immediately for existing orgs on next agent invocation. Signed-off-by: Arun Rajkumar <mickyarunr@gmail.com> * feat(api): expose GET /v1/buds/{id}/learning New endpoint returns the BUDLearningRead payload (retrospective_md + metrics envelope + cycle/estimated/bug counts + timestamps) for the BUD detail "Learnings" tab. Permission gate buds:view matches every other BUD-read endpoint. 404 when no FeatureLearning row exists yet — the FE only calls this after BUDRead.has_learning flips true so the missing case is the cold-state contract, not a normal path. BUDRead now carries has_learning: bool, set in _bud_response by a single get_for_bud check on the FeatureLearningRepository. The flag gates tab visibility on the FE without forcing the BUD detail GET to ship the full retrospective markdown — saves a few KB per BUD load. Set true only when both the row AND retrospective_md exist, so an in-flight Learning Agent task does not show an empty tab. 70 BUD API tests pass; mypy clean across 488 files. No schema migration — both additions are read-side only. Signed-off-by: Arun Rajkumar <mickyarunr@gmail.com> * feat(buds): add Learnings tab to BUD detail with summary cards New BUD detail tab renders the post-close retrospective whenever the backend's has_learning flag is true. The tab consumes the GET /v1/buds/{id}/learning endpoint via the new useBudLearning composable and shows four headline metrics (cycle time, original estimate, bugs, parallelism), a phase-drift table with red/amber/ green tinting on the drift percentage, a contributor table, and the LLM-written retrospective markdown rendered through the existing DOMPurify-backed renderMarkdown utility. The composable maps the 404 cold-state to a null payload so the panel can render the AppCallout empty-state ("the Learning Agent runs on close when auto_generate_phases.closed is enabled") rather than a red error banner. BUDDetail.vue subscribes to the existing bud:{id}:activity socket and bumps a refresh key when learning_recorded fires, so the panel re-fetches without us having to wire a separate useLearningSocket. The BUDDocument type carries the optional has_learning flag. BUDBoard.vue's advanced-settings dialog gains a "Learning recap" auto-generate switch wired to the closed phase. It defaults OFF (unlike the standard stages which default ON) so opt-in is explicit — recap generation costs an LLM call after every close and BYO-AI orgs shouldn't see us spawn that silently. vue-tsc clean. Empty state and populated state render correctly. Signed-off-by: Arun Rajkumar <mickyarunr@gmail.com> * feat(learnings): add org-level Learnings overview route Backend exposes GET /v1/learnings/overview returning four shapes for the new /learnings dashboard: complexity_buckets (read straight from velocity_aggregates so the hot path stays O(rows-per-org)), repeat_offender_phases (median drift_pct across the last 50 feature_learnings, threshold = 30%), velocity_trend (weekly avg cycle days over the last 12 weeks), and top_contributors (per-user buds shipped + commits + PRs over the last 30 days). All SQL lives in the new LearningsOverviewRepository per the sql_in_repositories rule. Aggregation that needs joining the metrics.contributors JSONB to user names happens in Python on the bounded (<=500 entries) recent window — a single bulk fetch by id is cheaper than a JSONB join in SQL. Permission gate is org:view_settings so the same admins who toggle the Learning Agent on / off see its aggregate output. Frontend lands a new /learnings route + sidebar entry (gated on canViewQAAutomation which maps to org:view_settings). The LearningsOverview.vue page renders four cards via useLearningsOverview with a 5-min in-memory SWR cache — overview data changes slowly so revalidating per visit instead of streaming saves bandwidth and keeps the Vue tree simple. Velocity-trend bars are inline-styled divs; no chart library dep for the initial drop. AppCallout empty-state for orgs with no closed BUDs yet. mypy clean across 491 files. vue-tsc clean. The trend_30d_pct field on PhaseRollupRead reads ``running_mean_30d_ago`` which is currently NULL on every row — populated by the daily snapshot job that lands with the cleanup commit; until then the FE renders "—". Signed-off-by: Arun Rajkumar <mickyarunr@gmail.com> * chore: wire daily velocity-snapshot roller and tidy migration formatting Closes the last loop in the Learnings overview: a daily roll-forward job advances each velocity_aggregates bucket's running_mean_30d_ago so the dashboard's trend_30d_pct field has a stable comparison baseline. Mirrors the existing mcp_audit_cleanup pattern — single asyncio task spawned in main.lifespan, idempotent across boots, consecutive-failure backoff with ALERT_AFTER threshold for observability. Also reflows the two autogenerated alembic migrations (af5af83d4327 feature_learnings_metrics_column and a9a073feecb7 velocity_aggregates_rollup) so every line fits the project's 99-col limit. Autogenerated upgrade() bodies tend to emit one column per line at the table's full nesting depth; ruff was correctly flagging those as too long. No DDL change, just wrapping. Final quality gates pass at this point in the branch: ruff + ruff format clean, mypy clean across 492 source files, vue-tsc clean, 84 backend pytest tests pass across estimation_context, estimation_engine, estimation_llm, bud_agent_trigger_phase_gate, agent_skill_map_alignment, and the BUD API surface. Signed-off-by: Arun Rajkumar <mickyarunr@gmail.com> * chore(scripts): add end-to-end Learning Agent pipeline simulation New scripts/simulate_learning_pipeline.py walks every scenario the PR introduced against the real dev DB so we can prove the pipeline end-to-end without depending on the actual Claude subprocess. Scenarios covered: - Opt-out close: feature_learnings + velocity_aggregates updated, NO BUDAgentTask spawned - Opt-in close: BUDAgentTask queued with task_type=closed, then handle_learning_result invoked with a synthetic recap (skips the Anthropic API) - PROD->CLOSED double-fire: single feature_learnings row preserved, velocity_aggregates contributing_bud_ids prevents double-counting - Bucket warming: 5 more BUDs at complexity=3, then 7 phases each show n=7 in the rollup - Estimator switch: complexity-3 hits aggregates, complexity-5 cold bucket falls back to the proportional split - Varied actuals (Scenario 5b): scaled phase durations produce a non-degenerate percentile spread - PERT triple correctness (Scenario 5c): a <= m <= b ordering plus Welford running mean / m2 visibility - Cross-BUD context: find_similar returns the prior recap via cosine Idempotent: the script wipes any [LEARN-SIM]-tagged data at startup AND at end so re-runs don't collide on uq_bud_org_number. Bootstraps setup_job_handlers + seed_skills_for_org + seed_stage_mappings_for_org so the simulation matches what FastAPI lifespan would do. Verified against the dev Postgres on 2026-05-30 — all scenarios pass; cleanup leaves no residual rows. Signed-off-by: Arun Rajkumar <mickyarunr@gmail.com> * fix(metrics): serialize VelocityAggregate.phase as enum .value not .name The bud_status Postgres enum holds lowercase value strings ('bud', 'design', 'tech_arch', ...), but SQLAlchemy was serializing BUDStatus enum members by their .name attribute ('BUD', 'DESIGN', 'TECH_ARCH', ...) — so every velocity_aggregates INSERT failed with InvalidTextRepresentationError: invalid input value for enum bud_status: "BUD". Mirror the values_callable kwarg that BUDDocument.status already uses (per bud.py:119) so the column emits .value strings the enum already accepts. The existing rows on disk store .value strings, so the migration doesn't need a backfill — only the mapper changes. Caught by scripts/simulate_learning_pipeline.py on the first real INSERT against velocity_aggregates. Pure unit tests didn't surface it because the AsyncMock skips the actual Postgres roundtrip; an integration-test pass would have caught it too. The simulation is now the canonical "does this pipeline actually work end-to-end" gate. mypy clean across 492 files. The pipeline simulation now passes every scenario including the bucket-warming and PERT-triple checks. Signed-off-by: Arun Rajkumar <mickyarunr@gmail.com> * fix(metrics): read expected_days from estimate snapshot + handle 0.0 estimate Two real bugs caught while inspecting a real closed BUD (BUD-246): 1. bud_metrics_phases._estimated_days_for_phase was reading ``p70_days`` from BUDEstimateSnapshot.phase_estimates, but the estimator in bud_estimation.build_estimated_dates actually writes ``expected_days`` (the PERT-derived mean duration). The p70_date field that exists is an ISO date string, not a numeric duration. The result: every per-phase ``estimated_days`` came back as None and every ``drift_pct`` was None, so estimate-vs-actual drift was invisible in the recap. 2. The chained ``or`` over candidate keys treated ``0.0`` as falsy and skipped to the next key, returning None. But ``0.0`` is a legitimate estimate ("this phase is instantaneous", which the estimator emits for the BUD-creation phase). Walk the keys explicitly with ``is not None`` so zero estimates persist. Plus three new helper scripts: - scripts/inspect_bud_learning.py — read-only printout of every row the Learning Agent pipeline should have written for one BUD. Reports feature_learnings + agent tasks + timeline events + velocity_aggregates buckets. - scripts/reprocess_bud_learning.py — purges a BUD's existing feature_learnings row and its contribution to velocity_aggregates (sample_window + contributing_bud_ids + Welford carry-over), then re-runs compute_and_persist. Use after fixing a metrics-shape bug to backfill an already-closed BUD without re-firing the rest of on_bud_closed (XP/SP/scan). - simulate_learning_pipeline.py — fixed to write the real production estimate payload (expected_days + p50/p70/p85_date strings) so the simulation can never again silently agree with a broken reader. Verified on BUD-246: drift now resolves correctly (development phase showed +602.7% drift — actual 2.108d against an estimated 0.3d, a real signal the original recap missed entirely). Signed-off-by: Arun Rajkumar <mickyarunr@gmail.com> * feat(buds): Learning recap defaults on + warning + BUD-detail skill picker User-driven defaults change after a real shipped BUD (BUD-246) closed without ever triggering the recap, because the closed-stage toggle defaulted off in the original PR and the user had no obvious way to enable it for existing BUDs. Changes: - BUDBoard.vue create-BUD dialog: the closed stage now defaults ON alongside every other stage. The exception clause that singled it out is gone; the consistent "every phase ON for new BUDs" rule applies everywhere. A new AppCallout (variant=warning) appears inline whenever auto_generate_phases.closed is flipped off so the user sees exactly what they lose — the written retrospective + its embedding for trend grounding — while keeping the per-phase velocity rollup that already powers future estimates either way. - BUDSkillSettingsDialog.vue: the closed stage is now part of the per-BUD skill override list, mirroring the backend's BUD_STAGE_AGENT_TYPE. The technical-writer skill shows up under the Learning recap row so the user can swap it for a custom recap skill without leaving BUD detail. Loading the dialog for BUDs whose auto_generate_phases lacks the closed key (every BUD created before this PR) treats it as ON rather than off so the warning isn't shown for "I just haven't been asked yet" — only when the user actively disables it. The same warning callout appears inline. - scripts/force_run_learning_agent.py now drives handle_bud_agent_job inline rather than relying on the in-process job_queue (which the dev backend can't see across processes). Used to retroactively generate the Learning recap for BUD-246; the Claude subprocess ran 4 turns, $0.16, and produced a 5,667-char retrospective which is now visible on the Learnings tab. vue-tsc clean. Signed-off-by: Arun Rajkumar <mickyarunr@gmail.com> * fix(metrics): count merged PRs from external authors via github_login Before: PullRequest rows with author_user_id=NULL were silently dropped from the contributor breakdown on the Learnings tab. BUD-246's PR #50 was merged on 30 May (visible on the Prod tab as "Released"), but its ``author_github_login=mickyarun`` never resolved to a local user row, so prs_merged came back as 0 for every contributor. Two coordinated changes: 1. github_webhook_handler._handle_pr_opened now best-effort resolves pr_data.user.login to a local user_id via _resolve_github_user (the same helper PR reviewers already use). When the login matches a known user (via ``users.github_username``) we record the user_id so internal authors line up across stage-XP, team context, and the contributor breakdown. 2. bud_metrics_contributors.build_contributor_breakdown now handles the NULL case explicitly. If author_user_id is set and matches the contributor union we count the PR under that user; otherwise we add an external-author row keyed by github_login with the PR count, leaving commits/todos/active_days at zero. External collaborators show up as "<login> (external)" in the contributor table. Internal users always rank ahead via the sort key. Frontend: - BudLearningContributor.user_id is now ``string | null``; new ``github_login`` field carries the GitHub handle for unresolved authors. - LearningsPanel table keys on ``user_id ?? github_login ?? name`` so external rows render without Vue key warnings. Verified on BUD-246: contributor_count=2 Arun commits=2 prs=0 todos=4 days=1 mickyarun (external) commits=0 prs=1 todos=0 days=0 A follow-up worth doing separately: nothing in the dev environment populates ``users.github_username``, so every PR author currently falls through to the external path. Once that column is wired (e.g. via a profile-settings dialog or a once-off backfill) internal authors will resolve to user_id and the (external) suffix will only appear for genuine outside collaborators. Signed-off-by: Arun Rajkumar <mickyarunr@gmail.com> * fix(scripts): reprocess_bud_learning preserves retrospective_md Original implementation deleted the FeatureLearning row to defeat compute_and_persist's idempotency guard. That worked for the structured metrics envelope but blew away the LLM-written retrospective_md + its 384-d embedding — both of which are expensive to regenerate (a Claude subprocess call at ~$0.16 a pop) and remain valid even when the metrics envelope shape changes. The visible symptom: after running reprocess on BUD-246, the Learnings tab disappeared from the BUD detail page. ``has_learning`` on the BUDRead response is gated on ``retrospective_md`` being non-empty, and the deletion cleared it. Fix: clear only ``feature_learnings.metrics = None`` and leave retrospective_md + embedding + cycle/estimated/bug fields in place. The compute_and_persist short-circuit (which checks ``existing.metrics is not None``) still releases on the next call so the metrics envelope is rewritten with the latest math. The upsert path inside ``FeatureLearningRepository.upsert_for_bud`` overwrites the numeric fields too, so no data is left stale. After this fix the workflow for an in-place metrics refresh is: python -m scripts.reprocess_bud_learning <n> …with no need to re-run force_run_learning_agent unless the user specifically wants a new recap reflecting the new metrics. Signed-off-by: Arun Rajkumar <mickyarunr@gmail.com> * fix(metrics): bug_count reads the real bugs table, not QA test cases The legacy _record_feature_learning had a name/value mismatch: ``bug_count = len(qa_automation_cases) + len(qa_manual_cases)``. Those are QA test cases (the test plan size), not bugs. A BUD with 0 actual bugs but a comprehensive 24-case test plan would show "Bugs: 24" on the Learnings tab while "No bugs linked to this BUD" on the Prod tab — two surfaces in the same UI contradicting each other. bud_metrics._bug_count now reads BugRepository.count_for_bud(bud.id) — the real number of Bug rows linked to the BUD, across every status (open / in_progress / resolved / closed). Includes resolved bugs because the retrospective is meant to surface total bug volume accumulated over the BUD's lifecycle, not just the currently-unresolved subset. The function is now async because the count requires a query. The single caller (compute_and_persist) wraps it in await. Verified on BUD-246: Learnings tab now shows Bugs=0, matching the Prod tab's "No bugs linked" state. The 24 was 16 automation cases + 8 manual cases, both unchanged. Signed-off-by: Arun Rajkumar <mickyarunr@gmail.com> * feat(learnings): redesign the overview page to look like a product page The original drop landed all the data but visually read as four plain v-tables glued together. Now matches the in-house BUDEstimateTimeline range-bar idiom and the rest of the product: - New hero KPI row across the top (BUDs shipped 30d, median cycle, repeat-offender count, complexity buckets populated). Each tile carries a 3px primary accent on the left and a soft surface-variant gradient — same visual contract as AppCallout. The "BUDs shipped" tile flips its accent to warning + shows a "cards get sharper after ~5 BUDs ship" hint when totals are below the threshold. - Phase rollup is now horizontal bars per phase, scoped to one complexity bucket. Bar width = p70 relative to the slowest phase in the bucket; a tick mark indicates where p50 sits inside the bar. Trend pill uses bad/good/neutral colours. - Repeat-offender phases get their own card with red-tinted rows, large drift percentage, and a horizontal bar visualizing how many of the recent BUDs overran. Empty when no phase has more than one BUD over the +30% threshold so the section disappears entirely on fresh orgs instead of rendering empty headers. - Velocity trend: bars are taller, anchored to the bottom, show the cycle-day value above the bar and the BUD count below the week label. The chart has a hairline base axis line so single-week data points read as "one column on an axis" rather than "lonely green sliver in space" (the original visual issue). - Contributors get rank numbers, deterministic colour avatars (hash-based so the same user_id always renders the same hue), name + stats summary line, and a horizontal bar proportional to buds shipped relative to the top contributor. Empty state callout updated to set expectations — "Close at least one BUD … fills in automatically … meaningful trends within a week or two" instead of the original "Close 5+ BUDs" line that suggested the page was broken below 5 closes. No new dependencies; all charts are hand-rolled CSS. vue-tsc clean. Signed-off-by: Arun Rajkumar <mickyarunr@gmail.com> * refactor(learnings): apply code-review fixes (Welford correctness, SQL boundary, file size, tests) code-reviewer findings on the post-MVP batch, addressed in priority order: 1. must-fix — reprocess_bud_learning was leaving Welford running_mean / running_m2 stale after removing a BUD from the bucket window. When compute_and_persist re-rolled the same BUD, the new Welford step ran against state that still included the prior contribution, silently double-counting it. Recompute mean / m2 / pert triple from the surviving window now, in both the populated and empty branches. 2. must-fix — LearningsOverview.vue was 847 lines (~4× the project's ~200-line file budget per feedback_code_principles). Split into five per-card components under components/learnings/: - LearningsKpiRow hero KPI tiles - PhaseRollupCard bucketed percentile bars - RepeatOffenderCard red-tinted drift rows - VelocityTrendCard weekly bar chart - TopContributorsCard ranked + avatar rows The view file is now ~120 lines and composes them with prop passing only — no behaviour change. 3. important — velocity_snapshot_roller.py was running raw ``select(Organization.id)`` inside a service, violating feedback_sql_in_repositories_only. Promoted to OrganizationRepository.list_all_ids() and routed the service through it. 4. important — added tests/services/test_bud_metrics_shapes.py with 13 regression tests pinning the data-shape bugs the simulation missed: expected_days vs p70_days (incl. the 0.0-falsy edge), nearest-rank percentile, Welford correctness on a known sequence, double-fire idempotency, bug_count async-ness, VelocityAggregate.phase serialising via values_callable, plus the (org_id, complexity, phase) unique-constraint pin. mypy clean across 493 files; 32 service-level tests pass; vue-tsc clean. Frontend split preserves every existing selector via scoped CSS — no leakage out of the per-card components. Signed-off-by: Arun Rajkumar <mickyarunr@gmail.com> --------- Signed-off-by: Arun Rajkumar <mickyarunr@gmail.com>
1 parent 479fcd1 commit 84ca5a6

60 files changed

Lines changed: 5925 additions & 157 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 106 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,106 @@
1+
"""velocity_aggregates_rollup
2+
3+
Revision ID: a9a073feecb7
4+
Revises: af5af83d4327
5+
Create Date: 2026-05-30 21:54:26.636330
6+
7+
"""
8+
from collections.abc import Sequence
9+
10+
import sqlalchemy as sa
11+
from sqlalchemy.dialects import postgresql
12+
13+
from alembic import op
14+
15+
# revision identifiers, used by Alembic.
16+
revision: str = 'a9a073feecb7'
17+
down_revision: str | None = 'af5af83d4327'
18+
branch_labels: str | Sequence[str] | None = None
19+
depends_on: str | Sequence[str] | None = None
20+
21+
22+
def upgrade() -> None:
23+
# ### commands auto generated by Alembic - please adjust! ###
24+
# The ``bud_status`` enum already exists in the DB (created by an
25+
# earlier migration). Use postgresql.ENUM with create_type=False so
26+
# we reuse the existing type rather than re-CREATE-ing it — per the
27+
# alembic_enum_create_table gotcha documented in project memory.
28+
bud_status = postgresql.ENUM(name='bud_status', create_type=False)
29+
jsonb_empty_list = sa.text("'[]'::jsonb")
30+
now_sql = sa.text('now()')
31+
op.create_table(
32+
'velocity_aggregates',
33+
sa.Column('org_id', sa.UUID(), nullable=False),
34+
sa.Column('complexity', sa.Integer(), nullable=False),
35+
sa.Column('phase', bud_status, nullable=False),
36+
sa.Column('n_samples', sa.Integer(), server_default='0', nullable=False),
37+
sa.Column(
38+
'sample_window',
39+
postgresql.JSONB(astext_type=sa.Text()),
40+
server_default=jsonb_empty_list,
41+
nullable=False,
42+
),
43+
sa.Column(
44+
'contributing_bud_ids',
45+
postgresql.JSONB(astext_type=sa.Text()),
46+
server_default=jsonb_empty_list,
47+
nullable=False,
48+
),
49+
sa.Column('p50_days', sa.Numeric(precision=8, scale=2), nullable=True),
50+
sa.Column('p70_days', sa.Numeric(precision=8, scale=2), nullable=True),
51+
sa.Column('p85_days', sa.Numeric(precision=8, scale=2), nullable=True),
52+
sa.Column('pert_optimistic', sa.Numeric(precision=8, scale=2), nullable=True),
53+
sa.Column('pert_most_likely', sa.Numeric(precision=8, scale=2), nullable=True),
54+
sa.Column('pert_pessimistic', sa.Numeric(precision=8, scale=2), nullable=True),
55+
sa.Column(
56+
'running_mean',
57+
sa.Numeric(precision=10, scale=4),
58+
server_default='0',
59+
nullable=False,
60+
),
61+
sa.Column(
62+
'running_m2',
63+
sa.Numeric(precision=12, scale=4),
64+
server_default='0',
65+
nullable=False,
66+
),
67+
sa.Column('running_mean_30d_ago', sa.Numeric(precision=10, scale=4), nullable=True),
68+
sa.Column('snapshot_taken_at', sa.DateTime(timezone=True), nullable=True),
69+
sa.Column('id', sa.UUID(), nullable=False),
70+
sa.Column(
71+
'created_at',
72+
sa.DateTime(timezone=True),
73+
server_default=now_sql,
74+
nullable=False,
75+
),
76+
sa.Column(
77+
'updated_at',
78+
sa.DateTime(timezone=True),
79+
server_default=now_sql,
80+
nullable=False,
81+
),
82+
sa.ForeignKeyConstraint(['org_id'], ['organizations.id']),
83+
sa.PrimaryKeyConstraint('id'),
84+
sa.UniqueConstraint('org_id', 'complexity', 'phase', name='uq_velocity_agg_bucket'),
85+
)
86+
op.create_index(
87+
'ix_velocity_agg_lookup',
88+
'velocity_aggregates',
89+
['org_id', 'complexity'],
90+
unique=False,
91+
)
92+
op.create_index(
93+
op.f('ix_velocity_aggregates_org_id'),
94+
'velocity_aggregates',
95+
['org_id'],
96+
unique=False,
97+
)
98+
# ### end Alembic commands ###
99+
100+
101+
def downgrade() -> None:
102+
# ### commands auto generated by Alembic - please adjust! ###
103+
op.drop_index(op.f('ix_velocity_aggregates_org_id'), table_name='velocity_aggregates')
104+
op.drop_index('ix_velocity_agg_lookup', table_name='velocity_aggregates')
105+
op.drop_table('velocity_aggregates')
106+
# ### end Alembic commands ###
Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
"""feature_learnings_metrics_column
2+
3+
Revision ID: af5af83d4327
4+
Revises: 0adad4d4dd8d
5+
Create Date: 2026-05-30 21:39:11.914718
6+
7+
"""
8+
from collections.abc import Sequence
9+
10+
import sqlalchemy as sa
11+
from sqlalchemy.dialects import postgresql
12+
13+
from alembic import op
14+
15+
# revision identifiers, used by Alembic.
16+
revision: str = 'af5af83d4327'
17+
down_revision: str | None = '0adad4d4dd8d'
18+
branch_labels: str | Sequence[str] | None = None
19+
depends_on: str | Sequence[str] | None = None
20+
21+
22+
def upgrade() -> None:
23+
# ### commands auto generated by Alembic - please adjust! ###
24+
op.add_column(
25+
'feature_learnings',
26+
sa.Column('metrics', postgresql.JSONB(astext_type=sa.Text()), nullable=True),
27+
)
28+
# ### end Alembic commands ###
29+
30+
31+
def downgrade() -> None:
32+
# ### commands auto generated by Alembic - please adjust! ###
33+
op.drop_column('feature_learnings', 'metrics')
34+
# ### end Alembic commands ###
Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
# Copyright 2025-2026 Arun Rajkumar
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
"""Prompt builders for stage-specific agents that don't fit the legacy
16+
``app.services.agent_prompts`` module's section-writing pattern.
17+
18+
The Learning Agent lives here rather than in ``agent_prompts`` because
19+
its output (a retrospective recap) is not a BUD-document section the
20+
existing builders manipulate — it consumes structured metrics and
21+
writes to ``feature_learnings.retrospective_md`` instead.
22+
"""
Lines changed: 155 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,155 @@
1+
# Copyright 2025-2026 Arun Rajkumar
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
"""Prompt builder for the post-close Learning Agent.
16+
17+
The recap is grounded entirely on data the prompt builder pre-fetches
18+
and inlines as JSON inside the prompt body — there is no MCP read tool
19+
for these metrics. Rationale: the agent reads, never writes, and the
20+
data is in scope at prompt-build time. An MCP tool would only add
21+
round-trip latency and an extra serialization contract to maintain.
22+
23+
Cross-BUD context: the builder pulls the top-N (default 3) most
24+
semantically similar prior retrospectives via cosine over
25+
``feature_learnings.embedding`` and injects them as few-shot examples
26+
so the recap can call out trends (e.g. "the design phase has dragged
27+
on the last 3 similar BUDs").
28+
"""
29+
30+
import json
31+
import uuid
32+
from typing import Any
33+
34+
import structlog
35+
36+
from app.models.bud import BUDDocument
37+
from app.repositories.feature_learning import FeatureLearningRepository
38+
from app.services.embedding_service import embedding_service
39+
from app.services.skill_loader import Skill
40+
41+
logger = structlog.get_logger(__name__)
42+
43+
PRIOR_RECAP_LIMIT = 3
44+
PRIOR_RECAP_SNIPPET_CHARS = 1_500
45+
46+
47+
def _summarize_bud_for_embedding(bud: BUDDocument) -> str:
48+
"""Concatenate the BUD's identity + brief into a single embedding input."""
49+
parts = [
50+
bud.title or "",
51+
(bud.requirements_md or "")[:1_500],
52+
]
53+
return "\n\n".join(p for p in parts if p)
54+
55+
56+
async def _fetch_prior_recaps(
57+
db: Any,
58+
org_id: uuid.UUID,
59+
bud: BUDDocument,
60+
) -> list[dict[str, str]]:
61+
"""Top-3 prior retrospectives most semantically similar to this BUD.
62+
63+
Embeds the current BUD's title + requirements (NOT the new metrics
64+
JSON) so the similarity hits are about the work itself, not about
65+
the numerical shape. Returns an empty list when the BUD's content
66+
can't be embedded — the prompt builder is resilient to this.
67+
"""
68+
try:
69+
embedding = await embedding_service.embed(_summarize_bud_for_embedding(bud))
70+
except Exception:
71+
logger.warning("learning_prompt_embed_failed", bud_id=str(bud.id))
72+
return []
73+
74+
repo = FeatureLearningRepository(db, org_id=org_id)
75+
similar = await repo.find_similar(
76+
embedding,
77+
limit=PRIOR_RECAP_LIMIT,
78+
exclude_bud_id=bud.id,
79+
)
80+
out: list[dict[str, str]] = []
81+
for row in similar:
82+
if not row.retrospective_md:
83+
continue
84+
out.append(
85+
{
86+
"bud_id": str(row.bud_id),
87+
"retrospective_md": row.retrospective_md[:PRIOR_RECAP_SNIPPET_CHARS],
88+
}
89+
)
90+
return out
91+
92+
93+
def _format_prompt_body(
94+
skill: Skill,
95+
bud: BUDDocument,
96+
metrics: dict[str, Any],
97+
prior_recaps: list[dict[str, str]],
98+
) -> str:
99+
"""Assemble the skill body, BUD identity, structured metrics, and prior recaps."""
100+
prior_block = (
101+
json.dumps(prior_recaps, indent=2)
102+
if prior_recaps
103+
else "(no prior retrospectives available — write the first one for this complexity bucket)"
104+
)
105+
return (
106+
f"{skill.prompt}\n\n"
107+
f"## BUD\n"
108+
f"Number: BUD-{bud.bud_number:03d}\n"
109+
f"Title: {bud.title}\n\n"
110+
f"## Original PRD (excerpt)\n"
111+
f"{(bud.requirements_md or '')[:2_000]}\n\n"
112+
f"## Structured metrics for this BUD\n"
113+
f"```json\n{json.dumps(metrics, indent=2, default=str)}\n```\n\n"
114+
f"## Prior retrospectives from similar BUDs (most-similar first)\n"
115+
f"{prior_block}\n\n"
116+
f"## Task\n"
117+
f"Write the retrospective markdown for this BUD. The output is read by\n"
118+
f"the team on the BUD detail Learnings tab and used as cross-BUD\n"
119+
f"context for future recaps. Follow the skill's workflow exactly.\n"
120+
)
121+
122+
123+
async def build_learning_prompt(
124+
bud: BUDDocument,
125+
skill: Skill,
126+
org_id: uuid.UUID,
127+
db: Any,
128+
) -> tuple[str, str | None]:
129+
"""Build the prompt for the post-close Learning Agent.
130+
131+
Pulls the freshly-persisted ``FeatureLearning.metrics`` envelope
132+
from the DB (written seconds earlier by
133+
``bud_metrics.compute_and_persist``) and injects it inline, then
134+
appends up to three vector-similar prior recaps for trend grounding.
135+
Returns ``(prompt, working_dir=None)`` — the agent doesn't run
136+
against a repo working directory.
137+
"""
138+
repo = FeatureLearningRepository(db, org_id=org_id)
139+
learning = await repo.get_for_bud(bud.id)
140+
if learning is None or not learning.metrics:
141+
logger.warning(
142+
"learning_prompt_no_metrics_envelope",
143+
bud_id=str(bud.id),
144+
bud_number=bud.bud_number,
145+
)
146+
# Defensive: build a usable prompt anyway so the agent doesn't
147+
# silently degrade into "(no data)" output — a missing envelope
148+
# is a bug we want surfaced as a written recap that flags it.
149+
metrics: dict[str, Any] = {"_warning": "metrics envelope missing"}
150+
else:
151+
metrics = dict(learning.metrics)
152+
153+
prior_recaps = await _fetch_prior_recaps(db, org_id, bud)
154+
prompt = _format_prompt_body(skill, bud, metrics, prior_recaps)
155+
return prompt, None

backend/app/agents/skill_mapping.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -77,6 +77,7 @@ def get_skill_for_agent(agent_name: str) -> str | None:
7777
BUDStatus.DESIGN: AgentType.DESIGN,
7878
BUDStatus.TECH_ARCH: AgentType.TECH_PLAN,
7979
BUDStatus.TESTING: AgentType.TEST_PLAN,
80+
BUDStatus.CLOSED: AgentType.LEARNING,
8081
}
8182

8283
# Maps a BUD section key to the agent type that handles chat for that

0 commit comments

Comments
 (0)