Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Two independent packages, no top-level workspace manager:

- `backend/` — FastAPI + SQLAlchemy 2 + Alembic, managed by `uv`, Python 3.14
- `frontend/` — Next.js 14 App Router, TypeScript, Tailwind CSS, Supabase SSR
- `frontend-v2/` — Next.js 16 + Tailwind v4 v2 frontend (per [ADR 0007](docs/adr/0007-frontend-v2-migration-strategy.md)). Phases 2 + 3 ship: OKLCH token system + shadcn primitives + ~17 Ascendra composites; pages for `/`, `/login`, `/signup`, `/onboarding`, `/dashboard`, `/arena`, `/arena/[mechanic]/play`, `/replay/[id]`, `/replay/[id]/share`; Supabase auth + TanStack Query; TopBar in the `(app)` layout. MVP pages stay in `frontend/` until parity is reached and the cutover happens. See `frontend-v2/CLAUDE.md` for v2 conventions (theme provider, `proxy.ts`, async cookies, design tokens, Vitest + Playwright test runner separation).
- `frontend-v2/` — Next.js 16 + Tailwind v4 v2 frontend (per [ADR 0007](docs/adr/0007-frontend-v2-migration-strategy.md)). Phases 2 + 3 (sub-plans #1 + #2) ship: OKLCH token system + shadcn primitives + ~19 Ascendra composites; pages for `/`, `/login`, `/signup`, `/onboarding`, `/dashboard`, `/arena`, `/arena/[mechanic]/play` (cv-battle + behavioural-duel + mental-maths-blitz), `/replay/[id]`, `/replay/[id]/share`; Supabase auth + TanStack Query; TopBar in the `(app)` layout. MVP pages stay in `frontend/` until parity is reached and the cutover happens. See `frontend-v2/CLAUDE.md` for v2 conventions (theme provider, `proxy.ts`, async cookies, design tokens, Vitest + Playwright test runner separation).

All commands below assume you `cd` into the relevant package first. Each has its own README with more detail (`backend/README.md` is the most accurate single source of truth for backend setup).

Expand Down
8 changes: 8 additions & 0 deletions backend/CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -231,6 +231,14 @@ See `.env.example`. Required: `DATABASE_URL` (Session pooler), `SUPABASE_URL`, `

For local development against the real DB, the recommended pattern is to point `DATABASE_URL` at a Supabase project's Session pooler URL on port 5432 with the `postgresql+psycopg://` prefix.

## Public template reads sanitize answer keys

`templates.services.challenge_templates` ships two read variants:
- `get_challenge_template_or_error` / `list_challenge_templates` — return the full ORM rubric (consumed by the scoring pipeline's `DeterministicNumericJudge` via `ctx.template.rubric_definition`).
- `get_challenge_template_or_error_for_public_api` / `list_challenge_templates_for_public_api` — detach the row and overwrite `rubric_definition` with a sanitized copy (no `correct_answers`, no `questions[i].answer`).

The `/api/v1/challenge-templates/*` router uses the public variants. The scoring pipeline reads the template via the ORM directly so deterministic mechanics keep their answer key. When adding a new template type that carries hidden answer/hints, extend `_sanitize_rubric_for_public_read` rather than introducing a parallel surface.

## Common Pitfalls Already Hit

- Forgetting to add a new model to `app/models/__init__.py` ⇒ test suite passes locally because the import happens via another route; CI catches it via `Base.metadata.create_all` missing the table.
Expand Down
8 changes: 4 additions & 4 deletions backend/app/contexts/templates/api/challenge_templates.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@
ChallengeTemplateRead,
)
from app.contexts.templates.services.challenge_templates import (
get_challenge_template_or_error,
list_challenge_templates,
get_challenge_template_for_public_api,
list_challenge_templates_for_public_api,
)
from app.core.enums import ApplicationCycle, Arena, ChallengeMode, ChallengeType, DomainPathway

Expand All @@ -29,7 +29,7 @@ def list_challenge_templates_endpoint(
mode: ChallengeMode | None = None,
calibration_only: bool | None = None,
) -> ChallengeTemplateListResponse:
items = list_challenge_templates(
items = list_challenge_templates_for_public_api(
db,
target_domain=target_domain,
application_cycle=application_cycle,
Expand All @@ -47,4 +47,4 @@ def get_challenge_template_endpoint(
db: DatabaseSession,
current_user: CurrentUser,
) -> ChallengeTemplateRead:
return get_challenge_template_or_error(db, template_id=template_id)
return get_challenge_template_for_public_api(db, template_id=template_id)
23 changes: 23 additions & 0 deletions backend/app/contexts/templates/schemas/challenge_template.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,12 +26,35 @@ class RubricCategory(BaseModel):
focus_keywords: list[str] = Field(default_factory=list)


class RubricQuestion(BaseModel):
"""One question entry on the rubric (Phase 3 sub-plan #2).

Only the ``prompt`` is exposed publicly — the corresponding
``answer`` (if present on the server-side rubric) is stripped by
``_sanitize_rubric_for_public_read`` in the templates service, AND
ignored here by Pydantic ``extra="ignore"`` default behaviour. Used
by ``mental_maths_blitz`` (numeric streak).
"""

prompt: str


class RubricDefinition(BaseModel):
categories: list[RubricCategory] = Field(default_factory=list)
focus_keywords: list[str] = Field(default_factory=list)
ideal_word_range_min: int | None = None
ideal_word_range_max: int | None = None
expected_answers: list[str] = Field(default_factory=list)
# Question battery exposed to the room composites (Phase 3 sub-plan
# #2). Two shapes are supported for backwards compatibility:
# - Plain strings (``behavioural_duel`` stores 4 STAR prompts).
# - ``RubricQuestion`` dicts (``mental_maths_blitz`` stores 30
# ``{"prompt": ..., "answer": ...}`` entries; ``answer`` is
# stripped by ``_sanitize_rubric_for_public_read``).
questions: list[RubricQuestion | str] = Field(default_factory=list)
# Optional time-bonus window (seconds) for deterministic numeric
# judges. Used by ``mental_maths_blitz``.
time_bonus_seconds: int | None = None
benchmark_anchor_score: float = 72.0


Expand Down
103 changes: 103 additions & 0 deletions backend/app/contexts/templates/services/bootstrap.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
from __future__ import annotations

import importlib.util
from pathlib import Path
from types import ModuleType

from sqlalchemy import select
from sqlalchemy.orm import Session

Expand All @@ -25,6 +29,38 @@
SkillNodeType,
)

# ---------------------------------------------------------------------------
# Phase 3 sub-plan #2 (Task A4): pull in seed scripts for the v2 mechanics so
# the conftest fixture + ``scripts/seed_reference_data.py`` produce a DB that
# includes ``behavioural_duel`` and ``mental_maths_blitz``.
#
# ``scripts/`` is intentionally NOT a Python package, so we use
# ``importlib.util`` to load the seed modules on first use. The modules are
# cached on the function so repeated calls (e.g. across multiple test cases)
# don't re-parse the file. Per ADR 0008 this stays inside the templates
# context — the scripts only mutate templates / mechanics / anchors rows.
# ---------------------------------------------------------------------------

_SCRIPTS_DIR = Path(__file__).resolve().parents[4] / "scripts"


def _load_script_module(filename: str, alias: str) -> ModuleType:
cache = _load_script_module._cache # type: ignore[attr-defined]
if alias in cache:
return cache[alias]
path = _SCRIPTS_DIR / filename
spec = importlib.util.spec_from_file_location(alias, path)
assert spec is not None and spec.loader is not None, (
f"Failed to locate seed script at {path}"
)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
cache[alias] = module
return module


_load_script_module._cache = {} # type: ignore[attr-defined]


def seed_reference_data(db: Session) -> None:
skill_nodes = {node.slug: node for node in _upsert_skill_nodes(db)}
Expand All @@ -37,9 +73,76 @@ def seed_reference_data(db: Session) -> None:
pack.slug: pack for pack in _upsert_benchmark_packs(db, templates)
}
_upsert_quests(db, skill_nodes, benchmark_packs)
_seed_phase3_mechanics(db)
db.commit()


def _seed_phase3_mechanics(db: Session) -> None:
"""Hook the Phase 3 sub-plan #2 mechanic seed scripts in. Idempotent
(both scripts upsert)."""

behavioural = _load_script_module(
"seed_phase3_behavioural_duel_mechanic.py",
"seed_phase3_behavioural_duel",
)
behavioural.seed_behavioural_duel(db, profile="dev")

mental_maths = _load_script_module(
"seed_phase3_mental_maths_blitz_mechanic.py",
"seed_phase3_mental_maths_blitz",
)
mental_maths.seed_mental_maths_blitz(db)
_link_phase3_templates_to_skill_nodes(db)


def _link_phase3_templates_to_skill_nodes(db: Session) -> None:
"""Wire the new Phase 3 sub-plan #2 templates into the existing skill
tree (Task A5).

Existing Professional + Technical leaves cover the relevant signals
for the two new mechanics, so no new skill nodes are added in this
sub-plan. Deeper skill-tree work lands in Phase 4.
"""

link_seeds = [
# Behavioural Duel — STAR-anchored, drives professional-arena
# storytelling + stakeholder judgement + the conflict /
# leadership leaves it covers via the four canonical prompts.
("behavioural-duel-phase3-default", "star-storytelling", 1.2),
("behavioural-duel-phase3-default", "stakeholder-judgement", 1.0),
("behavioural-duel-phase3-default", "conflict-resolution", 0.8),
("behavioural-duel-phase3-default", "leadership-framing", 0.8),
# Mental Maths Blitz — deterministic numeric accuracy under
# heavy time pressure; also exercises problem decomposition.
("mental-maths-blitz-phase3-default", "numerical-accuracy", 1.3),
("mental-maths-blitz-phase3-default", "problem-decomposition", 0.7),
]
for template_slug, node_slug, emphasis in link_seeds:
template = db.scalar(
select(ChallengeTemplate).where(ChallengeTemplate.slug == template_slug)
)
node = db.scalar(select(SkillNode).where(SkillNode.slug == node_slug))
if template is None or node is None:
# If the seed didn't run (e.g., partial DB state), skip the
# link silently — the test fixture asserts the seed ran.
continue
existing = db.scalar(
select(ChallengeTemplateSkillLink).where(
ChallengeTemplateSkillLink.challenge_template_id == template.id,
ChallengeTemplateSkillLink.skill_node_id == node.id,
)
)
if existing is None:
existing = ChallengeTemplateSkillLink(
challenge_template_id=template.id,
skill_node_id=node.id,
)
existing.emphasis_weight = emphasis
existing.is_repair_anchor = True
db.add(existing)
db.flush()


def _skill_seed(
*,
slug: str,
Expand Down
103 changes: 103 additions & 0 deletions backend/app/contexts/templates/services/challenge_templates.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
from __future__ import annotations

import copy
import uuid
from typing import Any

from fastapi import status
from sqlalchemy import select
Expand All @@ -11,6 +13,62 @@
from app.core.errors import AppError


def _sanitize_rubric_for_public_read(
rubric: dict[str, Any] | None,
) -> dict[str, Any] | None:
"""Deep-copy a rubric dict and strip fields that leak answer keys
(Phase 3 sub-plan #2 Task A3).

The mental_maths_blitz seed embeds the deterministic answer key
into ``rubric_definition.correct_answers`` and into each entry of
``rubric_definition.questions[i].answer``. These MUST never reach
the client — otherwise a user could read the page source and ace
the drill. This helper produces a defensive deep copy with both
fields removed so the caller can attach the sanitized dict to a
response object without mutating the original (or the
SQLAlchemy-attached ORM row).

The pipeline / workflow / scoring helpers read ``rubric_definition``
directly off the SQLAlchemy ORM object (see
``DeterministicNumericJudge.score``), bypassing this helper — so
deterministic scoring still works.
"""

if rubric is None:
return None
sanitized: dict[str, Any] = copy.deepcopy(rubric)
sanitized.pop("correct_answers", None)
questions = sanitized.get("questions")
if isinstance(questions, list):
cleaned_questions: list[Any] = []
for entry in questions:
if isinstance(entry, dict):
cleaned_entry = {k: v for k, v in entry.items() if k != "answer"}
cleaned_questions.append(cleaned_entry)
else:
cleaned_questions.append(entry)
sanitized["questions"] = cleaned_questions
return sanitized


def _detach_with_sanitized_rubric(
db: Session, template: ChallengeTemplate
) -> ChallengeTemplate:
"""Expunge the template from the session and overwrite
``rubric_definition`` with a sanitized deep copy.

Detaching avoids any risk that the in-memory mutation flushes back
to the database. Used by the public-API list/get wrappers so the
response model never sees the raw answer key.
"""

db.expunge(template)
template.rubric_definition = _sanitize_rubric_for_public_read(
template.rubric_definition
) or {}
return template


def list_challenge_templates(
db: Session,
*,
Expand Down Expand Up @@ -55,6 +113,35 @@ def is_visible(template: ChallengeTemplate) -> bool:
return [template for template in templates if is_visible(template)]


def list_challenge_templates_for_public_api(
db: Session,
*,
target_domain: DomainPathway | None = None,
application_cycle: ApplicationCycle | None = None,
arena: Arena | None = None,
challenge_type: ChallengeType | None = None,
mode: ChallengeMode | None = None,
calibration_only: bool | None = None,
) -> list[ChallengeTemplate]:
"""Wrapper around :func:`list_challenge_templates` that strips
answer-key fields out of each template's rubric (Task A3).

The pipeline / scoring helpers continue to call
:func:`list_challenge_templates` for the full rubric.
"""

templates = list_challenge_templates(
db,
target_domain=target_domain,
application_cycle=application_cycle,
arena=arena,
challenge_type=challenge_type,
mode=mode,
calibration_only=calibration_only,
)
return [_detach_with_sanitized_rubric(db, t) for t in templates]


def get_challenge_template_or_error(
db: Session,
*,
Expand All @@ -72,3 +159,19 @@ def get_challenge_template_or_error(
status_code=status.HTTP_404_NOT_FOUND,
)
return template


def get_challenge_template_for_public_api(
db: Session,
*,
template_id: uuid.UUID,
) -> ChallengeTemplate:
"""Wrapper around :func:`get_challenge_template_or_error` that strips
answer-key fields out of the rubric before returning (Task A3).

The pipeline / scoring helpers continue to call
:func:`get_challenge_template_or_error` for the full rubric.
"""

template = get_challenge_template_or_error(db, template_id=template_id)
return _detach_with_sanitized_rubric(db, template)
26 changes: 26 additions & 0 deletions backend/app/core/enums.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,11 @@ class ChallengeType(StrEnum):
CV_BATTLE = "cv_battle"
BEHAVIOURAL_DUEL = "behavioural_duel"
TIMED_DRILL = "timed_drill"
# Phase 3 sub-plan #2: mental_maths_blitz mechanic (deterministic
# numeric judge, Technical arena). Distinct from TIMED_DRILL so
# the frontend dispatcher and analytics can branch on the mechanic
# without consulting other columns.
MENTAL_MATHS = "mental_maths"


class DifficultyLevel(StrEnum):
Expand All @@ -65,19 +70,40 @@ class ScoringMode(StrEnum):
RUBRIC = "rubric"
PAIRWISE = "pairwise"
HYBRID = "hybrid"
# Phase 3 sub-plan #2: explicit semantic values used by new
# v2 mechanics. JUDGE_LLM signals an LLM-judge-only mix
# (no deterministic component); DETERMINISTIC signals a
# deterministic-judge-only mix (no LLM call). These coexist
# with HYBRID for backwards compatibility with cv_battle.
JUDGE_LLM = "judge_llm"
DETERMINISTIC = "deterministic"


class InputType(StrEnum):
TEXT = "text"
DOCUMENT = "document"
STRUCTURED_FORM = "structured_form"
NUMERIC = "numeric"
# Phase 3 sub-plan #2: behavioural_duel ships multi-paragraph
# text answers (one per behavioural question). LONG_TEXT lets
# the frontend pick a multi-textarea wizard vs. a single TEXT
# input.
LONG_TEXT = "long_text"
# Phase 3 sub-plan #2: mental_maths_blitz streams numeric
# answers one per question (30 sequential responses).
# Distinct from NUMERIC so the room composite can render the
# streak UI.
NUMERIC_STREAK = "numeric_streak"


class OutputType(StrEnum):
TEXT = "text"
NUMERIC = "numeric"
STRUCTURED_JSON = "structured_json"
# Phase 3 sub-plan #2: behavioural_duel emits a structured
# 4-answer payload (one per question). Distinct from
# STRUCTURED_JSON to signal a fixed Q/A array shape.
STRUCTURED = "structured"


class DomainProfileStatus(StrEnum):
Expand Down
Loading
Loading