refactor(container): one scorecard source, and a thin session route

CLAUDE.md

@@ -143,7 +143,7 @@ Key additions for production: `ENVIRONMENT=production`, `ADMIN_PASSWORD`, `CORS_
 - **BYOK via dashboard**: Claude OAuth tokens (from `claude setup-token`) stored per installation, encrypted with Fernet, injected as `CLAUDE_CODE_OAUTH_TOKEN` env var into containers. Zero API cost (uses user's Claude subscription).
 - **Dashboard over SQLAdmin**: user-facing operations (installation list, session history, token config) go through the dashboard UI; SQLAdmin is the superadmin escape hatch
 - **Open source target**: designed for self-hosting with own Claude licenses
-- **Post-results to PR**: after session completion, the API can post score card as a PR comment — opt-in per installation via `post_results_to_pr` boolean; extraction and formatting in `container/pr_comment.py`, triggered in `_event_stream()` after `mark_completed()`
+- **Post-results to PR**: after session completion the API can post the scorecard as a PR comment — opt-in per installation via `post_results_to_pr`. Rendered from the validated `Scorecard` model (`container/scorecard.py`), the same object stored on `cs.scorecard` and used for `xp_earned`, so the dashboard and the comment cannot disagree. Skills emit two things: markdown for the human watching the stream, and the `helprs-scorecard` JSON block for everything machine-read. Only the JSON is parsed; `pr_comment.py` formats, it no longer extracts.
 - **Coolify deployment**: Two-domain setup via Traefik: `helprs.tech` (web) and `api.helprs.tech` (API). TLS via Let's Encrypt, managed by Coolify. Domains are set in Coolify UI (General > Domains for api / Domains for web) — they may get cleared on redeploy, re-check after each deploy. "Preserve Repository During Deployment" must be enabled so skills are available on the host. The prod compose uses `./` paths (repo-root-relative) because Coolify sets `--project-directory` to the repo root.
 - **claude-runner image**: declared in both `docker-compose.yml` and `infra/coolify/docker-compose.prod.yml` as a **build-only service** — `entrypoint: ["/bin/true"]` + `restart: "no"` so the container exits immediately on `up`, leaving only the built image `claude-runner:latest` on the host. The API spawns containers from this image dynamically via the mounted Docker socket. Image tag is hard-coded in `container/service.py:CLAUDE_RUNNER_IMAGE` — it must stay `claude-runner:latest` (no namespace prefix). Earlier attempt with `profiles: [build-only]` was reverted: `profiles:` excludes a service from both build AND run unless the profile is explicitly activated, which Coolify does not do, causing `404 No such image` failures on session spawn.
 - **Non-root API container**: production Dockerfile uses `appuser` with `chown -R appuser:appuser /app` for uv cache writes. Docker socket access requires `group_add: ["${DOCKER_GID:-994}"]` in the compose to match the host's docker group GID.

apps/api/src/helprs/modules/container/pr_comment.py

@@ -1,68 +1,62 @@
-"""Score card extraction and PR comment formatting for challenge-me sessions."""
+"""Rendering a finished session as a GitHub PR comment.
 
-import re
-from uuid import UUID
+Built from the validated ``Scorecard``, not from the markdown the skill
+writes for the live stream. Two extractors used to coexist -- a validated
+JSON parser feeding the dashboard and an unvalidated regex over the markdown
+feeding this comment -- so one session could be scored in one place and
+silently uncommented in the other. There is one machine-readable source now,
+and this module no longer runs its own SQL over the events table.
+"""
 
-import structlog
-from sqlalchemy import select
-from sqlalchemy.ext.asyncio import AsyncSession
+from uuid import UUID
 
-from helprs.modules.container.models import SessionEvent
+from helprs.modules.container.scorecard import MAX_DIMENSION_SCORE, Scorecard
 
-logger = structlog.get_logger()
+_BAR_WIDTH = 10
 
-_RESULTS_PATTERN = re.compile(
-    r"---\s*\n\s*\n##\s+Results\b(.*?)^---\s*$",
-    re.DOTALL | re.MULTILINE,
-)
 
+def _score_bar(score: float) -> str:
+    filled = round(score * _BAR_WIDTH / MAX_DIMENSION_SCORE)
+    return "█" * filled + "░" * (_BAR_WIDTH - filled)
 
-async def extract_score_card(session_id: UUID, db: AsyncSession) -> str | None:
-    """Extract the score card markdown from persisted session events.
 
-    Queries the last ``result`` event for the session and extracts the
-    ``## Results`` section delimited by ``---`` markers (as defined in
-    ``skills/challenge-me/CLAUDE.md``).
-    """
-    result = await db.execute(
-        select(SessionEvent.data)
-        .where(
-            SessionEvent.session_id == session_id,
-            SessionEvent.data["type"].astext == "result",
-        )
-        .order_by(SessionEvent.event_id.desc())
-        .limit(1)
+def _dimensions_table(scorecard: Scorecard) -> str:
+    rows = "\n".join(
+        f"| {name.replace('_', ' ').title()} | {score:g} / {MAX_DIMENSION_SCORE} |"
+        for name, score in scorecard.dimensions.items()
     )
-    row = result.scalar_one_or_none()
-    if row is None:
-        return None
-
-    result_text = row.get("result")
-    if not result_text or not isinstance(result_text, str):
-        return None
+    return f"| Dimension | Score |\n|-----------|-------|\n{rows}"
 
-    match = _RESULTS_PATTERN.search(result_text)
-    if not match:
-        return None
 
-    return f"## Results{match.group(1).rstrip()}"
+def format_pr_comment(scorecard: Scorecard, session_url: str) -> str:
+    """Render the scorecard as the comment body."""
+    overall = scorecard.overall_score
+    sections = [
+        "### helPRs Challenge-Me Results",
+        "",
+        f"**Score: {overall:.1f} / {MAX_DIMENSION_SCORE}**  {_score_bar(overall)}",
+        "",
+        _dimensions_table(scorecard),
+        "",
+        scorecard.summary,
+    ]
 
+    if scorecard.highlights:
+        sections += ["", "**Highlights**", *(f"- {item}" for item in scorecard.highlights)]
 
-def format_pr_comment(
-    score_card: str,
-    session_url: str,
-) -> str:
-    """Format the score card as a GitHub PR comment."""
-    return (
-        f"### helPRs Challenge-Me Results\n\n"
-        f"{score_card}\n\n"
-        f"<details>\n"
-        f"<summary>View session</summary>\n\n"
-        f"[Open full Q&A session]({session_url})\n\n"
-        f"---\n"
-        f"*Posted by [helPRs](https://github.com/apps/helprs)*\n"
-        f"</details>\n"
-    )
+    sections += [
+        "",
+        "<details>",
+        "<summary>View session</summary>",
+        "",
+        f"[Open full Q&A session]({session_url})",
+        "",
+        "---",
+        "*Posted by [helPRs](https://github.com/apps/helprs)*",
+        "</details>",
+        "",
+    ]
+    return "\n".join(sections)
 
 
 def build_session_url(

apps/api/src/helprs/modules/container/router.py

@@ -14,9 +14,8 @@
 
 from helprs.core.database import get_db_context
 from helprs.core.dependencies import CurrentUser, DbSession, GetSettings, authenticate_token, stream_token
-from helprs.core.exceptions import ConflictError, NotFoundError
+from helprs.core.exceptions import ConflictError
 from helprs.core.middleware import limiter
-from helprs.core.security import fernet_decrypt
 from helprs.modules.container.docker_client import AioDockerClient, DockerClient
 from helprs.modules.container.models import ContainerStatus
 from helprs.modules.container.schemas import (
@@ -30,23 +29,16 @@
     StopSessionResponse,
 )
 from helprs.modules.container.service import (
-    create_session,
     delete_session,
     finalize_session,
     get_session_events,
     get_session_or_404,
+    open_session,
     send_message,
-    start_container,
     stop_container,
 )
 from helprs.modules.container.streaming import spawn_detached, stream_and_persist
-from helprs.modules.installation.github import RUNNER_TOKEN_PERMISSIONS
 from helprs.modules.installation.service import (
-    get_byok_config,
-    get_installation_by_github_id,
-    mint_installation_token,
-    verify_installation_access,
-    verify_repo_access,
     verify_session_access,
 )
 
@@ -70,53 +62,17 @@ async def create_container_session(
     user: CurrentUser,
 ) -> ContainerSessionResponse:
     """Create a session and start its container."""
-    installation = await get_installation_by_github_id(db, body.installation_id)
-    if not installation:
-        raise NotFoundError("Installation not found")
-
-    # Installation membership is not enough: an installation can cover repos
-    # this user cannot read, and the session streams the diff back to them.
-    await verify_installation_access(user, installation, settings)
-    await verify_repo_access(user, body.repo_full_name, settings)
-
-    byok_config = await get_byok_config(db, installation.id)
-    if not byok_config:
-        raise NotFoundError("No Claude token configured for this installation")
-
-    claude_oauth_token = fernet_decrypt(byok_config.encrypted_api_key, settings.fernet_keys)
-
-    # Narrowed to this repo, read-only: the container runs Claude Code over
-    # untrusted PR content with network egress.
-    github_token = await mint_installation_token(
-        installation.github_installation_id,
-        settings,
-        repositories=[body.repo_full_name.split("/")[-1]],
-        permissions=RUNNER_TOKEN_PERMISSIONS,
-    )
-
-    cs = await create_session(
-        db=db,
-        installation_id=installation.id,
-        pr_number=body.pr_number,
-        repo_full_name=body.repo_full_name,
-        skill_name=body.skill_name,
-        user_id=user.id,
-    )
-    # Committed before the container is touched. start_container marks the row
-    # FAILED on error and then raises, but that exception unwinds through
-    # get_db, which rolls back -- discarding the FAILED status *and* the row
-    # itself. A failed start left the user with a 502 and a dashboard showing
-    # that nothing had ever happened.
-    await db.commit()
-
     docker = _get_docker_client()
     try:
-        cs = await start_container(
-            db=db,
-            session_id=cs.id,
-            docker=docker,
-            claude_oauth_token=claude_oauth_token,
-            github_token=github_token,
+        cs = await open_session(
+            db,
+            docker,
+            user=user,
+            installation_github_id=body.installation_id,
+            pr_number=body.pr_number,
+            repo_full_name=body.repo_full_name,
+            skill_name=body.skill_name,
+            settings=settings,
         )
     finally:
         await docker.close()

apps/api/src/helprs/modules/container/scorecard.py

@@ -1,47 +1,74 @@
-"""Scorecard extraction from Claude session output.
+"""The structured scorecard a skill emits at the end of a session.
 
-Skills emit a structured JSON scorecard wrapped in a fenced code block
-tagged `helprs-scorecard`. This module parses it from the last assistant
-message text.
+Skills write two things when they finish: markdown results for the human
+watching the stream, and this JSON block for anything that has to *read* the
+outcome. Both are documented in ``skills/challenge-me/CLAUDE.md``.
+
+Everything machine-consumed goes through the model below, so the dashboard
+and the PR comment can never disagree about a session. The PR comment used to
+be regex-scraped out of the markdown instead, with no validation, so a small
+wording drift in a skill silently produced no comment at all.
 """
 
 import json
 import re
 
+from pydantic import BaseModel, ConfigDict, Field, ValidationError, field_validator
+
 SCORECARD_PATTERN = re.compile(
     r"```helprs-scorecard\s*\n(.*?)\n```",
     re.DOTALL,
 )
 
-REQUIRED_FIELDS = {"skill", "version", "dimensions", "summary"}
+DIMENSION_COUNT = 3
+MAX_DIMENSION_SCORE = 10
 
 
-def extract_scorecard(text: str) -> dict | None:
-    """Extract the helprs-scorecard JSON block from text.
+class Scorecard(BaseModel):
+    """A skill's verdict on one session.
 
-    Returns the parsed dict if found and valid, None otherwise.
+    ``extra="allow"`` because a skill may report more than helPRs reads: the
+    fields below are the contract, not the ceiling.
     """
-    match = SCORECARD_PATTERN.search(text)
-    if not match:
-        return None
 
-    try:
-        data = json.loads(match.group(1))
-    except (json.JSONDecodeError, ValueError):
-        return None
+    model_config = ConfigDict(extra="allow")
 
-    if not isinstance(data, dict):
-        return None
+    skill: str
+    version: int
+    dimensions: dict[str, float]
+    summary: str
+    questions_asked: int | None = None
+    questions_answered: int | None = None
+    highlights: list[str] = Field(default_factory=list)
 
-    if not REQUIRED_FIELDS.issubset(data.keys()):
-        return None
+    @field_validator("dimensions")
+    @classmethod
+    def check_dimensions(cls, value: dict[str, float]) -> dict[str, float]:
+        if len(value) != DIMENSION_COUNT:
+            raise ValueError(f"expected exactly {DIMENSION_COUNT} dimensions, got {len(value)}")
+        for name, score in value.items():
+            if not 0 <= score <= MAX_DIMENSION_SCORE:
+                raise ValueError(f"dimension '{name}' is {score}, outside 0-{MAX_DIMENSION_SCORE}")
+        return value
+
+    @property
+    def overall_score(self) -> float:
+        """Mean of the dimensions, on the same 0-10 scale."""
+        return sum(self.dimensions.values()) / len(self.dimensions)
 
-    dims = data.get("dimensions")
-    if not isinstance(dims, dict) or len(dims) != 3:
-        return None
 
-    for value in dims.values():
-        if not isinstance(value, (int, float)) or value < 0 or value > 10:
-            return None
+def extract_scorecard(text: str) -> Scorecard | None:
+    """Parse the ``helprs-scorecard`` block out of text.
 
-    return data
+    Returns ``None`` rather than raising: a session whose skill emitted no
+    scorecard, or emitted a malformed one, is a session without a scorecard --
+    not a failed session.
+    """
+    match = SCORECARD_PATTERN.search(text)
+    if not match:
+        return None
+
+    try:
+        return Scorecard.model_validate(json.loads(match.group(1)))
+    except (json.JSONDecodeError, ValueError, ValidationError):
+        return None