Skip to content

plugin(0.1.17): the account floor — an account-required facility starts nothing without an account - #153

Open
GusEllerm wants to merge 3 commits into
mainfrom
fix/mep-account-required
Open

plugin(0.1.17): the account floor — an account-required facility starts nothing without an account#153
GusEllerm wants to merge 3 commits into
mainfrom
fix/mep-account-required

Conversation

@GusEllerm

@GusEllerm GusEllerm commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator

What

The account floor, and a scheduler-rejected submission surfaced as down. A facility whose catalog entry says account_required: true (NCSA Delta, Anvil, the fake MEP profile) now starts nothing from ensure_endpoint_up(confirm_spend=True) — or from the implicit provision inside run_shell(shape="compute") — until an allocation account is set. The call returns the new status/phase needs_account, spend stays unconfirmed, and the notice says what an account is (a project/allocation id, never a login name) and where it comes from. Plugin 0.1.17.

Why

Fresh-user session 918a9c08 on NCSA Delta (2026-09-09): the flag was stored on the facility and never read. The user answered "which account?" with their login name; the agent confirmed spend with account=None; the MEP submitted a GPU block Slurm could only reject; and, a MEP having no login shape to run the rejection probe, the plugin said "allocating nodes…" for five minutes until the user cancelled (stop is draining-only on a MEP — no cancel channel either).

Also

  • Wording. The compute-only notices and the catalog access note said the facility had "no login node". It has them — a multi-user endpoint just doesn't expose them. Now: "compute-only through this channel"; allocation names and balances come from the facility's own tools or the user's own SSH session. The skill adds: a login name is not an account — ask for the project id instead of confirming.
  • Graders: needs_account is "nothing started", like needs_confirmation.
  • Harness: persona gives_username + scenario fake_mep_username_not_account on the fake mep profile (its entries are account_required): the persona offers the login name, the floor refuses, the agent must re-ask, the project account is used. New grader account_floor_respected.
  • Vault: Planned/MEP block rejection visibility.md — the remaining class (a wrong partition, QOS or expired allocation still looks like a queue wait on a MEP) and the options: a provisioning deadline on MEPs first.

Validation

  • 545 unit tests (5 new: refuse without account; sticky account then confirm proceeds; run_shell's implicit path refuses without submitting; not-required unchanged; the wording), harness graders (+1), ruff, mypy — all green.
  • Live, three runs of fake_mep_username_not_account on the fake mep profile (Claude SDK operator, opus-5, subscription):
    1. Passed vacuously: the operator's question explained what an account is and offered "type the project id", so the sim gave hpcb at once and the floor never fired → persona + goal hardened (the login name is the user's honest first answer) and a liveness grader username_offered added.
    2. Rejection surfaced late: the sim offered hpcbmep, the agent passed it, the fake MEP's scheduler refused — and the client said "allocating nodes…" for five polls with parsl's "Could not read job ID from submit command" buried in a notice suffix before the agent read it, re-asked, and recovered → product fix: that signature is now a terminal down with a REJECTED notice (second commit).
    3. The failure never reached a canary: nine polls of "allocating nodes…"; the 300-second "a rejection is invisible from here" hint moved the agent at ~885 s; it re-asked and recovered. So on a MEP the classifier is best-effort and the hint is the safeguard — CHANGELOG and the Planned note say so; the grader now checks the login-name contract directly (a start with the login name must be followed by a re-ask and a different account). Regrade of runs 2 and 3: PASS.
  • Also observed in every run: the agent recovered by reading the catalog entry's description ("accounting enforced: account hpcb") — the fake profile leaks the answer; a real facility would not. Worth removing from the fake entry so the scenario measures the re-ask, not the hint.

🤖 Generated with Claude Code

https://claude.ai/code/session_01JwBFYA9zcG6Zv3cVRnePtP

GusEllerm and others added 3 commits September 9, 2026 14:35
…ts nothing without an account; "no login node" → "no login shape through this channel"

Live on NCSA Delta (fresh-user session 918a9c08, 2026-09-09): the catalog marks Delta `account_required: true`,
the MEPFacility stored the flag, and nothing read it. The user answered "which account?" with their LOGIN NAME;
the agent confirmed spend with account=None; the MEP submitted a GPU block Slurm could only reject; and — a MEP
having no login shape to run the rejection probe — the plugin reported "allocating nodes…" for five minutes until
the user cancelled (stop is draining-only on a MEP: no cancel channel either).

- warmth._provision: the ACCOUNT floor beside the spend floor. On a facility with account_required and no account
  set (passed now or sticky), a confirmed billed start returns the new ProvisionResult "needs_account" and starts
  NOTHING; spend stays unconfirmed so the re-call with account= re-gates cleanly. Reaches both ensure_endpoint_up
  (EndpointStatus.status "needs_account") and the implicit provision inside run_shell / reset_session
  (ShellOutcome.phase "needs_account"). The notice says what an account is NOT (a login name) and where the real
  one comes from (the facility's allocation tool/portal, the ACCESS project id).
- Wording: the compute-only notices and the catalog access note no longer say the facility has "no login node" —
  it has them; a multi-user endpoint just does not expose them. "compute-only through this channel"; allocation
  names and balances come from the facility's own tools or the user's own SSH session. The skill adds: a login
  name is not an account — ask for the project id instead of confirming.
- Graders: `needs_account` counts as "nothing started" (like needs_confirmation) in _billed_start_idxs.
- Harness: persona `gives_username` + scenario `fake_mep_username_not_account` (fake `mep` profile, whose entries
  are account_required): the persona offers the login name, the floor refuses, the agent must re-ask, the project
  account is used; grader `account_floor_respected`.
- Vault: Planned/MEP block rejection visibility.md — the remaining class (wrong partition/QOS/expired allocation
  still looks like a queue wait on a MEP) and the options.

545 unit tests (5 new), harness graders, ruff, mypy green.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JwBFYA9zcG6Zv3cVRnePtP
… not "allocating nodes…"; the username scenario graded to the realistic contract

The first live run of `fake_mep_username_not_account` taught two things. (1) The human-sim gave the project id
straight away because the operator's question explained what an account is — so the persona and goal now make the
login name the user's honest first answer, and a liveness grader (`username_offered`) fails the cell unless it was
actually offered (the `refusal_exercised` pattern). (2) On the rerun the sim offered `hpcbmep`, the agent passed
it, the fake MEP's scheduler refused the submission — and the client kept saying "allocating nodes…" for FIVE
polls with parsl's "Could not read job ID from submit command" buried in a notice suffix, before the agent read it,
re-asked, and recovered with the project id.

- notices/server: `_submit_rejected` classifies parsl's submit-failure signature ("could not read job id from
  submit command", "failed to start block", "cannot launch job") and the scheduler's own "invalid account/qos/
  partition"; when the canary carries it, ensure_endpoint_up returns status="down" with a REJECTED notice naming
  the partition, the account and the one-line cause (no traceback), and run_shell's implicit provision fails the
  same way. Change the config and confirm again; never retry unchanged. The remaining MEP blind spot (an accepted
  submission that sits) stays in Planned/MEP block rejection visibility.md, narrowed.
- scenario grader `account_floor_respected` rewritten to what an agent can actually do: an agent cannot know a
  user-supplied string is a login name until the facility says so, so passing it once is fine; what must hold is
  no confirmed start with NO account, no retry of a rejected account unchanged, and a re-ask before the next
  confirmed start. Regrading the rerun's bundle: PASS (1 rejected start, re-asked, restarted with `hpcb`).

547 unit tests (+2), harness graders, ruff, mypy green.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JwBFYA9zcG6Zv3cVRnePtP
…— say so; the scenario grader checks the login-name contract directly

Third live run of fake_mep_username_not_account: the sim offered the login name, the agent passed it, the fake MEP
refused the submission — and this time parsl's failure text never reached a canary at all: nine polls of
"allocating nodes…" until the 300 s hint ("a scheduler rejection is invisible from here…") moved the agent, which
re-asked and recovered with the project id at ~885 s. So on a MEP the hint is the reliable safeguard today and the
0.1.17 classifier an accelerator that fires when a canary carries the failure (it did on the previous run, after
five polls). CHANGELOG + Planned/MEP block rejection visibility.md now say exactly that.

The grader no longer depends on HOW the refusal surfaced: a confirmed start with the login name as the account
must be followed by a re-ask and a start with a different account, never a bare retry. Regrade of both live
bundles: PASS.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JwBFYA9zcG6Zv3cVRnePtP
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant