Skip to content

Design: the Bee-Hive — cortex agents with worker-harnesses or code-drones, against M and H #44

Description

@OriNachum

Status: a design proposal, not a plan. Nothing here is pre-registered yet;
that happens after the design is argued down to something falsifiable. The
verdict that motivated it is preserved verbatim in
§0.

The problem with the verdict

It recommends flat-27B on evidence that is entirely about counsel and
sequential reasoning
, and then applies it to a question about execution and
width
. Four experiments agree that orchestration costs multiples and buys
nothing measurable — but every one of them measured a second mind advising or
deciding, never a second mind doing bulk work in parallel.

The one rung that would have tested the actual claim — rung F, the fan-out rung,
where the pre-registration says separation is structural rather than hopeful
because a flat arm cannot dispatch concurrent units at all — was never
reached. C1 ceilinged, and the ladder's budget went to rungs that measure a
mind thinking alone.

So the honest state is not "orchestration doesn't pay." It is: we measured the
axis where orchestration cannot win, and skipped the one where it wins by
construction.

Four architectures, and why B is a different kind of thing

arm who acts worker's role worker has a loop?
E cortex alone
W worker alone
M cortex orchestrates executes delegated subtasks as an agent yes
H cortex orchestrates and does complex work executes simple work as an agent yes
B cortex only a tool the cortex calls no

B — "Bee-Hive" is the operator's proposal and it is not a weaker M. It is a
different claim about where authority lives: the worker never runs a loop, never
decides how to do anything, and never holds a turn. It is a fast pure function
the cortex calls — transform(N things) -> N results.

That matters for three reasons:

  1. It is what the rig already says. lobes advertises
    worker.forbidden_responsibilities = [final_decision, security_decision].
    B is the only arm that takes that literally.
  2. It removes the component nobody can grade. M and H both require the
    worker to interpret an underspecified subtask — which is where the muse arms'
    NO_ANSWER failures came from (#32,
    #33). A tool call has
    a schema; a delegated goal does not.
  3. It is the cheapest thing that captures the ~9× parallelism. If B wins,
    M and H are unnecessary complexity. If B loses to M, that is a real finding
    about agency being worth its cost
    — and it is the first evidence this repo
    would have for it.

B is the arm most likely to make M and H unnecessary, so it should be
measured before either is built.

P — the arm the operator actually proposed, and the one I would prioritise

"a bot unit that is controllable by code created by the agent/model"

This is not a variant of the above. Every arm in the table delegates to a
mind
. This one delegates to a program:

model calls for an N-turn episode
E / W / M / H / B N (one or more per turn)
P — Programmed 1 — the mind writes a policy, the policy plays

The cortex writes a unit-control policy as code once, the policy drives the
units for the whole episode at ~0 ms per decision, and the mind intervenes only
at declared checkpoints. On the speed and efficiency axes this is not an
incremental difference, it is a different order of magnitude — and it is the
only proposal here that could beat the worker on latency without using the
worker at all
.

It also asks a question none of the other arms do, and one this repo is unusually
well-placed to answer: can the mind externalise its decision procedure?
Writing a policy that plays well is strictly harder than playing well — it
requires knowing why your moves are right, not just making them. A mind that
plays at 5/6 and writes a policy that plays at 1/6 has told us something real
about the difference between competence and articulable competence.

And we already have the machinery to grade it. challenge_coding.py runs
model-written source against committed cases inside a network-less workspace;
k1.py adapts it into the arms harness. P is that grader pointed at a policy
instead of a puzzle.

The Bee-Hive, named

Bee-hive is Cortex agents with Worker-harnesses or Code-Drones

That is the architecture in one line, and it settles the vocabulary:

caste what it is intelligence made by
Cortex agent a mind with open-ended judgement and final authority full, at play time dialled
Worker-harness code the cortex wrote, which calls the worker for scoped typed questions borrowed, bounded manufactured
Code-drone code the cortex wrote, which decides everything itself compiled, frozen manufactured

The structural claim this makes, which M and H do not

Every delegation is to something the cortex built. Nothing is delegated to a
raw mind.

In M and H the worker is a dialled peer — the cortex hands it a goal and hopes.
In the Bee-Hive the worker never appears on its own: it appears inside a
harness the cortex authored
. The cortex delegates to its own artifacts, and
those artifacts may internally borrow a mind.

That is not a stylistic difference, and it predicts a specific failure going
away. #32 and
#33 are both interface
failures — a model handed a contract someone else designed, producing the wrong
shape (17 of 23 calls refused on an argv). In a harness, the schema is one the
cortex wrote for its own use
. A delegator that authors the interface it
delegates across cannot be surprised by it in the same way, and the harness can
validate and fall back locally instead of losing the turn.

If the Bee-Hive's schema-adherence is materially better than M's on the same
work, that is the finding — and it would be the first mechanistic explanation
this repo has for why its tool arms kept coming back INCONCLUSIVE.

One rig constraint the name makes visible

"Cortex agents" is plural, and on this rig that is expensive in a way the other
castes are not. The cortex serves with --max-num-seqs=2 and measures ~23 tok/s;
its effective concurrency is 1. Drones and harnesses scale (unbounded and
~9-way); queens do not.

So a hive on this box is one or two cortex agents commanding an arbitrary number
of manufactured units — which is worth stating before anyone designs a scenario
that assumes a population of queens. It also sharpens what the sweep measures:
the scarce resource is cortex attention, and every arm is really answering how
far does one queen's attention stretch?

The bot-code IS a worker tier — the operator's correction, and the real design

I first wrote this as "the policy calls the worker". That is backwards. The
operator's point:

bot-code could be the semi-intelligent bot worker

The bot-code is not a client of the worker. It is a worker — a third executor
tier, one the mind manufactures rather than dials. That reframes the whole
experiment from "which architecture" to "which tier does each unit of work
belong on
", and it is what makes the Bee-Hive name literal: a hive with castes.

tier intelligence cost per decision concurrency who makes it
cortex 27B full, at play time ~23 tok/s, thousands of tokens 1 dialled
worker 35B A3B full, at play time 76.4 tok/s ~9 dialled
bot-code semi — the mind's strategy, compiled ~0 tokens, ~0 ms unbounded written, once

"Semi-intelligent" is exact rather than loose: the bot-code carries the cortex's
strategy but has no runtime judgement. It is the mind's reasoning frozen at
authoring time — right about everything it anticipated, blind to everything it
did not.

This is the Opus-to-Sonnet relationship stated more precisely than M or H state
it.
Not "the smart one delegates tasks to the fast one" but "the smart one
builds the thing that does the work" — and what it builds costs nothing to
run and scales without limit.

The tier is a spectrum, and the middle of it is the likely answer

Second operator refinement, and it is the one I would build first:

there is also "code that uses worker for scoped minor-intelligence" —
better than rigid code, but still fast

Rigid bot-code has one real weakness: it is blind to everything its author did
not anticipate
. The fix is not to hand control back to a mind — it is to let
the code ask a narrow question at the point where it cannot decide, and keep
control flow itself.

variant play-time calls what the delegate is asked who holds control flow
B0 — rigid 0 the code
B1 — scoped N small worker calls, ~9 concurrent "which of these three?", "is this safe?" — a typed question with a small answer space the code
B2 — agentic (this is M) N worker turns "handle this unit" — an open goal the worker

B1 is not a small M. The difference is not how much is delegated but what
kind of thing
is delegated: B1 delegates a question, M delegates a goal.
A question has a schema, a bounded answer space, and a wrong answer the code can
detect and fall back from. A goal has none of those.

Why scoping is what makes it both fast and safe

The scope is doing three jobs at once, and they are the same job:

  • Speed — a scoped question is a small prompt and a small completion. The
    cortex spends 5,000–14,000 tokens on a turn; a scoped worker call spends tens.
    At 76.4 tok/s and ~9-way concurrency, N scoped calls cost less wall clock than
    one cortex turn.
  • Reliability#33
    measured 17 of 23 calls refused because a model produced the wrong shape
    for an open-ended argv. A narrow answer space is a narrow schema, and a narrow
    schema is one a model gets right. Scoping is a schema-adherence strategy, not
    just a latency one
    — which is precisely the failure that made the muse tool
    arms INCONCLUSIVE.
  • Falsifiability — a scoped question has a correct answer often enough to
    grade the delegate separately from the strategy. M cannot do this: when an
    agentic delegation goes wrong there is no way to tell a bad goal from bad
    execution.

This shape is also not novel outside this repo, which is mild evidence for it:
it is how competent game AI has always worked — a behaviour tree or utility
system carrying an expensive evaluator at the specific leaves where judgement is
needed, and cheap deterministic logic everywhere else.

The dial this adds, and it is measurable

Scope size becomes an independent variable — how much of each decision is
compiled versus asked:

scope example question prompt tokens answer space
none (B0) 0
narrow "target A, B, or C?" ~50 3
medium "rank these 5 by threat" ~200 orderings
wide "what should this unit do?" ~1,500 free text
open (B2/M) "handle this unit" ~2,000+ a whole turn

The prediction worth pre-registering: quality rises then plateaus while cost
rises monotonically
, so the useful setting is the knee, not the maximum. That
is the same shape #40
argues for the coherence gate, and the same reason a sweep beats a binary — the
knee is a finding, and no on/off design can locate it.

The question this turns the experiment into

Not M or H or B. Rather: what fraction of the work can run on manufactured
intelligence before quality drops — and where exactly is that boundary?

That is a dose–response curve, not a binary, which is the single most useful
structural change here. Every prior arm-comparison in this repo died tied at a
ceiling (#40 argues the
same point for the coherence gate): a binary on/off design produces a tie and
stops, a sweep produces a curve, and a curve can be wrong in ways a tie
cannot.

With three units the mix is enumerable and the sweep is cheap:

mix bot-code worker cortex what a win here would mean
all-bot 3 0 0 the strategy fully compiles; play-time minds are unnecessary
mostly-bot 2 1 0 one unit needs judgement; the rest are mechanical
split 1 2 0 most work needs a mind; some is compilable
all-worker 0 3 0 manufacture buys nothing — code cannot hold the strategy
all-cortex 0 0 3 the flat control (E)

The interesting result is almost certainly not at either end. And the shape
of the curve answers the operator's original question directly: the point where
quality starts dropping is the boundary between work that can be compiled and
work that needs a mind — which is exactly what M and H are guessing at with
their routing rules.

Two things this tier can do that neither M nor H can

  1. Promotion and demotion between tiers. A unit can start on bot-code and be
    promoted to worker when the policy hits a case it cannot decide, which is a
    real product design and not just an experiment. The policy declares its own
    incompleteness — an I don't know return — and the escalation is earned
    rather than routed by a heuristic nobody can grade. This is the credible
    answer to H's parked risk
    ("hybrid routing quality has no precedent
    grader"
    ): a policy that says when it is out of its depth is a router that
    grades itself.
  2. Zero marginal cost at width. The 27B leaves ~9× parallelism unused; the
    35B saturates near 8. Bot-code has no such ceiling — dispatching 3 units or
    300 costs the same. If width is where orchestration wins, this tier wins it
    by the largest margin available.

What it must report, and the failure mode that will masquerade as success

A compiled strategy that is syntactically valid and strategically inert will
run to completion, produce legal moves, and look like a working arm. So:

  • The three P controls apply — random policy, hand-written baseline, no-op.
    Without the random control, "the model can compile a strategy" and "any program
    beats per-turn control" are indistinguishable, and those are different
    findings.
  • Escalation rate is a reported axis, not a footnote. A policy that escalates
    every decision has collapsed into the worker arm with extra latency; one that
    escalates never may simply be unable to detect its own ignorance. The rate is
    the measurement, and both extremes are informative.
  • If the policy calls a mind at all, report call-acceptance separately from
    outcome.
    #33 measured
    17 of 23 calls refused because a model sent command as a string where an
    array was required. Here the model designs that contract rather than
    satisfying one — a policy whose escalations are 74% refused would play like a
    random bot while running perfectly. Interface failure and strategy failure must
    never share a number.

The environment: league-of-agents, and why it is the right one

Reusing the league lane rather than inventing a task, for reasons this cycle
made expensive to relearn:

  • The outcome metric is not ours. corrections.md §2 exists because our own
    graders were wrong in the flattering direction four times in one cycle. M2
    in next-cycle-candidates.md asks specifically for a grader we did not
    author. League has one.
  • It is genuinely multi-unit, so width is real rather than simulated.
  • The harnesses existarch_league.py, arch_league_maps.py,
    league_seat.py, league_commander.py and the fog-scoped map renderer are
    all committed and tested.

One hard exclusion, learned the expensive way: league-h2h's ranking
(full-qwen > mixed > full-gemma) rested entirely on an optional team-message
field
while the outcome metric tied 0–0 in all six matches. It measured
interface compliance, not play. No metric in this experiment may be an
optional field that one arm happens to populate more often.

What to measure, and the order to measure it in

Run the width rung first. It is the cheapest decisive experiment available
and it is the one the ladder never reached. If width does not separate, the
whole M/H/B question is moot and the flat default stands on evidence rather than
on priors.

# rung asks why this order
1 F — width does dispatching N units concurrently beat doing them in sequence, on wall clock? separation is structural; a flat arm cannot dispatch at all. If this ties, stop — the whole question is moot
2 the scope sweep (B0 → B1 → B2) how much intelligence must stay at play time, and how narrow can the delegated question be? a curve, not a tie. B0 costs ~0 per decision and B1's calls are tens of tokens, so the whole sweep is cheaper than one arm of the ladder. Its knee locates the compile/judgement boundary M and H merely guess at
3 B1 vs M does the delegate need a goal, or does a question suffice? the sharpest available test of whether agency is worth its cost — same delegate, same work, differing only in what it is handed
4 H does routing by complexity beat routing by rule? last, and possibly never: if a policy can declare its own ignorance, escalation is earned rather than routed, and H's ungradeable router is answered without being built

Note what the reordering does to H. In the original framing H was deferred
because its router could not be graded. In the tier framing it may be
unnecessary: a bot-code policy that returns "I cannot decide this" is a router
whose decisions are self-evidencing. That is a better outcome than deferring the
problem — and it is only visible once the bot-code is treated as a tier rather
than as an alternative to one.

Axes, all three reported for every arm at every rung

  • Quality — league's own outcome metric. Plus, for P, whether the policy is
    correct code at all (compiles, runs, terminates) as a separate validity gate.
  • Efficiency — all four token counts, split prompt/completion.
    league-commander measured hierarchy's bill at ~95% prompt, so a
    completion-only comparison would miss where the money goes.
  • Speed — wall clock per decision and per episode, plus model calls per
    episode
    , which is the axis P attacks directly.

The controls that make P falsifiable

P's failure mode is specific and unusually easy to mistake for success: a policy
that is syntactically valid, runs to completion, and is strategically inert.
It will look like a working arm. Three controls, all pre-registered:

  1. A random policy — legal moves, no strategy. If the model's policy does not
    beat this, it wrote code but not a policy.
  2. A hand-written baseline policy — a few obvious heuristics. This separates
    "the model can program" from "any program beats per-turn model control",
    which is a completely different and also-interesting finding.
  3. A no-op policy — if this scores near the others, the scenario does not
    reward play and the rung measures nothing. The ceiling check, applied to the
    environment rather than to an arm.

Without (1) and (3), a mediocre policy and a good one are indistinguishable.

Guards carried in from this cycle

Each of these cost something to learn; none is hypothetical.

  • Timeouts from max_tokens / measured_rate, never a latency percentile
    (#42). A 300 s timeout
    silently deleted every generation past ~6,975 tokens and scored two of them as
    wrong answers. P is especially exposed — writing a policy is a long
    generation.
  • Fan-out width 8, not 14. The advertised ×14 is KV-pool arithmetic
    (KV pool ÷ max_model_len), not measured concurrency.
  • Dial the worker directly on Thor. The local proxy reports
    feasible: false.
  • Do not trust /capabilities for modality. It understated cortex vision;
    fixed upstream, not necessarily deployed.
  • Model-written code runs in the network-less workspace jail, reusing
    challenge_coding's container. P executes model output by design — this is
    the one arm with a real blast radius.
  • A ceiling check before comparing arms. Run E first; if it maxes the
    metric, escalate the scenario rather than reporting a tie.
  • Every declared rung must be runnable, not merely well-shaped. K1's pin
    built problems with statement="" and grade=lambda _raw: {} and passed for
    an entire task while the rung could not be dialled.

What this cannot settle

  • One rig, two model builds, one arena. A policy that plays a grid skirmish
    well says nothing about a policy that refactors a codebase.
  • P conflates two abilities — playing well and articulating play. A low P
    score does not distinguish them, and a follow-up would need a task where the
    optimal policy is known.
  • Width here is ~3 units, from the committed scenario. Claims about width 8
    need a scenario that fields 8.

Open questions for the operator

  1. Does P intervene? A pure "write once, never look" policy is the clean
    experiment; a checkpointed policy is the useful product. They are different
    arms and I would run the clean one first.
  2. Should B's tool be fanout as built (N workers, one call, partitioned
    grant), or a narrower transform verb? The existing tool is more capable
    than B's thesis requires, and testing B with a tool that permits agency
    would blur exactly the M/B line the experiment exists to draw.
  3. Is a new scenario needed for width? c-frontier-1 fields 3 roles, so
    MAX_FANOUT_WIDTH is never approached and P6 ("no batch wider than 3")
    holds trivially.

0. The verdict this came from, unchanged

Preserved verbatim as the motivation for the design above.

Short answer: 27B-only today, with the worker seam built and opt-in — not M, not H as a default. But the reasoning matters more than the verdict, because the evidence for that recommendation is weaker than it looks.

  What actually supports it

The M/H comparison produced no data. So the recommendation rests on priors from adjacent work in this repo, not on the series:

  - league-commander measured hierarchy at 2.4–4.4× the cost of a flat mind for identical results, with ~95% of the bill being prompt tokens, and the commander overriding 1 of 82 proposals.
  - The muse arms (the closest prior analogue to "a second mind in the loop") returned INCONCLUSIVE while measuring real harm — 9 of 16 tool-arm runs came back NO_ANSWER.
- C1-E just ceilinged: the flat 27B scored 5/5 heading to 5-or-6 of 6 on the easiest rung. On the one rung with data, there is no headroom for an orchestrated arm to win.

That's a consistent direction across four experiments: orchestration costs multiples and hasn't yet bought anything measurable on this rig.

Why that's not the whole story

The worker is a genuinely different proposition from the muse, and the priors above are all about counsel, not execution:

┌───────────────────────┬────────────┬───────────────────┐
│                       │ cortex 27B │  worker 35B A3B   │
├───────────────────────┼────────────┼───────────────────┤
│ single-stream         │ ~23 tok/s  │ 76.4 tok/s (3.3×) │
├───────────────────────┼────────────┼───────────────────┤
│ effective concurrency │ 1          │ ~9                │
├───────────────────────┼────────────┼───────────────────┤
│ errors in 52 calls    │ —          │ 0                 │
└───────────────────────┴────────────┴───────────────────┘

A flat mind cannot dispatch concurrent units at all. That's why the pre-registration says separation on the fan-out rung is structural rather than hopeful — it's the one place where the orchestrated arms should win by construction, not by luck. For work that is genuinely wide and independent (N file edits, N test runs, N searches), the 27B alone is leaving ~9× parallelism unused, and no amount of ceiling on a sequential reasoning task speaks to that.

The path I'd take

1. Default stays flat 27B. It's the only configuration with measured evidence, and the operator's standing rule cuts both ways — an unmeasured benefit shouldn't ship as default any more than a measured failure mode should.
  2. Wire the worker as an explicit fan-out tool, opt-in, off. Scope it to where the win is structural (wide, independent, bounded work), not to general delegation. This is M-shaped but deliberately narrower than M.
  3. Defer H's router. H is the more powerful design and matches the Opus–Sonnet framing you described, but it introduces a component nobody can currently evaluate — the plan itself parks "hybrid routing quality has no precedent grader" as an open risk. Shipping a router you can't grade means the architecture's quality becomes unfalsifiable.

  If forced to pick M or H to build first: M. Not because it's better, but because H contains an unmeasurable part and M doesn't.

Four operational facts colleague needs regardless of architecture

  • Set the request timeout from max_tokens / measured_rate, not a latency percentile. The cortex generates 11–14k tokens on real reasoning turns; a 300s timeout silently deleted every generation past ~6,975 tokens and scored two of them as wrong answers.
  • Fan-out width is 8, not 14. The advertised ×14 is KV-pool arithmetic, not measured concurrency.
  • Dial the worker directly on Thor. The local proxy reports feasible: false.
  • Don't trust /capabilities for modality. It understated cortex vision; fixed upstream but possibly not deployed.

What would change my answer

A single measured result on the fan-out rung showing wall-clock separation. That's the cheapest decisive experiment available, and it's the one the ladder never reached — if you want the M/H question settled rather than reasoned about, run rung F before rung C2.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions