compile_transaction allocated 13,148 bytes to produce a 1,248-byte transaction in a
helium-program-library measurement: five instructions, 73 account slots, 31 distinct accounts.
That ratio matters more than it looks, because the SBF heap is bump-allocated and dealloc is a
no-op, so a program is bounded by the bytes it ever allocates rather than its peak live set. Every
intermediate allocation is retained for the life of the instruction.
Where it goes
pubkeys_to_metadata: HashMap<Pubkey, AccountMeta> starts empty and grows by doubling. For 31
entries that is roughly 3 → 7 → 14 → 28 → 56 slots, and on a bump allocator all five tables stay
resident, not just the last.accounts_to_index: HashMap<Pubkey, u8> is a second map over the same keys.remaining_accounts is always built, and many callers discard it: let (compiled_tx, _) = compile_transaction(...) is the common shape.
Suggested fix
The slot count is knowable up front as instructions.iter().map(|ix| ix.accounts.len() + 1).sum(),
so both structures can be sized once. At these sizes a Vec with a linear scan also beats a
HashMap: hashing 32-byte pubkeys ~73 times costs more than scanning a vector that never exceeds
31 entries. Returning remaining_accounts lazily, or exposing a variant that skips it, would save
another Vec of AccountMeta per call.
A local reimplementation along those lines measured 8,672 bytes saved on the same input, holding
the ordering contract (writable signers, read-only signers, writable, read-only, with the three
counts marking the boundaries) and producing indices that resolve to the same accounts. One
incidental benefit: within a priority group the order becomes insertion order, which is
deterministic, where sort_by over HashMap iteration order currently is not.
Why it is worth doing upstream
queue_end_epoch in helium-program-library exhausted the 32KB heap in production once already,
and again recently when two accounts were added to the transaction it compiles: it runs at CPI depth
2 under run_task_v0 and shares one heap with it, so compile_transaction's overhead is most of the
budget. That one has been worked around locally, but the same call appears in eleven other places in
that repo, including mini-fanout/schedule_task_v0.rs and dc-auto-top/schedule_task_v0.rs, which
schedule and reschedule live cranks. They all pay the same overhead and have the same ceiling.
Happy to open a PR if the approach looks right.
compile_transactionallocated 13,148 bytes to produce a 1,248-byte transaction in ahelium-program-librarymeasurement: five instructions, 73 account slots, 31 distinct accounts.That ratio matters more than it looks, because the SBF heap is bump-allocated and
deallocis ano-op, so a program is bounded by the bytes it ever allocates rather than its peak live set. Every
intermediate allocation is retained for the life of the instruction.
Where it goes
pubkeys_to_metadata: HashMap<Pubkey, AccountMeta>starts empty and grows by doubling. For 31entries that is roughly 3 → 7 → 14 → 28 → 56 slots, and on a bump allocator all five tables stay
resident, not just the last.
accounts_to_index: HashMap<Pubkey, u8>is a second map over the same keys.remaining_accountsis always built, and many callers discard it:let (compiled_tx, _) = compile_transaction(...)is the common shape.Suggested fix
The slot count is knowable up front as
instructions.iter().map(|ix| ix.accounts.len() + 1).sum(),so both structures can be sized once. At these sizes a
Vecwith a linear scan also beats aHashMap: hashing 32-byte pubkeys ~73 times costs more than scanning a vector that never exceeds31 entries. Returning
remaining_accountslazily, or exposing a variant that skips it, would saveanother Vec of
AccountMetaper call.A local reimplementation along those lines measured 8,672 bytes saved on the same input, holding
the ordering contract (writable signers, read-only signers, writable, read-only, with the three
counts marking the boundaries) and producing indices that resolve to the same accounts. One
incidental benefit: within a priority group the order becomes insertion order, which is
deterministic, where
sort_byoverHashMapiteration order currently is not.Why it is worth doing upstream
queue_end_epochinhelium-program-libraryexhausted the 32KB heap in production once already,and again recently when two accounts were added to the transaction it compiles: it runs at CPI depth
2 under
run_task_v0and shares one heap with it, socompile_transaction's overhead is most of thebudget. That one has been worked around locally, but the same call appears in eleven other places in
that repo, including
mini-fanout/schedule_task_v0.rsanddc-auto-top/schedule_task_v0.rs, whichschedule and reschedule live cranks. They all pay the same overhead and have the same ceiling.
Happy to open a PR if the approach looks right.