contextvars: continuation-local storage

[coreyleavitt]

Summary

This PR adds chronos/contextvars: continuation-local storage for chronos, in the spirit of Python's contextvars/PEP 567. A context variable's binding follows a logical task through await suspensions, callback registrations, and combinators (race, allFutures, ...), while concurrent tasks stay isolated from each other. Typical uses are request IDs, authenticated users, tracing spans, and other "ambient" data that would otherwise have to be threaded through every procedure signature by hand.

We run this in production downstream and will maintain the branch regardless; it is offered upstream because both the feature and the queue fix it surfaced seem generally useful. No pressure or timeline attached: this is shared in the spirit of open source, and we're equally content if it lands, gets split up, or simply serves as a reference.

import chronos
import chronos/contextvars

type User = object
  name: string

let currentUser* {.contextVar.} = User(name: "anonymous")

proc audit(action: string) =
  echo currentUser.value.name, ": ", action

proc handleRequest(user: User) {.async.} =
  currentUser.withValue(user):
    await sleepAsync(10.milliseconds)   # binding survives suspension
    audit("query")                      # sees `user`, not the default

A context variable is a first-class typed key, ContextVar[T]. The {.contextVar.} pragma declares one as an ordinary let, deriving the variable's name and its introspection visibility from the declaration itself (let with a default, var without one for a required variable, enforced at compile time); the raw newContextVar and newRequiredContextVar constructors are the underlying primitives and stay public. The pragma requires Nim 2.x, since macro pragmas on let/var sections are a 2.x language capability; on 1.6 the constructors are the declaration path and the feature is otherwise identical, with the pragma compiled out entirely rather than emulated. Every operation is an ordinary generic defined once in the library: the ambient read cv.value, the snapshot read ctx[cv], the scoped binder cv.withValue(value): body (which binds for the dynamic extent of body and restores on every exit path: normal, exception, cancellation), and the boundness probes cv in ctx / cv.isBound. Nothing mints per-variable identifiers, so declarations cannot collide with existing API, and keys are ordinary values: they can be passed to procs, stored in collections, and used as Table keys. Reading a required variable while unbound raises a Defect (unbound-read is a contract violation, and using a Defect keeps readers out of raises effect tracking, so {.async: (raises: []).} procedures can read context variables freely). Bindings nest, innermost wins, and propagate into tasks spawned within the binder's extent. Two further primitives, currentContext() and withContext(ctx, body), snapshot and restore the full binding chain for cases that fall outside a single dynamic extent, such as a resource's independently-registered setup and teardown callbacks, which share no call stack. Snapshots are first-class values: they compare with == (chain identity), read with ctx[cv] without installing anything, and dumpContext(ctx)/$ render a snapshot's contents for logs and debugging, backed by a registry that keeps every key alive for the life of the process and lists only keys registered as visible. There is no imperative token API (PEP 567's ContextVar.set()/Token.reset()): within one logical task withValue already expresses every binding lifecycle, and across independently-scheduled callbacks a token could not work, since the dispatcher's restore-at-fire discipline unwinds any push a callback leaves behind regardless.

Full usage, semantics, and implementation notes are in docs/src/contextvars.md (added by this PR).

Performance

The design goal is cost proportional to use: a program that never declares a context variable should pay a cost indistinguishable, on each hot path, from a build without the feature at all.

Headline (refc: chronos's flagship consumer, nimbus-eth2, pins --mm:refc unconditionally). Cross-commit comparison of the submitted commits against their pre-contextvars base, callSoon schedule+fire (the hottest path a contextvars-free program pays on), 15 interleaved trials per side: refc 1.04x, bootstrap 95% CI [0.82, 1.26], Mann-Whitney p = 0.76, statistically indistinguishable from the base. orc 0.99x, CI [0.80, 1.20]. Earlier independent measurement sessions during development read 1.01x–1.11x on refc; the raw per-trial data behind the final numbers is retained and reproducible.

Per-call-class bill, both memory managers, verified by inspecting the generated C at each site (not inferred from throughput alone):

• leaf-callback fire: one TLS load + one predicted branch. On refc this compiles to a bare pointer load with no write barrier; on orc the same load is wrapped in ORC's owning-copy/destroy hooks, cheap for the common nil case.
• continuation resume: the above plus one unconditional TLS save/restore pair, required for correctness (a suspended continuation must not leak a stale binding into whatever fires next).
• capturing-callback construction (any callback that must observe registration-time bindings): one TLS load + one predicted branch; barrier-free on refc, and on orc the context-copy is skipped entirely when no binder is live.
• heap-field registration (e.g. registering a future's cancel callback): one write-barrier call per ref field under refc, paid once, at the point ownership transfers into the heap field.

Bound-value lookup walks the binding chain comparing key identities: one pointer compare per node, with no per-node runtime type test (measured: reading through a 16-deep binding chain costs ~12 ns on refc).

Struct growth: sizeof(AsyncCallback) +8 B (one pointer field); sizeof(Future) +16 B after accounting for two embedded callbacks.

Both-worlds intra-commit detail, measured on the submitted series (release builds; unused arm = no binder active in the whole program; bound arm = one context variable bound around the hot loop), refc:

metric	unused	bound
callSoon schedule+fire	31.3 ns/op	26.7 ns/op
sleepAsync(0) await chain	741.8 ns/op	913.3 ns/op
future create/await	105.6 ns/op	97.7 ns/op

orc:

metric	unused	bound
callSoon schedule+fire	36.9 ns/op	48.0 ns/op
sleepAsync(0) await chain	753.0 ns/op	910.2 ns/op
future create/await	89.6 ns/op	96.0 ns/op

(Rows where the bound column is at or below the unused column are within run-to-run noise of each other; construction skips the context copy entirely when no binder is live. The sleepAsync rows carry the largest run-to-run spread, with individual runs swinging between the bound arm faster and the bound arm slower, so their bound-vs-unused deltas are noise-dominated; the callSoon rows are the tighter signal for the schedule/fire path.)

benchmarks/bench_contextvars.nim (included in this PR) reproduces these tables; the cross-commit comparisons above used a small self-contained harness that needs a second checkout to compare against, which we are happy to provide if useful.

Queue transport: a named, separate improvement

While measuring the above, a pre-existing defect in chronos's callback queue surfaced: std/deques.popFirst returns its element by value, which is a proc, not a template, so every dequeue pays a copy-out and, under refc, three write-barrier calls per ref field (a dead zero-initialization of the hidden return slot, the copy-out itself, and clearing the vacated slot). This cost existed before this PR, on the queue's original single ref field (the callback's closure environment); adding contextvars' second ref field (the captured context) doubled it, and that doubling is what our own benchmarks caught.

This PR fixes it by replacing the three internal Deque fields on the dispatcher (callbacks, idlers, ticks) with a small purpose-built CallbackQueue, in chronos/internal/callbackqueue.nim, whose dequeue is a template with a barrier-free copy-out. Effect on refc: the dequeue side drops from three write-barrier calls per ref field to one, restoring per-hop cost to parity with what the pre-contextvars codebase already paid on its single ref field. orc is unaffected either way: its ownership model does not incur this cost regardless of queue shape, and the orc cross-commit ratio above shows no shift.

This change is the first, standalone commit of the series; it has no dependency on contextvars and is structured to be reviewable and cherry-pickable on its own. If maintainers would prefer to review and merge the queue fix as a preliminary PR before the contextvars feature itself, we are glad to split it out; the commit boundary already does the work.

Test coverage

• Synchronous semantics: declaration (pragma and raw constructor), defaults, nesting/shadowing, restore-on-every-exit-path (normal, exception, cancellation), and the binder contract.
• Async propagation: isolation across interleaved tasks, survival across sequential awaits, exception and cancellation paths, spawn-time inheritance into tasks started within a binder's extent, and per-scheduling-site capture coverage (callSoon, setTimer/ sleepAsync, callIdle, addReader, race/allFutures, closeSocket/closeHandle).
• Key identity and lifetime: same-name keys never alias (binding one is unobservable through the other), two same-type keys bound simultaneously each read back their own value, keys as Table keys and generic-proc parameters, and a captured context outliving a key's declaration scope stays sound.
• The "bridging independent callbacks" pattern: currentContext()/withContext() carrying a binding from an enter hook into a separately-scheduled exit hook.
• Compile-time drift detection: private-field/constructor-only enforcement on the internal callback types and chain nodes, the {.contextVar.} pragma's one-symbol expansion and its let/var grammar (on Nim 2.x, where the pragma exists), no custom == on keys, coexistence with std/tables.withValue, and constructor-string fidelity in dumpContext and UnboundContextVarDefect.varName, so a future refactor that reopens a capture bypass fails to compile rather than silently regressing.
• Public-surface pins: import chronos plus import chronos/contextvars expose exactly the intended API and none of the dispatcher internals; introspection-visibility semantics for pragma (star-driven) and raw-constructor (private param) declarations, including the registration/export decoupling case.
• A dedicated suite for the new CallbackQueue (growth during reentrant drain across capacity boundaries, zero-value-empty, sentinel fidelity, front-insertion ordering, queue integrity across a Defect unwind, and repeated-growth-under-memory-pressure coverage for the ref-field payload case).
• Restore-on-unwind is pinned on the load-bearing paths: a Defect raised from a callback fired under a binding that differs from the ambient context (the write-and-restore arm, for both the normal and the cancel-callback fire sites) leaves the prior ambient context intact; await inside withContext is pinned against the suspend-across-binder hazard; and the cross-thread callSoon empty-context contract has a dedicated test (bound origin thread, posting thread, receiving callback observes the default).
• Context snapshots: identity ==, ctx[cv] snapshot reads, boundness probes, required (must-bind) variables including async propagation, and dumpContext/$ rendering (including the placeholder path for value types without a $).
• The construction-ordering checks (keys are constructed on one thread, before threads start) run in a standalone driver binary that isolates each hostile suite in its own subprocess: one suite deliberately lets an AssertionDefect escape poll(), and the opt-in construction lock is one-way for the process's lifetime.
• The nimble test matrix gains one leg combining -d:chronosDebug with -d:chronosPreviewV5, so the strict-reentrancy callback-drain discipline is exercised together with the debug-mode invariant asserts rather than only apart from them.
• The full nimble test matrix passes unmodified on both --mm:refc and --mm:orc. The new queue module additionally compiles under --os:standalone, keeping it available to bare-metal configurations (cf. Support compiling timer on bare metal #697).
• One incidental fix: the nimble test task's benchmark-compile step matched startsWith("bench_") against paths returned by walkDirRec, which always carry the benchmarks/ prefix, so the step had never compiled anything. This PR corrects the match (and passes --threads:on explicitly, since Nim 1.6 has no threaded default), which puts the benchmark files back under CI's compile check; the re-enabled check exposed two latent 32-bit integer-division truncations in bench_http_fetch.nim, also fixed here.

Platform semantics, non-goals, and known gaps

• No compile-time opt-out flag is included. We measured the always-on cost first; if maintainers want a chronosFutureTracking-style build flag as a condition of merging, we can add one; happy to discuss the tradeoff (double test surface, a define that does not compose through nimble) given the measured numbers above.
• Windows completion paths carry the registration-time context: the completion-bearing state captures the active context when the operation is armed (registerWaitable, stream-server start(), explicit accept()), and the completion dispatch fires under it, so Windows waitable and stream-server callbacks propagate context like their epoll/kqueue counterparts (verified by a dedicated test on Windows CI). Low-level per-operation read/write completion trampolines intentionally stay context-free: they only drive an internal future, whose awaiter carries its own captured context.
• Cross-thread callSoon (the MPSC mechanism from add thread-safe dispatcher callback mechanism #694) fires its callbacks with an empty context, by construction: the queue's payload is a bare nimcall proc and a raw pointer, and a captured context chain is a GC ref that must not cross thread-local heaps under refc. This fails closed: bindings from another thread's tasks can never leak in; propagation begins at the first in-loop scheduling point on the receiving thread. This contract has a dedicated test.
• Reads inside a caller's own {.cast(gcsafe).} block interact with a Nim compiler defect: when a nested cast block exits, it cancels the enclosing block's cast for the statements that follow, so code after a .value read inside such a block is flagged as not GC-safe. The documentation carries a hoist-the-read workaround. The compiler defect is fixed upstream (Nested {.cast(gcsafe).} blocks: exiting the inner block cancels the outer block's cast for the statements after it nim-lang/Nim#26092, fix merged in fixes #26092; restore enclosing cast block state when a nested cast block exits nim-lang/Nim#26093); the fix lives on devel only, so the workaround stays relevant for released toolchains.
• Key construction is supported before thread creation (module-level declarations, as in every example). Every build, including release, automatically asserts that all keys are constructed on one thread, using a chronos-assigned per-thread generation rather than an OS thread id (which is recycled after a thread exits); the check is one compare-exchange on the cold construction path. A chronosDebug build additionally carries an opt-in assertion hook for embedders who want the stricter before-any-thread boundary checked at their own thread-creation point. Registered keys live for the life of the process, matching static-global semantics.
• No --os:standalone verification of the contextvars substrate itself (the queue module compiles standalone; asyncengine.nim does not, for a pre-existing, unrelated reason: an unconditional "operation system is not yet supported" import guard that predates this PR).

Public API surface

Frozen for this PR: ContextVar[T] with newContextVar (defaulted) and newRequiredContextVar (distinct procs, because a bool-typed default in the second positional slot would be ambiguous against the trailing private visibility param under a single overloaded name), the {.contextVar.} declaration pragma, value, [], withValue, contains/in and isBound, the name/hasDefault/private read-only accessors and an identity hash on keys, AsyncContext with ==, hash, and $, currentContext, withContext, dumpContext and ContextVarEntry, and UnboundContextVarDefect (carrying varName). No other new public identifiers. No version bump is included; that is left to maintainers' release process.

[coreyleavitt]

@vladopajic tagging you since I noticed #690 is circling the same problem and you've clearly been thinking about it. I built continuation-local storage for a downstream project and carried it on a fork of chronos for a while before folding it into this series; the core mechanic ended up the same as your experiment, capture a context reference and save/restore it around the resume, which I take as a good sign it's the right substrate.

The extra scope here came from requirements I ran into downstream rather than a different opinion on the design. I needed context to follow timer and callSoon callbacks as well as async continuations, so the capture point moved into the dispatcher's callback queue, and I needed several libraries to use the mechanism without coordinating on a shared context object, which is what pushed me from a single slot to typed keys (ContextVar[T], along the lines of Python's contextvars).

The pieces #690 has that this series doesn't, currentTaskFuture() and the context-switch callback hook, look orthogonal to me and could layer on top of this cleanly. If your experiment was headed somewhere specific, I'd rather line this up with that than run a parallel effort, so your take would be very welcome.

[vladopajic]

hey @coreyleavitt, notification from this PR found me in the bed, and i was intrigued to read it. reading it was interesting to see that there are others stumbling on this problem, and i was also puzzled why i was getting this notification (later realized the tag).

effort on my experiment was lazy, ai generated, and i haven't been dedicated enough to drive it forward with maintainers. may this be opportunity to come to a solution for this in chronos.

@arnetheduck is the wizard that can bring wisdom to this topic.

in the meantime i can get ready #690 (at least add some context to that PR explaining problem and solution). i will also read code from #702. one note about #702 is that i noticed that diff is big 6k lines so naturally every PR of this size is "wow, hold on! hold on! take it easy my friend" :)

[coreyleavitt]

@vladopajic thanks for taking a look! I certainly don't expect this to merge as is; I maintain a fork for downstream projects, so I tried to be as thorough as possible. The runtime code changes are about 550 lines, and the rest of the diff is tests, docs, and benchmarks.

If you want the short read, the second commit is the equivalent of your #690 experiment, and the typed API sits on top of it in the third.

Appreciate the warm response to a cold ping. Added context on #690 sounds great.