feat(host): typed host callbacks via {.ffiHost.} (WIP, increments 1-5)

[Ivansete-status]

Status: draft / work-in-progress. Roadmap item #1 from docs/future-work.md — pausing here to resume later. Functionally complete for the raw string case, from Nim all the way to an idiomatic Go API.

What this adds

A new pragma {.ffiHost.} — the inverse of {.ffi.}. A bodyless

proc fetchToken(key: string): Future[Result[string, string]] {.ffiHost.}

declares a function the host implements; a {.ffi.} handler can await it. This is the "a lower layer needs to read from a higher one" case from logos-delivery #3865. Events are lib→host fire-and-forget; this is lib→host and back, awaited.

Key constraint (see docs/design-host-callbacks.md): a host answering from its own thread can't complete a chronos Future directly, so completion mirrors the request path in reverse — park the answer + signal, drain and complete on the FFI loop thread. Each call carries a callId correlation id; a late/unknown completion is dropped, never a crash.

Increments (all landed, orc+refc green)

1. ffi/ffi_host.nim — FFIHostRegistry + FFIPendingTable (lock-guarded data structures) + unit tests.
2. Completion bridge on FFIContext — FFICompletionQueue (GC-free c_malloc nodes); completeHostCall parks the answer and fires the existing reqSignal (no second signal); the loop drains each iteration; failAllPending on shutdown so an unanswered call can't hang teardown.
3. The {.ffiHost.} macro — generates the async body (resolve thread-local registry/table → look up by snake_case wire name → newPending a callId → invoke host fn with the raw request → await).
4. C ABI — declareLibrary emits <lib>_register_host_fn / <lib>_host_complete; c.nim emits the FFIHostFn typedef + decls. Cross-thread e2e (test_ffi_host_e2e): a {.ffi.} handler awaits a host fn answered from a separate worker thread.
5. FFIHostMeta registry + Go generator — a //export cgo trampoline + cgo.Handle + per-host Set<Name>(func(string) (string, error)); the closure runs on a goroutine (non-blocking contract) then answers via host_complete. Validated by go run in examples/host_demo/.

Plus a token→callId rename for clarity.

Validation

• 16 host unit tests (orc + refc).
• 19 ffi_context integration tests stay green (no regression).
• examples/host_demo Go round-trip via go run: Go UseToken → Nim {.ffi.} → await fetchToken {.ffiHost.} → Go trampoline → goroutine → host_complete → result back in Go.

Not done yet (resume points)

• Increment 6 — struct/typed args + results, and the per-proc {.ffiHost: cbor.} format arg (decision recorded in docs/design-abi-format.md; default raw, library-wide declareLibrary(defaultAbiFormat = …)).
• Other language wrappers — C++/Rust (this branch), Swift (feat(codegen): native Swift generator + iOS example over the C ABI #59) / Kotlin (feat(codegen): native Kotlin/JNI generator + Android example over the C ABI #60) on their branches.
• The exported C shims are verified by a symbol check; a literal C-driver e2e (vs. the Nim cross-thread e2e) is a nice-to-have.

🤖 Generated with Claude Code