Skip to content

feat: {.ffiHandle.} — export complex live objects as opaque handles#81

Merged
gmelodie merged 5 commits into
masterfrom
feat/ffi-handle
Jun 16, 2026
Merged

feat: {.ffiHandle.} — export complex live objects as opaque handles#81
gmelodie merged 5 commits into
masterfrom
feat/ffi-handle

Conversation

@gmelodie

@gmelodie gmelodie commented Jun 12, 2026

Copy link
Copy Markdown
Collaborator

Summary

Implements #80. Adds {.ffiHandle.}, letting an .ffi. proc return — or receive — a complex, non-CBOR-serializable live object (a Waku, a Kernel, a session) by reference instead of by value.

The object never crosses the FFI boundary. Its wire form is a plain uint64; the live ref lives in a per-ctx registry and is reconstituted on the way back in. A given type is either a value type ({.ffi.}, copied) or a handle type ({.ffiHandle.}, referenced) — never both — so there is no per-proc "send the object vs send a handle" choice.

type Kernel {.ffiHandle.} = ref object
  ...

proc demoKernel*(d: Demo, req: MakeReq): Future[Result[Kernel, string]] {.ffi.}   # returns a handle
proc demoKernelSend*(k: Kernel, msg: string): Future[Result[string, string]] {.ffi.}  # handle is the receiver

Design

  • ffi/ffi_handles.nimFFIHandleRegistry, mirroring the FFIEventRegistry idiom: monotonic nextId (0 = null, ids never recycled) + a Table. register / lookup / release / releaseAll. Each entry records its type name, so a stale, forged, or wrong-type id misses cleanly → RET_ERR, never a use-after-free. Single-threaded (FFI-thread) access, so no locking. Handle types inherit an injected FFIHandleRoot base so heterogeneous refs are storable under one static type.
  • FFIContext — new handles field, init/deinit alongside eventRegistry. Refs are freed on the FFI thread (ffiThreadBody defer) because refc heaps are thread-local.
  • Macro forks in ffi/internal/ffi_macro.nim:
    • ffiHandle type macro — validates ref object, injects of FFIHandleRoot, records the name.
    • storageType — a handle param stores as uint64 in the per-proc Req.
    • Return path — a handle return is registered; only the id is CBOR-encoded.
    • Param path — a handle param is lookuped, returning RET_ERR on a miss.
    • First-param (receiver) fork — a handle receiver resolves FFIContext[LibType] / pool from the lib type recorded by declareLibrary.
    • Composes with {.ffiRaw.} with zero special-casing: the wire type is already uint64, so raw bodies call ctx.handles.lookup/register/release by hand.
  • Codegen (rust / cpp / cddl) — handle params and returns map to u64 / uint64_t / uint; no struct is emitted for a handle type. The flat kernel_send(ctx, handle, …) form works; the optional OO/RAII wrapper class is left for a follow-up.
  • declareLibrary now also declares the <LibType>FFIPool var (previously only ffiCtor/ffiDtor did) and records the lib type name.

Testing

  • tests/unit/test_ffi_handle.nim: handle round-trip (out as uint64 → reconstituted live object), handle-as-receiver, and forged / null id → RET_ERR. Passes under orc and refc, and is ASAN/UBSan-clean.
  • Existing suites pass; the timer/echo example bindings regenerate unchanged (no codegen regression); nph --check clean.

@gmelodie
gmelodie force-pushed the feat/ffi-handle branch 2 times, most recently from deb3be4 to 772aab2 Compare June 12, 2026 15:18
@gmelodie
gmelodie marked this pull request as ready for review June 12, 2026 18:04
@gmelodie
gmelodie changed the base branch from master to release/v0.1 June 12, 2026 19:50
@gmelodie
gmelodie requested a review from Ivansete-status June 12, 2026 20:00
@gmelodie
gmelodie changed the base branch from release/v0.1 to master June 16, 2026 14:32
@gmelodie
gmelodie changed the base branch from master to release/v0.1 June 16, 2026 14:44
@gmelodie
gmelodie changed the base branch from release/v0.1 to master June 16, 2026 14:47
(cherry picked from commit 498416b)
gmelodie and others added 3 commits June 16, 2026 12:16
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

@Ivansete-status Ivansete-status left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks for it! 💯

Comment thread ffi/ffi_handles.nim Outdated
Comment thread ffi/internal/ffi_macro.nim Outdated
Comment thread ffi/internal/ffi_macro.nim
Comment thread ffi/ffi_handles.nim Outdated
@gmelodie
gmelodie merged commit 021f469 into master Jun 16, 2026
83 checks passed
@gmelodie
gmelodie deleted the feat/ffi-handle branch June 16, 2026 17:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants