Skip to content

Add Go + Python codegen targets and a cross-target parity harness #94

Description

@gmelodie

Problem

Codegen targets are Rust and C++ only. genBindings() (ffi/internal/ffi_macro.nim) dispatches targetLanggenerateRustCrate / generateCppBindings / generateCddlBindings, else error. There's no Go or Python target and no test that the same type cases round-trip identically across wrappers.

The registries (ffiProcRegistry/ffiTypeRegistry/ffiEventRegistry) and the CBOR envelope are language-agnostic, so each new target is "render the same metadata in new syntax + a CBOR lib + a small runtime shim" — the rust.nim pattern.

Proposal

5a — Python (ctypes), codegen/python.nim. dataclass per type; typed methods per proc that CBOR-encode args (cbor2), call the exported symbol via ctypes.CDLL, decode the result; event registration mapping the C callback to a Python callable; create/destroy. Hand-written _ffi_runtime.py shim under codegen/templates/python/. Dispatch "python"/"py".

5b — Go (cgo), codegen/go.nim. struct per type with cbor tags (fxamacker/cbor/v2); method per proc returning (T, error); import "C" against the generated .h; callbacks via exported Go funcs + an integer-keyed handle map under sync.Mutex (cgo forbids passing Go pointers to C); Close() + runtime.SetFinalizer. Template dir codegen/templates/go/. Dispatch "go".

5c — parity harness, tests/parity/. One Nim "type zoo" lib (primitives, enums, distinct, seq, array, Option, nested objects, handles, events — extend the timer example); a driver per language calling every proc with known inputs/outputs; a task that builds the lib, regenerates all four wrappers, runs each driver. Add check_bindings_{python,go} (regenerate + git diff --exit-code) folded into check_bindings.

Gotchas

  • cgo pointer rule — never hand a Go pointer to C; use the integer-keyed handle registry for callback context (the common cgo footgun).
  • Python GIL — the C callback fires on the event thread; acquire the GIL before touching Python objects (verify CFUNCTYPE does this).
  • Pin and document CBOR libs (cbor2, fxamacker/cbor) in generated requirements.txt/go.mod.
  • CI cost: gate python/go parity behind a single non-sanitized job initially.
  • Update the genBindings() doc comment + the invalid-targetLang error message (currently rust/cpp/cddl only).

Tests

  • The parity harness is the test; require green in CI before merge.
  • check_bindings_{python,go} guard against drift, matching the rust/cpp discipline.

Effort: medium per language (each mirrors rust.nim). The parity harness is the most valuable, reusable piece — build it even if only one new language lands.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions