Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@ name: CI

on:
push:
branches: [master, main]
branches: [master, main, 'release/*']
pull_request:
branches: [master, main]
branches: [master, main, 'release/*']

jobs:
# Single source of truth for Nim / Nimble versions used by every job and
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -83,4 +83,4 @@ jobs:
if [ "$RUNNER_OS" == "Windows" ]; then
export PATH="$GITHUB_WORKSPACE/.nim_runtime/bin:$HOME/.nimble/bin:$PATH"
fi
nim c -r --mm:${{ matrix.mm }} -d:chronicles_log_level=WARN tests/unit/${{ inputs.test }}.nim
nim c -r --mm:${{ matrix.mm }} -d:chronicles_log_level=WARN -d:ffiAllowSignalHandler tests/unit/${{ inputs.test }}.nim
37 changes: 16 additions & 21 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,32 +4,12 @@ All notable changes to this project are documented in this file.

## [Unreleased]

### Changed
- User event callbacks now run on a dedicated event thread fed by a
bounded SPSC queue (default capacity 1024), so a slow listener can no
longer block the FFI thread or concurrent `add_event_listener` /
`remove_event_listener` calls
([#6](https://github.com/logos-messaging/nim-ffi/issues/6)).
- Replaced the dedicated watchdog thread with a heartbeat check that
runs on the event thread. The FFI thread advances an atomic heartbeat
each loop iteration; if it stalls for more than 1s past the start-up
grace window, the event thread emits the `not_responding` event.

### Added
- Queue-overflow handling: when the bounded event queue is full, the
library sets a sticky "stuck" flag, logs an error, fires
`not_responding` from the event thread, and rejects subsequent
`sendRequestToFFIThread` calls with `event queue stuck - library
cannot accept new requests`.

## [0.2.0] - 2026-06-04

Major release introducing the CBOR-based wire format, CBOR-backed FFI events
with a multi-listener registry, multi-language binding generation (C++, Rust,
CDDL), CI hardening with sanitizers, and several robustness fixes around
context lifetime and memory safety.

### Added
- **CBOR serialization** as the FFI wire format, replacing the previous
JSON/string-based `serial.nim`
([#23](https://github.com/logos-messaging/nim-ffi/pull/23)).
Expand Down Expand Up @@ -64,6 +44,21 @@ context lifetime and memory safety.
([#41](https://github.com/logos-messaging/nim-ffi/pull/41)).

### Changed
- Enforce `-d:noSignalHandler` at compile time: the build now fails with an
actionable message unless the consumer sets `-d:noSignalHandler` (libraries
embedded in a foreign host such as Go/Rust) or `-d:ffiAllowSignalHandler`
(standalone Nim binaries). Prevents the Nim runtime from installing its own
OS signal handlers and clobbering the host's — the cause of a real status-go
regression.
- User event callbacks now run on a dedicated event thread fed by a
bounded SPSC queue (default capacity 1024), so a slow listener can no
longer block the FFI thread or concurrent `add_event_listener` /
`remove_event_listener` calls
([#6](https://github.com/logos-messaging/nim-ffi/issues/6)).
- Replaced the dedicated watchdog thread with a heartbeat check that
runs on the event thread. The FFI thread advances an atomic heartbeat
each loop iteration; if it stalls for more than 1s past the start-up
grace window, the event thread emits the `not_responding` event.
- Removed the redundant `ffiType` macro; the `ffi` macro is now the single
authoring entry point
([#22](https://github.com/logos-messaging/nim-ffi/pull/22)).
Expand Down Expand Up @@ -91,7 +86,7 @@ context lifetime and memory safety.
and `rust_client`, the latter with a Tokio async variant) (#15).
- JSON/string-based FFI (de)serialization via `ffi/serial.nim`
(`ffiSerialize`/`ffiDeserialize`), with `tests/test_serial.nim` coverage.
(CBOR replaced this layer later, in 0.2.0.)
(CBOR replaced this layer later.)
- FFI context pool (`ffi/ffi_context_pool.nim`) using a fixed array of contexts.
- Test suite expansion: `test_alloc.nim`, `test_ctx_validation.nim`,
`test_ffi_context.nim`, `test_gc_compat.nim`.
Expand Down
1 change: 1 addition & 0 deletions examples/echo/cpp_bindings/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ add_custom_command(
COMMAND "${NIM_EXECUTABLE}" c
--mm:orc
-d:chronicles_log_level=WARN
-d:noSignalHandler
--app:lib
--noMain
"--nimMainPrefix:libecho"
Expand Down
11 changes: 9 additions & 2 deletions examples/echo/cpp_bindings/echo.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -406,7 +406,7 @@ typedef void (*FFICallback)(int ret, const char* msg, size_t len, void* user_dat
void* echo_create(const uint8_t* req_cbor, size_t req_cbor_len, FFICallback callback, void* user_data);
int echo_shout(void* ctx, FFICallback callback, void* user_data, const uint8_t* req_cbor, size_t req_cbor_len);
int echo_version(void* ctx, FFICallback callback, void* user_data, const uint8_t* req_cbor, size_t req_cbor_len);
int echo_destroy(void* ctx);
int echo_destroy(void* ctx, FFICallback callback, void* user_data);
uint64_t echo_add_event_listener(void* ctx, const char* event_name, FFICallback callback, void* user_data);
int echo_remove_event_listener(void* ctx, uint64_t listener_id);
} // extern "C"
Expand Down Expand Up @@ -516,7 +516,14 @@ class EchoCtx {
// context.
~EchoCtx() {
if (ptr_) {
echo_destroy(ptr_);
// `echo_destroy` is non-blocking at the C ABI: it parks the
// context for reuse and reports the outcome via the callback. Block
// here until that callback fires so the pool slot is fully drained
// and parked before this object goes away — otherwise a rapid
// create/destroy churn could outrun the recycle and exhaust the pool.
(void)ffi_call_([this](FFICallback cb, void* ud) {
return echo_destroy(ptr_, cb, ud);
}, timeout_);
ptr_ = nullptr;
}
}
Expand Down
2 changes: 1 addition & 1 deletion examples/echo/echo.nimble
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ requires "nim >= 2.2.4"
requires "chronos"
requires "chronicles"
requires "taskpools"
requires "https://github.com/logos-messaging/nim-ffi >= 0.2.0"
requires "https://github.com/logos-messaging/nim-ffi >= 0.1.4"

const nimFlags = "--mm:orc -d:chronicles_log_level=WARN"

Expand Down
1 change: 1 addition & 0 deletions examples/timer/cpp_bindings/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ add_custom_command(
COMMAND "${NIM_EXECUTABLE}" c
--mm:orc
-d:chronicles_log_level=WARN
-d:noSignalHandler
--app:lib
--noMain
"--nimMainPrefix:libmy_timer"
Expand Down
11 changes: 9 additions & 2 deletions examples/timer/cpp_bindings/my_timer.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -706,7 +706,7 @@ int my_timer_echo(void* ctx, FFICallback callback, void* user_data, const uint8_
int my_timer_version(void* ctx, FFICallback callback, void* user_data, const uint8_t* req_cbor, size_t req_cbor_len);
int my_timer_complex(void* ctx, FFICallback callback, void* user_data, const uint8_t* req_cbor, size_t req_cbor_len);
int my_timer_schedule(void* ctx, FFICallback callback, void* user_data, const uint8_t* req_cbor, size_t req_cbor_len);
int my_timer_destroy(void* ctx);
int my_timer_destroy(void* ctx, FFICallback callback, void* user_data);
uint64_t my_timer_add_event_listener(void* ctx, const char* event_name, FFICallback callback, void* user_data);
int my_timer_remove_event_listener(void* ctx, uint64_t listener_id);
} // extern "C"
Expand Down Expand Up @@ -816,7 +816,14 @@ class MyTimerCtx {
// context.
~MyTimerCtx() {
if (ptr_) {
my_timer_destroy(ptr_);
// `my_timer_destroy` is non-blocking at the C ABI: it parks the
// context for reuse and reports the outcome via the callback. Block
// here until that callback fires so the pool slot is fully drained
// and parked before this object goes away — otherwise a rapid
// create/destroy churn could outrun the recycle and exhaust the pool.
(void)ffi_call_([this](FFICallback cb, void* ud) {
return my_timer_destroy(ptr_, cb, ud);
}, timeout_);
ptr_ = nullptr;
}
}
Expand Down
1 change: 1 addition & 0 deletions examples/timer/rust_bindings/build.rs
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ fn main() {
cmd.arg("c")
.arg("--mm:orc")
.arg("-d:chronicles_log_level=WARN")
.arg("-d:noSignalHandler")
.arg("--app:lib")
.arg("--noMain")
.arg(format!("--nimMainPrefix:libmy_timer"))
Expand Down
4 changes: 3 additions & 1 deletion examples/timer/rust_bindings/src/api.rs
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,9 @@ unsafe impl Sync for MyTimerCtx {}
impl Drop for MyTimerCtx {
fn drop(&mut self) {
if !self.ptr.is_null() {
unsafe { ffi::my_timer_destroy(self.ptr); }
let _ = ffi_call_sync(self.timeout, |cb, ud| unsafe {
ffi::my_timer_destroy(self.ptr, cb, ud)
});
self.ptr = std::ptr::null_mut();
}
}
Expand Down
2 changes: 1 addition & 1 deletion examples/timer/rust_bindings/src/ffi.rs
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ extern "C" {
pub fn my_timer_version(ctx: *mut c_void, callback: FFICallback, user_data: *mut c_void, req_cbor: *const u8, req_cbor_len: usize) -> c_int;
pub fn my_timer_complex(ctx: *mut c_void, callback: FFICallback, user_data: *mut c_void, req_cbor: *const u8, req_cbor_len: usize) -> c_int;
pub fn my_timer_schedule(ctx: *mut c_void, callback: FFICallback, user_data: *mut c_void, req_cbor: *const u8, req_cbor_len: usize) -> c_int;
pub fn my_timer_destroy(ctx: *mut c_void) -> c_int;
pub fn my_timer_destroy(ctx: *mut c_void, callback: FFICallback, user_data: *mut c_void) -> c_int;
pub fn my_timer_add_event_listener(ctx: *mut c_void, event_name: *const c_char, callback: FFICallback, user_data: *mut c_void) -> u64;
pub fn my_timer_remove_event_listener(ctx: *mut c_void, listener_id: u64) -> c_int;
}
2 changes: 1 addition & 1 deletion examples/timer/timer.nimble
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ requires "nim >= 2.2.4"
requires "chronos"
requires "chronicles"
requires "taskpools"
requires "https://github.com/logos-messaging/nim-ffi >= 0.2.0"
requires "https://github.com/logos-messaging/nim-ffi >= 0.1.4"

const nimFlags = "--mm:orc -d:chronicles_log_level=WARN"

Expand Down
6 changes: 3 additions & 3 deletions ffi.nimble
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# ffi.nimble

version = "0.2.0"
version = "0.1.4"
author = "Institute of Free Technology"
description = "FFI framework with custom header generation"
license = "MIT or Apache License 2.0"
Expand All @@ -13,8 +13,8 @@ requires "chronicles"
requires "taskpools"
requires "cbor_serialization"

const nimFlagsOrc = "--mm:orc -d:chronicles_log_level=WARN"
const nimFlagsRefc = "--mm:refc -d:chronicles_log_level=WARN"
const nimFlagsOrc = "--mm:orc -d:chronicles_log_level=WARN -d:ffiAllowSignalHandler"
const nimFlagsRefc = "--mm:refc -d:chronicles_log_level=WARN -d:ffiAllowSignalHandler"

import std/[algorithm, os, strutils]

Expand Down
8 changes: 5 additions & 3 deletions ffi/codegen/cpp.nim
Original file line number Diff line number Diff line change
Expand Up @@ -339,7 +339,9 @@ proc generateCppHeader*(
[p.procName]
)
of FFIKind.DTOR:
lines.add("int $1(void* ctx);" % [p.procName])
lines.add(
"int $1(void* ctx, FFICallback callback, void* user_data);" % [p.procName]
)
# `declareLibrary` always exports the listener-registration ABI. Declare
# it here so the typed event-handler wiring below can call into it.
lines.add(
Expand Down Expand Up @@ -570,8 +572,8 @@ proc generateCppHeader*(
lines.add(" std::chrono::milliseconds timeout_;")
if events.len > 0:
# One owning entry per live listener, keyed by id. Destroyed after
# the destructor body runs `<lib>_destroy(ptr_)`, by which point the
# FFI side has joined its threads so no callback is mid-flight.
# the destructor blocks on `<lib>_destroy`'s recycle callback, by which
# point the FFI side has drained/parked the slot so no callback is mid-flight.
lines.add(
" std::unordered_map<std::uint64_t, std::unique_ptr<ListenerBase>> listeners_;"
)
Expand Down
12 changes: 11 additions & 1 deletion ffi/codegen/rust.nim
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,7 @@ fn main() {
cmd.arg("c")
.arg("--mm:orc")
.arg("-d:chronicles_log_level=WARN")
.arg("-d:noSignalHandler")
.arg("--app:lib")
.arg("--noMain")
.arg(format!("--nimMainPrefix:lib$2"))
Expand Down Expand Up @@ -206,6 +207,8 @@ proc generateFFIRs*(procs: seq[FFIProcMeta]): string =
lines.add(" pub fn $1($2) -> *mut c_void;" % [p.procName, params.join(", ")])
of FFIKind.DTOR:
params.add("ctx: *mut c_void")
params.add("callback: FFICallback")
params.add("user_data: *mut c_void")
lines.add(" pub fn $1($2) -> c_int;" % [p.procName, params.join(", ")])

# Listener-registration ABI — emitted on the Nim side by `declareLibrary`,
Expand Down Expand Up @@ -530,7 +533,14 @@ proc generateApiRs*(
lines.add("impl Drop for $1 {" % [ctxTypeName])
lines.add(" fn drop(&mut self) {")
lines.add(" if !self.ptr.is_null() {")
lines.add(" unsafe { ffi::$1(self.ptr); }" % [dtorProcName])
# `<lib>_destroy` is non-blocking at the C ABI: it parks the context for
# reuse and reports the outcome via the callback. Block until that callback
# fires so the pool slot is fully drained and parked before this handle goes
# away — otherwise rapid create/destroy churn could outrun the recycle and
# exhaust the pool. The recycle outcome is best-effort on drop, so discard it.
lines.add(" let _ = ffi_call_sync(self.timeout, |cb, ud| unsafe {")
lines.add(" ffi::$1(self.ptr, cb, ud)" % [dtorProcName])
lines.add(" });")
lines.add(" self.ptr = std::ptr::null_mut();")
lines.add(" }")
# `listeners` is dropped automatically after this body returns. By
Expand Down
1 change: 1 addition & 0 deletions ffi/codegen/templates/cpp/CMakeLists.txt.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ add_custom_command(
COMMAND "${NIM_EXECUTABLE}" c
--mm:orc
-d:chronicles_log_level=WARN
-d:noSignalHandler
--app:lib
--noMain
"--nimMainPrefix:lib{{LIB}}"
Expand Down
9 changes: 8 additions & 1 deletion ffi/codegen/templates/cpp/context_rule_of_5.hpp.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,14 @@
// context.
~{{CTX}}() {
if (ptr_) {
{{LIB}}_destroy(ptr_);
// `{{LIB}}_destroy` is non-blocking at the C ABI: it parks the
// context for reuse and reports the outcome via the callback. Block
// here until that callback fires so the pool slot is fully drained
// and parked before this object goes away — otherwise a rapid
// create/destroy churn could outrun the recycle and exhaust the pool.
(void)ffi_call_([this](FFICallback cb, void* ud) {
return {{LIB}}_destroy(ptr_, cb, ud);
}, timeout_);
ptr_ = nullptr;
}
}
Expand Down
Loading
Loading