Skip to content

Commit a557c65

Browse files
feat(examples): in-library chronos CBOR server + cross-platform IPC CI
Adds the "remote channel" of the in-process/remote split: the library serving itself over a CBOR socket. Rather than an external process linking the CBOR C ABI, examples/timer/ipc/serve.nim is compiled into libmy_timer (only under -d:ffiIpcServe, so every other build is untouched) and runs a chronos socket server that, per request, decodes CBOR at the socket edge and calls the library's own async procs *directly* — native, in-process, zero serialization between the socket layer and the logic. The server *is* the library, so there is no FFI boundary and no callback bridge inside it. Exposed as `my_timer_serve(address)`. Why CBOR (not native) at the lib boundary here: over a wire the data arrives serialized regardless, so a native ABI would only relocate the decode and add POD marshalling for no gain. CBOR-on-the-wire / direct-call-in-process is the correct shape for IPC — "native locally, CBOR for IPC." Portable client + host: serve_host.nim starts the server and client.nim is a lib-free chronos client speaking the same proto.h framing. Both use chronos sockets, so the example builds and runs on Linux, macOS and Windows over TCP (unix sockets remain a POSIX bonus). The original POSIX C server.c/client.c are kept as the C-language variant. CI: a compiled cross-platform driver (tests/e2e/ipc/run_roundtrip.nim) builds the dylib + host + client, spawns the server, round-trips over loopback TCP and asserts the replies; wired as `nimble test_ipc` and a 3-OS CI matrix (ubuntu/macos/windows) — closing the gap where the IPC example had no automated validation at all. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent f08cb79 commit a557c65

10 files changed

Lines changed: 468 additions & 0 deletions

File tree

.github/workflows/ci.yml

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -208,6 +208,53 @@ jobs:
208208
nim-versions: ${{ needs.versions.outputs.nim-versions }}
209209
nimble-version: ${{ needs.versions.outputs.nimble }}
210210

211+
ipc:
212+
name: IPC round-trip
213+
needs: versions
214+
strategy:
215+
fail-fast: false
216+
matrix:
217+
os: [ubuntu-22.04, macos-15, windows-latest]
218+
runs-on: ${{ matrix.os }}
219+
steps:
220+
- uses: actions/checkout@v4
221+
222+
- name: Setup Nim
223+
uses: jiro4989/setup-nim-action@v2
224+
with:
225+
nim-version: "2.2.4"
226+
repo-token: ${{ secrets.GITHUB_TOKEN }}
227+
228+
- name: Install Nimble ${{ needs.versions.outputs.nimble }}
229+
shell: bash
230+
run: |
231+
if [ "$RUNNER_OS" == "Windows" ]; then
232+
export PATH="$GITHUB_WORKSPACE/.nim_runtime/bin:$PATH"
233+
fi
234+
cd /tmp && nimble install "nimble@${{ needs.versions.outputs.nimble }}" -y
235+
echo "$HOME/.nimble/bin" >> $GITHUB_PATH
236+
237+
- name: Cache nimble deps
238+
id: cache-nimbledeps
239+
uses: actions/cache@v4
240+
with:
241+
path: |
242+
nimbledeps/
243+
nimble.paths
244+
key: ${{ runner.os }}-nimbledeps-2.2.4-${{ hashFiles('*.nimble') }}
245+
restore-keys: |
246+
${{ runner.os }}-nimbledeps-2.2.4-
247+
${{ runner.os }}-nimbledeps-
248+
249+
- name: Install nimble deps
250+
if: steps.cache-nimbledeps.outputs.cache-hit != 'true'
251+
shell: bash
252+
run: nimble setup --localdeps -y
253+
254+
- name: IPC round-trip
255+
shell: bash
256+
run: nimble test_ipc
257+
211258
auto-assign:
212259
name: Auto-assign PR author
213260
if: github.event_name == 'pull_request' && github.event.action == 'opened'

examples/timer/ipc/.gitignore

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,3 +3,5 @@
33
/libmy_timer.dylib
44
/libmy_timer.so
55
/.srv.pid
6+
/serve_host
7+
/libmy_timer.dll

examples/timer/ipc/README.md

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -86,6 +86,40 @@ two machines may differ in OS, architecture, or endianness. The client needs
8686
only `my_timer_cbor.h` (or a CBOR library in its own language) — not the
8787
compiled timer library.
8888

89+
## Portable variant — the library serves *itself* (Nim / chronos)
90+
91+
The C `server`/`client` above use POSIX sockets. The portable counterpart runs
92+
on **Linux, macOS and Windows** and demonstrates a cleaner shape: instead of an
93+
external process linking the CBOR ABI, the **library exposes itself over the
94+
socket**.
95+
96+
- **`serve.nim`** is compiled into `libmy_timer` only with `-d:ffiIpcServe`. It
97+
runs a chronos socket server and, for each decoded request, calls the
98+
library's own async procs **directly** — a plain in-process call, native, with
99+
**zero serialization between the socket layer and the logic**. CBOR exists
100+
only at the socket edge. There is no FFI boundary inside the server because
101+
the server *is* the library. Exposed as `my_timer_serve(address)`.
102+
- **`serve_host.nim`** is a trivial host that links the library and starts it.
103+
- **`client.nim`** is the cross-platform client (chronos sockets), the Nim
104+
analogue of `client.c`; it does not link the library.
105+
106+
This is the **remote channel** of the in-process / remote split: native locally
107+
(the serve loop never serializes to reach the logic), CBOR only on the wire.
108+
109+
```sh
110+
# one-command, asserted round-trip over loopback TCP (what CI runs):
111+
nimble test_ipc
112+
113+
# or by hand (TCP works everywhere; unix:<path> also works on POSIX):
114+
nim c --app:lib --noMain --nimMainPrefix:libmy_timer -d:ffiIpcServe \
115+
-o:examples/timer/ipc/libmy_timer.dylib examples/timer/timer.nim
116+
nim c --passL:-Lexamples/timer/ipc --passL:-lmy_timer \
117+
-o:examples/timer/ipc/serve_host examples/timer/ipc/serve_host.nim
118+
nim c -o:examples/timer/ipc/client examples/timer/ipc/client.nim
119+
examples/timer/ipc/serve_host tcp:127.0.0.1:9099 &
120+
examples/timer/ipc/client tcp:127.0.0.1:9099
121+
```
122+
89123
## Notes
90124

91125
- Every `{.ffi.}` call is dispatched on the library's FFI thread, so the server

examples/timer/ipc/client.nim

Lines changed: 118 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,118 @@
1+
## Cross-platform CBOR-over-socket client for the timer IPC server.
2+
##
3+
## Speaks the same `proto.h` wire format as the C client, but uses chronos for
4+
## the socket so it builds and runs on Linux, macOS and Windows. It does not
5+
## link the library — it only needs the CBOR codec to build requests and read
6+
## replies (exactly what a remote client has).
7+
##
8+
## client tcp:127.0.0.1:9099 # any platform
9+
## client unix:/tmp/timer.sock # POSIX
10+
##
11+
## Exits 0 only if the round-trip values match the expectations, so it doubles
12+
## as the integration check the CI driver runs.
13+
import std/[os, strutils]
14+
import chronos
15+
import chronos/transports/stream
16+
import results
17+
import ffi/cbor_serial
18+
19+
# Local mirrors of the wire shapes (a remote client defines its own — it never
20+
# imports the library). Field names match the generated CBOR ABI.
21+
type
22+
EchoReqWire = object
23+
message: string
24+
delayMs: int
25+
EchoReqEnv = object
26+
req: EchoReqWire
27+
EchoRespWire = object
28+
echoed: string
29+
timerName: string
30+
31+
proc writeU32(t: StreamTransport, v: uint32) {.async.} =
32+
let b = @[byte(v shr 24), byte(v shr 16), byte(v shr 8), byte(v and 0xFF)]
33+
discard await t.write(b)
34+
35+
proc readU32(t: StreamTransport): Future[uint32] {.async.} =
36+
var b: array[4, byte]
37+
await t.readExactly(addr b[0], 4)
38+
return
39+
(uint32(b[0]) shl 24) or (uint32(b[1]) shl 16) or (uint32(b[2]) shl 8) or
40+
uint32(b[3])
41+
42+
proc call(
43+
t: StreamTransport, meth: string, payload: seq[byte]
44+
): Future[(int32, seq[byte])] {.async.} =
45+
# Request frame: [u32 method_len][method][u32 payload_len][payload].
46+
await writeU32(t, uint32(meth.len))
47+
if meth.len > 0:
48+
discard await t.write(meth)
49+
await writeU32(t, uint32(payload.len))
50+
if payload.len > 0:
51+
discard await t.write(payload)
52+
# Response frame: [i32 ret][u32 len][body].
53+
let ret = cast[int32](await readU32(t))
54+
let blen = await readU32(t)
55+
var body = newSeq[byte](int(blen))
56+
if blen > 0'u32:
57+
await t.readExactly(addr body[0], int(blen))
58+
return (ret, body)
59+
60+
proc parseAddress(a: string): TransportAddress =
61+
if a.startsWith("unix:"):
62+
return initTAddress(a[5 ..^ 1])
63+
elif a.startsWith("tcp:"):
64+
let hp = a[4 ..^ 1]
65+
let sep = hp.rfind(':')
66+
doAssert sep > 0, "tcp address must be tcp:<host>:<port>"
67+
return initTAddress(hp[0 ..< sep], Port(parseInt(hp[sep + 1 ..^ 1])))
68+
else:
69+
raise newException(ValueError, "address must be unix:<path> or tcp:<host>:<port>")
70+
71+
proc connectWithRetry(ta: TransportAddress): Future[StreamTransport] {.async.} =
72+
# The server may still be starting; retry briefly so the example/CI is not
73+
# racing socket setup.
74+
for _ in 0 ..< 100:
75+
try:
76+
return await connect(ta)
77+
except CatchableError:
78+
await sleepAsync(50.milliseconds)
79+
return await connect(ta) # last attempt: surface the real error
80+
81+
proc run(address: string): Future[bool] {.async.} =
82+
let t = await connectWithRetry(parseAddress(address))
83+
var ok = true
84+
echo "[client] connected"
85+
86+
# 1) version — empty request, response is a CBOR text string.
87+
block:
88+
let (ret, body) = await t.call("version", @[byte 0xA0]) # empty CBOR map {}
89+
let v = cborDecode(body, string)
90+
if ret == 0 and v.isOk:
91+
echo "[client] version = ", v.get()
92+
ok = ok and v.get() == "nim-timer v0.1.0"
93+
else:
94+
echo "[client] version failed (ret=", ret, ")"
95+
ok = false
96+
97+
# 2) echo — nested request, response is an EchoResponse map.
98+
block:
99+
let req = cborEncode(EchoReqEnv(req: EchoReqWire(message: "hello over the wire", delayMs: 5)))
100+
let (ret, body) = await t.call("echo", req)
101+
let r = cborDecode(body, EchoRespWire)
102+
if ret == 0 and r.isOk:
103+
echo "[client] echo.echoed= ", r.get().echoed
104+
echo "[client] echo.timer = ", r.get().timerName
105+
ok = ok and r.get().echoed == "hello over the wire" and r.get().timerName == "ipc-server"
106+
else:
107+
echo "[client] echo failed (ret=", ret, ")"
108+
ok = false
109+
110+
await t.closeWait()
111+
echo "[client] done"
112+
return ok
113+
114+
when isMainModule:
115+
if paramCount() != 1:
116+
stderr.writeLine "usage: client <tcp:host:port | unix:path>"
117+
quit(2)
118+
quit(if waitFor run(paramStr(1)): 0 else: 1)

examples/timer/ipc/serve.nim

Lines changed: 177 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,177 @@
1+
## In-library CBOR-over-socket server — the *remote channel* of the
2+
## in-process/remote split.
3+
##
4+
## Included into `timer.nim` only under `-d:ffiIpcServe`, so it shares the
5+
## module scope: it can build a `MyTimer` and call the library's async procs
6+
## (`myTimerEcho`, …) **directly** — a plain Nim call, native, in-process, with
7+
## zero serialization between the socket layer and the logic. CBOR exists only
8+
## at the socket edge: each request is decoded, dispatched to the matching proc,
9+
## and the result re-encoded. There is no FFI boundary and no callback bridge
10+
## inside the server because the server *is* the library.
11+
##
12+
## Exposed as `my_timer_serve(address)` (C ABI) so a trivial host can start it;
13+
## the wire protocol is the same `proto.h` framing the C client/server already
14+
## speak, so the existing CBOR client works against this server unchanged.
15+
16+
import std/[os, strutils]
17+
import chronos
18+
import chronos/transports/stream
19+
import results
20+
import ffi/cbor_serial
21+
import ffi/ffi_events
22+
23+
# The async procs fire library events (e.g. echo -> onEchoFired). Off the FFI
24+
# thread that would log "event registry not set"; give this thread an empty
25+
# registry so dispatch finds zero listeners and stays quiet. Delivering events
26+
# to remote clients is separate, future work.
27+
var serveEventRegistry: FFIEventRegistry
28+
29+
# Wire request envelopes. The field names mirror the generated CBOR request
30+
# ABI (each `{.ffi.}` proc packs its args under their Nim parameter names), so
31+
# the same bytes the C client builds decode straight into these.
32+
type
33+
EchoReqEnv = object
34+
req: EchoRequest
35+
ComplexReqEnv = object
36+
req: ComplexRequest
37+
ScheduleReqEnv = object
38+
job: JobSpec
39+
retry: RetryPolicy
40+
schedule: ScheduleConfig
41+
42+
const
43+
RetOk: int32 = 0 # mirrors RET_OK / RET_ERR in my_timer_cbor.h
44+
RetErr: int32 = 1
45+
46+
proc toBytes(s: string): seq[byte] =
47+
var b = newSeq[byte](s.len)
48+
if s.len > 0:
49+
copyMem(addr b[0], unsafeAddr s[0], s.len)
50+
return b
51+
52+
proc toStr(b: openArray[byte]): string =
53+
var s = newString(b.len)
54+
if b.len > 0:
55+
copyMem(addr s[0], unsafeAddr b[0], b.len)
56+
return s
57+
58+
proc readU32(t: StreamTransport): Future[uint32] {.async.} =
59+
# Frame integers are network byte order (big-endian), per proto.h.
60+
var b: array[4, byte]
61+
await t.readExactly(addr b[0], 4)
62+
return
63+
(uint32(b[0]) shl 24) or (uint32(b[1]) shl 16) or (uint32(b[2]) shl 8) or
64+
uint32(b[3])
65+
66+
proc writeResponse(t: StreamTransport, ret: int32, body: seq[byte]) {.async.} =
67+
# Response frame: [i32 ret][u32 len][body], big-endian.
68+
var hdr = newSeq[byte](8)
69+
let r = cast[uint32](ret)
70+
hdr[0] = byte(r shr 24)
71+
hdr[1] = byte(r shr 16)
72+
hdr[2] = byte(r shr 8)
73+
hdr[3] = byte(r and 0xFF)
74+
let l = uint32(body.len)
75+
hdr[4] = byte(l shr 24)
76+
hdr[5] = byte(l shr 16)
77+
hdr[6] = byte(l shr 8)
78+
hdr[7] = byte(l and 0xFF)
79+
discard await t.write(hdr)
80+
if body.len > 0:
81+
discard await t.write(body)
82+
83+
proc dispatch(meth: string, payload: seq[byte]): Future[(int32, seq[byte])] {.async.} =
84+
# The library's own state object — no FFIContext needed, we call the async
85+
# procs directly. `MyTimer` only carries a name, so building it per request
86+
# is free and keeps the handler stateless.
87+
let timer = MyTimer(name: "ipc-server")
88+
case meth
89+
of "version":
90+
let r = await myTimerVersion(timer)
91+
if r.isOk:
92+
return (RetOk, cborEncode(r.get())) # string -> CBOR text
93+
return (RetErr, toBytes(r.error()))
94+
of "echo":
95+
let env = cborDecode(payload, EchoReqEnv)
96+
if env.isErr:
97+
return (RetErr, toBytes(env.error()))
98+
let r = await myTimerEcho(timer, env.get().req)
99+
if r.isOk:
100+
return (RetOk, cborEncode(r.get()))
101+
return (RetErr, toBytes(r.error()))
102+
of "complex":
103+
let env = cborDecode(payload, ComplexReqEnv)
104+
if env.isErr:
105+
return (RetErr, toBytes(env.error()))
106+
let r = await myTimerComplex(timer, env.get().req)
107+
if r.isOk:
108+
return (RetOk, cborEncode(r.get()))
109+
return (RetErr, toBytes(r.error()))
110+
of "schedule":
111+
let env = cborDecode(payload, ScheduleReqEnv)
112+
if env.isErr:
113+
return (RetErr, toBytes(env.error()))
114+
let r =
115+
await myTimerSchedule(timer, env.get().job, env.get().retry, env.get().schedule)
116+
if r.isOk:
117+
return (RetOk, cborEncode(r.get()))
118+
return (RetErr, toBytes(r.error()))
119+
else:
120+
return (RetErr, toBytes("unknown method: " & meth))
121+
122+
proc onConnection(
123+
server: StreamServer, transp: StreamTransport
124+
) {.async: (raises: []).} =
125+
try:
126+
while not transp.atEof():
127+
let mlen = await readU32(transp)
128+
var mbytes = newSeq[byte](int(mlen))
129+
if mlen > 0'u32:
130+
await transp.readExactly(addr mbytes[0], int(mlen))
131+
let plen = await readU32(transp)
132+
var payload = newSeq[byte](int(plen))
133+
if plen > 0'u32:
134+
await transp.readExactly(addr payload[0], int(plen))
135+
let (ret, body) = await dispatch(toStr(mbytes), payload)
136+
await writeResponse(transp, ret, body)
137+
except CatchableError:
138+
discard # peer closed or malformed frame: drop the connection
139+
try:
140+
await transp.closeWait()
141+
except CatchableError:
142+
discard
143+
144+
proc serveLoop(address: string) {.async.} =
145+
var server: StreamServer
146+
if address.startsWith("unix:"):
147+
let path = address[5 ..^ 1]
148+
removeFile(path) # clear a stale socket so bind() succeeds
149+
server = createStreamServer(initTAddress(path), onConnection, {ReuseAddr})
150+
elif address.startsWith("tcp:"):
151+
let hp = address[4 ..^ 1]
152+
let sep = hp.rfind(':')
153+
doAssert sep > 0, "tcp address must be tcp:<host>:<port>"
154+
server =
155+
createStreamServer(
156+
initTAddress(hp[0 ..< sep], Port(parseInt(hp[sep + 1 ..^ 1]))),
157+
onConnection,
158+
{ReuseAddr},
159+
)
160+
else:
161+
raise newException(ValueError, "address must be unix:<path> or tcp:<host>:<port>")
162+
server.start()
163+
echo "[serve] listening on ", address
164+
await server.join()
165+
166+
proc my_timer_serve(address: cstring): cint {.exportc, cdecl, dynlib.} =
167+
## C entry point: boot the library runtime and run the socket server forever.
168+
## Blocks the calling thread. `address` is "unix:<path>" or "tcp:<host>:<port>".
169+
initializeLibrary()
170+
initEventRegistry(serveEventRegistry)
171+
ffiCurrentEventRegistry = addr serveEventRegistry
172+
try:
173+
waitFor serveLoop($address)
174+
except CatchableError as e:
175+
echo "[serve] error: ", e.msg
176+
return 1
177+
return 0

0 commit comments

Comments
 (0)