Skip to content

Commit fc799d2

Browse files
committed
cleanup json and cbor readme
1 parent dd00098 commit fc799d2

3 files changed

Lines changed: 198 additions & 89 deletions

File tree

serde/cbor/README.md

Lines changed: 13 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,18 @@
11
# nim-serde CBOR
22

3-
This README details the usage of CBOR serialization and deserialization features offered by nim-serde, in compliance with [RFC 8949](https://datatracker.ietf.org/doc/html/rfc8949).
3+
The CBOR module in nim-serde provides serialization and deserialization for Nim values following [RFC 8949](https://www.rfc-editor.org/rfc/rfc8949.html). Unlike the JSON implementation, CBOR offers a stream-based API that enables direct binary serialization without intermediate representations.
4+
5+
> Note: While the JSON implementation supports serde modes via pragmas, the current CBOR implementation does not support the `serialize` and `deserialize` pragmas. The library will raise an assertion error if you try to use these pragmas with CBOR serialization.
6+
47

58
## Table of Contents
69
- [nim-serde CBOR](#nim-serde-cbor)
710
- [Table of Contents](#table-of-contents)
811
- [Serialization API](#serialization-api)
9-
- [Basic Serialization with Stream API](#basic-serialization-with-stream-api)
10-
- [Object Serialization](#object-serialization)
11-
- [Custom Type Serialization](#custom-type-serialization)
12-
- [Converting to CBOR with `toCbor`](#converting-to-cbor-with-tocbor)
12+
- [Stream API: Primitive Type and Sequence Serialization](#stream-api-primitive-type-and-sequence-serialization)
13+
- [Stream API: Object Serialization](#stream-api-object-serialization)
14+
- [Stream API: Custom Type Serialization](#stream-api-custom-type-serialization)
15+
- [Serialization without Stream API: `toCbor`](#serialization-without-stream-api-tocbor)
1316
- [Working with CborNode](#working-with-cbornode)
1417
- [Convenience Functions for CborNode](#convenience-functions-for-cbornode)
1518
- [Deserialization API](#deserialization-api)
@@ -24,7 +27,7 @@ This README details the usage of CBOR serialization and deserialization features
2427

2528
The nim-serde CBOR serialization API provides several ways to convert Nim values to CBOR.
2629

27-
### Basic Serialization with Stream API
30+
### Stream API: Primitive Type and Sequence Serialization
2831

2932
The `writeCbor` function writes Nim values to a stream in CBOR format:
3033

@@ -50,7 +53,7 @@ discard stream.writeCbor(@[1, 2, 3]) # Sequence
5053
let cborData = stream.data
5154
```
5255

53-
### Object Serialization
56+
### Stream API: Object Serialization
5457

5558
Objects can be serialized to CBOR format using the stream API:
5659

@@ -78,7 +81,7 @@ discard stream.writeCbor(person)
7881
let cborData = stream.data
7982
```
8083

81-
### Custom Type Serialization
84+
### Stream API: Custom Type Serialization
8285

8386
You can extend nim-serde to support custom types by defining your own `writeCbor` procs:
8487

@@ -114,7 +117,7 @@ discard userStream.writeCbor(user)
114117
let userCborData = userStream.data
115118
```
116119

117-
### Converting to CBOR with `toCbor`
120+
### Serialization without Stream API: `toCbor`
118121

119122
The `toCbor` function can be used to directly convert a Nim value to CBOR binary data:
120123

@@ -363,7 +366,7 @@ For deserialization, the library parses CBOR data into a `CborNode` representati
363366

364367
### Current Limitations
365368

366-
While the JSON implementation supports serde modes via pragmas, the current CBOR implementation does not support the `serialize` and `deserialize` pragmas. The library will raise an assertion error if you try to use these pragmas with CBOR serialization.
369+
This implementation does not support the `serialize` and `deserialize` pragmas. The library will raise an assertion error if you try to use these pragmas with CBOR serialization.
367370

368371
```nim
369372
import pkg/serde/cbor

serde/cbor/helpers.nim

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -36,10 +36,6 @@ template assertNoPragma*(value, pragma, msg) =
3636
when value.hasCustomPragma(pragma):
3737
raiseAssert(msg)
3838

39-
macro dot*(obj: object, fld: string): untyped =
40-
## Turn ``obj.dot("fld")`` into ``obj.fld``.
41-
newDotExpr(obj, newIdentNode(fld.strVal))
42-
4339
func floatSingle*(half: uint16): float32 =
4440
## Convert a 16-bit float to 32-bits.
4541
func ldexp(

serde/json/README.md

Lines changed: 185 additions & 75 deletions
Original file line numberDiff line numberDiff line change
@@ -1,27 +1,134 @@
11
# nim-serde JSON
22

3-
Explore JSON serialization and deserialization using nim-serde, an improved alternative to `std/json`.
3+
The JSON module in nim-serde provides serialization and deserialization for Nim values, offering an improved alternative to the standard `std/json` library. Unlike the standard library, nim-serde JSON implements a flexible system of serialization/deserialization modes that give developers precise control over how Nim objects are converted to and from JSON.
44

55
## Table of Contents
66
- [nim-serde JSON](#nim-serde-json)
77
- [Table of Contents](#table-of-contents)
8+
- [Serde Modes](#serde-modes)
9+
- [Modes Overview](#modes-overview)
10+
- [Default Modes](#default-modes)
11+
- [Field Options](#field-options)
812
- [Serialization API](#serialization-api)
913
- [Basic Serialization with `%` operator](#basic-serialization-with--operator)
1014
- [Object Serialization](#object-serialization)
11-
- [Serialization with `%*`](#serialization-with-)
15+
- [Inlining JSON Directly in Code with `%*`](#inlining-json-directly-in-code-with-)
1216
- [Converting to JSON String with `toJson`](#converting-to-json-string-with-tojson)
1317
- [Serialization Modes](#serialization-modes)
1418
- [Field Customization for Serialization](#field-customization-for-serialization)
19+
- [Custom Type Serialization](#custom-type-serialization)
1520
- [Deserialization API](#deserialization-api)
1621
- [Basic Deserialization with `fromJson`](#basic-deserialization-with-fromjson)
1722
- [Error Handling](#error-handling)
1823
- [Parsing JSON with `JsonNode.parse`](#parsing-json-with-jsonnodeparse)
1924
- [Deserialization Modes](#deserialization-modes)
2025
- [Field Customization for Deserialization](#field-customization-for-deserialization)
2126
- [Using as a Drop-in Replacement for std/json](#using-as-a-drop-in-replacement-for-stdjson)
22-
- [Custom Type Serialization](#custom-type-serialization)
2327
- [Implementation Details](#implementation-details)
2428

29+
30+
## Serde Modes
31+
This implementation supports three different modes to control de/serialization:
32+
33+
```nim
34+
OptIn
35+
OptOut
36+
Strict
37+
```
38+
39+
Modes can be set in the `{.serialize.}` and/or `{.deserialize.}` pragmas on type
40+
definitions. Each mode has a different meaning depending on if the type is being
41+
serialized or deserialized. Modes can be set by setting `mode` in the `serialize` or
42+
`deserialize` pragma annotation, eg:
43+
44+
```nim
45+
type MyType {.serialize(mode=Strict).} = object
46+
field1: bool
47+
field2: bool
48+
```
49+
50+
### Modes Overview
51+
52+
| Mode | Serialize | Deserialize |
53+
|:-----|:----------|:------------|
54+
| `OptOut` | All object fields will be serialized, except fields marked with `{.serialize(ignore=true).}`. | All JSON keys will be deserialized, except fields marked with `{.deserialize(ignore=true).}`. No error if extra JSON fields exist. |
55+
| `OptIn` | Only fields marked with `{.serialize.}` will be serialized. Fields marked with `{.serialize(ignore=true).}` will not be serialized. | Only fields marked with `{.deserialize.}` will be deserialized. Fields marked with `{.deserialize(ignore=true).}` will not be deserialized. A `SerdeError` is raised if the field is missing in JSON. |
56+
| `Strict` | All object fields will be serialized, regardless if the field is marked with `{.serialize(ignore=true).}`. | Object fields and JSON fields must match exactly, otherwise a `SerdeError` is raised. |
57+
58+
### Default Modes
59+
60+
Types can be serialized and deserialized even without explicit annotations, using default modes. Without any pragmas, types are serialized in OptIn mode and deserialized in OptOut mode. When types have pragmas but no specific mode is set, OptOut mode is used for both serialization and deserialization.
61+
62+
63+
| Context | Serialize | Deserialize |
64+
|:--------|:----------|:------------|
65+
| Default (no pragma) | `OptIn` | `OptOut` |
66+
| Default (pragma, but no mode) | `OptOut` | `OptOut` |
67+
68+
```nim
69+
# Type is not annotated
70+
# A default mode of OptIn (for serialize) and OptOut (for deserialize) is assumed.
71+
type MyObj1 = object
72+
field1: bool
73+
field2: bool
74+
75+
# Type is annotated, but mode not specified
76+
# A default mode of OptOut is assumed for both serialize and deserialize.
77+
type MyObj2 {.serialize, deserialize.} = object
78+
field1: bool
79+
field2: bool
80+
```
81+
82+
### Field Options
83+
84+
Individual fields can be customized using the `{.serialize.}` and `{.deserialize.}` pragmas with additional options that control how each field is processed during serialization and deserialization
85+
86+
87+
| | serialize | deserialize |
88+
|:---------|:-----------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------|
89+
| `key` | aliases the field name in json | deserializes the field if json contains `key` |
90+
| `ignore` | <li>**OptOut:** field not serialized</li><li>**OptIn:** field not serialized</li><li>**Strict:** field serialized</li> | <li>**OptOut:** field not deserialized</li><li>**OptIn:** field not deserialized</li><li>**Strict:** field deserialized</li> |
91+
92+
93+
Example with field options:
94+
95+
```nim
96+
import pkg/serde/json
97+
98+
type
99+
Person {.serialize(mode=OptOut), deserialize(mode=OptIn).} = object
100+
id {.serialize(ignore=true), deserialize(key="personid").}: int
101+
name: string
102+
birthYear: int
103+
address: string
104+
phone: string
105+
106+
let person = Person(
107+
name: "Lloyd Christmas",
108+
birthYear: 1970,
109+
address: "123 Sesame Street, Providence, Rhode Island 12345",
110+
phone: "555-905-justgivemethedamnnumber!⛽️🔥")
111+
112+
let createRequest = """{
113+
"name": "Lloyd Christmas",
114+
"birthYear": 1970,
115+
"address": "123 Sesame Street, Providence, Rhode Island 12345",
116+
"phone": "555-905-justgivemethedamnnumber!⛽️🔥"
117+
}"""
118+
assert person.toJson(pretty=true) == createRequest
119+
120+
let createResponse = """{
121+
"personid": 1,
122+
"name": "Lloyd Christmas",
123+
"birthYear": 1970,
124+
"address": "123 Sesame Street, Providence, Rhode Island 12345",
125+
"phone": "555-905-justgivemethedamnnumber!⛽️🔥"
126+
}"""
127+
assert !Person.fromJson(createResponse) == Person(id: 1)
128+
```
129+
130+
More examples can be found in [Serialization Modes](#serialization-modes) and [Deserialization Modes](#deserialization-modes).
131+
25132
## Serialization API
26133

27134
The nim-serde JSON serialization API provides several ways to convert Nim values to JSON.
@@ -67,16 +174,19 @@ assert jsonNode["age"].getInt == 30
67174
assert "address" notin jsonNode
68175
```
69176

70-
### Serialization with `%*`
177+
### Inlining JSON Directly in Code with `%*`
71178

72179
The `%*` macro provides a more convenient way to create JSON objects:
73180

74181
```nim
75182
import pkg/serde/json
76183
77-
let jsonObj = %*{
78-
"name": "John",
79-
"age": 30,
184+
let
185+
name = "John"
186+
age = 30
187+
jsonObj = %*{
188+
"name": name,
189+
"age": age,
80190
"hobbies": ["reading", "coding"],
81191
"address": {
82192
"street": "123 Main St",
@@ -85,8 +195,8 @@ let jsonObj = %*{
85195
}
86196
87197
assert jsonObj.kind == JObject
88-
assert jsonObj["name"].getStr == "John"
89-
assert jsonObj["age"].getInt == 30
198+
assert jsonObj["name"].getStr == name
199+
assert jsonObj["age"].getInt == age
90200
assert jsonObj["hobbies"].kind == JArray
91201
assert jsonObj["hobbies"][0].getStr == "reading"
92202
assert jsonObj["address"]["street"].getStr == "123 Main St"
@@ -171,6 +281,72 @@ let jsonStr = toJson(person)
171281
assert jsonStr == """{"first_name":"John","last_name":"Doe","age":30}"""
172282
```
173283

284+
## Custom Type Serialization
285+
286+
You can extend nim-serde to support custom types by defining your own `%` operator overloads and `fromJson` procs:
287+
288+
```nim
289+
import pkg/serde/json
290+
import pkg/serde/utils/errors
291+
import pkg/questionable/results
292+
import std/strutils
293+
294+
# Define a custom type
295+
type
296+
UserId = distinct int
297+
298+
# Custom serialization for UserId
299+
proc `%`*(id: UserId): JsonNode =
300+
%("user-" & $int(id))
301+
302+
# Custom deserialization for UserId
303+
proc fromJson*(_: type UserId, json: JsonNode): ?!UserId =
304+
if json.kind != JString:
305+
return failure(newSerdeError("Expected string for UserId, got " & $json.kind))
306+
307+
let str = json.getStr()
308+
if str.startsWith("user-"):
309+
let idStr = str[5..^1]
310+
try:
311+
let id = parseInt(idStr)
312+
success(UserId(id))
313+
except ValueError:
314+
failure(newSerdeError("Invalid UserId format: " & str))
315+
else:
316+
failure(newSerdeError("UserId must start with 'user-' prefix"))
317+
318+
# Test serialization
319+
let userId = UserId(42)
320+
let jsonNode = %userId
321+
assert jsonNode.kind == JString
322+
assert jsonNode.getStr() == "user-42"
323+
324+
# Test deserialization
325+
let jsonStr = "\"user-42\""
326+
let parsedJson = !JsonNode.parse(jsonStr)
327+
let result = UserId.fromJson(parsedJson)
328+
assert result.isSuccess
329+
assert int(!result) == 42
330+
331+
# Test in object context
332+
type User {.serialize(mode = OptOut).} = object
333+
id: UserId
334+
name: string
335+
336+
let user = User(id: UserId(123), name: "John")
337+
let userJson = %user
338+
assert userJson.kind == JObject
339+
assert userJson["id"].getStr() == "user-123"
340+
assert userJson["name"].getStr() == "John"
341+
342+
# Test deserialization of object with custom type
343+
let userJsonStr = """{"id":"user-123","name":"John"}"""
344+
let userResult = User.fromJson(userJsonStr)
345+
assert userResult.isSuccess
346+
assert int((!userResult).id) == 123
347+
assert (!userResult).name == "John"
348+
```
349+
174350
## Deserialization API
175351

176352
nim-serde provides a type-safe way to convert JSON data back into Nim types.
@@ -342,72 +518,6 @@ assert parsedNode["name"].getStr == "John"
342518
let prettyJson = pretty(jsonNode)
343519
```
344520

345-
## Custom Type Serialization
346-
347-
You can extend nim-serde to support custom types by defining your own `%` operator overloads and `fromJson` procs:
348-
349-
```nim
350-
import pkg/serde/json
351-
import pkg/serde/utils/errors
352-
import pkg/questionable/results
353-
import std/strutils
354-
355-
# Define a custom type
356-
type
357-
UserId = distinct int
358-
359-
# Custom serialization for UserId
360-
proc `%`*(id: UserId): JsonNode =
361-
%("user-" & $int(id))
362-
363-
# Custom deserialization for UserId
364-
proc fromJson*(_: type UserId, json: JsonNode): ?!UserId =
365-
if json.kind != JString:
366-
return failure(newSerdeError("Expected string for UserId, got " & $json.kind))
367-
368-
let str = json.getStr()
369-
if str.startsWith("user-"):
370-
let idStr = str[5..^1]
371-
try:
372-
let id = parseInt(idStr)
373-
success(UserId(id))
374-
except ValueError:
375-
failure(newSerdeError("Invalid UserId format: " & str))
376-
else:
377-
failure(newSerdeError("UserId must start with 'user-' prefix"))
378-
379-
# Test serialization
380-
let userId = UserId(42)
381-
let jsonNode = %userId
382-
assert jsonNode.kind == JString
383-
assert jsonNode.getStr() == "user-42"
384-
385-
# Test deserialization
386-
let jsonStr = "\"user-42\""
387-
let parsedJson = !JsonNode.parse(jsonStr)
388-
let result = UserId.fromJson(parsedJson)
389-
assert result.isSuccess
390-
assert int(!result) == 42
391-
392-
# Test in object context
393-
type User {.serialize(mode = OptOut).} = object
394-
id: UserId
395-
name: string
396-
397-
let user = User(id: UserId(123), name: "John")
398-
let userJson = %user
399-
assert userJson.kind == JObject
400-
assert userJson["id"].getStr() == "user-123"
401-
assert userJson["name"].getStr() == "John"
402-
403-
# Test deserialization of object with custom type
404-
let userJsonStr = """{"id":"user-123","name":"John"}"""
405-
let userResult = User.fromJson(userJsonStr)
406-
assert userResult.isSuccess
407-
assert int((!userResult).id) == 123
408-
assert (!userResult).name == "John"
409-
```
410-
411521
## Implementation Details
412522

413523
The JSON serialization in nim-serde is based on the `%` operator pattern:

0 commit comments

Comments
 (0)