Skip to content

Commit bc295d0

Browse files
committed
Support integer map keys and add encode_to_iodata!
Phoenix's `:json_library` contract calls `encode_to_iodata!/1` from its socket serializers, controllers, and longpoll transport, so configuring Torque as the JSON library raised UndefinedFunctionError on every message. `encode_to_iodata/2` already raises on error, so the new function is a straight alias that also forwards the `:dirty` option. LiveView renders diffs as maps with integer keys, which encoding rejected with `:invalid_key`. JSON object names must be strings (RFC 8259 §4), so an integer key has no direct representation: reject or stringify are the only options, and Jason stringifies. Route integer keys through the existing `encode_integer`, which already handles the i64/u64/bignum cascade, so proplists get the same treatment for free. The conversion is lossy — `%{1 => "a"}` round-trips to `%{"1" => "a"}`, and a map mixing both forms encodes to duplicate names — so document that in Limitations rather than leave it to be found in production. Reported in #47.
1 parent de85562 commit bc295d0

6 files changed

Lines changed: 107 additions & 13 deletions

File tree

CLAUDE.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -93,7 +93,7 @@ Torque is a high-performance JSON library for Elixir using Rustler NIFs backed b
9393

9494
### Encoding
9595

96-
`encode/1` walks Elixir terms directly (no intermediate representation) and writes JSON bytes to a buffer. Supports maps (atom/binary keys), lists, numbers, booleans, nil, and jiffy-style `{proplist}` tuples.
96+
`encode/1` walks Elixir terms directly (no intermediate representation) and writes JSON bytes to a buffer. Supports maps (atom/binary/integer keys — integer keys are stringified, since JSON object names must be strings), lists, numbers, booleans, nil, and jiffy-style `{proplist}` tuples.
9797

9898
### Scheduler Awareness
9999

README.md

Lines changed: 10 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -104,6 +104,10 @@ already-parsed document via `Torque.get_many_nil(doc, pointers)`.
104104
{:ok, json} = Torque.encode(%{id: "abc", price: 1.5})
105105
# "{\"id\":\"abc\",\"price\":1.5}"
106106

107+
# Integer keys are stringified — JSON object names must be strings
108+
{:ok, json} = Torque.encode(%{0 => "a", 1 => "b"})
109+
# "{\"0\":\"a\",\"1\":\"b\"}"
110+
107111
# Bang variant
108112
json = Torque.encode!(%{id: "abc"})
109113

@@ -116,8 +120,8 @@ json = Torque.encode_to_iodata(%{id: "abc"})
116120

117121
Unlike decoding, encoding cannot cheaply predict its output size, so dirty
118122
scheduler dispatch is opt-in. Pass `dirty: true` (accepted by `encode/2`,
119-
`encode!/2`, and `encode_to_iodata/2`) when terms are expected to encode to
120-
large output (more than roughly 20 KB):
123+
`encode!/2`, `encode_to_iodata/2`, and `encode_to_iodata!/2`) when terms are
124+
expected to encode to large output (more than roughly 20 KB):
121125

122126
```elixir
123127
{:ok, json} = Torque.encode(big_term, dirty: true)
@@ -133,6 +137,7 @@ large output (more than roughly 20 KB):
133137
| `Torque.encode(term, opts)` | Encode term to JSON binary |
134138
| `Torque.encode!(term, opts)` | Encode term, raising on error |
135139
| `Torque.encode_to_iodata(term, opts)` | Encode term, returns binary directly (fastest) |
140+
| `Torque.encode_to_iodata!(term, opts)` | Alias for `encode_to_iodata/2` (Phoenix `:json_library`) |
136141
| `Torque.get(doc, path)` | Extract field by JSON Pointer path |
137142
| `Torque.get(doc, path, default)` | Extract field with default for missing paths |
138143
| `Torque.get_many(doc, paths)` | Extract multiple fields in one NIF call |
@@ -163,7 +168,7 @@ Integers outside the signed/unsigned 64-bit range decode as exact arbitrary-prec
163168

164169
| Elixir | JSON |
165170
|--------|------|
166-
| map (atom/binary keys) | object |
171+
| map (atom/binary/integer keys) | object |
167172
| list | array |
168173
| binary | string |
169174
| integer | number |
@@ -191,7 +196,7 @@ Functions return `{:error, reason}` tuples (or raise `ArgumentError` for bang/io
191196
|------|-------------|---------|
192197
| `:unsupported_type` | `encode/1` | Term has no JSON representation (PID, reference, port, …) |
193198
| `:invalid_utf8` | `encode/1` | Binary string or map key is not valid UTF-8 |
194-
| `:invalid_key` | `encode/1` | Map key is not an atom or binary (e.g. integer key) |
199+
| `:invalid_key` | `encode/1` | Map key is not an atom, binary, or integer (e.g. float or tuple key) |
195200
| `:malformed_proplist` | `encode/1` | `{proplist}` contains a non-`{key, value}` element |
196201
| `:non_finite_float` | `encode/1` | Float is infinity or NaN (unreachable from normal BEAM code) |
197202
| `:nesting_too_deep` | `encode/1` | Term exceeds 128 nesting levels |
@@ -286,6 +291,7 @@ MIX_ENV=bench mix run bench/torque_bench.exs
286291

287292
## Limitations
288293

294+
- **Integer map keys are lossy**: JSON object names must be strings (RFC 8259 §4), so `encode/1` stringifies integer keys and `decode/1` gives them back as binaries — `%{1 => "a"}` round-trips to `%{"1" => "a"}`. A map mixing both forms, like `%{1 => "a", "1" => "b"}`, encodes to duplicate names (`{"1":"a","1":"b"}`); RFC 8259 says names *should* be unique, and decoders resolve the collision however they choose. Jason behaves identically.
289295
- **Nesting depth**: JSON documents nested deeper than 128 levels return `{:error, :nesting_too_deep}` from `decode/1`, `parse/1`, `get/2`, `get_many/2`, and `encode/1` rather than crashing the VM. Real-world documents are never this deep; the limit exists to prevent stack overflow in the NIF (the dirty CPU scheduler, used for inputs over 20 KB, has a small stack).
290296

291297
## License

lib/torque.ex

Lines changed: 25 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -20,17 +20,18 @@ defmodule Torque do
2020
2121
## Encoding
2222
23-
`encode/1` serializes Elixir terms to JSON. Supports maps (atom or
24-
binary keys), lists, binaries, numbers, booleans, `nil`, and
25-
jiffy-style `{proplist}` tuples.
23+
`encode/1` serializes Elixir terms to JSON. Supports maps (atom,
24+
binary, or integer keys), lists, binaries, numbers, booleans, `nil`,
25+
and jiffy-style `{proplist}` tuples.
2626
2727
## Scheduler awareness
2828
2929
Decoding and parsing automatically dispatch inputs larger than 20 KB to a
3030
dirty CPU scheduler to avoid blocking normal BEAM schedulers. Encoding
3131
cannot cheaply predict its output size up front, so dirty dispatch is
32-
opt-in there: pass `dirty: true` to `encode/2`, `encode!/2`, or
33-
`encode_to_iodata/2` when terms are expected to produce large output.
32+
opt-in there: pass `dirty: true` to `encode/2`, `encode!/2`,
33+
`encode_to_iodata/2`, or `encode_to_iodata!/2` when terms are expected
34+
to produce large output.
3435
3536
## Type conversion
3637
@@ -116,7 +117,8 @@ defmodule Torque do
116117
117118
## Supported terms
118119
119-
* Maps with atom or binary keys
120+
* Maps with atom, binary, or integer keys (integer keys are
121+
stringified — JSON object names must be strings)
120122
* Lists (JSON arrays)
121123
* Binaries (JSON strings)
122124
* Integers and floats
@@ -212,6 +214,23 @@ defmodule Torque do
212214
:error, value -> raise ArgumentError, "encode error: #{inspect(value)}"
213215
end
214216

217+
@doc """
218+
Alias for `encode_to_iodata/2`, which already raises on error.
219+
220+
Exists to satisfy Phoenix's `:json_library` contract, which calls
221+
`encode_to_iodata!/1` from its socket serializers, controllers, and
222+
longpoll transport. Set `config :phoenix, :json_library, Torque` to use
223+
Torque there.
224+
225+
## Examples
226+
227+
iex> Torque.encode_to_iodata!(%{ok: true})
228+
~s({"ok":true})
229+
"""
230+
@doc group: :encode
231+
@spec encode_to_iodata!(term(), keyword()) :: binary()
232+
def encode_to_iodata!(term, opts \\ []), do: encode_to_iodata(term, opts)
233+
215234
# --- Parse + Get ---
216235

217236
@doc """

native/torque_nif/src/encoder.rs

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -261,6 +261,10 @@ fn encode_map_key(
261261
.map_err(|_| EncodeError::InvalidUtf8)?;
262262
}
263263
}
264+
// Object names must be strings (RFC 8259 §4), so integer keys are
265+
// stringified rather than rejected, matching Jason. Skips escaping
266+
// because a decimal integer is only digits and a leading '-'.
267+
TermType::Integer => encode_integer(env_raw, key, buf)?,
264268
_ => return Err(EncodeError::InvalidKey),
265269
}
266270
buf.push(b'"');

test/encode_test.exs

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -151,6 +151,47 @@ defmodule Torque.EncodeTest do
151151
test "invalid UTF-8 binary map key returns error" do
152152
assert {:error, :invalid_utf8} = Torque.encode(%{<<0x80>> => "value"})
153153
end
154+
155+
test "map with integer keys stringifies them" do
156+
assert {:ok, json} = Torque.encode(%{0 => "a", 1 => "b"})
157+
assert Jason.decode!(json) == %{"0" => "a", "1" => "b"}
158+
end
159+
160+
test "negative integer map key" do
161+
assert {:ok, ~s({"-1":"x"})} = Torque.encode(%{-1 => "x"})
162+
end
163+
164+
test "integer map key beyond i64 uses the u64 path" do
165+
assert {:ok, json} = Torque.encode(%{9_223_372_036_854_775_808 => "x"})
166+
assert Jason.decode!(json) == %{"9223372036854775808" => "x"}
167+
end
168+
169+
test "bignum map key encodes exactly" do
170+
assert {:ok, json} = Torque.encode(%{1_180_591_620_717_411_303_424 => "x"})
171+
assert Jason.decode!(json) == %{"1180591620717411303424" => "x"}
172+
end
173+
174+
test "negative bignum map key encodes exactly" do
175+
assert {:ok, json} = Torque.encode(%{-1_180_591_620_717_411_303_424 => "x"})
176+
assert Jason.decode!(json) == %{"-1180591620717411303424" => "x"}
177+
end
178+
179+
test "proplist with integer keys stringifies them" do
180+
assert {:ok, ~s({"1":"a","2":"b"})} = Torque.encode({[{1, "a"}, {2, "b"}]})
181+
end
182+
183+
test "integer and binary keys that collide emit duplicate names" do
184+
assert {:ok, json} = Torque.encode(%{1 => "a", "1" => "b"})
185+
assert json in [~s({"1":"a","1":"b"}), ~s({"1":"b","1":"a"})]
186+
end
187+
188+
test "float map key is still rejected" do
189+
assert {:error, :invalid_key} = Torque.encode(%{1.5 => "x"})
190+
end
191+
192+
test "tuple map key is still rejected" do
193+
assert {:error, :invalid_key} = Torque.encode(%{{:a, :b} => "x"})
194+
end
154195
end
155196

156197
describe "encode/2 with dirty: true" do
@@ -227,4 +268,28 @@ defmodule Torque.EncodeTest do
227268
end
228269
end
229270
end
271+
272+
describe "encode_to_iodata!/2" do
273+
test "matches encode_to_iodata/1 output" do
274+
term = %{nested: %{list: [1, 2, 3], str: "hello"}}
275+
assert Torque.encode_to_iodata!(term) == Torque.encode_to_iodata(term)
276+
end
277+
278+
test "unsupported term raises ArgumentError" do
279+
assert_raise ArgumentError, ~r/unsupported_type/, fn ->
280+
Torque.encode_to_iodata!(self())
281+
end
282+
end
283+
284+
test "accepts dirty: true" do
285+
term = %{"a" => [1, 2, 3], "b" => "hello"}
286+
assert Torque.encode_to_iodata!(term, dirty: true) == Torque.encode_to_iodata!(term)
287+
end
288+
289+
test "is exported at arity 1 for Phoenix's :json_library contract" do
290+
Code.ensure_loaded!(Torque)
291+
assert function_exported?(Torque, :encode_to_iodata!, 1)
292+
assert function_exported?(Torque, :decode!, 1)
293+
end
294+
end
230295
end

test/property_test.exs

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -546,8 +546,8 @@ defmodule Torque.PropertyTest do
546546
assert {:error, :unsupported_type} = Torque.encode({"not_a_list"})
547547
end
548548

549-
test "integer map key returns invalid_key" do
550-
assert {:error, :invalid_key} = Torque.encode(%{42 => "v"})
549+
test "float map key returns invalid_key" do
550+
assert {:error, :invalid_key} = Torque.encode(%{4.2 => "v"})
551551
end
552552

553553
test "proplist item not a tuple returns malformed_proplist" do

0 commit comments

Comments
 (0)