Skip to content

Commit 1f7bcd7

Browse files
authored
Record recent benchmark results and add AGENTS.md (#19)
1 parent b819c80 commit 1f7bcd7

4 files changed

Lines changed: 135 additions & 0 deletions

File tree

‎AGENTS.md‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
# AGENTS.md
2+
3+
Guidance for coding agents working in this repository. The design itself (write and read
4+
paths, buffer ownership, map keys, the parser's accessor contract) is in `docs/DESIGN.md`;
5+
read it before changing the generator, parser, writer or reader.
6+
7+
## Project
8+
9+
A Jackson 3 dataformat for MessagePack, `org.komamitsu:jackson-dataformat-msgpack`, with its
10+
own MessagePack encoder and decoder. msgpack-core is a test dependency only, used as the
11+
reference implementation in tests.
12+
13+
## Commands
14+
15+
```bash
16+
./gradlew clean build # Compile, checkstyle, all tests (Java 17 toolchain)
17+
./gradlew build -PtestJavaVersion=21 # Run the tests on another JDK, as CI does for 17, 21 and 24
18+
./gradlew soakTest -Psoak.seconds=120 # Memory-growth soak tests, excluded from the normal build
19+
./gradlew :jmh:jmhJar # Build the benchmark jar
20+
```
21+
22+
Tests run with `-Dfile.encoding=windows-1252`, so a conversion that forgets to name a charset
23+
fails in the tests.
24+
25+
## Rules
26+
27+
- **Round-trip safety comes first.** Whatever this library writes, it must be able to read
28+
back. A type that cannot round-trip is refused on write rather than written in a form that
29+
fails on read (see `UnreadableKeyGuard` and DESIGN.md 2.6).
30+
- **Compatibility with the Jackson 2 module (`msgpack-jackson` in msgpack-java) is not a
31+
goal.** Do not keep or add behaviour only because that module had it.
32+
- **Output after a serialization failure is the caller's to discard.** Do not add rollback or
33+
cleanup machinery for callers that catch the exception and keep using the generator.
34+
- **Bug fixes come with a test that fails without the fix.** Assert exact values and bytes,
35+
not only types or non-null.
36+
- **Benchmark changes to a hot path before pushing.** Build the change and `main` and run them
37+
back to back:
38+
`java -jar jmh/build/libs/jmh-jmh.jar <Benchmark> -f 5 -wi 5 -w 1 -i 10 -r 2`, adding
39+
`--add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED`
40+
through `-jvmArgs`. Measure with a reused mapper; a benchmark that creates a mapper per
41+
operation measures nothing users see. Numbers from different days are not comparable on
42+
this machine, and a short run is not enough to call a change free.
43+
- **Record meaningful results in `jmh/results/`**, one file per change, with the command, the
44+
compared commits and the JSON control.
45+
- **Reviews** go through GitHub Copilot, requested with
46+
`gh api -X POST repos/komamitsu/jackson-dataformat-msgpack/pulls/<n>/requested_reviewers -f "reviewers[]=copilot-pull-request-reviewer[bot]"`.
47+
Check each finding against the code, with a test where it makes a claim about behaviour,
48+
before fixing or declining it.
Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
# One wrap helper for keys and container headers
2+
3+
Measured on the same machine as the earlier files, run back to back:
4+
5+
- Before: `c2d1248` (Write scalar values through one verify-and-wrap helper)
6+
- After: `1af13b5`. Two commits on top of the before state: `f8695a4` (the change below) and
7+
a read-side change that builds repeated parser errors in one place, which no write benchmark
8+
runs
9+
10+
The JMH command of this run was not recorded, so it is not given here.
11+
12+
## Change
13+
14+
`f8695a4`: `writeName`, `openContainer` and `closeContainer` each wrapped their writer call in
15+
their own try/catch that turned an `IOException` into a Jackson exception. The try/catch now
16+
lives in one `pack()` helper, which makes those three hot methods small enough to inline
17+
better.
18+
19+
## Result
20+
21+
| Benchmark | Before | After | Change |
22+
|---|---|---|---|
23+
| writePojoJson | 1099800 ± 14428 | 1088509 ± 13948 | control |
24+
| writePojoMsgpack | 1170317 ± 9411 | **1273380 ± 12473** | **+8.8%** |
25+
26+
The error bars do not overlap and the control is flat.
Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
# Reading an integer without boxing it
2+
3+
Same machine as the earlier files, with `-prof gc` for the allocation figures, run back to
4+
back:
5+
6+
- Before: `d2d774e` (Name the context methods for what they do)
7+
- After: `1037af9` below
8+
9+
The rest of the JMH command of this run was not recorded, so it is not given here.
10+
11+
## Change
12+
13+
`1037af9`: the parser boxed every integer it read into an `Integer` or `Long`. It now keeps the
14+
value in a primitive field.
15+
16+
## Result
17+
18+
| Benchmark | Before | After |
19+
|---|---|---|
20+
| readPojoMsgpack allocation | 2056 B/op | **1912 B/op** |
21+
| readPojoJson allocation (control) | 2096 B/op | 2096 B/op |
22+
23+
144 bytes per operation are gone. MessagePack read now allocates about 9% less than Jackson's JSON reader on the same POJO.
24+
25+
Throughput did not resolve on this machine (694079 ± 11508 before, 680121 ± 7908 after, with
26+
the JSON control moving from 674k to 677k), so the allocation figure is the result here.
Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
# PR #9 against main after the map-key and UUID changes
2+
3+
A check that the changes merged after the Jackson 3 rewrite (#9) did not cost performance:
4+
#10 (string tokens read as binary), #16 (MessagePackKeySerializer removed, key guard added) and
5+
#17 (UUID written as a string).
6+
7+
Same machine as the earlier files. Both commits were built and run back to back with
8+
`java -jar jmh/build/libs/jmh-jmh.jar "MsgpackWriteBenchmark|MsgpackReadBenchmark|WriteUTF8StringBenchmark|WriteStringBenchmark" -f 5 -wi 5 -w 1 -i 10 -r 2 -bm thrpt -tu s`
9+
(50 samples per benchmark), PR #9 (`bb4158f`) first, then main (`cff8bfc`).
10+
11+
The numbers are higher than the ones #9 recorded in `2026-09-21-single-pass-short-strings.md`
12+
for the same code (writePojoMsgpack 1242839 here, 1092242 there), so only runs made back to
13+
back are compared.
14+
15+
## Result
16+
17+
| Benchmark | PR #9 | main | Change |
18+
|---|---|---|---|
19+
| readPojoJson | 686406 ± 11655 | 682182 ± 10299 | control |
20+
| readPojoMsgpack | 686441 ± 7242 | 689769 ± 5651 | +0.5%, within error |
21+
| writePojoJson | 1091002 ± 5201 | 1072677 ± 11429 | control, -1.7% |
22+
| writePojoMsgpack | 1242839 ± 9565 | 1237674 ± 8973 | -0.4%, within error |
23+
| WriteStringBenchmark.shortAscii | 128397 ± 2511 | 128497 ± 2274 | within error |
24+
| WriteStringBenchmark.shortNonAscii | 85536 ± 3778 | 89744 ± 2307 | within error |
25+
| WriteStringBenchmark.shortNonAsciiShifted | 48127 ± 1499 | 46294 ± 1721 | within error |
26+
| WriteStringBenchmark.longAscii | 81484 ± 1344 | 80275 ± 265 | within error |
27+
| WriteStringBenchmark.longNonAscii | 55262 ± 518 | 55197 ± 187 | within error |
28+
| WriteUTF8StringBenchmark.writeUTF8StringAscii | 118398 ± 1288 | 117607 ± 667 | within error |
29+
| WriteUTF8StringBenchmark.writeUTF8StringNonAscii | 115120 ± 1381 | 112199 ± 1320 | -2.5% |
30+
31+
POJO read and write are on par with #9. The only MessagePack row whose error bars do not
32+
overlap, writeUTF8StringNonAscii at -2.5%, moved while the JSON write control, whose code is
33+
the same in both builds, dropped 1.7%, so the machine was slower during the second run. That
34+
row is inconclusive rather than a confirmed regression; running the string benchmarks again
35+
with main first would tell drift from a real cost.

0 commit comments

Comments
 (0)