Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 24 additions & 9 deletions packages/conformance/conformance/docs/rules/by-id/O001.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,8 @@ Suppress a finding on the violating line or the line directly above it:

**Rationale:** orjson is already a core SDK dependency — zero incremental cost — and on hot paths the
~10x throughput advantage compounds at fleet scale. WARN (not block) because orjson
returns bytes not str and has a different option API, so each site needs human judgment
before migrating.
returns bytes not str and has a different option API, and rejects deeply nested values
and integers above 64 bits, so each site needs human judgment before migrating.

### What correct looks like

Expand Down Expand Up @@ -59,13 +59,28 @@ APIs orjson does not provide), and custom `JSONEncoder` subclasses are out of sc
Autofixable per-site, not mechanically: `orjson` is not a drop-in replacement.
`orjson.dumps` returns `bytes` (not `str`) and has no `indent=` / `sort_keys=` keyword
surface (use `option=orjson.OPT_INDENT_2 | orjson.OPT_SORT_KEYS` for those); `default=`
may be passed by keyword or positionally. It rejects some inputs stdlib accepts. A
blind `json.`→`orjson.` swap silently changes `str`→`bytes` and breaks callers — each
site needs human judgement. The encoded bytes also change on any call that does not
already pass `separators=(",", ":")` and `ensure_ascii=False`: orjson is always compact
and always writes non-ASCII as UTF-8, with no option for either. The parsed value is
identical, so tests that compare parsed JSON pass; a consumer that hashes, commits or
byte-compares the output sees the difference.
may be passed by keyword or positionally. It rejects some inputs stdlib accepts, and
unit tests with small values rarely reach them: a value nested 254 or more levels deep
raises `TypeError: Recursion limit reached`, and an integer above 64 bits cannot be
serialized. A blind `json.`→`orjson.` swap silently changes `str`→`bytes` and breaks
callers — each site needs human judgement. The encoded bytes also change on any call
that does not already pass `separators=(",", ":")` and `ensure_ascii=False`: orjson is
always compact and always writes non-ASCII as UTF-8, with no option for either. The
parsed value is identical, so tests that compare parsed JSON pass; a consumer that
hashes, commits or byte-compares the output sees the difference.

Before swapping a call that serializes source-system or API payloads, check whether the
value can reach those limits. If it can, do not keep stdlib `json` as a fallback by
default. First ask whether the app needs the deeply nested part at all. Source
payloads often carry subtrees the app never reads (for example formula expression trees
in a BI tool's metadata), and dropping them before serialization removes the limit,
shrinks the stored output and keeps the orjson path. In a prior incident, a bare swap
on such a path turned a successful extract into a failure on every retry, and the only
field in that payload that grows without bound was a subtree the app stored but never
read. If the app genuinely needs the deeply nested value, or must carry integers above
64 bits, the site may have to stay on stdlib `json`. That is a design decision for a
human reviewer, not a remediated end state: it does not meet *Already correct when*, so
the finding stays open unless the reviewer chooses to suppress it.

No orjson call reproduces stdlib's default bytes: there is no separators option, no
`ensure_ascii` option, `orjson.dumps` cannot serialize an integer above 64 bits, and
Expand Down
33 changes: 24 additions & 9 deletions packages/conformance/conformance/docs/rules/optimizations.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,8 @@ reassigned.

**Rationale:** orjson is already a core SDK dependency — zero incremental cost — and on hot paths the
~10x throughput advantage compounds at fleet scale. WARN (not block) because orjson
returns bytes not str and has a different option API, so each site needs human judgment
before migrating.
returns bytes not str and has a different option API, and rejects deeply nested values
and integers above 64 bits, so each site needs human judgment before migrating.

### What correct looks like

Expand Down Expand Up @@ -78,13 +78,28 @@ APIs orjson does not provide), and custom `JSONEncoder` subclasses are out of sc
Autofixable per-site, not mechanically: `orjson` is not a drop-in replacement.
`orjson.dumps` returns `bytes` (not `str`) and has no `indent=` / `sort_keys=` keyword
surface (use `option=orjson.OPT_INDENT_2 | orjson.OPT_SORT_KEYS` for those); `default=`
may be passed by keyword or positionally. It rejects some inputs stdlib accepts. A
blind `json.`→`orjson.` swap silently changes `str`→`bytes` and breaks callers — each
site needs human judgement. The encoded bytes also change on any call that does not
already pass `separators=(",", ":")` and `ensure_ascii=False`: orjson is always compact
and always writes non-ASCII as UTF-8, with no option for either. The parsed value is
identical, so tests that compare parsed JSON pass; a consumer that hashes, commits or
byte-compares the output sees the difference.
may be passed by keyword or positionally. It rejects some inputs stdlib accepts, and
unit tests with small values rarely reach them: a value nested 254 or more levels deep
raises `TypeError: Recursion limit reached`, and an integer above 64 bits cannot be
serialized. A blind `json.`→`orjson.` swap silently changes `str`→`bytes` and breaks
callers — each site needs human judgement. The encoded bytes also change on any call
that does not already pass `separators=(",", ":")` and `ensure_ascii=False`: orjson is
always compact and always writes non-ASCII as UTF-8, with no option for either. The
parsed value is identical, so tests that compare parsed JSON pass; a consumer that
hashes, commits or byte-compares the output sees the difference.

Before swapping a call that serializes source-system or API payloads, check whether the
value can reach those limits. If it can, do not keep stdlib `json` as a fallback by
default. First ask whether the app needs the deeply nested part at all. Source
payloads often carry subtrees the app never reads (for example formula expression trees
in a BI tool's metadata), and dropping them before serialization removes the limit,
shrinks the stored output and keeps the orjson path. In a prior incident, a bare swap
on such a path turned a successful extract into a failure on every retry, and the only
field in that payload that grows without bound was a subtree the app stored but never
read. If the app genuinely needs the deeply nested value, or must carry integers above
64 bits, the site may have to stay on stdlib `json`. That is a design decision for a
human reviewer, not a remediated end state: it does not meet *Already correct when*, so
the finding stays open unless the reviewer chooses to suppress it.

No orjson call reproduces stdlib's default bytes: there is no separators option, no
`ensure_ascii` option, `orjson.dumps` cannot serialize an integer above 64 bits, and
Expand Down
23 changes: 21 additions & 2 deletions packages/conformance/conformance/suite/rules/optimizations.py
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,8 @@
rationale=(
"orjson is already a core SDK dependency — zero incremental cost — and on hot "
"paths the ~10x throughput advantage compounds at fleet scale. WARN (not block) "
"because orjson returns bytes not str and has a different option API, so each "
"because orjson returns bytes not str and has a different option API, and rejects "
"deeply nested values and integers above 64 bits, so each "
"site needs human judgment before migrating."
),
short_description="json.dumps()/json.loads() — prefer orjson (a core SDK dependency, ~10x faster)",
Expand All @@ -62,7 +63,10 @@
"returns ``bytes`` (not ``str``) and has no ``indent=`` / ``sort_keys=``\n"
"keyword surface (use ``option=orjson.OPT_INDENT_2 |\n"
"orjson.OPT_SORT_KEYS`` for those); ``default=`` may be passed by keyword\n"
"or positionally. It rejects some inputs stdlib accepts. A blind\n"
"or positionally. It rejects some inputs stdlib accepts, and unit tests\n"
"with small values rarely reach them: a value nested 254 or more levels\n"
"deep raises ``TypeError: Recursion limit reached``, and an integer above\n"
"64 bits cannot be serialized. A blind\n"
"``json.``→``orjson.`` swap silently changes\n"
"``str``→``bytes`` and breaks callers — each site needs human judgement.\n"
"The encoded bytes also change on any call that does not already pass\n"
Expand All @@ -72,6 +76,21 @@
"JSON pass; a consumer that hashes, commits or byte-compares the output\n"
"sees the difference.\n"
"\n"
"Before swapping a call that serializes source-system or API payloads,\n"
"check whether the value can reach those limits. If it can, do not keep\n"
"stdlib ``json`` as a fallback by default. First ask whether the app\n"
"needs the deeply nested part at all. Source payloads often carry\n"
"subtrees the app never reads (for example formula expression trees in a\n"
"BI tool's metadata), and dropping them before serialization removes the\n"
"limit, shrinks the stored output and keeps the orjson path. In a prior\n"
"incident, a bare swap on such a path turned a successful extract into a\n"
"failure on every retry, and the only field in that payload that grows\n"
"without bound was a subtree the app stored but never read. If the app genuinely needs the deeply nested value, or\n"
"must carry integers above 64 bits, the site may have to stay on stdlib\n"
"``json``. That is a design decision for a human reviewer, not a\n"
"remediated end state: it does not meet *Already correct when*, so the\n"
"finding stays open unless the reviewer chooses to suppress it.\n"
"\n"
"No orjson call reproduces stdlib's default bytes: there is no separators\n"
"option, no ``ensure_ascii`` option, ``orjson.dumps`` cannot serialize an\n"
"integer above 64 bits, and\n"
Expand Down
Loading