Encode std::byte ranges compactly and read fixed char arrays in BEVE#2649
Merged
Conversation
Fixes #2647. BEVE mishandled arrays of two primitive element types: - std::array<char, N> failed to compile on read: from<BEVE, str_t> unconditionally called value.resize(n), which std::array lacks. - std::vector<std::byte> / std::array<std::byte, N> serialized as a generic_array (a one-byte type header per element, ~2x the size), inconsistent with std::vector<uint8_t> (a compact u8 typed array) and with a scalar std::byte (already a u8 number). std::byte is now encoded as a u8 typed array, byte-for-byte identical to uint8_t (the existing numeric formulas yield byte_count 0 and an unsigned u8 header for it). A new beve_num_t concept opts std::byte into the numeric typed-array paths in write/read/size. The generic-enum reader gains a tagged overload (delegating to its underlying integer reader) so the typed-array conversion paths work for std::byte. from<BEVE, str_t> now stores into resizable strings, string views, and fixed std::array<char, N> (bounds-checked, remainder zero-filled) via a shared helper used by both the tagged and untagged read paths. Note: this changes the BEVE wire format for std::byte arrays; data written by older versions using the generic-array form is rejected cleanly with a syntax_error.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #2647.
Problem
The BEVE codec mishandled arrays of two primitive element types:
std::array<char, N>failed to compile on read. It is classified asstr_t(viaarray_char_t), andfrom<BEVE, str_t>unconditionally calledvalue.resize(n), whichstd::arraydoes not have.std::vector<std::byte>/std::array<std::byte, N>used ~2× the space.std::byteis a scoped enum (notnum_t), so it fell through to thegeneric_arraybranch — a one-byte type header per element. This was inconsistent withstd::vector<uint8_t>(a compactu8typed array) and with a scalarstd::byte, which was already encoded as au8number.Fix
beve_num_t = num_t<T> || std::same_as<T, std::byte>concept optsstd::byteinto the numeric typed-array dispatch inwrite/read/size. The existing numeric formulas already yieldbyte_count == 0and an unsignedu8header forstd::byte, so the wire output is byte-for-byte identical touint8_t(e.g. a 16-element vector goes 34 → 18 bytes).std::byteanduint8_tranges are now cross-readable in both directions.std::byte.from<BEVE, str_t>now stores into resizable strings, string views, and fixedstd::array<char, N>(bounds-checked, remainder zero-filled) through a shared helper used by both the tagged and untagged read paths.Wire-driven paths (
skip,beve_to_json,ptr) need no change — they handle the resultingu8typed array purely from the tag.Wire-format note
This intentionally changes the BEVE encoding of
std::bytearrays. Data written by older versions using the generic-array form is rejected cleanly witherror_code::syntax_error(no UB). This alignsstd::bytearrays with both scalarstd::byteandstd::vector<uint8_t>.Tests
11 regression tests added to
beve_test.cppcovering: compact encoding + wire-equality withuint8_t, cross-readability,std::array<std::byte, N>, empty and non-contiguous (std::deque) byte ranges,std::bytestruct members,beve_to_jsonrendering, full and short (zero-filled)std::array<char, N>round trips, oversized-payload rejection, and untagged mode. Fullbeve_testsuite passes (381 tests), along withlazy_beve_testandskip_null_on_read_test.