Skip to content

SDDL1: any non-ASCII byte in a description aborts sddl_compiler with a "Corruption detected" error #943

Description

@avionicharshit-byte

One byte >= 0x80 anywhere in an SDDL1 description kills compilation, comments included.
sddl_compiler aborts (SIGABRT), zli --profile sddl exits 1 with a CBOR corruption error.
Neither message mentions encoding.

Repro (built from a9de25e, macOS arm64):

printf '# caf\xc3\xa9\n: Float32LE[_rem / 4]\n' | build/tools/sddl/sddl_compiler   # rc=134
printf '# cafe\n: Float32LE[_rem / 4]\n'         | build/tools/sddl/sddl_compiler   # rc=0
OpenZL error string: Corruption detected
Message: Encountered error in A1CBOR library with code "writeFailed".
	#0 ??? (src/openzl/shared/a1cbor.c:1404)
	#2 A1C_convert_cbor_to_json (src/openzl/common/a1cbor_helpers.c:185)

Any high byte does it, valid UTF-8 or not, anywhere in the file. SDDL2 is unaffected.

Cause looks like: sddl_compiler embeds the description source, comments included, into the
compiled compressor, and serializing that to JSON hits

// src/openzl/shared/a1cbor.c:1698
if ((uint8_t)c >= 0x80) {
    return A1C_Encoder_error(encoder, A1C_ErrorType_jsonUTF8Unsupported);
}

jsonUTF8Unsupported is already defined and mapped, but what surfaces is writeFailed under
ZL_ErrorCode_corruption, so the message blames CBOR instead of encoding.

Expected: surface the real error type, or reject non-ASCII at parse time with a line number.
Not aborting out of the CLI tool would help on its own.

Matters because the repo ships an LLM-targeted SDDL spec and LLM-written comments are full of
em-dashes.

Happy to send a PR if you have a preference.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions