All notable changes to this project are documented here. The format follows Keep a Changelog and this project adheres to Semantic Versioning.
First public release under github.com/dotcommander/chunker. Chunking is
delegated to github.com/dotcommander/reliquary.
- Module path renamed from
chunkertogithub.com/dotcommander/chunkerso the tool installs viago install github.com/dotcommander/chunker/cmd/chunker@latest. - Unknown
strategyandtoken_encodingvalues are now rejected with a clear error instead of being silently remapped to defaults. Behavior change for callers that relied on the silent-default leniency. total_tokensmetadata is computed from the original request text (consistent withtotal_chars); previously it summed per-chunk counts and double-counted the overlap region fortoken_basedchunking.start_char/end_charoffsets are derived from the library's authoritative byte spans rather than re-derived by substring search.
http.Servernow setsReadHeaderTimeout/ReadTimeout/WriteTimeout/IdleTimeout(Slowloris defense); previously only the handler-level timeout was set.- The
filessubcommand now bounds per-file reads (100 MiB cap, matching stdin), preventing memory exhaustion on large glob matches. The HTTP/chunkbody is bounded at 10 MiB. - Corrected the
cl100k_baseencoding comment (GPT-3.5/GPT-4, not gpt-5). - Stopped git-ignoring the
cmd/chunker/source directory — a barechunkerignore pattern had hidden 9 production files (including the embedded config) from version control; a fresh clone previously would not build.
v1.0.0 and v1.1.0 predate this changelog; see git log for their history.