Support deprecated Gerber syntax via a refactored tokenizer#25
Open
nicolube wants to merge 3 commits into
Open
Support deprecated Gerber syntax via a refactored tokenizer#25nicolube wants to merge 3 commits into
nicolube wants to merge 3 commits into
Conversation
added 3 commits
May 21, 2026 10:07
Commands are delimited by `*`, not newlines (spec 4.1). The old line-based tokenizer merged single-line files into one block, causing a spurious CoordinateDataWithoutOperationCode error. Now splits on `*`, treats `%...%` spans (incl. macros) as one block, and ends a block at a newline outside such spans.
Parse errors carried only a (line, content) tuple. Replace it with an ErrorContext { line, offset, token } so callers can pinpoint which command failed, which matters when several are packed onto one line. Also funnel outer parse_line errors through the context path instead of dropping them.
ViewMate and other tools emit G1/G2/G3 instead of G01/G02/G03 (spec 8.3 style variation). These fell through to UnknownCommand, so the interpolate never registered and modal D01 never armed, cascading into spurious CoordinateDataWithoutOperationCode errors across every region. Accept the single-digit forms, disambiguating G3 from the G36/G37 region commands.
Contributor
Author
|
@hydra Pls review the PR carefully, this is a hevy one with breaking API changes. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Some copper layers exported from ViewMate Pro failed to load — a flood of CoordinateDataWithoutOperationCode errors. Root cause was a mix of newline-based tokenizing and a couple of unsupported deprecated constructs from the Gerber spec. This PR
refactors the tokenizer and adds support for the missing syntax.
Refactored tokenizer (Gerber spec §4.1)
Per §4.1, commands are delimited by the end-of-block character *; newlines are insignificant whitespace. The old tokenizer split input line-by-line, so files that pack the whole stream onto one physical line were parsed as a single command and fell
apart.
Reworked it into a byte-level, */%-aware tokenizer:
Support deprecated single-digit G-codes (Gerber spec §8.3)
§8.3 lists G1/G2/G3 as a tolerated style variation of G01/G02/G03. ViewMate emits these, and we treated them as UnknownCommand — so the interpolation never registered, modal D01 never armed, and every coordinate line in every region errored. Now
supported, including the combined G1X…D1* form, with G3 disambiguated from the G36/G37 region commands.
Richer error context
Errors now carry a column offset and the failing token (Line 14:28: 'G1X1576795Y1969D1*'), not just a line number — which is what made the single-digit G-code issue obvious. Also fixed a path where some errors were silently dropped.
Breaking: GerberParserErrorWithContext.line: Option<(usize, String)> is replaced by context: Option ({ line, offset, token }).
Verification
ViewMate copper layers that previously threw ~597k errors now parse with zero. Regression tests added for each change; full suite, clippy, and fmt are green.