Skip to content

Improved bash json parser#869

Merged
rcannood merged 2 commits into
switch_to_arraysfrom
better_json_parser
Feb 10, 2026
Merged

Improved bash json parser#869
rcannood merged 2 commits into
switch_to_arraysfrom
better_json_parser

Conversation

@rcannood
Copy link
Copy Markdown
Contributor

@rcannood rcannood commented Feb 10, 2026

Describe your changes

Rewrites the Bash JSON parser (ViashParseJson.sh) as a recursive descent parser modelled after the Scala ViashParseJson.scala. The old line-based regex parser silently failed on compact JSON and had fragile string/array handling.

Related issue(s)

Part of the switch_to_arrays branch (PR #762).

Type of Change

  • Major change (non-breaking change which modifies existing functionality)
  • Bug fix (non-breaking change which fixes an issue)

Changes

Problems with the old parser:

  • Silent failure on compact/minified JSON (assumed one key-value pair per line)
  • Array parsing logic duplicated 3×
  • Embedded commas, colons, braces in strings could confuse regex patterns
  • Unrecognised lines silently skipped

New parser: Recursive descent with _viash_peek/_viash_consume primitives, type-specific parsers (_viash_parse_string, _viash_parse_number, etc.), serialization functions for deep nesting, and proper error reporting. Input is pre-split into a character array via fold -w1 for O(1) access.

Performance:

Dimension Old New
500 KV pairs 573ms 481ms
500 array elements 1040ms 295ms
10k char strings N/A 177ms

Compatibility

  • Bash 3.2+ (macOS default) — no declare -g, local -n, or readarray
  • All 64 parser tests + 70 render tests pass

Checklist

  • I have read the CONTRIBUTING doc.
  • I have performed a self-review of my code.
  • My code follows the code style of this project.
  • I have added tests that prove my fix is effective or that my feature works.
  • New and existing unit tests pass locally with my changes.
  • Proposed changes are described in the CHANGELOG.md.

@rcannood rcannood changed the title add improved json parser Improved bash json parser Feb 10, 2026
Copy link
Copy Markdown
Collaborator

@hcannoodt hcannoodt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's starting to become all a bit crazy ...

@rcannood rcannood marked this pull request as ready for review February 10, 2026 13:00
@rcannood rcannood merged commit 3d7ebdf into switch_to_arrays Feb 10, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants