Skip to content

Add grammar-based payload generation via barkus#1838

Draft
GeorgeHahn wants to merge 5 commits intomainfrom
hahn/integrate-barkus-grammar-payload
Draft

Add grammar-based payload generation via barkus#1838
GeorgeHahn wants to merge 5 commits intomainfrom
hahn/integrate-barkus-grammar-payload

Conversation

@GeorgeHahn
Copy link
Contributor

Summary

  • Integrates barkus as a new grammar payload type in lading_payload
  • Enables any existing generator (TCP, UDP, HTTP, file, unix socket, etc.) to send grammar-generated structured data from EBNF, PEG, or ANTLR v4 grammar files
  • Bridges rand 0.9 (lading) to rand 0.10 (barkus) by seeding a local SmallRng, preserving determinism
  • Validates at construction time that the grammar is reachable within the configured depth budget
  • Caps consecutive generation failures to prevent infinite loops on pathological grammars

Config example

generator:
  - tcp:
      seed: [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1]
      addr: "127.0.0.1:8080"
      variant:
        grammar:
          grammar_path: /path/to/json.ebnf
          format: ebnf
          max_depth: 40
      maximum_block_size: "1 MiB"
      maximum_prebuild_cache_size_bytes: "8 MiB"

@GeorgeHahn GeorgeHahn force-pushed the hahn/integrate-barkus-grammar-payload branch 2 times, most recently from 9b5b1cf to f179a8b Compare March 25, 2026 21:22
GeorgeHahn and others added 5 commits March 26, 2026 09:58
Integrates barkus as a new payload type in lading_payload, enabling any
existing generator (TCP, UDP, HTTP, file, unix socket, etc.) to send
grammar-generated structured data (JSON, CSV, SQL, arbitrary formats)
from EBNF, PEG, or ANTLR v4 grammar files.

Key design decisions:
- Bridges rand 0.9 (lading) to rand 0.8 (barkus) by seeding a local
  SmallRng from the lading RNG, preserving determinism
- Validates at construction time that the grammar's start production
  is reachable within the configured depth budget
- Caps consecutive generation failures to prevent infinite loops on
  pathological grammars

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Fix ast-grep no-long-paths: shorten 4+ segment paths via imports
- Fix ast-grep no-as-imports: use helper fn instead of `as _` import
- Fix clippy unnecessary_wraps: replace default_max_depth() with const
- Add barkus to deny.toml allow-git and clarify license entries
- Add example config (examples/lading-grammar.yaml) with sample EBNF
- Add payload README (lading_payload/README.grammar.md)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Barkus updated from rand 0.8 to rand 0.10. Update the pinned rev to
45e307e and switch the renamed dependency from rand_0_8 to rand_0_10.

The rand 0.9 (lading) -> rand 0.10 (barkus) bridge still uses the same
seed_from_u64 approach since rand_core 0.9 and 0.10 are incompatible.
This bridge can be removed entirely once lading upgrades to rand 0.10.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Fix comment saying "rand 0.8" to correctly say "rand 0.10"
- Add DEFAULT_MAX_TOTAL_NODES (20000) constant and always apply it,
  matching the documented default instead of relying on barkus internal default
- Add comment documenting barkus invariant for productions[start] (ADR-0010)
- Fix example YAML blackhole port (8080 → 8282) to match generator target

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The grammar generator should not inject delimiters between samples —
the grammar itself defines the output format. Users who want
newline-delimited output can include "\n" in their start production.

Updated example grammars, tests, and docs accordingly.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@GeorgeHahn GeorgeHahn force-pushed the hahn/integrate-barkus-grammar-payload branch from f179a8b to 5f267b9 Compare March 26, 2026 10:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant