Optional schema validation system for type-safe data handling

[hermanndelcampo-theaiqueen]

Hey TOON community! 👋

I've been working with TOON in production and loving the token savings, but I kept running into situations where I needed type validation and better self-documentation for LLMs. So I prototyped a schema validation system that I'd love to get your feedback on!

The Idea

Add optional schema definitions to TOON that provide:

• Type validation at encode/decode time
• Self-documenting data that LLMs understand better
• Even more token savings (early tests show 56% reduction!)

Proposed Syntax

@schema User {
  id: int!
  name: string(3..50)
  email: email!
  age?: int(0..120)
}

users[100]{...@User}:
1,Alice,[email protected],25
2,Bob,[email protected],30

Instead of repeating field names 100 times, we define them once! The schema also helps LLMs understand the data structure without additional prompting.

🧪 Early Results

I've built a working prototype and the results are promising:

• Token reduction: 56% fewer tokens for 100+ records
• Performance: Only ~0.2ms overhead for validation
• Backward compatible: Completely optional feature
• LLM accuracy: 15% better data extraction in my tests

Questions for You

1. Syntax preferences: Is @schema intuitive? Would you prefer something else?

2. Validation levels: Should we support:

   • Strict (fail on first error)
   • Lenient (collect all errors)
   • Coerce (auto-fix types when possible)

3. Schema location: Where should schemas live?

   • Inline with data (current approach)
   • Separate .toon-schema files
   • Both?

4. Default behavior: Should validation be:

   • Opt-in with validateOnEncode: true
   • On by default with opt-out
   • Only in development mode

5. Error format: What's most useful?

   // Option A: Flat list
   ["field 'age' must be number", "field 'email' is required"]
   
   // Option B: Structured
   [{field: "age", error: "type", expected: "int", got: "string"}]

6. Features to add/remove:

   • Custom validators? (@validate("price > 0"))
   • Schema inheritance? (@schema Admin extends User)
   • Computed fields? (fullName = firstName + lastName)
   • Too complex? What could be simpler?

Your Thoughts?

• Would this be useful for your use cases?
• Any concerns about complexity or performance?
• Missing features that would make this killer?
• Alternative approaches I should consider?

Really excited to hear what everyone thinks! TOON has been game-changing plattform, happy found it 2 weeks ago and I think schemas could make it even better.

Let me know if you love it, hate it, or have ideas to make it better! All feedback welcome 🙏

My PR: #126

[smortezah]

#6

[johannschopplich]

Hi there, thanks for the initial PR! Well done. 👏

I like the idea of having some kind of schema validation built into the format itself. Having said that, to keep the format lean and implementations across languages feasable, the feature requires some back and forth in the spec first.

Planning-wise, I'd love to have:
1️⃣ an RFC first in the spec repo,
2️⃣ accompanied with comformance tests that can be used across languages.

Finally, we can overhaul the TS implementation. Most importantly, I have some concers over the current implementation:

• I'd prefer Standard Schema instead of a custom validation engine. That would enable users to use Zod/Valibot etc. Dunno how that works well for TOON in other languages, tho.
• Tests should be part of the conformance test suite.

---

Regarding your questions:

1. Syntax preferences: Is @schema is plain and simple, I like it.

2. Validation levels: strict and lenient feels like a good place to start.

3. Inline schemas for simpler parsing for now.

4. Depends on the spec version. We could keep it opt-in during v2 and enable it by default for v3.

5. A structured list.

I'd prefer a clean and simple interface for the initial version that the other ports (Python, etc.) can jump on without too much of a hassle.

[hermanndelcampo-theaiqueen]

Hello, I will directly work on it and adapt your suggestions to my fPR. Best regards Hermann Hermann del Campo Managing Director & CTO Mobile: +49 160 553 80 13 E-Mail: ***@***.*** Raiva Technologies UG (haftungsbeschränkt) C/o UFA Dianastraße 21 14482 Potsdam Germany Managing Director: Carsten Probst, Hermann del Campo Registration Court: AG Potsdam Registration number: in progress VAT number: in progress Am 14.11.2025 um 13:12 schrieb Johann Schopplich ***@***.***>: Sie erhalten nicht häufig E-Mails von ***@***.*** Erfahren Sie, warum dies wichtig ist<https://aka.ms/LearnAboutSenderIdentification> Hi there, thanks for the initial PR! Well done. 👏 I like the idea of having some kind of schema validation built into the format itself. Having said that, to keep the format lean and implementations across languages feasable, the feature requires some back and forth in the spec<https://github.com/toon-format/spec> first. Planning-wise, I'd love to have: 1️⃣ an RFC first in the spec repo<https://github.com/toon-format/spec>, 2️⃣ accompanied with comformance tests that can be used across languages. Finally, we can overhaul the TS implementation. Most importantly, I have some concers over the current implementation: * I'd prefer Standard Schema<https://github.com/standard-schema/standard-schema> instead of a custom validation engine. That would enable users to use Zod/Valibot etc. Dunno how that works well for TOON in other languages, tho. * Tests should be part of the conformance test suite.

…

________________________________ Regarding your questions: 1. Syntax preferences: Is @Schema is plain and simple, I like it. 2. Validation levels: strict and lenient feels like a good place to start. 3. Inline schemas for simpler parsing for now. 4. Depends on the spec version. We could keep it opt-in during v2 and enable it by default for v3. 5. A structured list. I'd prefer a clean and simple interface for the initial version that the other ports (Python, etc.) can jump on without too much of a hassle. — Reply to this email directly, view it on GitHub<#127 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BZN4JLFP33AYX2PW2IHVUXT34XBMRAVCNFSM6AAAAACMBLXYL2VHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTIOJWHEYDANI>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[johannschopplich]

Amazing, thanks! Regarding the idea about Standard Schema – maybe the standard is not that useful to the TOON format, since only a subset of validations are required to reflect JSON values.