This file provides specific guidance for working with the RIDDL project. For general ossuminc organization patterns, see ../CLAUDE.md (parent directory).
RIDDL documentation has moved to ossum.tech/riddl
The Hugo-based documentation site at riddl.tech has been consolidated into the ossum.tech MkDocs site. Key documentation:
- Language Reference: https://ossum.tech/riddl/references/language-reference/
- EBNF Grammar: https://ossum.tech/riddl/references/ebnf-grammar/
- Tutorials: https://ossum.tech/riddl/tutorials/
- Tools (riddlc): https://ossum.tech/riddl/tools/riddlc/
The doc/ directory in this repository contains legacy Hugo content that
redirects to ossum.tech. Do not add new documentation here.
RIDDL (Reactive Interface to Domain Definition Language) is a specification language for designing distributed, reactive, cloud-native systems using DDD principles. It's a monorepo containing multiple cross-platform Scala modules.
RIDDL is a heavily used library both by Ossum Inc. and external consumers. Never make incompatible changes to public APIs without following this process:
- No removal of public API — Do not remove public methods, classes,
traits, or extension methods. If functionality must be retired, add
@deprecatedannotations with a migration message and the target major version for removal (e.g.,@deprecated("Use flatten() instead", "2.0.0")). - No breaking signature changes — Do not change parameter types, return types, or add required parameters to existing public methods. New parameters must have defaults.
- Deprecation warnings until next major release — Deprecated APIs must remain functional through the current major version (1.x). They may only be removed in the next major release (2.0.0).
- Additive changes only — New methods, extension methods, classes, and traits are always safe. Prefer adding new APIs alongside old ones rather than modifying existing ones.
When in doubt, add, don't change.
- Scala 3.8.3 (not Scala 2!) — overrides sbt-ossuminc's 3.3.7 LTS default. RIDDL originally pinned 3.8.3 to dodge a 3.3.x compiler infinite loop on opaque + intersection types; the override has been kept while we ride ahead of LTS.
- ALWAYS use Scala 3 syntax:
while i < end do ... end while(NOTwhile (i < end) { ... })- No
nullchecks — useOption(x)instead - New control flow syntax with
do/then/end
Current version: 1.4.0 (1.21.0 upgrade).
// Scala 3.8.3 (overridden from sbt-ossuminc's 3.3.7 LTS default)
.configure(With.scala3) // Sets scalaVersion to 3.3.7 LTS
// Then override: scalaVersion := "3.8.3"
// Scala.js configuration
.jsConfigure(With.ScalaJS(
header = "RIDDL: module-name",
hasMain = false,
forProd = true,
withCommonJSModule = true
))
// Scala Native configuration
.nativeConfigure(With.Native(
mode = "fast", // "debug", "fast", "full", "size", "release"
buildTarget = "static", // or "application"
gc = "none",
lto = "none"
))
// BuildInfo with custom keys
.jvmConfigure(With.BuildInfo.withKeys(
"key1" -> value1,
"key2" -> value2
))utils → language → passes → commands → riddlc
↓
testkit
Note: The diagrams and hugo modules have been moved to the riddl-gen repository.
Purpose: Binary AST serialization for fast module imports. Status: Complete; ~6-10x faster than reparsing source; output ~63-67% of source size on non-trivial inputs.
- Package:
com.ossuminc.riddl.language.bastinlanguage/shared/src/main/scala/com/ossuminc/riddl/language/bast/ - Cross-platform: JVM, JS, Native
- Pass:
passes/shared/.../BASTWriterPass.scala - CLI:
riddlc bastify <file.riddl>(write);riddlc unbastify(read — pending) - Format docs: live at ossum.tech/riddl, not in this repo
Key files in the bast package:
package.scala— constants and node type tags (NODE_, TYPE_, STREAMLET_*, …)BASTWriter.scala— serialization (extends HierarchyPass)BASTReader.scala— deserializationBASTLoader.scala— import-loading utilityBASTUtils.scala— shared utilitiesStringTable.scala,PathTable.scala— interning tables
HAZARD — disjoint tag sets: readNode() only handles NODE_*
tags; readTypeExpression() only handles TYPE_* tags. Crossing
them causes byte misalignment that surfaces as "Invalid string
table index" errors during deserialization.
The riddlLib module exports a TypeScript-friendly API via RiddlAPI object.
Key features:
- All method names preserved (not minified) via
@JSExport - JavaScript-friendly return types:
{ succeeded: boolean, value?: object, errors?: Array<object> } - All Scala types converted to plain JS:
List→Array- Case classes → Plain objects
Either→{ succeeded, value, errors }
Building npm packages (via sbt-ossuminc 1.4.0 helpers):
sbt riddlLibJS/npmPrepare # Assemble package (pure sbt)
sbt riddlLibJS/npmPack # Create .tgz tarball
sbt riddlLibJS/npmPublishGithub # Publish to GH Packages
sbt riddlLibJS/npmPublishNpmjs # Publish to npmjs.comCI Workflow: .github/workflows/npm-publish.yml triggers on
release or manual dispatch, uses sbt tasks directly.
Module format: ESModule ("type": "module" in package.json).
Consumers use import { RiddlAPI } from '@ossuminc/riddl-lib'.
Documentation:
NPM_PACKAGING.md- npm build and installation guideTYPESCRIPT_API.md- Complete TypeScript API reference
Published: @ossuminc/riddl-lib on GitHub Packages npm registry
CRITICAL DISTINCTION:
- Can appear anywhere in hierarchy
- Parser rules determined by enclosing container
include "entities.riddl"in a Context → must contain Context-valid content- Already implemented
- Loads BAST-serialized content into RIDDL models
- Full import:
import "file.bast"— loads all Nebula contents - Selective import:
import domain X from "file.bast" - Aliased import:
import type T from "file.bast" as MyT - Allowed locations: Root level, inside domains, inside contexts
- 14 definition kinds supported (domain, context, entity, type, etc.)
- Key files:
CommonParser.scala—bastImport(),selectiveBastImport()TopLevelParser.scala—loadBASTImports()post-parse loadingBASTLoader.scala— BAST file reading and content populationAST.scala—BASTImportcase class
- Tests: 4 passing in
BASTLoaderTest.scala - Validation: Integrated into
ValidationPass
- Wraps
ArrayBuffer[CV]for efficient modification - Extension methods:
.toSeq,.isEmpty,.nonEmpty - Do NOT use:
.toList,.iteratordirectly (not available) - Pattern:
contents.toSeq.map { ... }.toJSArrayfor JS conversion
- Scala 3 enum, not case classes
- Get type name:
token.getClass.getSimpleName.replace("$", "") - Extract text:
token.loc.source.data.substring(token.loc.offset, token.loc.endOffset)
- Fields:
line,col,offset,endOffset,source - Always 1-based (not 0-based)
- Delta encoding for BAST: compress by storing differences
Prefer HierarchyPass for maintaining parent context:
class MyPass extends HierarchyPass {
override def process(value: RiddlValue, parents: ParentStack): Unit = {
value match {
case d: Domain => processDomain(d, parents)
case c: Context => processContext(c, parents)
// ... pattern match all node types
}
}
override def result: MyPassOutput = MyPassOutput(...)
}BAST Writer Pattern:
BASTWriterPass(in passes module) extendsHierarchyPass- Uses
BASTWriterutilities (in language module) for byte writing - Sacrifice write speed for read speed
- String interning for deduplication
Updated: Jan 2026 for improved reliability and performance
- Triggers:
main,developmentbranches - Parallelized: JVM/Native/JS builds using matrix strategy
- Timeout: 60 minutes
- Dependency scanning with SARIF upload
- Auto-triggers on PRs and pushes (not manual-only)
- Timeout: 45 minutes
- Fixed artifact paths (was broken in earlier versions)
- Triggers only on Hugo/doc changes (NOT all .scala files)
- ScalaDoc caching for faster builds
- Timeouts: 30min build, 10min deploy
All workflows use JDK 25 (standardized)
When the Scala LTS version changes (either directly in build.sbt or when sbt-ossuminc updates its default), the following files MUST be updated:
GitHub Workflows (.github/workflows/):
-
scala.yml:
RIDDLC_PATHenv var:riddlc/native/target/scala-X.Y.Z/riddlc- Cache paths:
**/target/scala-X.Y.Z - Native artifact path:
riddlc/native/target/scala-X.Y.Z/riddlc - Native artifact path:
riddlLib/native/target/scala-X.Y.Z/libriddl-lib.a - JS artifact path:
riddlLib/js/target/scala-X.Y.Z/riddl-lib-opt/main.js
-
coverage.yml:
- Coverage report paths:
**/target/scala-X.Y.Z/scoverage-report/
- Coverage report paths:
sbt-ossuminc Version Policy:
- sbt-ossuminc always defaults to the latest Scala LTS version
- When sbt-ossuminc is updated, check if its default Scala version changed
- Current LTS: 3.3.7; riddl runs ahead on 3.8.3
- Next LTS expected: 3.9.x (Q2 2026)
Quick Search to Find All References:
grep -r "scala-3\." .github/workflows/Example Fix (3.8.3 → 3.9.0):
sed -i 's/scala-3.8.3/scala-3.9.0/g' .github/workflows/*.ymlAny change to the fastparse parser MUST have a corresponding change to the EBNF grammar.
The EBNF grammar at language/shared/src/main/resources/riddl/grammar/ebnf-grammar.ebnf
is the canonical specification of RIDDL syntax. It is validated by a TatSu-based parser
that runs in CI on all **/input/**/*.riddl test files.
When modifying the fastparse parser:
- Update the corresponding rule(s) in
ebnf-grammar.ebnf - Run the EBNF validator locally:
cd language/jvm/src/test/python pip install -r requirements.txt # first time only python ebnf_tatsu_validator.py
- Ensure both parsers accept the same inputs
- CI will fail if the EBNF parser cannot parse test files that fastparse accepts
This ensures the documented grammar stays in sync with the actual implementation.
When implementing new code:
- Write the code
- ALWAYS run
sbt "project <module>" compile - Fix Scala 3 syntax errors immediately
- Then proceed to next step
- Input test files:
language/input/<category>/<file>.riddl - Examples:
language/input/import/import.riddl
Cause: Using Scala 2 syntax
Fix: Use Scala 3 syntax with do/end
Cause: Token is an enum
Fix: Use token.getClass.getSimpleName
Cause: Contents is opaque type with limited extensions
Fix: Use .toSeq extension method
Cause: sbt-ossuminc 1.0.0 API change
Fix: Use With.ScalaJS instead
Cause: Scala 3.8.3 limitation — default parameter values in a case
class's first parameter list cannot resolve given instances from a
subsequent using clause in the generated companion apply method.
Fix: Remove the default value. May be fixed in 3.9.x LTS.
Example:
// This fails in 3.8.3:
case class Foo(x: Bar = Bar())(using PlatformContext)
// Fix: remove default (or provide explicit given)
case class Foo(x: Bar)(using PlatformContext)Cause: @JSExportTopLevel on a case class with (using PlatformContext) in a second parameter list. The JS export sees the
context parameter as a non-default parameter after defaulted params.
Fix: Remove @JSExportTopLevel from internal data structures that
don't need to be constructed from JS code.
Cause: System.lineSeparator() returns \0 in Scala.js
Fix: Use PlatformContext.newline instead. Never use
System.lineSeparator() in shared code. The FileBuilder trait
and its entire hierarchy use (using PlatformContext) for this.
- Create directory:
<moduleName>/shared/src/{main,test}/scala/... - Add to
build.sbtusingCrossModule - Add variants to root aggregation
- Dependencies go in both directions of
cpDep()
- Shared code:
<module>/shared/src/ - Platform-specific:
<module>/{jvm,js,native}/src/ - Avoid platform-specific APIs in shared code
- Use
PlatformContextfor platform abstraction
- sbt-dynver generates versions from git tags
- Format:
MAJOR.MINOR.PATCH-commits-hash-YYYYMMDD-HHMM - Clean tag:
git tag -a 1.0.0 -m "Release 1.0.0"(novprefix - it interferes with sbt-dynver) - Always run
sbt publishLocalafter tagging to make the new version available locally
Short description (imperative mood)
Detailed explanation of what changed and why.
Focus on "why" rather than "what".
Co-Authored-By: Claude <model-name> <noreply@anthropic.com>
- main: Production releases
- development: Active development (current work)
- Always push to
developmentunless releasing
# Compile specific module
sbt "project bast" compile
# Run tests for module
sbt "project language" test
# Build npm package
./scripts/pack-npm-modules.sh riddlLib
# Format code
sbt scalafmt
# Check all platforms compile
sbt cJVM cJS cNative
# Run all tests
sbt tJVM tJS tNative
# Package riddlc executable
sbt riddlc/stage
# Result: riddlc/jvm/target/universal/stage/bin/riddlcCore parsing/validation logic lives in RiddlLib (shared trait +
companion object) at riddlLib/shared/.../RiddlLib.scala. This is
usable on JVM, JS, and Native. The JS-only RiddlAPI.scala is a
thin facade that delegates to RiddlLib and converts results to
plain JavaScript objects.
- Cross-platform code: Use
RiddlLib.parseString(...)etc. with agiven PlatformContextin scope (provided by each platform'scom.ossuminc.riddl.utils.pc) - JS facade:
RiddlAPIadds@JSExportmethods,getDomains,inspectRoot, and JS-only helpers likeformatErrorArray
CRITICAL: All methods that accept an origin parameter use
RiddlLib.originToURL() to convert strings to URLs.
def originToURL(origin: String): URL =
if origin.startsWith("/") then
URL.fromFullPath(origin)
else
URL(URL.fileScheme, "", "", origin)
end ifWrong (Scala 2 style):
lines.foreach(pc.log.info) // Error: type mismatchCorrect (Scala 3):
lines.foreach(line => pc.log.info(line))Reason: Scala 3 doesn't automatically convert by-name parameters (=> String) to function parameters (String => Unit).
When code needs to be shared between JVM (riddlc commands) and JS (RiddlAPI), put it in utils/shared/:
Example: InfoFormatter is used by both:
commands/InfoCommand.scala(JVM)riddlLib/RiddlAPI.scala(JS via@JSExport)
// utils/shared/src/main/scala/com/ossuminc/riddl/utils/InfoFormatter.scala
object InfoFormatter {
def formatInfo: String = {
// Build info formatting logic
}
}After staging (sbt riddlc/stage), the riddlc executable provides:
riddlc help # Show all available commands
riddlc version # Version information
riddlc info # Build information
riddlc parse <file> # Parse RIDDL file
riddlc validate <file> # Validate RIDDL fileCommands can load options from HOCON config files.
Executable location: riddlc/jvm/target/universal/stage/bin/riddlc
lazy val mymodule_cp = CrossModule("mymodule", "riddl-mymodule")(JVM, JS, Native)
.dependsOn(cpDep(utils_cp), cpDep(language_cp))
.configure(With.typical, With.GithubPublishing)
.settings(
description := "Description here"
)
.jvmConfigure(With.coverage(50))
.jsConfigure(With.ScalaJS("RIDDL: mymodule", withCommonJSModule = true))
.nativeConfigure(With.Native(mode = "fast"))
lazy val mymodule = mymodule_cp.jvm
lazy val mymoduleJS = mymodule_cp.js
lazy val mymoduleNative = mymodule_cp.nativeThen add to root aggregation: .aggregate(..., mymodule, mymoduleJS, mymoduleNative)
Note: Use With.ScalaJS(...) for sbt-ossuminc 1.0.0+, not With.Javascript(...)
- Extend
Pass,DepthFirstPass, orHierarchyPass - Implement
process()method for each AST node type - Declare dependencies via
def requires(): Seq[Pass] = Seq(...) - Override
result()to return yourPassOutputsubclass - Add to standard passes or invoke explicitly
- Define options:
case class MyOptions(...) extends CommandOptions - Define command:
class MyCommand extends Command[MyOptions] - Implement:
def name: Stringdef getOptionsParser: OptionParser[MyOptions]def run(options: MyOptions, context: PlatformContext): Either[Messages, PassesResult]
- Register with
CommandLoaderif using plugin system
Each subsection is a topic, not a serial number — add new entries to the right group rather than appending to a list.
- VERSION is a single integer (
VERSION: Int = 1) and stays at 1 until the schema is finalized for external users. - FORMAT_REVISION must be incremented whenever a BASTWriter
change produces output that an older BASTReader can't read
correctly: new statement subtypes, wire-format changes,
reordered fields, new node tags. Constant lives in
language/shared/.../bast/package.scala. - Location comparisons use offsets, not
line/col. - BASTImport in HierarchyPass —
openBASTImport/closeBASTImporthooks plustraverseBASTImportContents(bi). AllPassVisitorimplementors must define these (even as no-ops);BASTImportextendsContainerbut notBranch, so without the hooks it falls through and its contents are never visited.
- AST.Set shadows scala.Set — use selective imports or
qualify as
scala.collection.immutable.Set. - Schema match ordering — Schema extends
Leaf(Definition) but is also in theNonDefinitionValuesunion. Its case must appear BEFOREcase _: NonDefinitionValues. Same trap forRelationshipvscase _: Definition. - State is a Branch, not a Leaf, of
Branch[StateContents]whereStateContents = Handler | Comment.PassVisitorusesopenState/closeState(notdoState). ResolutionPass prepends State to parents (as with all Branches), so refMap keys for State's type ref use State as parent, not Entity. do "..."is an alias forprompt "..."— both producePromptStatement.- walkStatements helper — private in ValidationPass; walks
into
WhenStatement/MatchStatementnesting. - Definition hashCode/equals override —
Definitiontrait overrides both:hashCodecheap (id + loc + class);equalsstructural viaproductEquals, skippingContentsfields. Prevents O(subtree) hashing in anyHashMap[Definition, X]. Opaque typeContents[?]erases toArrayBufferat runtime, socase (_: Contents[?], …)matches correctly.
- OutlinePass / TreePass — lightweight
HierarchyPasssubclasses inpasses/shared/.../passes/. OutlinePass → flatSeq[OutlineEntry]. TreePass → recursiveSeq[TreeNode], exposed viaRiddlAPI.getOutline()/getTree(). TreePass uses amutable.Stack[ListBuffer[TreeNode]]for pure O(n) building (not aHashMap[Definition, ListBuffer]). - Analysis passes — MessageFlowPass, EntityLifecyclePass,
DependencyAnalysisPass, AIHelperPass (1.22.0). All in
passes/shared/.../analysis/(orpasses/ai/); each extendsCollectingPass(or HierarchyPass for AIHelperPass) and requires ResolutionPass. - MessageFlowPass —
MessageFlowEdge.messageTypeisOption[Type](adaptor declarations produceNone; typed handler edges produceSome). Direction-aware:InboundAdaptor("from") → producer=referent, consumer=source;OutboundAdaptor("to") → producer=source, consumer=referent.MessageFlowOutput.edgesForDomain()/edgesForContext()take aSymbolsOutputparameter for parent-chain walking. - UsageResolution uses
mutable.Set[Definition]foruses/usedBy(wasSeq). API boundary methods (getUsers,getUses) return.toSeq. - ParentStack is a class, not a type alias. Use
ParentStack.empty(notmutable.Stack.empty). Same API (push, pop, toParents). It cachestoParents(toSeq). - ValidationMode enum —
FullorQuick. Quick skipscheckStreamingandclassifyHandlersin postProcess. - IncrementalValidator — caches messages per-Context using
FNV-1a fingerprints.
validator.reset()forces a full recheck. - RecognizedOptions registry — validates option names, argument counts, parent types. Unrecognized → StyleWarning.
- RiddlLib analysis API —
getHandlerCompleteness(),getMessageFlow(),getEntityLifecycles()on the shared RiddlLib trait and JS facade. JS facade returns""for the untyped (None) MessageFlow edges.
- Streamlet shape check — guard on
nonEmptybefore checking inlet/outlet counts (empty = placeholder). - Adaptor cross-context type resolution — use the
parent-independent
resolution.refMap.definitionOf[Type](pathId). - Schema parser —
schemaKinduses"time-series"(hyphenated). Consecutive schemas needwith { ... }blocks. - CheckMessagesTest
.checkfile format — lines starting with space are continuation lines; non-space lines begin new entries. Don't insert mid-continuation. - RiddlResult[T] replaces
Either[Messages, T]— sealed ADT withSuccess[T]/Failure; useresult.toEitherfor backward compat.
- Container.flatten() recursively removes Include / BASTImport
wrappers in place. Use base
Pass, notDepthFirstPass— mutating contents during traversal corrupts ArrayBuffer iteration. - FileBuilder requires PlatformContext —
trait FileBuilder (using PlatformContext). All subclasses must propagate theusingclause.
- Multi-file mode —
flatten=false(default) preserves include/import structure;-s truecollapses to single file. PrettifyState.toDestination()strips leading/trailing/fromoutDir(URL basis can't start with/).- Include paths —
openIncludeusesurl.path(relative filename), noturl.toExternalForm(absolute URL). RiddlFileEmitter.trimTrailingNewline()— used incloseTypeto join}withwith {on the same line.
- parseString returns an opaque Root in JS — use
getDomains(root)orinspectRoot(root)to access data; TypeScript type is brandedRootAST. - RiddlLib.ast2bast(root) returns
RiddlResult[Array[Byte]]on the shared side /RiddlResult<Int8Array>in TS. - riddlLibJS tests override
Test / scalaJSLinkerConfigtoCommonJSModule. Production stays ESModule. - ESM shim hazard — never put
import ',import ", orimport(in shared string literals; ESM shim plugins rewrite these patterns. Use string concatenation.ESMSafetyTestenforces it. - npm prerelease publishing — sbt-dynver versions like
1.2.3-1-hashare prerelease per npm semver; pass--tag dev. - GitHub Packages npm auth —
gh auth refresh -s write:packagesis required.
- release.yml — triggered by
gh release create. Builds native riddlc (macOS ARM64, Linux x86_64) + JVM universal. Sendsrepository_dispatchto homebrew-tap with SHA256s. Requires theHOMEBREW_TAP_SECRETrepo secret. - sbt-dynver wants a clean working tree —
git stashmodified files beforesbt publishon a release tag. - External-repo tests — download at construction time (not
in
beforeAll) for ScalaTestAnyWordSpec. - TatSu pin —
TatSu>=5.12.0,<5.17.0. 5.17.0 has a missingrichdependency that breaks import. - EBNF TatSu syntax —
{rule}+notrule+for positive closure; TatSu requires curly braces around the repeated element. - ScalaDoc + inline + opaque types — keep
inlineoffContentsextension methods (NPE inScalaSignatureProvider.methodSignature). Filed: scala/scala3#25306. - sbt-riddl auto-downloads riddlc — caches in
~/.cache/riddlc/<version>/; three-tier resolution: explicit path > download > PATH. Use--no-ansi-messagesand strip ANSI for version parsing. PinriddlcVersionto a real release tag in scripted tests, not the dynver snapshot. - sbt plugin visibility — use
private[plugin] def(notprivate def) so Scala 2.12 doesn't warn "private method never used" when sbt macros generate the usage.
- PR merge with branch protection —
gh pr merge --admin --merge --delete-branch=false.