|
1 | 1 | # Developers Guide |
2 | 2 |
|
3 | | -!!! info "Work In Progress" |
4 | | - This section is being developed elsewhere. When we feel it is able to not |
5 | | - confuse you, it will appear in stead of this notice. |
| 3 | +This guide is for developers who want to contribute to RIDDL itself—the |
| 4 | +compiler, tools, and ecosystem. If you're looking to use RIDDL to specify |
| 5 | +systems, see the [Author's Guide](../authors/index.md) or |
| 6 | +[Implementor's Guide](../implementors/index.md). |
| 7 | + |
| 8 | +## Prerequisites |
| 9 | + |
| 10 | +To work on RIDDL development, you'll need: |
| 11 | + |
| 12 | +- **JDK 21 or later** (Temurin recommended) |
| 13 | +- **Scala 3.6.x** (managed by sbt) |
| 14 | +- **sbt 1.10+** (Scala Build Tool) |
| 15 | +- **Git** for version control |
| 16 | + |
| 17 | +## Getting Started |
| 18 | + |
| 19 | +### Clone the Repository |
| 20 | + |
| 21 | +```bash |
| 22 | +git clone https://github.com/ossuminc/riddl.git |
| 23 | +cd riddl |
| 24 | +``` |
| 25 | + |
| 26 | +### Build the Project |
| 27 | + |
| 28 | +```bash |
| 29 | +# Compile all modules |
| 30 | +sbt compile |
| 31 | + |
| 32 | +# Run tests |
| 33 | +sbt test |
| 34 | + |
| 35 | +# Build riddlc executable |
| 36 | +sbt stage |
| 37 | +``` |
| 38 | + |
| 39 | +The staged executable will be at `riddlc/target/universal/stage/bin/riddlc`. |
| 40 | + |
| 41 | +## Project Structure |
| 42 | + |
| 43 | +RIDDL is organized as a multi-module sbt project: |
| 44 | + |
| 45 | +``` |
| 46 | +riddl/ |
| 47 | +├── language/ # Core language model and AST |
| 48 | +├── passes/ # Compiler passes (parsing, validation, etc.) |
| 49 | +├── commands/ # riddlc command implementations |
| 50 | +├── riddlc/ # Command-line interface |
| 51 | +├── hugo/ # Hugo documentation generator |
| 52 | +├── testkit/ # Testing utilities |
| 53 | +└── project/ # sbt build configuration |
| 54 | +``` |
| 55 | + |
| 56 | +### Key Modules |
| 57 | + |
| 58 | +**language** - The foundation of RIDDL: |
| 59 | + |
| 60 | +- AST definitions in `com.ossuminc.riddl.language.AST` |
| 61 | +- Parser combinators using fastparse |
| 62 | +- Source location tracking |
| 63 | + |
| 64 | +**passes** - Compiler transformation passes: |
| 65 | + |
| 66 | +- Parsing pass (text → AST) |
| 67 | +- Resolution pass (resolve references) |
| 68 | +- Validation pass (semantic checks) |
| 69 | +- Symbol table construction |
| 70 | + |
| 71 | +**commands** - The riddlc CLI: |
| 72 | + |
| 73 | +- Command parsing and dispatch |
| 74 | +- Option handling |
| 75 | +- Output formatting |
| 76 | + |
| 77 | +**hugo** - Documentation generator: |
| 78 | + |
| 79 | +- Converts RIDDL models to Hugo-compatible markdown |
| 80 | +- Generates diagrams (Mermaid) |
| 81 | +- Produces glossaries and indexes |
| 82 | + |
| 83 | +## Development Workflow |
| 84 | + |
| 85 | +### Running Tests |
| 86 | + |
| 87 | +```bash |
| 88 | +# Run all tests |
| 89 | +sbt test |
| 90 | + |
| 91 | +# Run tests for a specific module |
| 92 | +sbt "language/test" |
| 93 | +sbt "passes/test" |
| 94 | + |
| 95 | +# Run a specific test class |
| 96 | +sbt "testOnly com.ossuminc.riddl.language.ParserSpec" |
| 97 | + |
| 98 | +# Run tests continuously on file changes |
| 99 | +sbt ~test |
| 100 | +``` |
| 101 | + |
| 102 | +### Code Coverage |
| 103 | + |
| 104 | +```bash |
| 105 | +sbt coverage test coverageReport |
| 106 | +``` |
| 107 | + |
| 108 | +Coverage reports are generated in `target/scala-3.x/scoverage-report/`. |
| 109 | + |
| 110 | +### Code Formatting |
| 111 | + |
| 112 | +RIDDL uses scalafmt for consistent formatting: |
| 113 | + |
| 114 | +```bash |
| 115 | +# Format all code |
| 116 | +sbt scalafmt test:scalafmt |
| 117 | + |
| 118 | +# Check formatting without changes |
| 119 | +sbt scalafmtCheck test:scalafmtCheck |
| 120 | +``` |
| 121 | + |
| 122 | +## Architecture Overview |
| 123 | + |
| 124 | +### Compiler Pipeline |
| 125 | + |
| 126 | +RIDDL compilation follows a multi-pass architecture: |
| 127 | + |
| 128 | +``` |
| 129 | +Source Text |
| 130 | + │ |
| 131 | + ▼ |
| 132 | +┌─────────┐ |
| 133 | +│ Parsing │ → AST (Abstract Syntax Tree) |
| 134 | +└────┬────┘ |
| 135 | + ▼ |
| 136 | +┌────────────┐ |
| 137 | +│ Resolution │ → Resolved references |
| 138 | +└─────┬──────┘ |
| 139 | + ▼ |
| 140 | +┌────────────┐ |
| 141 | +│ Validation │ → Errors/Warnings |
| 142 | +└─────┬──────┘ |
| 143 | + ▼ |
| 144 | +┌────────────┐ |
| 145 | +│ Translation│ → Output (Hugo, etc.) |
| 146 | +└────────────┘ |
| 147 | +``` |
| 148 | + |
| 149 | +### AST Design |
| 150 | + |
| 151 | +The AST is defined in `language/src/main/scala/com/ossuminc/riddl/language/AST.scala`. |
| 152 | +Key design principles: |
| 153 | + |
| 154 | +- **Immutable** - All AST nodes are immutable case classes |
| 155 | +- **Hierarchical** - Nodes form a tree with `Definition` as the base |
| 156 | +- **Location-tracked** - Every node carries source location information |
| 157 | + |
| 158 | +### Parser Design |
| 159 | + |
| 160 | +RIDDL uses fastparse for its parser, defined in |
| 161 | +`language/src/main/scala/com/ossuminc/riddl/language/parsing/`. |
| 162 | + |
| 163 | +The parser is organized by language construct: |
| 164 | + |
| 165 | +- `CommonParser.scala` - Shared parsing utilities |
| 166 | +- `TypeParser.scala` - Type expression parsing |
| 167 | +- `StatementParser.scala` - Statement parsing |
| 168 | +- `DefinitionParser.scala` - Definition parsing |
| 169 | + |
| 170 | +## Adding New Features |
| 171 | + |
| 172 | +### Adding a New Statement Type |
| 173 | + |
| 174 | +1. **Define the AST node** in `AST.scala`: |
| 175 | + ```scala |
| 176 | + case class MyStatement( |
| 177 | + loc: At, |
| 178 | + // ... fields |
| 179 | + ) extends Statement |
| 180 | + ``` |
| 181 | + |
| 182 | +2. **Add parser rule** in `StatementParser.scala`: |
| 183 | + ```scala |
| 184 | + def myStatement[u: P]: P[MyStatement] = ... |
| 185 | + ``` |
| 186 | + |
| 187 | +3. **Add validation** in the validation pass |
| 188 | + |
| 189 | +4. **Add translation** in output generators (Hugo, etc.) |
| 190 | + |
| 191 | +5. **Write tests** covering parsing, validation, and output |
| 192 | + |
| 193 | +### Adding a New Pass |
| 194 | + |
| 195 | +1. Create a new pass class extending `Pass`: |
| 196 | + ```scala |
| 197 | + class MyPass extends Pass { |
| 198 | + def name: String = "MyPass" |
| 199 | + def process(root: Root): PassOutput = ... |
| 200 | + } |
| 201 | + ``` |
| 202 | + |
| 203 | +2. Register the pass in the compiler pipeline |
| 204 | + |
| 205 | +3. Write tests for the pass behavior |
| 206 | + |
| 207 | +## Testing Guidelines |
| 208 | + |
| 209 | +### Test Organization |
| 210 | + |
| 211 | +- **Unit tests** - Test individual components in isolation |
| 212 | +- **Integration tests** - Test component interactions |
| 213 | +- **Example tests** - Test full RIDDL specifications |
| 214 | + |
| 215 | +### Writing Parser Tests |
| 216 | + |
| 217 | +```scala |
| 218 | +class MyParserSpec extends ParsingTestBase { |
| 219 | + "MyParser" should { |
| 220 | + "parse valid input" in { |
| 221 | + val input = """my construct { ... }""" |
| 222 | + parseDefinition(input) shouldBe a[MyConstruct] |
| 223 | + } |
| 224 | + |
| 225 | + "report errors for invalid input" in { |
| 226 | + val input = """invalid syntax""" |
| 227 | + parseDefinition(input) shouldBe a[ParsingError] |
| 228 | + } |
| 229 | + } |
| 230 | +} |
| 231 | +``` |
| 232 | + |
| 233 | +### Writing Validation Tests |
| 234 | + |
| 235 | +```scala |
| 236 | +class MyValidationSpec extends ValidatingTestBase { |
| 237 | + "MyValidation" should { |
| 238 | + "detect invalid references" in { |
| 239 | + val input = """domain D { context C { entity E { ??? } } }""" |
| 240 | + val messages = validate(input) |
| 241 | + messages should contain(a[MissingError]) |
| 242 | + } |
| 243 | + } |
| 244 | +} |
| 245 | +``` |
| 246 | + |
| 247 | +## Contributing |
| 248 | + |
| 249 | +### Contribution Process |
| 250 | + |
| 251 | +1. **Fork** the repository |
| 252 | +2. **Create a branch** for your feature: `git checkout -b feature/my-feature` |
| 253 | +3. **Make changes** with tests and documentation |
| 254 | +4. **Ensure tests pass**: `sbt test` |
| 255 | +5. **Format code**: `sbt scalafmt` |
| 256 | +6. **Submit a pull request** to the `development` branch |
| 257 | + |
| 258 | +### Code Review |
| 259 | + |
| 260 | +Pull requests are reviewed for: |
| 261 | + |
| 262 | +- Correctness and test coverage |
| 263 | +- Code style consistency |
| 264 | +- Documentation completeness |
| 265 | +- Performance considerations |
| 266 | + |
| 267 | +### Commit Messages |
| 268 | + |
| 269 | +Follow conventional commit style: |
| 270 | + |
| 271 | +``` |
| 272 | +feat: add new statement type for async operations |
| 273 | +
|
| 274 | +Add AsyncStatement to support asynchronous message handling. |
| 275 | +Includes parser, validation, and Hugo output support. |
| 276 | +
|
| 277 | +Co-Authored-By: Your Name <email> |
| 278 | +``` |
| 279 | + |
| 280 | +## Debugging Tips |
| 281 | + |
| 282 | +### Parser Debugging |
| 283 | + |
| 284 | +Enable parse tracing for detailed output: |
| 285 | + |
| 286 | +```scala |
| 287 | +implicit val ctx = ParsingContext(tracing = true) |
| 288 | +``` |
| 289 | + |
| 290 | +### AST Inspection |
| 291 | + |
| 292 | +Use the `dump` command to see parsed AST: |
| 293 | + |
| 294 | +```bash |
| 295 | +riddlc dump mymodel.riddl |
| 296 | +``` |
| 297 | + |
| 298 | +### Logging |
| 299 | + |
| 300 | +RIDDL uses SLF4J logging. Configure log levels in `logback.xml`: |
| 301 | + |
| 302 | +```xml |
| 303 | +<logger name="com.ossuminc.riddl" level="DEBUG"/> |
| 304 | +``` |
| 305 | + |
| 306 | +## Resources |
| 307 | + |
| 308 | +- [GitHub Repository](https://github.com/ossuminc/riddl) |
| 309 | +- [EBNF Grammar](../../references/ebnf-grammar.md) |
| 310 | +- [Language Reference](../../references/language-reference.md) |
| 311 | +- [Scala 3 Documentation](https://docs.scala-lang.org/scala3/) |
| 312 | +- [fastparse Documentation](https://com-lihaoyi.github.io/fastparse/) |
0 commit comments