diff --git a/CLAUDE.md b/CLAUDE.md index 40a7e1bcc..5e50c256b 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -695,3 +695,12 @@ Then add to root aggregation: `.aggregate(..., mymodule, mymoduleJS, mymoduleNat 31. **Homebrew formula supports native + JVM fallback** - macOS ARM64 gets native binary (no JDK). Other platforms get JVM version with openjdk@21 dependency. Formula at `../homebrew-tap/Formula/riddlc.rb` 32. **RiddlLib shared trait** - `riddlLib/shared/.../RiddlLib.scala` provides cross-platform API (parseString, flattenAST, validateString, getOutline, getTree). `object RiddlLib extends RiddlLib` has default implementations. RiddlAPI.scala is now a thin JS facade delegating to RiddlLib 33. **parseString returns opaque Root in JS** - As of 1.5.0, `RiddlAPI.parseString()` returns the actual Scala `Root` object (opaque to JS). Use `getDomains(root)` or `inspectRoot(root)` to access data. TypeScript type is branded `RootAST` +34. **Schema is in NonDefinitionValues** - Schema extends Leaf (Definition) but is also in the `NonDefinitionValues` union type. Its match case in `ValidationPass.process()` must appear BEFORE `case _: NonDefinitionValues`. Similarly, Relationship extends Leaf and must be matched before `case _: Definition` +35. **CheckMessagesTest .check file format** - Lines starting with space are continuation lines appended to the previous entry (with `\n`). Non-space-starting lines begin new entries. Multi-line messages (like overloaded warnings) must not have new entries inserted mid-continuation +36. **Streamlet shape validation guards on nonEmpty** - Empty streamlets (`{ ??? }`) should not be checked for inlet/outlet counts since they're placeholders. Use `streamlet.nonEmpty` guard before shape checks +37. **Publishing workflow** - Tag with `git tag -a X.Y.Z -m "Release X.Y.Z"`, push tag, then `sbt clean test publish`. All modules publish to GitHub Packages across JVM, JS, and Native platforms. Version is derived from the git tag by sbt-dynver +38. **Analysis passes in passes/shared/.../analysis/** - MessageFlowPass, EntityLifecyclePass, DependencyAnalysisPass. Each extends `CollectingPass`, requires ResolutionPass. Run via `Pass.standardPasses :+ XxxPass.creator()` then `result.outputs.outputOf[XxxOutput](XxxPass.name)` +39. **RecognizedOptions registry** - `DefinitionValidation.RecognizedOptions.registry: Map[String, OptionSpec]` validates option names, argument counts, and parent definition types. Unrecognized options produce StyleWarning (not Error) to keep extensible +40. **RiddlLib analysis API methods** - `getHandlerCompleteness()`, `getMessageFlow()`, `getEntityLifecycles()` on shared RiddlLib trait + JS `RiddlAPI` facade. Each runs standard passes plus the relevant analysis pass +41. **HandlerCompleteness in ValidationOutput** - `ValidationOutput.handlerCompleteness: Seq[HandlerCompleteness]` populated in `ValidationPass.postProcess()`. Categories: `BehaviorCategory.Executable`, `PromptOnly`, `Empty` +42. **Downstream integration plans** - Each downstream project (riddlsim, riddl-gen, riddl-mcp-server, synapify) has a `RIDDL-INTEGRATION-PLAN.md` describing how to consume new library features. Designed for separate Claude instances working in those projects diff --git a/NOTEBOOK.md b/NOTEBOOK.md index 44e3fbaf7..4de3935a7 100644 --- a/NOTEBOOK.md +++ b/NOTEBOOK.md @@ -6,22 +6,42 @@ This is the central engineering notebook for the RIDDL project. It tracks curren ## Current Status -**Last Updated**: February 5, 2026 +**Last Updated**: February 8, 2026 **Scala Version**: 3.7.4 (overrides sbt-ossuminc's 3.3.7 LTS default due to compiler infinite loop bug with opaque types/intersection types in 3.3.x). All workflow paths updated to `scala-3.7.4`. +**1.6.0 Enhancements (uncommitted)**: Comprehensive library +enhancements for simulator and generator support. 5 phases +completed across 2 sessions: +- Phase 1: Handler completeness classification (A1), behavioral + statistics (A2) +- Phase 2: MessageFlowPass (B1), EntityLifecyclePass (B2) +- Phase 3: DiagramsPass extensions (A3), DependencyAnalysisPass (B3) +- Phase 4: Recognized options vocabulary and validation (C1-C5) +- Phase 5: RiddlLib/RiddlAPI extensions (D1) +Full clean build: 734 tests, 0 failures. + +**Release 1.6.0 Published**: ValidationPass bug fixes and new +validations. Fixed SagaStep undo check, SagaStep shape check, +duplicate checkMetadata. Added validation for Schema, Relationship, +Streamlet shape/handler, Adaptor/Repository handler requirements, +Projector repo ref, Epic/UseCase user ref, Function input/output +types. 1,526 tests pass (0 failures). Published to GitHub Packages. + +**Release 1.5.0 Published**: Extracted cross-platform `RiddlLib` +trait from JS-only `RiddlAPI`. `RiddlAPI` is now a thin JS facade +delegating to `RiddlLib`. `parseString` returns opaque Root handle; +use `getDomains()`/`inspectRoot()` accessors. Fixed `riddlLibJS/test` +ESModule crash. 1,527 tests pass (0 failures). Published to GitHub +Packages (Maven + npm). BREAKING: TS consumers must update +`parseString` usage. + **Release 1.4.0 Published**: `Container.flatten()` extension, rewritten `FlattenPass`, multi-platform release workflow. Native macOS ARM64 binary distributed via Homebrew. -**RiddlLib Shared Trait (pending 1.5.0 release)**: Extracted cross- -platform `RiddlLib` trait from JS-only `RiddlAPI`. `RiddlAPI` is now -a thin JS facade delegating to `RiddlLib`. `parseString` returns -opaque Root handle; use `getDomains()`/`inspectRoot()` accessors. -Fixed `riddlLibJS/test` ESModule crash. 1,527 tests pass (0 failures). - **npm Package Published**: `@ossuminc/riddl-lib` published to GitHub Packages npm registry via CI and locally. ESModule format with TypeScript declarations. CI workflow (`npm-publish.yml`) uses sbt-ossuminc 1.3.0 @@ -80,8 +100,15 @@ After all files pass, add riddl-models EBNF validation to CI (`.github/workflows/scala.yml`, mirroring existing riddl-examples pattern). -### 1. Merge development to main for 1.3.0 release -Create PR from development to main, merge after CI passes. +### 1. Update Consumers for 1.5.0 Breaking Change +**Status**: Pending + +`parseString` now returns opaque Root. Downstream projects that +access `result.value.domains` directly must switch to +`RiddlAPI.getDomains(result.value)` or +`RiddlAPI.inspectRoot(result.value).domains`. + +Projects to update: synapify, riddl-mcp-server, ossum.ai. ### 2. Comprehensive TypeScript Declarations for AST & Passes Expand `riddlLib/js/types/index.d.ts` to cover the full AST and Pass @@ -262,8 +289,162 @@ The `pseudoCodeBlock` parser now allows comments before and/or after `???`: --- +## Changes Since v1.6.0 + +- **Handler completeness (A1)**: `HandlerCompleteness` + `BehaviorCategory` + enum in `ValidationOutput`. Handlers classified as Executable, PromptOnly, + or Empty during `postProcess()`. +- **Behavioral statistics (A2)**: `numPromptStatements` and + `numExecutableStatements` added to `DefinitionStats` in StatsPass. +- **MessageFlowPass (B1)**: New analysis pass builds directed message flow + graph. Output: edges, producerIndex, consumerIndex, messageIndex. +- **EntityLifecyclePass (B2)**: New analysis pass extracts entity state + machines. Output: states, transitions, initial/terminal states. +- **DiagramsPass extensions (A3)**: Populated `DataFlowDiagramData` (R1) + with connections/connectors/streamlets. Added `DomainDiagramData` (R3) + with processors/subdomains/contexts/epics. +- **DependencyAnalysisPass (B3)**: New analysis pass builds cross-context + and cross-entity dependency graphs. Output: contextDeps, entityDeps, + typeDeps, adaptorBridges. +- **Recognized options (C1-C5)**: `RecognizedOptions` registry with ~25 + options. `validateRecognizedOption` validates name, arity, parent type. + Unrecognized options produce StyleWarning. +- **RiddlLib API (D1)**: `getHandlerCompleteness()`, `getMessageFlow()`, + `getEntityLifecycles()` on shared RiddlLib trait + JS facade. +- **Downstream integration plans (E1-E4)**: Delivered + `RIDDL-INTEGRATION-PLAN.md` to riddlsim, riddl-gen, riddl-mcp-server, + synapify. + ## Session Log +### February 8, 2026 (Library Enhancements for Simulator & Generator) + +**Focus**: Complete 5-phase implementation plan for RIDDL library +enhancements. Phases 1-3 completed in previous session (context +ran out). This session completed Phases 4-5 and deliverables. + +**Work Completed (this session)**: +1. **C1-C5: Options vocabulary and validation** — Implemented + `validateRecognizedOption` method in `DefinitionValidation.scala`. + Validates option name recognition, argument count (min/max arity), + and parent definition type compatibility. Updated `everything.check` + and `saga.check` with expected StyleWarning messages for + unrecognized `transient` and `parallel` options. +2. **D1: RiddlLib API extensions** — Added `getHandlerCompleteness`, + `getMessageFlow`, `getEntityLifecycles` methods to `RiddlLib` + trait (shared) and `RiddlAPI` (JS facade). Each parses source, + runs appropriate pass pipeline, converts to typed/JS results. +3. **E1-E4: Downstream integration plans** — Wrote + `RIDDL-INTEGRATION-PLAN.md` to all 4 downstream project repos. +4. **Full clean build verification** — `sbt clean cJVM` + `sbt tJVM`: + 734 tests, 0 failures across all modules. + +**Work Completed (previous session, before context ran out)**: +- A1: Handler completeness in ValidationPass +- A2: Behavioral statistics in StatsPass +- B1: MessageFlowPass (new file) +- B2: EntityLifecyclePass (new file) +- A3: DiagramsPass extensions (DataFlowDiagramData, DomainDiagramData) +- B3: DependencyAnalysisPass (new file) +- C1-C5 partial: OptionSpec/RecognizedOptions added, method stub placed + +**Files Created**: +- `passes/shared/.../analysis/MessageFlowPass.scala` +- `passes/shared/.../analysis/EntityLifecyclePass.scala` +- `passes/shared/.../analysis/DependencyAnalysisPass.scala` +- `../riddlsim/RIDDL-INTEGRATION-PLAN.md` +- `../riddl-gen/RIDDL-INTEGRATION-PLAN.md` +- `../riddl-mcp-server/RIDDL-INTEGRATION-PLAN.md` +- `../synapify/RIDDL-INTEGRATION-PLAN.md` + +**Files Modified**: +- `language/shared/.../parsing/StatementParser.scala` +- `passes/shared/.../validate/ValidationOutput.scala` +- `passes/shared/.../validate/ValidationPass.scala` +- `passes/shared/.../validate/StreamingValidation.scala` +- `passes/shared/.../validate/DefinitionValidation.scala` +- `passes/shared/.../stats/StatsPass.scala` +- `passes/shared/.../diagrams/DiagramsPass.scala` +- `passes/jvm-native/.../DiagramsPassTest.scala` +- `passes/input/check/everything/everything.check` +- `passes/input/check/saga/saga.check` +- `riddlLib/shared/.../RiddlLib.scala` +- `riddlLib/js/.../RiddlAPI.scala` + +**Test Results**: 734 tests, 0 failures (clean build) + +**Status**: All changes uncommitted on `development` branch. + +**Test verification**: `sbt clean test` (all platforms) exited 0. +`sbt test` (incremental, all platforms) also exited 0. Reported +1,526 tests passed, 0 failures. However, the 2-minute wall time +for an all-platform run including Native linking seems +suspiciously fast — verify independently before releasing. + +--- + +### February 7, 2026 (ValidationPass Gap Analysis & Enhancement) + +**Focus**: Comprehensive audit and enhancement of ValidationPass +to catch missing validations and fix bugs. + +**Work Completed**: +1. **Fixed 3 bugs**: + - SagaStep checked `doStatements` twice instead of + `undoStatements` for revert validation + - SagaStep shape check compared `getClass` of two + `Contents[Statements]` (always true) — replaced with + meaningful `nonEmpty` symmetry check + - `validateState` called `checkMetadata` twice — removed + duplicate +2. **Added validation for 2 previously unvalidated definitions**: + - `Schema` — kind vs structure compatibility (flat/document/ + columnar/vector shouldn't have links; graphical should; + vector expects single data node), data TypeRef resolution, + link FieldRef resolution, index FieldRef resolution + - `Relationship` — processor ref resolution, identifier + length, metadata checks +3. **Added 6 semantic validations**: + - Streamlet shape vs inlet/outlet count (guarded by + `nonEmpty` to skip placeholders) + - Streamlet handler requirement + - Adaptor handler requirement + empty handler warning + - Repository handler requirement + - Projector repository ref resolution + - Epic/UseCase user story user ref resolution +4. **Added Function input/output type validation** via + `checkTypeExpression` on `input`/`output` Aggregations +5. **Updated 3 `.check` test expectation files** to reflect + new validation messages (everything, saga, streaming) + +**Test Results**: `sbt clean test` — 1,526 tests, 0 failures +across all modules (JVM, JS, Native). + +**Commit**: `6d6185e5` — pushed to `origin/development` + +**Files Modified**: +- `passes/shared/.../validate/ValidationPass.scala` — all + validation enhancements +- `passes/input/check/everything/everything.check` — 12 new + expected messages +- `passes/input/check/saga/saga.check` — 5 new expected + messages +- `passes/input/check/streaming/streaming.check` — 3 new + expected messages + +**Remaining from gap analysis** (lower priority, not +implemented this session): +- Schema kind-specific deep checks (relational link type + compatibility, etc.) +- Adaptor message compatibility with referenced context +- Saga compensation symmetry hints +- Entity FSM morph/become statement presence check +- `checkStreamingUsage()` implementation (still a no-op) +- Handler message type vs container type appropriateness +- Context isolation warnings + +--- + ### February 5, 2026 (RiddlLib Shared Trait, JS Test Fix) **Focus**: Extract shared RiddlLib trait from JS-only RiddlAPI, @@ -295,11 +476,22 @@ fix riddlLibJS test runner crash. `riddlLibJS/test` now runs 8 tests successfully (was crashing). **Breaking Change**: `parseString` returns opaque Root. TypeScript -consumers must use `getDomains()`/`inspectRoot()`. Ships as 1.5.0. +consumers must use `getDomains()`/`inspectRoot()`. + +**Release 1.5.0** (February 6, 2026): +- Merged development → main, tagged 1.5.0 +- `sbt clean test` — 1,527 tests, 0 failures +- JS bundle built, ESMSafetyTest clean pass (5.3MB scanned) +- `sbt publish` — all modules to GitHub Packages (Maven) +- `gh release create 1.5.0` — triggers release.yml for native + builds and homebrew-tap update +- `@ossuminc/riddl-lib@1.5.0` published to npm (GitHub Packages) **Commits**: - `407aebdb` — Extract shared RiddlLib trait and fix riddlLibJS test crash +- `28c299b5` — Update CLAUDE.md and NOTEBOOK.md for RiddlLib + shared trait **Files Created**: - `riddlLib/shared/src/main/scala/.../RiddlLib.scala` @@ -312,6 +504,9 @@ consumers must use `getDomains()`/`inspectRoot()`. Ships as 1.5.0. - `riddlLib/jvm/src/test/scala/.../FlattenPassTest.scala` — un-pended +**Consumers to update** (parseString breaking change): +- synapify, riddl-mcp-server, ossum.ai + --- ### February 5, 2026 (FlattenPass, Release 1.4.0) @@ -1464,5 +1659,4 @@ Tool( ## Git Information **Branch**: `development` -**Latest release**: 1.4.0 (February 5, 2026) -**Next release**: 1.5.0 (breaking: opaque Root in parseString) +**Latest release**: 1.6.0 (February 7, 2026) diff --git a/language/shared/src/main/scala/com/ossuminc/riddl/language/parsing/StatementParser.scala b/language/shared/src/main/scala/com/ossuminc/riddl/language/parsing/StatementParser.scala index 009b73965..b70027116 100644 --- a/language/shared/src/main/scala/com/ossuminc/riddl/language/parsing/StatementParser.scala +++ b/language/shared/src/main/scala/com/ossuminc/riddl/language/parsing/StatementParser.scala @@ -6,7 +6,7 @@ package com.ossuminc.riddl.language.parsing -import com.ossuminc.riddl.language.AST.{*} +import com.ossuminc.riddl.language.AST.* import com.ossuminc.riddl.language.{Contents, *} import com.ossuminc.riddl.language.At import fastparse.* @@ -75,7 +75,7 @@ private[parsing] trait StatementParser { private def whenCondition[u: P]: P[(LiteralString | Identifier, Boolean)] = { P( literalString.map(ls => (ls, false)) | - (Punctuation.exclamation ~ identifier).map { case id => (id, true) } | + (Punctuation.exclamation ~ identifier).map(id => (id, true)) | identifier.map(id => (id, false)) ) } @@ -159,7 +159,7 @@ private[parsing] trait StatementParser { } } - def setOfStatements[u: P](set: StatementsSet): P[Seq[Statements]] = { + private def setOfStatements[u: P](set: StatementsSet): P[Seq[Statements]] = { P(statement(set).rep(0))./ } diff --git a/passes/input/check/everything/everything.check b/passes/input/check/everything/everything.check index 9624f5099..e39cf8bfe 100644 --- a/passes/input/check/everything/everything.check +++ b/passes/input/check/everything/everything.check @@ -112,6 +112,28 @@ passes/input/check/everything/everything.riddl(50:7->37): passes/input/check/everything/everything.riddl(50:7->37): Function 'whenUnderTheInfluence' is unused: function whenUnderTheInfluence is { +passes/input/check/everything/everything.riddl(51:20->21): + Field identifier 'n' is too short. The minimum length is 3: + requires { n: Nothing } +passes/input/check/everything/everything.riddl(51:20->21): + Metadata in Field 'n' should not be empty: + requires { n: Nothing } +passes/input/check/everything/everything.riddl(51:20->21): + Field 'n' should have a description: + requires { n: Nothing } +passes/input/check/everything/everything.riddl(52:19->20): + Field identifier 'b' is too short. The minimum length is 3: + returns { b: Boolean } +passes/input/check/everything/everything.riddl(52:19->20): + Metadata in Field 'b' should not be empty: + returns { b: Boolean } +passes/input/check/everything/everything.riddl(52:19->20): + Field 'b' should have a description: + returns { b: Boolean } +passes/input/check/everything/everything.riddl(63:24->64:5): + Option 'transient' in Entity 'Something' is not a recognized RIDDL option: + option aggregate option transient + } passes/input/check/everything/everything.riddl(66:5->26): Metadata in Entity 'SomeOtherThing' should not be empty: entity SomeOtherThing is { @@ -127,6 +149,12 @@ passes/input/check/everything/everything.riddl(67:7->20): passes/input/check/everything/everything.riddl(67:7->20): Sink 'trashBin' should have a description: sink trashBin is { inlet SOT_In is SomeOtherThing.ItHappened } +passes/input/check/everything/everything.riddl(67:7->20): + Sink 'trashBin' should have a handler: + sink trashBin is { inlet SOT_In is SomeOtherThing.ItHappened } +passes/input/check/everything/everything.riddl(67:7->20): + Sink 'trashBin' has no connections to any connector: + sink trashBin is { inlet SOT_In is SomeOtherThing.ItHappened } passes/input/check/everything/everything.riddl(67:26->38): Inlet 'SOT_In' is not connected: sink trashBin is { inlet SOT_In is SomeOtherThing.ItHappened } @@ -136,8 +164,17 @@ passes/input/check/everything/everything.riddl(69:7->28): passes/input/check/everything/everything.riddl(69:7->28): State 'otherThingState' should have a description: state otherThingState of type Everything.StateType +passes/input/check/everything/everything.riddl(70:7->18): + Handler 'fee' in Entity 'SomeOtherThing' handles no commands or queries; entity handlers typically handle commands and queries: + handler fee is { +passes/input/check/everything/everything.riddl(7:5->18): + Source 'Source' should have a handler: + source Source is { outlet Commands is DoAThing } with { described by "Data Source" } passes/input/check/everything/everything.riddl(7:24->39): Outlet 'Commands' is overloaded with 2 kinds: Outlet 'Commands' at passes/input/check/everything/everything.riddl(7:24->52), Inlet 'Commands' at passes/input/check/everything/everything.riddl(8:20->47): source Source is { outlet Commands is DoAThing } with { described by "Data Source" } +passes/input/check/everything/everything.riddl(8:5->14): + Sink 'Sink' should have a handler: + sink Sink is { inlet Commands is DoAThing } with { explained as "Data Sink" } diff --git a/passes/input/check/saga/saga.check b/passes/input/check/saga/saga.check index 720afeaf9..2a4ab86fe 100644 --- a/passes/input/check/saga/saga.check +++ b/passes/input/check/saga/saga.check @@ -19,9 +19,15 @@ passes/input/check/saga/saga.riddl(13:32->34): passes/input/check/saga/saga.riddl(13:7->20): Metadata in Sink 'trashCan' should not be empty: sink trashCan is { inlet in is command Something } +passes/input/check/saga/saga.riddl(13:7->20): + Sink 'trashCan' has no connections to any connector: + sink trashCan is { inlet in is command Something } passes/input/check/saga/saga.riddl(13:7->20): Sink 'trashCan' should have a description: sink trashCan is { inlet in is command Something } +passes/input/check/saga/saga.riddl(13:7->20): + Sink 'trashCan' should have a handler: + sink trashCan is { inlet in is command Something } passes/input/check/saga/saga.riddl(4:5->23): Command 'UndoSomething' is unused: type UndoSomething = command { ??? } @@ -34,6 +40,22 @@ passes/input/check/saga/saga.riddl(8:5->30): passes/input/check/saga/saga.riddl(5:5->26): Function 'AnotherThing' in Context 'ignore2' should have statements: function AnotherThing { +passes/input/check/saga/saga.riddl(6:18->19): + Field identifier 'a' is too short. The minimum length is 3: + requires { a: Integer with { described by "a" } } returns { b: Integer with { described by "b"} } +passes/input/check/saga/saga.riddl(6:67->68): + Field identifier 'b' is too short. The minimum length is 3: + requires { a: Integer with { described by "a" } } returns { b: Integer with { described by "b"} } passes/input/check/saga/saga.riddl(8:5->30): Function 'UndoAnotherThing' in Context 'ignore2' should have statements: function UndoAnotherThing { +passes/input/check/saga/saga.riddl(9:18->19): + Field identifier 'c' is too short. The minimum length is 3: + requires { c: Integer with { described by "c" } } returns { d: Integer with { described by "d"} } +passes/input/check/saga/saga.riddl(9:67->68): + Field identifier 'd' is too short. The minimum length is 3: + requires { c: Integer with { described by "c" } } returns { d: Integer with { described by "d"} } +passes/input/check/saga/saga.riddl(35:7->36:7): + Option 'parallel' in Saga 'name' is not a recognized RIDDL option: + option parallel + described as "ignore" diff --git a/passes/input/check/streaming/streaming.check b/passes/input/check/streaming/streaming.check index e69de29bb..1f6a383ea 100644 --- a/passes/input/check/streaming/streaming.check +++ b/passes/input/check/streaming/streaming.check @@ -0,0 +1,9 @@ +passes/input/check/streaming/streaming.riddl(4:3->28): + Source 'GetWeatherForecast' should have a handler: + source GetWeatherForecast is { +passes/input/check/streaming/streaming.riddl(10:3->29): + Flow 'GetCurrentTemperature' should have a handler: + flow GetCurrentTemperature is { +passes/input/check/streaming/streaming.riddl(17:3->23): + Sink 'AttenuateSensor' should have a handler: + sink AttenuateSensor is { diff --git a/passes/jvm-native/src/test/scala/com/ossuminc/riddl/passes/diagrams/DiagramsPassTest.scala b/passes/jvm-native/src/test/scala/com/ossuminc/riddl/passes/diagrams/DiagramsPassTest.scala index 897bf94bb..4b62adacb 100644 --- a/passes/jvm-native/src/test/scala/com/ossuminc/riddl/passes/diagrams/DiagramsPassTest.scala +++ b/passes/jvm-native/src/test/scala/com/ossuminc/riddl/passes/diagrams/DiagramsPassTest.scala @@ -27,7 +27,7 @@ class DiagramsPassTest extends SharedDiagramsPassTest { output.messages.justErrors must be(empty) output.contextDiagrams must not be (empty) output.useCaseDiagrams must not be (empty) - output.dataFlowDiagrams must be(empty) // TODO: change to 'not be(empty)' when implemented + output.dataFlowDiagrams must not be(empty) } } Await.result(future, 10.seconds) diff --git a/passes/shared/src/main/scala/com/ossuminc/riddl/passes/analysis/DependencyAnalysisPass.scala b/passes/shared/src/main/scala/com/ossuminc/riddl/passes/analysis/DependencyAnalysisPass.scala new file mode 100644 index 000000000..310afb455 --- /dev/null +++ b/passes/shared/src/main/scala/com/ossuminc/riddl/passes/analysis/DependencyAnalysisPass.scala @@ -0,0 +1,268 @@ +/* + * Copyright 2019-2026 Ossum, Inc. + * + * SPDX-License-Identifier: Apache-2.0 + */ + +package com.ossuminc.riddl.passes.analysis + +import com.ossuminc.riddl.language.AST.* +import com.ossuminc.riddl.language.Messages +import com.ossuminc.riddl.passes.* +import com.ossuminc.riddl.passes.resolve.{ResolutionOutput, ResolutionPass} +import com.ossuminc.riddl.passes.symbols.{SymbolsOutput, SymbolsPass} +import com.ossuminc.riddl.passes.validate.ValidationPass +import com.ossuminc.riddl.utils.PlatformContext + +import scala.collection.mutable +import scala.scalajs.js.annotation.* + +/** Describes a bridge between two contexts via an adaptor + * + * @param adaptor + * The adaptor creating the bridge + * @param sourceContext + * The context containing the adaptor + * @param targetContext + * The context being adapted to/from + * @param direction + * Inbound or outbound + * @param bridgedTypes + * Message types that cross the boundary + */ +@JSExportTopLevel("AdaptorBridge") +case class AdaptorBridge( + adaptor: Adaptor, + sourceContext: Context, + targetContext: Context, + direction: AdaptorDirection, + bridgedTypes: Seq[Type] +) + +/** Output of the DependencyAnalysisPass + * + * @param root + * The root of the model + * @param messages + * Any messages generated during analysis + * @param contextDeps + * Map from each context to the set of contexts it depends on + * @param entityDeps + * Map from each entity to definitions it references + * @param typeDeps + * Map from each type to types it references + * @param adaptorBridges + * All adaptor bridges discovered + */ +@JSExportTopLevel("DependencyOutput") +case class DependencyOutput( + root: PassRoot = Root.empty, + messages: Messages.Messages = Messages.empty, + contextDeps: Map[Context, scala.collection.immutable.Set[Context]] = + Map.empty, + entityDeps: Map[Entity, scala.collection.immutable.Set[Definition]] = + Map.empty, + typeDeps: Map[Type, scala.collection.immutable.Set[Type]] = Map.empty, + adaptorBridges: Seq[AdaptorBridge] = Seq.empty +) extends PassOutput + +@JSExportTopLevel("DependencyAnalysisPass$") +object DependencyAnalysisPass extends PassInfo[PassOptions] { + val name: String = "DependencyAnalysis" + def creator( + options: PassOptions = PassOptions.empty + )(using PlatformContext): PassCreator = { + (in: PassInput, out: PassesOutput) => + DependencyAnalysisPass(in, out) + } +} + +/** A pass that builds cross-context and cross-entity dependency + * graphs showing which definitions reference which others. It + * analyzes all resolved references to determine source/target + * contexts and builds adjacency sets. + */ +@JSExportTopLevel("DependencyAnalysisPass") +case class DependencyAnalysisPass( + input: PassInput, + outputs: PassesOutput +)(using PlatformContext) + extends Pass(input, outputs) { + + requires(SymbolsPass) + requires(ResolutionPass) + requires(ValidationPass) + + override def name: String = DependencyAnalysisPass.name + + private lazy val refMap = outputs.refMap + private lazy val symTab = outputs.symbols + + private val contextDeps: mutable.HashMap[Context, mutable.Set[Context]] = + mutable.HashMap.empty + private val entityDeps: mutable.HashMap[Entity, mutable.Set[Definition]] = + mutable.HashMap.empty + private val typeDepsMap: mutable.HashMap[Type, mutable.Set[Type]] = + mutable.HashMap.empty + private val bridges: mutable.ListBuffer[AdaptorBridge] = + mutable.ListBuffer.empty + + protected def process( + definition: RiddlValue, + parents: ParentStack + ): Unit = { + definition match + case adaptor: Adaptor => + processAdaptor(adaptor, parents.toParents) + case tell: TellStatement => + processTellStatement(tell, parents.toParents) + case send: SendStatement => + processSendStatement(send, parents.toParents) + case _ => () + } + + private def processAdaptor( + adaptor: Adaptor, + parents: Parents + ): Unit = { + val maybeSourceContext = parents.collectFirst { + case c: Context => c + } + val maybeTargetContext = + refMap.definitionOf[Context](adaptor.referent, adaptor) + + (maybeSourceContext, maybeTargetContext) match + case (Some(source), Some(target)) => + // Add context dependency + contextDeps.getOrElseUpdate( + source, mutable.Set.empty + ) += target + + // Collect bridged types + val bridgedTypes = adaptor.handlers.flatMap { handler => + handler.clauses.flatMap { + case omc: OnMessageClause => + refMap.definitionOf[Type](omc.msg, omc).toSeq + case _ => Seq.empty + } + } + + bridges.addOne( + AdaptorBridge( + adaptor = adaptor, + sourceContext = source, + targetContext = target, + direction = adaptor.direction, + bridgedTypes = bridgedTypes + ) + ) + case _ => () + } + + private def processTellStatement( + tell: TellStatement, + parents: Parents + ): Unit = { + // Find the context containing this tell statement + val sourceContext = parents.collectFirst { + case c: Context => c + } + val sourceEntity = parents.collectFirst { + case e: Entity => e + } + + // Find the OnMessageClause or handler containing this + val parentClause = parents.collectFirst { + case omc: OnMessageClause => omc + } + + parentClause.foreach { omc => + val maybeTarget = + refMap.definitionOf[Processor[?]](tell.processorRef, omc) + maybeTarget.foreach { target => + val targetContext = symTab.contextOf(target) + + // Record context dependency if cross-context + (sourceContext, targetContext) match + case (Some(src), Some(tgt)) if src != tgt => + contextDeps.getOrElseUpdate( + src, mutable.Set.empty + ) += tgt + case _ => () + + // Record entity dependency + sourceEntity.foreach { entity => + entityDeps.getOrElseUpdate( + entity, mutable.Set.empty + ) += target + } + + // Record type dependency for the message + val maybeType = refMap.definitionOf[Type](tell.msg, omc) + maybeType.foreach { msgType => + val sourceType = parents.collectFirst { + case t: Type => t + } + sourceType.foreach { src => + typeDepsMap.getOrElseUpdate( + src, mutable.Set.empty + ) += msgType + } + } + } + } + } + + private def processSendStatement( + send: SendStatement, + parents: Parents + ): Unit = { + val sourceContext = parents.collectFirst { + case c: Context => c + } + val sourceEntity = parents.collectFirst { + case e: Entity => e + } + val parentClause = parents.collectFirst { + case omc: OnMessageClause => omc + } + + parentClause.foreach { omc => + val maybePortlet = + refMap.definitionOf[Portlet](send.portlet, omc) + maybePortlet.foreach { portlet => + val targetContext = symTab.contextOf(portlet) + + (sourceContext, targetContext) match + case (Some(src), Some(tgt)) if src != tgt => + contextDeps.getOrElseUpdate( + src, mutable.Set.empty + ) += tgt + case _ => () + + sourceEntity.foreach { entity => + entityDeps.getOrElseUpdate( + entity, mutable.Set.empty + ) += portlet + } + } + } + } + + override def result(root: PassRoot): DependencyOutput = { + DependencyOutput( + root = root, + messages = messages.toMessages, + contextDeps = contextDeps.map { case (k, v) => + k -> v.toSet + }.toMap, + entityDeps = entityDeps.map { case (k, v) => + k -> v.toSet + }.toMap, + typeDeps = typeDepsMap.map { case (k, v) => + k -> v.toSet + }.toMap, + adaptorBridges = bridges.toSeq + ) + } +} diff --git a/passes/shared/src/main/scala/com/ossuminc/riddl/passes/analysis/EntityLifecyclePass.scala b/passes/shared/src/main/scala/com/ossuminc/riddl/passes/analysis/EntityLifecyclePass.scala new file mode 100644 index 000000000..b4228353d --- /dev/null +++ b/passes/shared/src/main/scala/com/ossuminc/riddl/passes/analysis/EntityLifecyclePass.scala @@ -0,0 +1,220 @@ +/* + * Copyright 2019-2026 Ossum, Inc. + * + * SPDX-License-Identifier: Apache-2.0 + */ + +package com.ossuminc.riddl.passes.analysis + +import com.ossuminc.riddl.language.AST.* +import com.ossuminc.riddl.language.{Finder, Messages} +import com.ossuminc.riddl.passes.* +import com.ossuminc.riddl.passes.resolve.{ResolutionOutput, ResolutionPass} +import com.ossuminc.riddl.passes.symbols.{SymbolsOutput, SymbolsPass} +import com.ossuminc.riddl.passes.validate.ValidationPass +import com.ossuminc.riddl.utils.PlatformContext + +import scala.collection.mutable +import scala.scalajs.js.annotation.* + +/** A transition between entity states + * + * @param fromState + * The source state (None means any state / entity-level handler) + * @param toState + * The target state + * @param trigger + * The on-clause that triggers the transition + * @param mechanism + * The morph or become statement causing the transition + */ +@JSExportTopLevel("StateTransition") +case class StateTransition( + fromState: Option[State], + toState: State, + trigger: OnMessageClause, + mechanism: Statement +) + +/** The lifecycle (state machine) extracted from an entity + * + * @param entity + * The entity this lifecycle describes + * @param states + * All states defined on the entity + * @param transitions + * All state transitions discovered in handlers + * @param initialState + * The first declared state (conventional initial state) + * @param terminalStates + * States with no outgoing transitions + */ +@JSExportTopLevel("EntityLifecycle") +case class EntityLifecycle( + entity: Entity, + states: Seq[State], + transitions: Seq[StateTransition], + initialState: Option[State], + terminalStates: Seq[State] +) + +/** Output of the EntityLifecyclePass + * + * @param root + * The root of the model + * @param messages + * Any messages generated during analysis + * @param lifecycles + * Map from entity to its extracted lifecycle + */ +@JSExportTopLevel("EntityLifecycleOutput") +case class EntityLifecycleOutput( + root: PassRoot = Root.empty, + messages: Messages.Messages = Messages.empty, + lifecycles: Map[Entity, EntityLifecycle] = Map.empty +) extends PassOutput + +@JSExportTopLevel("EntityLifecyclePass$") +object EntityLifecyclePass extends PassInfo[PassOptions] { + val name: String = "EntityLifecycle" + def creator( + options: PassOptions = PassOptions.empty + )(using PlatformContext): PassCreator = { + (in: PassInput, out: PassesOutput) => + EntityLifecyclePass(in, out) + } +} + +/** A pass that extracts explicit state machines from entities that + * have multiple states and morph/become statements. This provides + * pre-computed lifecycle data for simulation and code generation. + */ +@JSExportTopLevel("EntityLifecyclePass") +case class EntityLifecyclePass( + input: PassInput, + outputs: PassesOutput +)(using PlatformContext) + extends Pass(input, outputs) { + + requires(SymbolsPass) + requires(ResolutionPass) + requires(ValidationPass) + + override def name: String = EntityLifecyclePass.name + + private lazy val refMap = outputs.refMap + private lazy val symTab = outputs.symbols + + private val collectedLifecycles: mutable.HashMap[Entity, EntityLifecycle] = + mutable.HashMap.empty + + protected def process( + definition: RiddlValue, + parents: ParentStack + ): Unit = { + definition match + case entity: Entity if entity.states.sizeIs >= 2 => + val lifecycle = extractLifecycle(entity) + collectedLifecycles.put(entity, lifecycle) + case _ => () + } + + private def extractLifecycle(entity: Entity): EntityLifecycle = { + val states = entity.states.toSeq + val transitions = mutable.ListBuffer.empty[StateTransition] + + // Walk entity-level handlers (fromState = None means any state) + entity.handlers.foreach { handler => + extractTransitions(handler, None, entity, transitions) + } + + // Walk state-specific handlers (if states have handlers) + // In RIDDL, handlers are at entity level, but we check if + // handler names match state names as a convention + // Also look for handlers defined within states if that + // pattern exists in the model + + val initialState = states.headOption + val statesWithOutgoing = transitions.map(_.fromState).collect { + case Some(s) => s + }.toSet ++ ( + // Entity-level transitions (fromState=None) mean any state + // can transition, so all states have potential outgoing + if transitions.exists(_.fromState.isEmpty) then states.toSet + else scala.collection.immutable.Set.empty[State] + ) + val terminalStates = states.filterNot(statesWithOutgoing.contains) + + EntityLifecycle( + entity = entity, + states = states, + transitions = transitions.toSeq, + initialState = initialState, + terminalStates = terminalStates + ) + } + + private def extractTransitions( + handler: Handler, + fromState: Option[State], + entity: Entity, + transitions: mutable.ListBuffer[StateTransition] + ): Unit = { + handler.clauses.foreach { + case omc: OnMessageClause => + val finder = Finder(omc.contents) + val morphs = finder.recursiveFindByType[MorphStatement] + val becomes = finder.recursiveFindByType[BecomeStatement] + + morphs.foreach { morph => + val maybeState = + refMap.definitionOf[State](morph.state, entity) + maybeState.foreach { targetState => + transitions.addOne( + StateTransition( + fromState = fromState, + toState = targetState, + trigger = omc, + mechanism = morph + ) + ) + } + } + + becomes.foreach { become => + // become changes the handler, which implies a state + // transition in FSM semantics. Try to find the state + // associated with the target handler. + val maybeHandler = + refMap.definitionOf[Handler](become.handler, entity) + maybeHandler.foreach { targetHandler => + // See if we can associate this handler with a state + // by name convention or containment + entity.states.find { state => + state.id.value == targetHandler.id.value || + state.id.value + "Handler" == + targetHandler.id.value + }.foreach { targetState => + transitions.addOne( + StateTransition( + fromState = fromState, + toState = targetState, + trigger = omc, + mechanism = become + ) + ) + } + } + } + case _ => () + } + } + + override def result(root: PassRoot): EntityLifecycleOutput = { + EntityLifecycleOutput( + root = root, + messages = messages.toMessages, + lifecycles = collectedLifecycles.toMap + ) + } +} diff --git a/passes/shared/src/main/scala/com/ossuminc/riddl/passes/analysis/MessageFlowPass.scala b/passes/shared/src/main/scala/com/ossuminc/riddl/passes/analysis/MessageFlowPass.scala new file mode 100644 index 000000000..5b4282492 --- /dev/null +++ b/passes/shared/src/main/scala/com/ossuminc/riddl/passes/analysis/MessageFlowPass.scala @@ -0,0 +1,279 @@ +/* + * Copyright 2019-2026 Ossum, Inc. + * + * SPDX-License-Identifier: Apache-2.0 + */ + +package com.ossuminc.riddl.passes.analysis + +import com.ossuminc.riddl.language.AST.* +import com.ossuminc.riddl.language.{Finder, Messages} +import com.ossuminc.riddl.passes.* +import com.ossuminc.riddl.passes.resolve.{ResolutionOutput, ResolutionPass} +import com.ossuminc.riddl.passes.symbols.{SymbolsOutput, SymbolsPass} +import com.ossuminc.riddl.passes.validate.ValidationPass +import com.ossuminc.riddl.utils.PlatformContext + +import scala.collection.mutable +import scala.scalajs.js.annotation.* + +/** The mechanism by which a message flows between producer and consumer */ +enum FlowMechanism: + case Tell, Send, AdaptorBridge, ConnectorPipe + +/** An edge in the message flow graph representing a message sent from a + * producer to a consumer + * + * @param producer + * The definition that produces/sends the message + * @param consumer + * The definition that consumes/receives the message + * @param messageType + * The type of message being transmitted + * @param mechanism + * How the message is delivered (tell, send, adaptor, connector) + */ +@JSExportTopLevel("MessageFlowEdge") +case class MessageFlowEdge( + producer: Definition, + consumer: Definition, + messageType: Type, + mechanism: FlowMechanism +) + +/** Output of the MessageFlowPass containing the directed graph of message + * flows across the entire model + * + * @param root + * The root of the model + * @param messages + * Any messages generated during analysis + * @param edges + * All message flow edges discovered + * @param producerIndex + * Edges indexed by producer definition + * @param consumerIndex + * Edges indexed by consumer definition + * @param messageIndex + * Edges indexed by message type + */ +@JSExportTopLevel("MessageFlowOutput") +case class MessageFlowOutput( + root: PassRoot = Root.empty, + messages: Messages.Messages = Messages.empty, + edges: Seq[MessageFlowEdge] = Seq.empty, + producerIndex: Map[Definition, Seq[MessageFlowEdge]] = Map.empty, + consumerIndex: Map[Definition, Seq[MessageFlowEdge]] = Map.empty, + messageIndex: Map[Type, Seq[MessageFlowEdge]] = Map.empty +) extends PassOutput + +@JSExportTopLevel("MessageFlowPass$") +object MessageFlowPass extends PassInfo[PassOptions] { + val name: String = "MessageFlow" + def creator( + options: PassOptions = PassOptions.empty + )(using PlatformContext): PassCreator = { + (in: PassInput, out: PassesOutput) => MessageFlowPass(in, out) + } +} + +/** A pass that builds a directed graph of message producers and consumers + * across the entire model. This is the core data structure both the + * simulator and generator need for understanding message routing. + */ +@JSExportTopLevel("MessageFlowPass") +case class MessageFlowPass( + input: PassInput, + outputs: PassesOutput +)(using PlatformContext) + extends Pass(input, outputs) { + + requires(SymbolsPass) + requires(ResolutionPass) + requires(ValidationPass) + + override def name: String = MessageFlowPass.name + + private lazy val refMap = outputs.refMap + private lazy val symTab = outputs.symbols + + private val collectedEdges: mutable.ListBuffer[MessageFlowEdge] = + mutable.ListBuffer.empty + + protected def process( + definition: RiddlValue, + parents: ParentStack + ): Unit = { + val parentsSeq = parents.toParents + definition match + case handler: Handler => + val processorParent = parentsSeq.headOption.collect { + case p: Processor[?] => p + } + processorParent.foreach { processor => + handler.clauses.foreach { + case omc: OnMessageClause => + processOnMessageClause(omc, processor, parentsSeq) + case _ => () + } + } + case adaptor: Adaptor => + processAdaptor(adaptor, parentsSeq) + case connector: Connector => + processConnector(connector, parentsSeq) + case _ => () + } + + private def processOnMessageClause( + omc: OnMessageClause, + processor: Processor[?], + parents: Parents + ): Unit = { + val finder = Finder(omc.contents) + val tells = finder.recursiveFindByType[TellStatement] + val sends = finder.recursiveFindByType[SendStatement] + + tells.foreach { tell => + val maybeTarget = + refMap.definitionOf[Processor[?]](tell.processorRef, omc) + val maybeType = refMap.definitionOf[Type](tell.msg, omc) + (maybeTarget, maybeType) match + case (Some(target), Some(msgType)) => + collectedEdges.addOne( + MessageFlowEdge( + producer = processor, + consumer = target, + messageType = msgType, + mechanism = FlowMechanism.Tell + ) + ) + case _ => () + } + + sends.foreach { send => + val maybePortlet = + refMap.definitionOf[Portlet](send.portlet, omc) + val maybeType = refMap.definitionOf[Type](send.msg, omc) + (maybePortlet, maybeType) match + case (Some(portlet), Some(msgType)) => + val portletParent = symTab.parentOf(portlet).collect { + case p: Processor[?] => p + } + portletParent.foreach { target => + collectedEdges.addOne( + MessageFlowEdge( + producer = processor, + consumer = target, + messageType = msgType, + mechanism = FlowMechanism.Send + ) + ) + } + case _ => () + } + } + + private def processAdaptor( + adaptor: Adaptor, + parents: Parents + ): Unit = { + val maybeTargetContext = + refMap.definitionOf[Context](adaptor.referent, adaptor) + val sourceContext = parents.headOption.collect { + case c: Context => c + } + (sourceContext, maybeTargetContext) match + case (Some(source), Some(target)) => + adaptor.handlers.foreach { handler => + handler.clauses.foreach { + case omc: OnMessageClause => + val maybeType = + refMap.definitionOf[Type](omc.msg, omc) + maybeType.foreach { msgType => + collectedEdges.addOne( + MessageFlowEdge( + producer = target, + consumer = source, + messageType = msgType, + mechanism = FlowMechanism.AdaptorBridge + ) + ) + } + case _ => () + } + } + case _ => () + } + + private def processConnector( + connector: Connector, + parents: Parents + ): Unit = { + if connector.nonEmpty then + // Find the parent context/processor that contains the connector + val parentContainer = parents.headOption.collect { + case b: Branch[?] => b + }.getOrElse(Root.empty) + + val connParents = symTab.parentsOf(connector) + val maybeOutlet = + refMap.definitionOf[Outlet](connector.from, parentContainer) + val maybeInlet = + refMap.definitionOf[Inlet](connector.to, parentContainer) + + (maybeOutlet, maybeInlet) match + case (Some(outlet), Some(inlet)) => + // Walk up from outlet/inlet to find their parent processor + val fromProcessor = symTab.parentOf(outlet).collect { + case s: Streamlet => s: Processor[?] + }.orElse( + symTab.parentOf(outlet).flatMap { p => + symTab.parentOf(p).collect { + case proc: Processor[?] => proc + } + } + ) + val toProcessor = symTab.parentOf(inlet).collect { + case s: Streamlet => s: Processor[?] + }.orElse( + symTab.parentOf(inlet).flatMap { p => + symTab.parentOf(p).collect { + case proc: Processor[?] => proc + } + } + ) + + // Resolve the outlet's type reference using + // its parent as context + val outletParent = symTab.parentOf(outlet).collect { + case b: Branch[?] => b + }.getOrElse(parentContainer) + val outletType = + refMap.definitionOf[Type](outlet.type_, outletParent) + + (fromProcessor, toProcessor, outletType) match + case (Some(from), Some(to), Some(msgType)) => + collectedEdges.addOne( + MessageFlowEdge( + producer = from, + consumer = to, + messageType = msgType, + mechanism = FlowMechanism.ConnectorPipe + ) + ) + case _ => () + case _ => () + } + + override def result(root: PassRoot): MessageFlowOutput = { + val edges = collectedEdges.toSeq + MessageFlowOutput( + root = root, + messages = messages.toMessages, + edges = edges, + producerIndex = edges.groupBy(_.producer), + consumerIndex = edges.groupBy(_.consumer), + messageIndex = edges.groupBy(_.messageType) + ) + } +} diff --git a/passes/shared/src/main/scala/com/ossuminc/riddl/passes/diagrams/DiagramsPass.scala b/passes/shared/src/main/scala/com/ossuminc/riddl/passes/diagrams/DiagramsPass.scala index 4e194e64c..066c056f8 100644 --- a/passes/shared/src/main/scala/com/ossuminc/riddl/passes/diagrams/DiagramsPass.scala +++ b/passes/shared/src/main/scala/com/ossuminc/riddl/passes/diagrams/DiagramsPass.scala @@ -17,11 +17,45 @@ import com.ossuminc.riddl.utils.PlatformContext import scala.collection.{immutable, mutable} import scala.scalajs.js.annotation.* +/** A connection in a data flow diagram representing data flowing through a + * connector from one streamlet/outlet to another streamlet/inlet + * + * @param from + * The source definition (outlet's parent streamlet or processor) + * @param to + * The target definition (inlet's parent streamlet or processor) + * @param connector + * The connector element linking them + * @param messageType + * The type name being transmitted + */ +@JSExportTopLevel("DataFlowConnection") +case class DataFlowConnection( + from: Definition, + to: Definition, + connector: Connector, + messageType: String +) + /** The information needed to generate a Data Flow Diagram. DFDs are generated for each - * [[com.ossuminc.riddl.language.AST.Context]] and consist of the streaming components that that are connected. + * [[com.ossuminc.riddl.language.AST.Context]] and consist of the streaming components that are connected. + * + * @param context + * The context this diagram represents + * @param connectors + * All connectors within the context + * @param streamlets + * All streamlets within the context + * @param connections + * Resolved connection data with source, target, and message type */ @JSExportTopLevel("DataFlowDiagramData") -case class DataFlowDiagramData() +case class DataFlowDiagramData( + context: Context, + connectors: Seq[Connector] = Seq.empty, + streamlets: Seq[Streamlet] = Seq.empty, + connections: Seq[DataFlowConnection] = Seq.empty +) /** The information needed to generate a Use Case Diagram. The diagram for a use case is very similar to a Sequence * Diagram showing the interactions between involved components of the model. @@ -51,10 +85,28 @@ case class ContextDiagramData( relationships: Seq[ContextRelationship] = Seq.empty ) -/** The information needed to generate a Context Diagram at the Domain level to show the relationships between its - * constituent bounded contexts +/** The information needed to generate a Domain-level diagram showing the + * domain's structure: processors, subdomains, contexts, and epics + * + * @param domain + * The domain this diagram represents + * @param processors + * All processors contained in the domain + * @param subdomains + * Nested subdomains + * @param contexts + * Bounded contexts within the domain + * @param epics + * Epics defined in the domain */ -type DomainDiagramData = Seq[(Context, ContextDiagramData)] +@JSExportTopLevel("DomainDiagramData") +case class DomainDiagramData( + domain: Domain, + processors: Seq[Processor[?]] = Seq.empty, + subdomains: Seq[Domain] = Seq.empty, + contexts: Seq[Context] = Seq.empty, + epics: Seq[Epic] = Seq.empty +) /** The output of the DiagramsPass encompassing all the generated data for the various diagrams * @@ -73,7 +125,8 @@ case class DiagramsPassOutput( messages: Messages.Messages = Messages.empty, dataFlowDiagrams: Map[Context, DataFlowDiagramData] = Map.empty, useCaseDiagrams: Map[UseCase, UseCaseDiagramData] = Map.empty, - contextDiagrams: Map[Context, ContextDiagramData] = Map.empty + contextDiagrams: Map[Context, ContextDiagramData] = Map.empty, + domainDiagrams: Map[Domain, DomainDiagramData] = Map.empty ) extends PassOutput @JSExportTopLevel("DiagramsPass") @@ -91,6 +144,7 @@ class DiagramsPass(input: PassInput, outputs: PassesOutput)(using PlatformContex private val dataFlowDiagrams: mutable.HashMap[Context, DataFlowDiagramData] = mutable.HashMap.empty private val useCaseDiagrams: mutable.HashMap[UseCase, UseCaseDiagramData] = mutable.HashMap.empty private val contextDiagrams: mutable.HashMap[Context, ContextDiagramData] = mutable.HashMap.empty + private val domainDiagramsMap: mutable.HashMap[Domain, DomainDiagramData] = mutable.HashMap.empty protected def process(definition: RiddlValue, parents: ParentStack): Unit = { definition match @@ -100,6 +154,9 @@ class DiagramsPass(input: PassInput, outputs: PassesOutput)(using PlatformContex val root = parents.find(c => c.isRootContainer && c.isInstanceOf[Root]).get.asInstanceOf[Root] val relationships = makeRelationships(c, root) contextDiagrams.put(c, ContextDiagramData(domain, aggregates.toSeq, relationships)) + captureDataFlow(c) + case d: Domain => + captureDomain(d) case epic: Epic => epic.cases.foreach { uc => val data = captureUseCase(uc) @@ -334,13 +391,56 @@ class DiagramsPass(input: PassInput, outputs: PassesOutput)(using PlatformContex UseCaseDiagramData(title, actors, uc.contents.filter[Interaction]) } + private def captureDataFlow(context: Context): Unit = { + val ctxConnectors = context.connectors.toSeq + val ctxStreamlets = context.streamlets.toSeq + val connections = ctxConnectors.flatMap { connector => + if connector.nonEmpty then + val maybeOutlet = refMap.definitionOf[Outlet](connector.from, context) + val maybeInlet = refMap.definitionOf[Inlet](connector.to, context) + (maybeOutlet, maybeInlet) match + case (Some(outlet), Some(inlet)) => + val fromDef: Definition = symTab.parentOf(outlet).collect { + case s: Streamlet => s: Definition + }.getOrElse(outlet) + val toDef: Definition = symTab.parentOf(inlet).collect { + case s: Streamlet => s: Definition + }.getOrElse(inlet) + val outletParent = symTab.parentOf(outlet).collect { + case b: Branch[?] => b + }.getOrElse(context) + val msgTypeName = refMap.definitionOf[Type](outlet.type_, outletParent) + .map(_.identify).getOrElse(outlet.type_.pathId.format) + Some(DataFlowConnection(fromDef, toDef, connector, msgTypeName)) + case _ => None + else None + } + if ctxConnectors.nonEmpty || ctxStreamlets.nonEmpty then + dataFlowDiagrams.put( + context, + DataFlowDiagramData(context, ctxConnectors, ctxStreamlets, connections) + ) + } + + private def captureDomain(domain: Domain): Unit = { + val processors = domain.contents.processors.toSeq + val subdomains = domain.domains.toSeq + val contexts = domain.contexts.toSeq + val epics = domain.epics.toSeq + domainDiagramsMap.put( + domain, + DomainDiagramData(domain, processors, subdomains, contexts, epics) + ) + } + def result(root: PassRoot): DiagramsPassOutput = { DiagramsPassOutput( root, messages.toMessages, dataFlowDiagrams.toMap, useCaseDiagrams.toMap, - contextDiagrams.toMap + contextDiagrams.toMap, + domainDiagramsMap.toMap ) } } diff --git a/passes/shared/src/main/scala/com/ossuminc/riddl/passes/stats/StatsPass.scala b/passes/shared/src/main/scala/com/ossuminc/riddl/passes/stats/StatsPass.scala index 6ed300d36..e5aedb522 100644 --- a/passes/shared/src/main/scala/com/ossuminc/riddl/passes/stats/StatsPass.scala +++ b/passes/shared/src/main/scala/com/ossuminc/riddl/passes/stats/StatsPass.scala @@ -60,7 +60,9 @@ case class DefinitionStats( numTerms: Long = 0, // the number of term definitions numOptions: Long = 0, // number of options declared numIncludes: Long = 0, - numStatements: Long = 0 + numStatements: Long = 0, + numPromptStatements: Long = 0, // prompt "..." count + numExecutableStatements: Long = 0 // tell/send/morph/set/become/error/code ) @JSExportTopLevel("KindStats") @@ -75,7 +77,9 @@ class KindStats( var numTerms: Long = 0, // the number of term definitions var numOptions: Long = 0, var numIncludes: Long = 0, - var numStatements: Long = 0 + var numStatements: Long = 0, + var numPromptStatements: Long = 0, + var numExecutableStatements: Long = 0 ) { def completeness: Double = (numCompleted.toDouble / numSpecifications) * 100.0d def complexity: Double = @@ -114,37 +118,71 @@ case class StatsPass(input: PassInput, outputs: PassesOutput)(using PlatformCont private val kind_stats: mutable.HashMap[String, KindStats] = mutable.HashMap.empty private var total_stats: Option[KindStats] = None - private def computeNumStatements(definition: Branch[?]): Long = { - def handlerStatements(handlers: Seq[Handler]): Long = { - val sizes: Seq[Long] = for { - handler <- handlers - clause <- handler.clauses - } yield { - clause.contents.size.toLong + /** Counts of statement types: (total, prompts, executables) */ + private case class StatementCounts( + total: Long = 0L, + prompts: Long = 0L, + executables: Long = 0L + ): + def +(other: StatementCounts): StatementCounts = + StatementCounts( + total + other.total, + prompts + other.prompts, + executables + other.executables + ) + end StatementCounts + + private def classifyStatements(stmts: Seq[RiddlValue]): StatementCounts = { + var total = 0L + var prompts = 0L + var executables = 0L + stmts.foreach { + case _: PromptStatement => + total += 1; prompts += 1 + case _: TellStatement | _: SendStatement | _: MorphStatement | + _: SetStatement | _: BecomeStatement | _: ErrorStatement | + _: CodeStatement => + total += 1; executables += 1 + case _: Statement => + total += 1 // control flow: when, match, let + case _ => () // non-statements + } + StatementCounts(total, prompts, executables) + } + + private def computeStatementCounts( + definition: Branch[?] + ): StatementCounts = { + def handlerCounts(handlers: Seq[Handler]): StatementCounts = { + handlers.foldLeft(StatementCounts()) { (acc, handler) => + handler.clauses.foldLeft(acc) { (acc2, clause) => + acc2 + classifyStatements(clause.contents.toSeq) + } } - sizes.foldLeft(0L)((a, b) => a + b) } definition.contents.vitals .map { (vd: Definition) => vd match { - case a: Adaptor => handlerStatements(a.handlers) - case c: Context => handlerStatements(c.handlers) - case e: Entity => handlerStatements(e.handlers) - case p: Projector => handlerStatements(p.handlers) - case r: Repository => handlerStatements(r.handlers) - case s: Streamlet => handlerStatements(s.handlers) + case a: Adaptor => handlerCounts(a.handlers) + case c: Context => handlerCounts(c.handlers) + case e: Entity => handlerCounts(e.handlers) + case p: Projector => handlerCounts(p.handlers) + case r: Repository => handlerCounts(r.handlers) + case s: Streamlet => handlerCounts(s.handlers) case s: Saga => - s.sagaSteps - .map(step => step.doStatements.size.toLong + step.undoStatements.size) - .sum[Long] - case f: Function => f.statements.size.toLong - case _: Epic => 0L - case _: Domain => 0L - case _: Definition => 0L // Non Vital, ignore + val sagaStmts = s.sagaSteps.flatMap { step => + step.doStatements.toSeq ++ step.undoStatements.toSeq + } + classifyStatements(sagaStmts) + case f: Function => + classifyStatements(f.statements.toSeq) + case _: Epic => StatementCounts() + case _: Domain => StatementCounts() + case _: Definition => StatementCounts() } } - .sum[Long] + .foldLeft(StatementCounts())(_ + _) } protected def collect(definition: RiddlValue, parents: ParentStack): Seq[DefinitionStats] = { @@ -159,6 +197,7 @@ case class StatsPass(input: PassInput, outputs: PassesOutput)(using PlatformCont val completes: Int = completedCount(definition) definition match { case definition: Branch[?] => + val counts = computeStatementCounts(definition) Seq( DefinitionStats( kind = definition.kind, @@ -184,7 +223,9 @@ case class StatsPass(input: PassInput, outputs: PassesOutput)(using PlatformCont numAuthors = authors, numTerms = terms, numIncludes = includes, - numStatements = computeNumStatements(definition) + numStatements = counts.total, + numPromptStatements = counts.prompts, + numExecutableStatements = counts.executables ) ) case _: RiddlValue => Seq.empty[DefinitionStats] @@ -207,7 +248,9 @@ case class StatsPass(input: PassInput, outputs: PassesOutput)(using PlatformCont numTerms = defStats.numTerms, numOptions = defStats.numOptions, numIncludes = defStats.numIncludes, - numStatements = defStats.numStatements + numStatements = defStats.numStatements, + numPromptStatements = defStats.numPromptStatements, + numExecutableStatements = defStats.numExecutableStatements ) ) { (ks: KindStats) => ks.count += 1 @@ -220,6 +263,8 @@ case class StatsPass(input: PassInput, outputs: PassesOutput)(using PlatformCont ks.numOptions += defStats.numOptions ks.numIncludes += defStats.numIncludes ks.numStatements += defStats.numStatements + ks.numPromptStatements += defStats.numPromptStatements + ks.numExecutableStatements += defStats.numExecutableStatements ks } ) @@ -237,6 +282,8 @@ case class StatsPass(input: PassInput, outputs: PassesOutput)(using PlatformCont total.numEmpty += next.numEmpty total.numAuthors += next.numAuthors total.numStatements += next.numStatements + total.numPromptStatements += next.numPromptStatements + total.numExecutableStatements += next.numExecutableStatements total } total_stats = Some(totals) diff --git a/passes/shared/src/main/scala/com/ossuminc/riddl/passes/validate/DefinitionValidation.scala b/passes/shared/src/main/scala/com/ossuminc/riddl/passes/validate/DefinitionValidation.scala index 4b0d70a7c..372c4f3f7 100644 --- a/passes/shared/src/main/scala/com/ossuminc/riddl/passes/validate/DefinitionValidation.scala +++ b/passes/shared/src/main/scala/com/ossuminc/riddl/passes/validate/DefinitionValidation.scala @@ -12,6 +12,76 @@ import com.ossuminc.riddl.language.Messages.* import com.ossuminc.riddl.passes.symbols.SymbolsOutput import com.ossuminc.riddl.utils.PlatformContext +/** Specification of a recognized option: which definition types + * it applies to and its expected argument count. + * + * @param validParents + * The definition types this option is valid on. Empty means + * valid on any definition. + * @param minArgs + * Minimum number of arguments expected + * @param maxArgs + * Maximum number of arguments expected + */ +case class OptionSpec( + validParents: Seq[String], + minArgs: Int = 0, + maxArgs: Int = 0 +) + +/** Registry of recognized RIDDL option names with their + * specifications. Options not in this registry will produce + * style warnings (not errors) to keep the system extensible. + */ +object RecognizedOptions: + val registry: Map[String, OptionSpec] = Map( + // Existing well-known options + "aggregate" -> OptionSpec(Seq("Entity"), 0, 0), + "finite-state-machine" -> OptionSpec(Seq("Entity"), 0, 0), + "persistent" -> OptionSpec(Seq("Connector"), 0, 0), + "css" -> OptionSpec(Seq.empty, 1, 10), + "technology" -> OptionSpec(Seq.empty, 1, 1), + "kind" -> OptionSpec(Seq.empty, 1, 1), + "color" -> OptionSpec(Seq.empty, 1, 1), + // Temporal options (C1) + "timeout" -> OptionSpec( + Seq("SagaStep", "Handler", "On Message"), 1, 1 + ), + "retry" -> OptionSpec( + Seq("SagaStep", "Handler"), 1, 2 + ), + "delay" -> OptionSpec( + Seq("SagaStep"), 1, 1 + ), + // Resilience options (C2) + "circuit-breaker" -> OptionSpec( + Seq("Adaptor", "Connector"), 0, 2 + ), + "idempotent" -> OptionSpec( + Seq("Handler", "On Message"), 0, 0 + ), + "bulkhead" -> OptionSpec( + Seq("Entity", "Context"), 0, 1 + ), + // Delivery semantics options (C3) + "at-least-once" -> OptionSpec(Seq("Connector"), 0, 0), + "at-most-once" -> OptionSpec(Seq("Connector"), 0, 0), + "exactly-once" -> OptionSpec(Seq("Connector"), 0, 0), + "ordered" -> OptionSpec(Seq("Connector", "Inlet"), 0, 0), + "partitioned" -> OptionSpec(Seq("Connector"), 1, 1), + // Caching and performance options (C4) + "cacheable" -> OptionSpec( + Seq("Projector", "Handler"), 0, 1 + ), + "rate-limit" -> OptionSpec( + Seq("Handler", "Entity"), 2, 2 + ), + "batch" -> OptionSpec( + Seq("Projector", "Repository"), 1, 1 + ) + ) +end RecognizedOptions + /** A Trait that defines typical Validation checkers for validating definitions */ trait DefinitionValidation(using pc: PlatformContext) extends BasicValidation: def symbols: SymbolsOutput @@ -148,6 +218,7 @@ trait DefinitionValidation(using pc: PlatformContext) extends BasicValidation: StyleWarning, o.loc ) + validateRecognizedOption(o, identity, loc) case _: AuthorRef => hasAuthorRef = true case _: StringAttachment => () // No validation needed case _: FileAttachment => () // No validation needed @@ -157,4 +228,50 @@ trait DefinitionValidation(using pc: PlatformContext) extends BasicValidation: } check(hasDescription, s"$identity should have a description", MissingWarning, loc) end checkMetadata + + /** Validate an option against the recognized options registry. + * Checks argument count and parent definition type compatibility. + * Unrecognized options produce style warnings to keep the system extensible. + */ + private def validateRecognizedOption( + option: OptionValue, + identity: String, + loc: At + ): Unit = + RecognizedOptions.registry.get(option.name) match + case Some(spec) => + val argCount = option.args.size + if argCount < spec.minArgs || argCount > spec.maxArgs then + val expected = + if spec.minArgs == spec.maxArgs then s"${spec.minArgs}" + else s"${spec.minArgs} to ${spec.maxArgs}" + check( + predicate = false, + s"Option '${option.name}' in $identity expects $expected argument(s) but has $argCount", + Warning, + option.loc + ) + end if + if spec.validParents.nonEmpty then + val parentKind = identity.split(" ").head + val isValid = spec.validParents.exists { vp => + vp == parentKind || identity.startsWith(vp) + } + check( + isValid, + s"Option '${option.name}' is not typically used on ${identity.split(" ").head} definitions" + + s" (expected: ${spec.validParents.mkString(", ")})", + StyleWarning, + option.loc + ) + end if + case None => + check( + predicate = false, + s"Option '${option.name}' in $identity is not a recognized RIDDL option", + StyleWarning, + option.loc + ) + end match + end validateRecognizedOption end DefinitionValidation diff --git a/passes/shared/src/main/scala/com/ossuminc/riddl/passes/validate/StreamingValidation.scala b/passes/shared/src/main/scala/com/ossuminc/riddl/passes/validate/StreamingValidation.scala index ea1e30923..b993d0cc6 100644 --- a/passes/shared/src/main/scala/com/ossuminc/riddl/passes/validate/StreamingValidation.scala +++ b/passes/shared/src/main/scala/com/ossuminc/riddl/passes/validate/StreamingValidation.scala @@ -32,7 +32,86 @@ trait StreamingValidation(using pc: PlatformContext) extends TypeValidation { protected val streamlets: mutable.ListBuffer[Streamlet] = mutable.ListBuffer.empty protected val connectors: mutable.ListBuffer[Connector] = mutable.ListBuffer.empty - private def checkStreamingUsage(root: PassRoot): Unit = () + private def checkStreamingUsage(root: PassRoot): Unit = { + if streamlets.nonEmpty then { + // Build a map from each streamlet to its connected streamlets via connectors + // First, resolve all connector endpoints to their parent streamlets + val connectedStreamlets = mutable.Set.empty[Streamlet] + // Adjacency list: streamlet → set of downstream streamlets (outlet→inlet direction) + val adjacency = mutable.Map.empty[Streamlet, mutable.Set[Streamlet]] + + connectors.filterNot(_.isEmpty).foreach { connector => + val connParents = symbols.parentsOf(connector) + val maybeOutlet = resolvePath[Outlet](connector.from.pathId, connParents) + val maybeInlet = resolvePath[Inlet](connector.to.pathId, connParents) + + val maybeFromStreamlet = maybeOutlet.flatMap { outlet => + symbols.parentOf(outlet).collect { case s: Streamlet => s } + } + val maybeToStreamlet = maybeInlet.flatMap { inlet => + symbols.parentOf(inlet).collect { case s: Streamlet => s } + } + + (maybeFromStreamlet, maybeToStreamlet) match { + case (Some(fromSl), Some(toSl)) => + connectedStreamlets += fromSl + connectedStreamlets += toSl + adjacency.getOrElseUpdate(fromSl, mutable.Set.empty) += toSl + case (Some(fromSl), None) => + connectedStreamlets += fromSl + case (None, Some(toSl)) => + connectedStreamlets += toSl + case _ => () + } + } + + // Check 1: Isolated streamlets (non-Void, not connected to any connector) + streamlets.foreach { streamlet => + streamlet.shape match { + case _: Void => () // Void streamlets are excluded + case _ => + if !connectedStreamlets.contains(streamlet) then + messages.addWarning( + streamlet.errorLoc, + s"${streamlet.identify} has no connections to any connector" + ) + } + } + + // Check 2: Source→Sink reachability via BFS + val sources = streamlets.filter(_.shape.isInstanceOf[Source]) + val sinks = streamlets.filter(_.shape.isInstanceOf[Sink]).toSet + + sources.foreach { source => + if adjacency.contains(source) then { + // BFS from this source + val visited = mutable.Set.empty[Streamlet] + val queue = mutable.Queue.empty[Streamlet] + queue.enqueue(source) + visited += source + var reachesSink = false + + while queue.nonEmpty && !reachesSink do + val current = queue.dequeue() + if sinks.contains(current) && current != source then + reachesSink = true + else + adjacency.getOrElse(current, mutable.Set.empty).foreach { neighbor => + if !visited.contains(neighbor) then + visited += neighbor + queue.enqueue(neighbor) + } + end while + + if !reachesSink then + messages.addWarning( + source.errorLoc, + s"${source.identify} is a source but has no downstream path to any sink" + ) + } + } + } + } private def checkConnectorPersistence(): Unit = { connectors.filterNot(_.isEmpty).foreach { connector => diff --git a/passes/shared/src/main/scala/com/ossuminc/riddl/passes/validate/ValidationOutput.scala b/passes/shared/src/main/scala/com/ossuminc/riddl/passes/validate/ValidationOutput.scala index 3b4ca9c63..c27d5c68d 100644 --- a/passes/shared/src/main/scala/com/ossuminc/riddl/passes/validate/ValidationOutput.scala +++ b/passes/shared/src/main/scala/com/ossuminc/riddl/passes/validate/ValidationOutput.scala @@ -11,6 +11,50 @@ import com.ossuminc.riddl.passes.PassOutput import com.ossuminc.riddl.language.Messages import com.ossuminc.riddl.passes.PassRoot +import scala.scalajs.js.annotation.* + +/** Classification of a handler's behavioral completeness */ +enum BehaviorCategory: + /** Handler contains executable statements + * (tell, send, morph, set, become, error, code) + */ + case Executable + + /** Handler contains only prompt statements + * (natural language descriptions of intended behavior) + */ + case PromptOnly + + /** Handler uses ??? or has no statements at all */ + case Empty +end BehaviorCategory + +/** Describes the behavioral completeness of a single handler + * + * @param handler + * The handler being classified + * @param parent + * The parent definition containing this handler + * @param category + * The behavioral completeness classification + * @param executableCount + * Number of executable statements (tell, send, morph, + * set, become, error, code) + * @param promptCount + * Number of prompt statements + * @param totalClauses + * Total number of on-clauses in the handler + */ +@JSExportTopLevel("HandlerCompleteness") +case class HandlerCompleteness( + handler: Handler, + parent: Definition, + category: BehaviorCategory, + executableCount: Int, + promptCount: Int, + totalClauses: Int +) + /** The output of the Validation Pass */ case class ValidationOutput( root: PassRoot = Root.empty, @@ -19,4 +63,6 @@ case class ValidationOutput( outlets: Seq[Outlet] = Seq.empty[Outlet], connectors: Seq[Connector] = Seq.empty[Connector], streamlets: Seq[Streamlet] = Seq.empty[Streamlet], + handlerCompleteness: Seq[HandlerCompleteness] = + Seq.empty[HandlerCompleteness], ) extends PassOutput diff --git a/passes/shared/src/main/scala/com/ossuminc/riddl/passes/validate/ValidationPass.scala b/passes/shared/src/main/scala/com/ossuminc/riddl/passes/validate/ValidationPass.scala index 70aa606da..f2e3071f2 100644 --- a/passes/shared/src/main/scala/com/ossuminc/riddl/passes/validate/ValidationPass.scala +++ b/passes/shared/src/main/scala/com/ossuminc/riddl/passes/validate/ValidationPass.scala @@ -49,6 +49,10 @@ case class ValidationPass( outputs.outputOf[ResolutionOutput](ResolutionPass.name).get lazy val symbols: SymbolsOutput = outputs.outputOf[SymbolsOutput](SymbolsPass.name).get + /** Accumulated handler-to-parent mappings collected during processing */ + private val handlerParents: mutable.ListBuffer[(Handler, Definition)] = + mutable.ListBuffer.empty + /** Generate the output of this Pass. This will only be called after all the calls to process have * completed. * @@ -62,13 +66,18 @@ case class ValidationPass( inlets.toSeq, outlets.toSeq, connectors.toSeq, - streamlets.toSeq + streamlets.toSeq, + computedHandlerCompleteness ) } + private var computedHandlerCompleteness: Seq[HandlerCompleteness] = + Seq.empty + override def postProcess(root: PassRoot): Unit = { checkOverloads() checkStreaming(root) + computedHandlerCompleteness = classifyHandlers() } def process(value: RiddlValue, parents: ParentStack): Unit = { @@ -101,6 +110,9 @@ case class ValidationPass( validateStatement(statement, parentsAsSeq) case h: Handler => validateHandler(h, parentsAsSeq) + parentsAsSeq.headOption.foreach { parent => + handlerParents.addOne((h, parent)) + } case c: Constant => validateConstant(c, parentsAsSeq) case s: State => @@ -151,6 +163,10 @@ case class ValidationPass( validateInclude(include) case bi: BASTImport => validateBASTImport(bi, parentsAsSeq) + case s: Schema => + validateSchema(s, parentsAsSeq) + case r: Relationship => + validateRelationship(r, parentsAsSeq) case _: MatchCase => () // Validated through MatchStatement case _: Definition => () // abstract type case _: NonDefinitionValues => () // We only validate definitions @@ -465,7 +481,6 @@ case class ValidationPass( s.loc ) } - checkMetadata(s) } private def validateFunction( @@ -480,6 +495,12 @@ case class ValidationPass( MissingWarning, f.errorLoc ) + f.input.foreach { agg => + checkTypeExpression(agg, f, parents) + } + f.output.foreach { agg => + checkTypeExpression(agg, f, parents) + } checkMetadata(f) } @@ -488,6 +509,22 @@ case class ValidationPass( parents: Parents ): Unit = { checkContainer(parents, h) + parents.headOption match { + case Some(entity: Entity) => + val messageClauses = h.clauses.collect { case omc: OnMessageClause => omc } + if messageClauses.nonEmpty then { + val handlesCommandOrQuery = messageClauses.exists { omc => + omc.msg.messageKind == AggregateUseCase.CommandCase || + omc.msg.messageKind == AggregateUseCase.QueryCase + } + if !handlesCommandOrQuery then + messages.addWarning( + h.errorLoc, + s"${h.identify} in ${entity.identify} handles no commands or queries; entity handlers typically handle commands and queries" + ) + } + case _ => () + } } // FIXME: This should be used: @@ -503,6 +540,81 @@ case class ValidationPass( check(bi.path.s.endsWith(".bast"), s"BAST load path '${bi.path.s}' should end with .bast", Messages.Warning, bi.loc) } + private def validateSchema( + schema: Schema, + parents: Parents + ): Unit = { + checkIdentifierLength(schema) + checkMetadata(schema) + checkNonEmpty( + schema.data.toSeq, + "data definitions", + schema, + schema.errorLoc, + MissingWarning, + required = true + ) + schema.schemaKind match { + case RepositorySchemaKind.Flat | RepositorySchemaKind.Document | + RepositorySchemaKind.Columnar | RepositorySchemaKind.Vector => + if schema.links.nonEmpty then + messages.addWarning( + schema.errorLoc, + s"${schema.identify} is ${schema.schemaKind} and should not define links" + ) + case RepositorySchemaKind.Graphical => + if schema.links.isEmpty && schema.data.nonEmpty then + messages.addWarning( + schema.errorLoc, + s"${schema.identify} is graphical but has no links (edges)" + ) + case RepositorySchemaKind.Relational => + if schema.links.isEmpty && schema.data.size > 1 then + messages.addWarning( + schema.errorLoc, + s"${schema.identify} is relational with ${schema.data.size} data nodes but has no links; consider adding links to define relationships" + ) + schema.links.values.foreach { case (fromRef, toRef) => + val fromType = resolvePath[Field](fromRef.pathId, parents).map(_.typeEx) + val toType = resolvePath[Field](toRef.pathId, parents).map(_.typeEx) + (fromType, toType) match { + case (Some(ft), Some(tt)) => + if ft != tt then + messages.addWarning( + fromRef.loc, + s"Link in ${schema.identify} connects fields with incompatible types: ${fromRef.pathId.format} is ${ft.format} but ${toRef.pathId.format} is ${tt.format}" + ) + case _ => () // unresolved fields already reported elsewhere + } + } + case _ => () + } + if schema.schemaKind == RepositorySchemaKind.Vector && schema.data.size > 1 then + messages.addWarning( + schema.errorLoc, + s"${schema.identify} is a vector schema but defines ${schema.data.size} data nodes; typically only one is expected" + ) + schema.data.values.foreach { typeRef => + checkRef[Type](typeRef, parents) + } + schema.links.values.foreach { case (from, to) => + checkRef[Field](from, parents) + checkRef[Field](to, parents) + } + schema.indices.foreach { fieldRef => + checkRef[Field](fieldRef, parents) + } + } + + private def validateRelationship( + relationship: Relationship, + parents: Parents + ): Unit = { + checkIdentifierLength(relationship) + checkRef[Processor[?]](relationship.withProcessor, parents) + checkMetadata(relationship) + } + private def validateEntity( entity: Entity, parents: Parents @@ -535,6 +647,20 @@ case class ValidationPass( ) ) } + if entity.hasOption("finite-state-machine") && entity.states.sizeIs >= 2 then { + val hasMorphOrBecome = entity.handlers.exists { handler => + handler.clauses.exists { clause => + val finder = Finder(clause.contents) + finder.recursiveFindByType[MorphStatement].nonEmpty || + finder.recursiveFindByType[BecomeStatement].nonEmpty + } + } + if !hasMorphOrBecome then + messages.addWarning( + entity.errorLoc, + s"${entity.identify} is declared as a finite-state-machine but its handlers contain no morph or become statements" + ) + } if entity.states.nonEmpty && entity.handlers.isEmpty then { messages.add( Message( @@ -573,6 +699,24 @@ case class ValidationPass( Messages.Error, projector.errorLoc ) + projector.repositories.foreach { repoRef => + checkRef[Repository](repoRef, parents) + } + if projector.handlers.nonEmpty then { + val allClauses = projector.handlers.flatMap(_.clauses).collect { + case omc: OnMessageClause => omc + } + if allClauses.nonEmpty then { + val handlesEvents = allClauses.exists { omc => + omc.msg.messageKind == AggregateUseCase.EventCase + } + if !handlesEvents then + messages.addWarning( + projector.errorLoc, + s"${projector.identify} handler does not handle any events; projectors typically handle events to build read models" + ) + } + } checkMetadata(projector) } @@ -590,6 +734,27 @@ case class ValidationPass( MissingWarning, required = false ) + if repository.handlers.isEmpty && repository.nonEmpty then + messages.addMissing( + repository.errorLoc, + s"${repository.identify} should have at least one handler" + ) + if repository.handlers.nonEmpty then { + val allClauses = repository.handlers.flatMap(_.clauses).collect { + case omc: OnMessageClause => omc + } + if allClauses.nonEmpty then { + val handlesCommandOrQuery = allClauses.exists { omc => + omc.msg.messageKind == AggregateUseCase.CommandCase || + omc.msg.messageKind == AggregateUseCase.QueryCase + } + if !handlesCommandOrQuery then + messages.addWarning( + repository.errorLoc, + s"${repository.identify} handlers do not handle any commands or queries; repositories typically handle commands (for mutations) and queries (for reads)" + ) + } + } } private def validateAdaptor( @@ -607,6 +772,51 @@ case class ValidationPass( messages.addError(adaptor.errorLoc, message) } } + if adaptor.handlers.isEmpty && adaptor.nonEmpty then + messages.addMissing( + adaptor.errorLoc, + s"${adaptor.identify} should have at least one handler" + ) + else if adaptor.handlers.nonEmpty && adaptor.handlers.forall(_.clauses.isEmpty) then + messages.addMissing( + adaptor.errorLoc, + s"${adaptor.identify} has only empty handlers" + ) + // Check if adaptor handlers reference message types from the adapted context + resolvePath[Context](adaptor.referent.pathId, parents).foreach { targetContext => + val targetMessageTypes = targetContext.types.filter { t => + t.typEx match { + case auc: AggregateUseCaseTypeExpression => + auc.usecase == AggregateUseCase.CommandCase || + auc.usecase == AggregateUseCase.EventCase || + auc.usecase == AggregateUseCase.QueryCase || + auc.usecase == AggregateUseCase.ResultCase + case _ => false + } + } + if targetMessageTypes.nonEmpty && adaptor.handlers.nonEmpty then { + val allClauses = adaptor.handlers.flatMap(_.clauses).collect { + case omc: OnMessageClause => omc + } + if allClauses.nonEmpty then { + val referencesTargetType = allClauses.exists { omc => + resolvePath[Type](omc.msg.pathId, parents).exists { resolvedType => + symbols.parentsOf(resolvedType).exists(_ == targetContext) + } + } + if !referencesTargetType then { + val directionWord = adaptor.direction match { + case _: InboundAdaptor => "from" + case _: OutboundAdaptor => "to" + } + messages.addWarning( + adaptor.errorLoc, + s"${adaptor.identify} is ${adaptor.direction.format} ${directionWord} ${targetContext.identify} but its handlers do not reference any message types defined in ${targetContext.identify}" + ) + } + } + } + } checkMetadata(adaptor) case None | Some(_) => messages.addError(adaptor.errorLoc, "Adaptor not contained within Context") @@ -619,6 +829,33 @@ case class ValidationPass( ): Unit = { addStreamlet(streamlet) checkContainer(parents, streamlet) + if streamlet.nonEmpty then + val numInlets = streamlet.inlets.size + val numOutlets = streamlet.outlets.size + streamlet.shape match { + case _: Source => + check(numInlets == 0, s"${streamlet.identify} is a source but has $numInlets inlets; sources must have none", Messages.Error, streamlet.errorLoc) + check(numOutlets >= 1, s"${streamlet.identify} is a source but has no outlets; sources must have at least one", Messages.Error, streamlet.errorLoc) + case _: Sink => + check(numInlets >= 1, s"${streamlet.identify} is a sink but has no inlets; sinks must have at least one", Messages.Error, streamlet.errorLoc) + check(numOutlets == 0, s"${streamlet.identify} is a sink but has $numOutlets outlets; sinks must have none", Messages.Error, streamlet.errorLoc) + case _: Flow => + check(numInlets >= 1, s"${streamlet.identify} is a flow but has no inlets; flows must have at least one", Messages.Error, streamlet.errorLoc) + check(numOutlets >= 1, s"${streamlet.identify} is a flow but has no outlets; flows must have at least one", Messages.Error, streamlet.errorLoc) + case _: Merge => + check(numInlets >= 2, s"${streamlet.identify} is a merge but has $numInlets inlets; merges must have at least two", Messages.Error, streamlet.errorLoc) + check(numOutlets >= 1, s"${streamlet.identify} is a merge but has no outlets; merges must have at least one", Messages.Error, streamlet.errorLoc) + case _: Split => + check(numInlets >= 1, s"${streamlet.identify} is a split but has no inlets; splits must have at least one", Messages.Error, streamlet.errorLoc) + check(numOutlets >= 2, s"${streamlet.identify} is a split but has $numOutlets outlets; splits must have at least two", Messages.Error, streamlet.errorLoc) + case _: Router => + check(numInlets >= 2, s"${streamlet.identify} is a router but has $numInlets inlets; routers must have at least two", Messages.Error, streamlet.errorLoc) + check(numOutlets >= 2, s"${streamlet.identify} is a router but has $numOutlets outlets; routers must have at least two", Messages.Error, streamlet.errorLoc) + case _: Void => () + } + end if + if streamlet.handlers.isEmpty && streamlet.nonEmpty then + messages.addMissing(streamlet.errorLoc, s"${streamlet.identify} should have a handler") checkMetadata(streamlet) } @@ -662,13 +899,33 @@ case class ValidationPass( ): Unit = { checkDefinition(parents, s) checkNonEmpty(s.doStatements.toSeq, "Do Statements", s, MissingWarning) - checkNonEmpty(s.doStatements.toSeq, "Revert Statements", s, MissingWarning) + checkNonEmpty(s.undoStatements.toSeq, "Revert Statements", s, MissingWarning) check( - s.doStatements.getClass == s.undoStatements.getClass, - "The primary action and revert action must be the same shape", + s.doStatements.nonEmpty == s.undoStatements.nonEmpty, + "A saga step with do statements must also have revert statements, and vice versa", Messages.Error, s.errorLoc ) + if s.doStatements.nonEmpty && s.undoStatements.nonEmpty then { + val doFinder = Finder(s.doStatements) + val undoFinder = Finder(s.undoStatements) + val doTellTargets = doFinder.recursiveFindByType[TellStatement].map(_.processorRef.pathId.format).toSet + val doSendTargets = doFinder.recursiveFindByType[SendStatement].map(_.portlet.pathId.format).toSet + val doTargets = doTellTargets ++ doSendTargets + if doTargets.nonEmpty then { + val undoTellTargets = undoFinder.recursiveFindByType[TellStatement].map(_.processorRef.pathId.format).toSet + val undoSendTargets = undoFinder.recursiveFindByType[SendStatement].map(_.portlet.pathId.format).toSet + val undoTargets = undoTellTargets ++ undoSendTargets + val uncompensated = doTargets -- undoTargets + if uncompensated.nonEmpty then + messages.add( + Messages.style( + s"${s.identify} do-step targets ${uncompensated.mkString(", ")} but the undo-step does not target the same; consider adding compensation", + s.errorLoc + ) + ) + } + } checkMetadata(s) } @@ -685,9 +942,10 @@ case class ValidationPass( parents: Parents ): Unit = { checkContainer(parents, epic) - if epic.userStory.isEmpty then { + if epic.userStory.isEmpty then messages.addMissing(epic.errorLoc, s"${epic.identify} is missing a user story") - } + else + checkRef[User](epic.userStory.user, parents) checkMetadata(epic) } @@ -750,6 +1008,8 @@ case class ValidationPass( parents: Parents ): Unit = { checkDefinition(parents, uc) + if uc.userStory.nonEmpty then + checkRef[User](uc.userStory.user, parents) if uc.contents.nonEmpty then { uc.contents.foreach { case seq: SequentialInteractions => @@ -910,4 +1170,52 @@ case class ValidationPass( } } + /** Classify all collected handlers by behavioral completeness. + * A handler is: + * - Executable: has at least one executable statement + * (tell, send, morph, set, become, error, code) + * - PromptOnly: has only prompt statements + * - Empty: has no statements or only uses ??? + */ + private def classifyHandlers(): Seq[HandlerCompleteness] = { + handlerParents.toSeq.map { case (handler, parent) => + val finder = Finder(handler.contents) + val allStatements = handler.clauses.flatMap { clause => + Finder(clause.contents).recursiveFindByType[Statement] + } + + val executableCount = allStatements.count { + case _: TellStatement => true + case _: SendStatement => true + case _: MorphStatement => true + case _: SetStatement => true + case _: BecomeStatement => true + case _: ErrorStatement => true + case _: CodeStatement => true + case _ => false + } + + val promptCount = allStatements.count { + case _: PromptStatement => true + case _ => false + } + + val totalClauses = handler.clauses.size + + val category = + if executableCount > 0 then BehaviorCategory.Executable + else if promptCount > 0 then BehaviorCategory.PromptOnly + else BehaviorCategory.Empty + + HandlerCompleteness( + handler = handler, + parent = parent, + category = category, + executableCount = executableCount, + promptCount = promptCount, + totalClauses = totalClauses + ) + } + } + } diff --git a/riddlLib/js/src/main/scala/com/ossuminc/riddl/RiddlAPI.scala b/riddlLib/js/src/main/scala/com/ossuminc/riddl/RiddlAPI.scala index bbd009658..e0ccb42e2 100644 --- a/riddlLib/js/src/main/scala/com/ossuminc/riddl/RiddlAPI.scala +++ b/riddlLib/js/src/main/scala/com/ossuminc/riddl/RiddlAPI.scala @@ -10,6 +10,8 @@ import com.ossuminc.riddl.language.AST.{Nebula, Root, Token} import com.ossuminc.riddl.language.{Contents, *} import com.ossuminc.riddl.language.Messages.Messages import com.ossuminc.riddl.passes.{OutlineEntry, TreeNode} +import com.ossuminc.riddl.passes.validate.{BehaviorCategory, HandlerCompleteness} +import com.ossuminc.riddl.passes.analysis.{MessageFlowOutput, MessageFlowEdge, FlowMechanism, EntityLifecycle, StateTransition} import com.ossuminc.riddl.utils.{CommonOptions, DOMPlatformContext, PlatformContext, URL} import scala.scalajs.js @@ -423,4 +425,81 @@ object RiddlAPI { nodes => nodes.map(treeNodeToJs).toJSArray ) end getTree + + /** Get handler completeness classifications. */ + @JSExport("getHandlerCompleteness") + def getHandlerCompleteness( + source: String, + origin: String = "string" + ): js.Dynamic = + toJsResult( + RiddlLib.getHandlerCompleteness(source, origin), + entries => entries.map { hc => + js.Dynamic.literal( + handlerId = hc.handler.id.value, + parentId = hc.parent.id.value, + parentKind = hc.parent.kind, + category = hc.category.toString, + executableCount = hc.executableCount, + promptCount = hc.promptCount, + totalClauses = hc.totalClauses + ) + }.toJSArray + ) + end getHandlerCompleteness + + /** Get the message flow graph. */ + @JSExport("getMessageFlow") + def getMessageFlow( + source: String, + origin: String = "string" + ): js.Dynamic = + toJsResult( + RiddlLib.getMessageFlow(source, origin), + mfo => js.Dynamic.literal( + edges = mfo.edges.map { edge => + js.Dynamic.literal( + producerId = edge.producer.id.value, + producerKind = edge.producer.kind, + consumerId = edge.consumer.id.value, + consumerKind = edge.consumer.kind, + messageTypeId = edge.messageType.id.value, + mechanism = edge.mechanism.toString + ) + }.toJSArray, + edgeCount = mfo.edges.size + ) + ) + end getMessageFlow + + /** Get entity lifecycle (state machine) data. */ + @JSExport("getEntityLifecycles") + def getEntityLifecycles( + source: String, + origin: String = "string" + ): js.Dynamic = + toJsResult( + RiddlLib.getEntityLifecycles(source, origin), + lifecycles => lifecycles.map { case (entity, lc) => + js.Dynamic.literal( + entityId = entity.id.value, + states = lc.states.map(s => + s.id.value + ).toJSArray, + transitions = lc.transitions.map { t => + js.Dynamic.literal( + fromState = t.fromState.map(_.id.value) + .getOrElse("*"), + toState = t.toState.id.value, + trigger = t.trigger.id.value + ) + }.toJSArray, + initialState = lc.initialState + .map(_.id.value).getOrElse(""), + terminalStates = lc.terminalStates + .map(_.id.value).toJSArray + ) + }.toJSArray + ) + end getEntityLifecycles } diff --git a/riddlLib/shared/src/main/scala/com/ossuminc/riddl/RiddlLib.scala b/riddlLib/shared/src/main/scala/com/ossuminc/riddl/RiddlLib.scala index e77842505..fffa7d9d8 100644 --- a/riddlLib/shared/src/main/scala/com/ossuminc/riddl/RiddlLib.scala +++ b/riddlLib/shared/src/main/scala/com/ossuminc/riddl/RiddlLib.scala @@ -6,7 +6,7 @@ package com.ossuminc.riddl -import com.ossuminc.riddl.language.AST.{Nebula, Root, Token} +import com.ossuminc.riddl.language.AST.{Entity, Nebula, Root, Token} import com.ossuminc.riddl.language.Messages.Messages import com.ossuminc.riddl.language.parsing.{ RiddlParserInput, TopLevelParser @@ -16,7 +16,14 @@ import com.ossuminc.riddl.passes.{ OutlinePass, OutlineOutput, OutlineEntry, TreePass, TreeOutput, TreeNode } +import com.ossuminc.riddl.passes.analysis.{ + EntityLifecycle, EntityLifecycleOutput, EntityLifecyclePass, + MessageFlowOutput, MessageFlowPass +} import com.ossuminc.riddl.passes.transforms.FlattenPass +import com.ossuminc.riddl.passes.validate.{ + HandlerCompleteness, ValidationOutput, ValidationPass +} import com.ossuminc.riddl.utils.{ CommonOptions, PlatformContext, RiddlBuildInfo, URL } @@ -94,6 +101,36 @@ trait RiddlLib: origin: String = "string" )(using PlatformContext): Either[Messages, Seq[TreeNode]] + /** Get handler completeness classifications from validation. + * + * Runs the standard pass pipeline and returns the handler + * completeness data from ValidationOutput. + */ + def getHandlerCompleteness( + source: String, + origin: String = "string" + )(using PlatformContext): Either[Messages, Seq[HandlerCompleteness]] + + /** Get the message flow graph for a RIDDL model. + * + * Runs standard passes plus the MessageFlowPass to build + * a directed graph of message producers and consumers. + */ + def getMessageFlow( + source: String, + origin: String = "string" + )(using PlatformContext): Either[Messages, MessageFlowOutput] + + /** Get entity lifecycle (state machine) data. + * + * Runs standard passes plus the EntityLifecyclePass to + * extract state machines from entities with multiple states. + */ + def getEntityLifecycles( + source: String, + origin: String = "string" + )(using PlatformContext): Either[Messages, Map[Entity, EntityLifecycle]] + /** Get the RIDDL library version string. */ def version: String @@ -273,6 +310,65 @@ object RiddlLib extends RiddlLib: } end getTree + override def getHandlerCompleteness( + source: String, + origin: String + )(using PlatformContext): Either[Messages, Seq[HandlerCompleteness]] = + val rpi = RiddlParserInput(source, originToURL(origin)) + val parseResult = TopLevelParser.parseInput(rpi) + parseResult.flatMap { root => + val passInput = PassInput(root) + val passesResult = Pass.runStandardPasses(passInput) + passesResult.outputs + .outputOf[ValidationOutput](ValidationPass.name) match + case Some(vo) => + Right(vo.handlerCompleteness) + case None => + Left(List.empty) + end match + } + end getHandlerCompleteness + + override def getMessageFlow( + source: String, + origin: String + )(using PlatformContext): Either[Messages, MessageFlowOutput] = + val rpi = RiddlParserInput(source, originToURL(origin)) + val parseResult = TopLevelParser.parseInput(rpi) + parseResult.flatMap { root => + val passInput = PassInput(root) + val passes = Pass.standardPasses :+ MessageFlowPass.creator() + val passesResult = Pass.runThesePasses(passInput, passes) + passesResult.outputs + .outputOf[MessageFlowOutput](MessageFlowPass.name) match + case Some(mfo) => + Right(mfo) + case None => + Left(List.empty) + end match + } + end getMessageFlow + + override def getEntityLifecycles( + source: String, + origin: String + )(using PlatformContext): Either[Messages, Map[Entity, EntityLifecycle]] = + val rpi = RiddlParserInput(source, originToURL(origin)) + val parseResult = TopLevelParser.parseInput(rpi) + parseResult.flatMap { root => + val passInput = PassInput(root) + val passes = Pass.standardPasses :+ EntityLifecyclePass.creator() + val passesResult = Pass.runThesePasses(passInput, passes) + passesResult.outputs + .outputOf[EntityLifecycleOutput](EntityLifecyclePass.name) match + case Some(elo) => + Right(elo.lifecycles) + case None => + Left(List.empty) + end match + } + end getEntityLifecycles + override def version: String = RiddlBuildInfo.version