Skip to content

Scala 3.7.4 upgrade and module reorganization#719

Merged
reid-spencer merged 12 commits into
mainfrom
feature/scala-bug
Jan 31, 2026
Merged

Scala 3.7.4 upgrade and module reorganization#719
reid-spencer merged 12 commits into
mainfrom
feature/scala-bug

Conversation

@reid-spencer
Copy link
Copy Markdown
Contributor

Summary

  • Upgrade from Scala 3.3.7 LTS to Scala 3.7.4 to fix compiler hang bug
  • Move hugo and diagrams modules to riddl-gen repository
  • Add selective BAST import support
  • Migrate documentation to ossum.tech

Changes

  • Fix Scala 3.7.4 compiler hang by extracting Contents to package level
  • Fix test compilation errors (fastparse context functions, Contents imports)
  • Remove docsite from build aggregation
  • Add redirect site for riddl.tech → ossum.tech migration
  • Update EBNF grammar for selective BAST imports

Test plan

  • JVM tests pass locally
  • Native tests pass locally
  • JS tests pass locally
  • CI verification

🤖 Generated with Claude Code

respencer-ncl and others added 11 commits January 30, 2026 17:44
Document Scala version upgrade blocker:
- Compiler infinite loop bug with intersection types
- Parser lambda syntax changes completed
- AST extension restructuring completed
- Both 3.7.4 and 3.6.3 trigger hasClassSymbol recursion

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add grammar rules for selective import syntax:
- import domain X from "file.bast"
- import type T from "file.bast" as MyT
- Support for kind, selector, and alias components

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
BASTLoader:
- Handle selective imports that filter by kind and name
- Support aliasing of imported definitions
- Process imports in contexts as well as domains

BASTReader/BASTWriter:
- Read/write new BASTImport fields (kind, selector, alias)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
CommonParser:
- Add importableKind parser for definition types
- Add selectiveBastImport for "import kind X from file" syntax
- Add fullBastImport for simple "import file" syntax
- Combined bastImport parser handles both forms

ParsingContext:
- Update doBASTImport to accept kind, selector, and alias
- Validate .bast extension

Syntax variants supported:
- import "path/to/file.bast"
- import domain X from "file.bast"
- import type T from "file.bast" as MyT

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Test input files:
- rbbq.riddl: Updated domain example
- bast-import.riddl: Test cases for selective imports

Python validation:
- riddl_grammar.lark: Lark grammar derived from EBNF
- ebnf_validator.py: Validation script for testing RIDDL files

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
WHY SCALA 3.7.4?
Scala 3.7 changed how underscore syntax works with context-bound methods.
The pattern `include[u, XxxContents](xxxDefinitions(_))` no longer works
because Scala 3.7 tries to resolve `AST.apply` extension method first,
interfering with fastparse's method resolution.

CHANGES MADE:

build.sbt:
- Override scalaVersion to 3.7.4 in all 7 modules
- TASTy is forward-compatible (3.7.4 output works with 3.3.7 consumers)

AST.scala - Extension method restructuring:
- Moved apply(n: Int) extension into Contents companion object
- This prevents AST.apply from polluting the namespace
- Separated merge extension (with CV & CV2 intersection type) to its own block
- Other extension methods remain at top level for availability within AST

Parser files - Explicit lambda syntax:
- Changed all `methodName(_)` to `p => methodName(using p)`
- This works correctly with Scala 3.7's new extension resolution rules
- 14 parser files updated with this pattern

JVMPlatformContext.scala:
- Updated implicit Codec parameter to use explicit `using` keyword

KNOWN ISSUE - COMPILER BUG:
Both Scala 3.7.4 and 3.6.3 exhibit an infinite loop when compiling:
- hasClassSymbol recursion in Types.scala:648
- computeAndOrType recursion in SymDenotations.scala:2312
The intersection type `Contents[CV & CV2]` in the merge method triggers this.
Investigation needed to find a Scala version or workaround.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add commit hashes for both branches pushed to GitHub.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The compiler was entering an infinite loop due to duplicate Contents
opaque type definitions - one in AST.scala object and one in Contents.scala
at package level. Scala 3.7.4 has a known bug with opaque types inside
objects, particularly with intersection types.

Changes:
- Removed Contents definition from AST.scala object (lines 107-229)
- Kept only package-level Contents.scala with all extensions
- Renamed Contents.map extension to mapValue to avoid ambiguity
- Changed merge extension to return Contents[RiddlValue] instead of CV & CV2
- Fixed BASTImport: renamed kind field to kindOpt, added kind method override
- Updated all parser files to import Contents from package level
- Fixed Root.empty to use def instead of val
- Updated BASTWriter and BASTLoader to use bi.kindOpt

Result: Language module compiles successfully with Scala 3.7.4

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Changes for Scala 3.7.4 compatibility:

1. Context function syntax in tests (3 files)
   - Changed `tp.root` to `p => tp.root(using p)` for explicit context params
   - TestParserTest.scala, TestParsingRules.scala, CommonParserTest.scala

2. Contents extension imports (32+ files)
   - Added `import com.ossuminc.riddl.language.{Contents, *}` for extension methods
   - Required because Contents opaque type extensions are at package level

3. Intersection type syntax (BASTWriterPass.scala)
   - Changed `Branch[?] with WithMetaData` to `(Branch[?] & WithMetaData)`

4. Unreachable pattern matches (ReferenceMapTest.scala)
   - Removed `case x => fail(...)` after exhaustive Option matches

All 714 JVM tests pass.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Required for Scala 3.7.4 compatibility - Contents opaque type extensions
are at package level and need explicit wildcard import.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Documentation has been moved to ossum.tech. The docsite module was
causing a Scala 3.7.4 compiler bug (ClassCastException in type system).

The redirect-site in doc/ remains for URL redirects from riddl.tech.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@CLAassistant
Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@coveralls
Copy link
Copy Markdown

coveralls commented Jan 31, 2026

Pull Request Test Coverage Report for Build 21551926127

Details

  • 61 of 172 (35.47%) changed or added relevant lines in 20 files are covered.
  • 74 unchanged lines in 22 files lost coverage.
  • Overall coverage decreased (-0.8%) to 64.09%

Changes Missing Coverage Covered Lines Changed/Added Lines %
language/shared/src/main/scala/com/ossuminc/riddl/language/parsing/AdaptorParser.scala 0 1 0.0%
language/shared/src/main/scala/com/ossuminc/riddl/language/parsing/FunctionParser.scala 0 1 0.0%
language/shared/src/main/scala/com/ossuminc/riddl/language/parsing/ModuleParser.scala 0 1 0.0%
language/shared/src/main/scala/com/ossuminc/riddl/language/parsing/ProjectorParser.scala 0 1 0.0%
language/shared/src/main/scala/com/ossuminc/riddl/language/parsing/RepositoryParser.scala 0 1 0.0%
language/shared/src/main/scala/com/ossuminc/riddl/language/parsing/StreamingParser.scala 0 1 0.0%
language/shared/src/main/scala/com/ossuminc/riddl/language/parsing/CommonParser.scala 3 5 60.0%
language/shared/src/main/scala/com/ossuminc/riddl/language/bast/BASTWriter.scala 0 3 0.0%
language/shared/src/main/scala/com/ossuminc/riddl/language/bast/BASTReader.scala 0 4 0.0%
language/shared/src/main/scala/com/ossuminc/riddl/language/parsing/ParsingContext.scala 0 4 0.0%
Files with Coverage Reduction New Missed Lines %
commands/shared/src/main/scala/com/ossuminc/riddl/command/PassCommand.scala 1 71.88%
language/shared/src/main/scala/com/ossuminc/riddl/language/At.scala 1 69.81%
language/shared/src/main/scala/com/ossuminc/riddl/language/bast/BASTUtils.scala 1 24.32%
language/shared/src/main/scala/com/ossuminc/riddl/language/bast/ByteBufferReader.scala 1 57.5%
language/shared/src/main/scala/com/ossuminc/riddl/language/bast/VarIntCodec.scala 1 62.16%
language/shared/src/main/scala/com/ossuminc/riddl/language/Finder.scala 1 90.57%
language/shared/src/main/scala/com/ossuminc/riddl/language/parsing/Keywords.scala 1 82.41%
passes/shared/src/main/scala/com/ossuminc/riddl/passes/validate/DefinitionValidation.scala 1 83.95%
passes/shared/src/main/scala/com/ossuminc/riddl/passes/validate/TypeValidation.scala 1 68.82%
language/shared/src/main/scala/com/ossuminc/riddl/language/parsing/ParsingErrors.scala 2 93.55%
Totals Coverage Status
Change from base Build 21487418570: -0.8%
Covered Lines: 5713
Relevant Lines: 8914

💛 - Coveralls

The test was using Scala 3.4.2 but riddl-commands is now built with
Scala 3.7.4, causing TASTy incompatibility.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@sonarqubecloud
Copy link
Copy Markdown

@reid-spencer reid-spencer merged commit 237b0f8 into main Jan 31, 2026
8 checks passed
@reid-spencer reid-spencer deleted the feature/scala-bug branch January 31, 2026 22:42
scala-steward pushed a commit to scala-steward/riddl-1 that referenced this pull request Feb 20, 2026
Scala 3.7.4 upgrade and module reorganization
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants