Skip to content

Comments

Add metals-extract module for dependency extraction from build tools#8132

Draft
olafurpg wants to merge 1 commit intomain-v2from
olafurpg/mbt-extract
Draft

Add metals-extract module for dependency extraction from build tools#8132
olafurpg wants to merge 1 commit intomain-v2from
olafurpg/mbt-extract

Conversation

@olafurpg
Copy link
Member

Previously, users had to install Bun and run the mbt-extract.ts script to extract build data. This commit introduces a new alternative to create mbt.json, a Scala CLI using more official APIs:

  • Maven Artifact Resolver API (Eclipse Aether) for Maven projects
  • Gradle Tooling API for Gradle projects
  • Parses maven_install.json for Bazel projects (bazel query/aspects for full support)

Implements extractors for Maven, Gradle, and Bazel projects that generate mbt.json files containing dependency information for Metals.

MavenExtractor features:

  • Uses Maven Artifact Resolver API (Eclipse Aether)
  • Support for parent POM inheritance
  • Support for multi-module projects (recursive submodule processing)
  • DependencyManagement version resolution
  • Property placeholder resolution (${project.version}, etc.)
  • Transitive dependency resolution
  • Skips test, provided, and system scope dependencies
  • Uses ~/.m2/repository (or M2_REPO env var) as local cache
  • Supports custom repositories defined in pom.xml

GradleExtractor features:

  • Uses Gradle Tooling API with EclipseProject model
  • Support for build.gradle and build.gradle.kts
  • Auto-detects and uses Gradle wrapper when available
  • Recursively processes child projects in multi-project builds
  • Finds sources JARs in Gradle cache structure
  • Deduplicates dependencies by artifact ID

BazelExtractor features:

  • Support for Bazel 7.x Bzlmod naming (~ separators)
  • Support for Bazel 8.x Bzlmod naming (++ and + separators)
  • Support for legacy dependency_tree format (rules_jvm_external v4)
  • Support for WORKSPACE mode
  • Process ALL maven_install*.json files and merge results
  • Download missing JARs from Maven Central using URLs in JSON
  • Fallback to .m2 cache when Bazel JARs not found

CLI options:

  • --scala-version Filter by Scala version (2.12, 2.13, or 3) Excludes artifacts with non-matching Scala suffixes (_2.12, _2.13, _3)

Tested against:

  • bazel-bsp: 100 deps (Bazel 7.0.2)
  • rules_jvm_external/bzlmod: 66 deps (Bazel 8.3.1)
  • salesforce/rules_spring: 183 deps (Bazel 7.4.1)
  • linafx/digital-assets: 322 deps with --scala-version 2.13
  • ValdemarGr/mezel: 23 deps with --scala-version 2.13

Previously, users had to install Bun and run the mbt-extract.ts script to
extract build data. This commit introduces a Scala CLI using official APIs:
- Maven Artifact Resolver API (Eclipse Aether) for Maven projects
- Gradle Tooling API for Gradle projects
- Parses maven_install.json for Bazel projects (bazel query/aspects for full support)

Implements extractors for Maven, Gradle, and Bazel projects that generate
mbt.json files containing dependency information for Metals.

MavenExtractor features:
- Uses Maven Artifact Resolver API (Eclipse Aether)
- Support for parent POM inheritance
- Support for multi-module projects (recursive submodule processing)
- DependencyManagement version resolution
- Property placeholder resolution (${project.version}, etc.)
- Transitive dependency resolution
- Skips test, provided, and system scope dependencies
- Uses ~/.m2/repository (or M2_REPO env var) as local cache
- Supports custom repositories defined in pom.xml

GradleExtractor features:
- Uses Gradle Tooling API with EclipseProject model
- Support for build.gradle and build.gradle.kts
- Auto-detects and uses Gradle wrapper when available
- Recursively processes child projects in multi-project builds
- Finds sources JARs in Gradle cache structure
- Deduplicates dependencies by artifact ID

BazelExtractor features:
- Support for Bazel 7.x Bzlmod naming (~ separators)
- Support for Bazel 8.x Bzlmod naming (++ and + separators)
- Support for legacy dependency_tree format (rules_jvm_external v4)
- Support for WORKSPACE mode
- Process ALL maven_install*.json files and merge results
- Download missing JARs from Maven Central using URLs in JSON
- Fallback to .m2 cache when Bazel JARs not found

CLI options:
- --scala-version <ver>  Filter by Scala version (2.12, 2.13, or 3)
  Excludes artifacts with non-matching Scala suffixes (_2.12, _2.13, _3)

Tested against:
- bazel-bsp: 100 deps (Bazel 7.0.2)
- rules_jvm_external/bzlmod: 66 deps (Bazel 8.3.1)
- salesforce/rules_spring: 183 deps (Bazel 7.4.1)
- linafx/digital-assets: 322 deps with --scala-version 2.13
- ValdemarGr/mezel: 23 deps with --scala-version 2.13

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
olafurpg

This comment was marked as resolved.

.configure(JavaPcSettings.settings(sharedSettings))
.dependsOn(interfaces, mtagsShared, `semanticdb-javac`, turbine)

lazy val `metals-extract` = project
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test comment

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reply

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant