Move Forest storage system into a plugin

[jflo]

Overview

Move the Forest world state storage implementation out of the core ethereum/core module and into a standalone plugin, following the same pattern as the RocksDB storage plugin. This will improve modularity, reduce core dependencies, and establish a pattern for future storage format plugins.

Task Breakdown

This is a parent issue tracking the overall Forest plugin migration. The work is broken down into phases with separate child issues.

Phase 1: Plugin System Prerequisites ⚙️

Goal: Prepare the plugin system to support pluggable world state archives

• [Plugin System] Create WorldStateArchiveProvider interface #9448 - Create WorldStateArchiveProvider interface (plugin-api)
• [Plugin System] Create WorldStateArchiveService for provider registration #9449 - Create WorldStateArchiveService interface (plugin-api)
• [Plugin System] Implement WorldStateArchiveService registry in core #9450 - Implement WorldStateArchiveService registry (core)
• [Plugin System] Refactor WorldStateStorageCoordinator to support provider pattern #9451 - Refactor WorldStateStorageCoordinator to remove Forest coupling
• [Plugin System] Update BesuControllerBuilder to use provider pattern #9452 - Update BesuControllerBuilder to use provider pattern
• [Plugin System] Update KeyValueStorageProvider to use provider pattern #9453 - Update KeyValueStorageProvider (discussion/deferred)

Status: These changes prepare the core to work with pluggable storage formats without breaking existing functionality.

Phase 2: Forest Plugin Creation 🔌

Goal: Create the Forest plugin module and move Forest classes

• Create plugins/forest/ module structure
• Implement ForestStoragePlugin class
• Implement ForestWorldStateArchiveProvider
• Move ForestWorldStateKeyValueStorage to plugin
• Move ForestMutableWorldState to plugin
• Move ForestWorldStateArchive to plugin
• Update package references and dependencies

Status: Not started - depends on Phase 1 completion

Phase 3: Integration & Testing 🧪

Goal: Integrate Forest plugin and ensure everything works

• Bundle Forest plugin in default distribution
• Update test infrastructure to load plugin
• Update all Forest-dependent tests
• Add plugin loading/registration tests
• Test error cases (missing plugin)
• Migration testing (embedded → plugin)

Status: Not started - depends on Phase 2 completion

Phase 4: Documentation & Cleanup 📚

Goal: Document the change and establish deprecation plan

• Update architecture documentation
• Create plugin development guide
• Document Forest plugin usage
• Create migration guide for users
• Add deprecation warnings
• Update CLI help text

[sridhar-panigrahi]

Hi @jflo — I've been reading through this Phase 1 plan and the related sub-issues and wanted to offer some help, if there's a piece that would genuinely be useful right now.

For context: I'm planning to apply for the LFDT 2026 mentorship Project #82 (Plugin API refactoring, mentored by @fab-10), which is the broader architectural work around the same module. I wanted to engage openly on the concrete engineering epic here so it's clear I'm not trying to cherry-pick anything from either scope — just keen to help where it's useful.

I can see @AliZDev-v0 is driving #9448 through the #9561 split series (#9617 and #9897 merged, #10135 in review) and is the natural owner for the Provider interface itself — I don't want to step on that work. But I also don't want to pick a downstream sub-issue (#9449–9453) blindly since they all depend on the interface landing first, and Ali's the natural owner for the immediate next piece too.

A few directions I'd be happy to take on, whichever fits what you actually need:

1. Phase 2 prep — the Forest plugin module itself (plugins/forest/ scaffolding, ForestStoragePlugin skeleton, build wiring). Nothing on the list is blocked by Phase 1's remaining PRs if we design the plugin entrypoint against the interface shape Ali is converging on.
2. Test / migration scaffolding — plugin loading and registration tests for the new provider pattern, or embedded-to-plugin migration tests listed under Phase 3. These can be written against the interface-as-designed and land once the implementation is in.
3. A parallel prerequisite that isn't already on Ali's backlog, if you see one I've missed.
4. Or genuinely: "wait until Phase 1 lands" — also a fine answer, just wanted to surface myself so you know there's a second pair of hands for Phase 2/3 when you're ready.

Happy to coordinate with @AliZDev-v0 directly if that's easier. No rush.