Modular plugable architecture #1496
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Introduced two new documents outlining the design and quick summary for the IPC library extraction initiative. The design document details the architecture, goals, and implementation phases for creating a reusable `ipc-lib` crate, while the quick summary provides an overview of the current issues, proposed solutions, and benefits of the extraction. Additionally, updated the Cargo.lock file to include new dependencies related to the IPC library.
Added multiple storage actors including `storage_blob_reader`, `storage_blobs`, `storage_bucket`, and `storage_timehub`, along with their respective Cargo configurations. Implemented foundational structures and methods for managing blob storage and retrieval. Updated Cargo.toml files to include new dependencies and features, enhancing the overall functionality of the storage system. Additionally, modified the Cargo.lock file to reflect these changes.
…gurations Added optional storage-node features across multiple Cargo.toml files, enabling conditional compilation for storage-related actors and dependencies. Updated the implementation to include new features for managing storage nodes, including the addition of relevant dependencies and configurations. This enhancement improves modularity and allows for more flexible usage of storage functionalities within the application.
…ments Introduced comprehensive documentation for Phase 5 testing results, detailing the outcomes of build and unit tests, binary analysis, and integration verification. The results highlight successes and limitations of the modularization efforts. Additionally, added a new design document outlining a proposed plugin architecture to replace hard-coded conditional compilations with a dynamic, compile-time plugin system. This architecture aims to enhance modularity and maintainability while ensuring zero runtime overhead. Updated multiple Cargo.toml files to support new feature configurations for the plugin system.
…framework Introduced a detailed implementation plan for a new plugin system, outlining design decisions, phases, and tasks for integrating a multi-trait hook system with zero-cost generics. The plan includes the creation of various plugin traits (Executor, MessageHandler, Genesis, Service, and CLI) and their respective implementations, along with a no-op plugin bundle for testing. Additionally, the core Fendermint components have been updated to support generics over the PluginBundle, ensuring modularity and flexibility. This commit sets the foundation for future plugin development and integration.
… framework Introduced a comprehensive module system for Fendermint, enabling functionality extension through a trait-based architecture. This commit includes the implementation of five core traits: ExecutorModule, MessageHandlerModule, GenesisModule, ServiceModule, and CliModule, along with their respective no-op implementations. A new crate, `fendermint_module`, has been created, and the Cargo configurations have been updated to support this modular architecture. Additionally, a detailed documentation file has been added to outline the module system's design, features, and usage examples, setting the foundation for future module development and integration.
This commit adds the `storage_node_executor` dependency to the `fendermint/module` crate, enabling enhanced functionality for managing storage nodes. Additionally, the `fendermint_module` is now included in the `fendermint/vm/interpreter` crate, facilitating the integration of the module system with the interpreter. The changes improve modularity and prepare the codebase for further development of the module system, ensuring a more flexible architecture moving forward.
This commit completes the implementation of the Fendermint module system, ensuring full functionality and extensibility. Key updates include the addition of the `fendermint_module` dependency across various crates, integration of the `NoOpModuleBundle` in the application logic, and enhancements to the `FvmExecState` to support module lifecycle hooks. The changes improve modularity, facilitate better state management, and prepare the codebase for future module development. Comprehensive documentation has also been added to outline the completed module system's design and usage.
…system This commit introduces the `StorageNodeModule`, integrating storage-node functionality into the Fendermint module system. Key updates include the addition of the `storage_node_module` dependency in the Cargo configurations, modifications to the `default_module.rs` for conditional module selection, and enhancements to the application logic to utilize the new module. Comprehensive documentation has been added to outline the module's implementation and usage, ensuring a robust foundation for future development and integration of storage-node features.
This commit introduces a build script for auto-discovering plugins in the Fendermint application, eliminating hardcoded plugin references. Key changes include the addition of a new `build.rs` script that scans the `plugins/` directory, generates glue code for enabled plugins, and updates the Cargo configurations to support dynamic loading. The `StorageNodeModule` is now integrated as a plugin, enhancing modularity and allowing for easier extension of functionalities. Comprehensive documentation has been added to guide future plugin development and usage.
This commit introduces a plugin discovery system for the Fendermint application, allowing for dynamic loading of plugins. Key changes include the addition of a new `plugins.rs` file for managing plugins, updates to the `Cargo.toml` files to include the `tracing` dependency, and the creation of documentation files (`PLUGIN_EXTRACTION_COMPLETE.md` and `PLUGIN_EXTRACTION_STATUS.md`) to guide future plugin development. These enhancements improve modularity and provide a clearer framework for integrating additional functionalities.
…tegration This commit introduces a comprehensive documentation file, `FINAL_STATUS.md`, detailing the achievements and remaining work related to the plugin extraction process in the Fendermint application. Key highlights include the successful implementation of a plugin-free core interpreter, the status of the plugin infrastructure, and the completion of the `StorageNodeModule`. Additionally, the application logic has been updated to conditionally load modules based on enabled features, enhancing modularity and providing a clear path forward for future plugin integration. This documentation serves as a valuable resource for understanding the current state and future directions of the project.
…ementation status This commit introduces several new documentation files, including `BUILD_VERIFICATION.md`, `IMPLEMENTATION_COMPLETE.md`, `PLUGIN_SUMMARY.md`, `PLUGIN_SYSTEM_SUCCESS.md`, `PLUGIN_USAGE.md`, and `QUICK_START_PLUGINS.md`. These documents provide detailed insights into the verification process, implementation status, and usage guidelines for the plugin system in the Fendermint application. The updates enhance the overall documentation quality, ensuring clarity and accessibility for future development and integration efforts.
This commit introduces a comprehensive reorganization of the IPC documentation, moving files into a structured hierarchy within the `docs/` directory. Key additions include `DOCUMENTATION_REORGANIZATION.md` summarizing the changes, a new `README.md` for the IPC documentation, and various development and feature-specific documentation files. This restructuring aims to improve accessibility and clarity for contributors, ensuring that documentation is easy to navigate and maintain as the project evolves.
This commit removes the `recall_sol_facade` dependency from the `Cargo.toml` file, streamlining the project's dependency management. The change reflects an update to the project's structure, ensuring that only necessary dependencies are included.
This commit reorganizes the storage node actors by moving several components to a new structure under `storage-node/actors/`. Key changes include the addition of new actors such as `storage_blob_reader`, `storage_adm`, and `storage_timehub`, along with their respective dependencies in the `Cargo.toml` files. The `Cargo.lock` has been updated to reflect these changes, ensuring all dependencies are correctly managed. Additionally, several unused files have been removed to streamline the project structure. This refactor enhances modularity and prepares the codebase for future development.
… report for storage plugin migration This commit adds several key documents to guide the ongoing migration of storage functionality to a plugin-based architecture. The `ARCHITECTURE_DECISION_NEEDED.md` outlines the context, options, and recommendations for plugin isolation levels, while `PHASE_1_COMPLETE.md` details the successful completion of phase 1, including actor interface migration and trait extensions. Additionally, `STORAGE_DEPENDENCIES_MAP.md` provides a visual representation of storage dependencies within the Fendermint core, and `STORAGE_MIGRATION_PROGRESS.md` tracks the progress and remaining tasks for the migration. These documents enhance clarity and direction for future development efforts.
This commit finalizes the migration of all storage-related components from the core Fendermint codebase to a modular plugin system. Key changes include the removal of the `fendermint_vm_storage_resolver` and the relocation of various storage actors and interfaces to `storage-node/actors/` and `plugins/storage-node/`. The `Cargo.lock` and `Cargo.toml` files have been updated to reflect these changes, ensuring proper dependency management. Additionally, comprehensive documentation has been created to summarize the migration process and verify the successful implementation of a truly modular architecture. This enhances the overall maintainability and extensibility of the project.
This commit completes the migration of storage-node functionality to a modular plugin architecture, ensuring no hardcoded references remain in the core Fendermint codebase. Key changes include the removal of the `iroh-blobs` dependency, the relocation of storage-specific types to `plugins/storage-node/src/topdown_types.rs`, and the introduction of a new `service_resources` module to manage storage resources generically. Comprehensive documentation has been created to summarize the migration process, verify the successful implementation, and outline the architecture's modularity. This enhances maintainability and prepares the codebase for future extensibility.
…tion This commit completes the integration of the storage-node plugin, ensuring all dependencies are correctly managed and the module system is fully operational. Key changes include the addition of the `rand` dependency for testing, updates to the `Cargo.toml` for proper dependency management, and the removal of unused test code in `lib.rs`. Comprehensive documentation has been created to summarize the completion status, verify successful implementation, and outline the architecture's modularity, paving the way for future extensibility and integration testing.
This commit introduces two new documentation files: `MODULE_SYSTEM_BUILD_SUCCESS.md` and `STORAGE_TESTING_NEXT_STEPS.md`. The build success report details the completion of the module system implementation, including the resolution of all compilation errors and successful test results. It outlines the architecture, build verification, and next steps for integration testing. The storage testing document provides a roadmap for verifying storage functionality, including options for testing with Docker and Anvil, and highlights the current status of the testing framework. These additions enhance the project's documentation and prepare for upcoming testing phases.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.