Update documentation for contexts

Author: altavir

README.md

@@ -50,7 +50,7 @@ In this section, we will try to cover DataForge main ideas in the form of questi
 
 * **Modularisation**. Contrary to lot other frameworks, DataForge is intrinsically modular. The mandatory part is a rather tiny core module. Everything else could be customized.
 
-* **Context encapsulation**. Every DataForge task is executed in some context. The context isolates environment for the task and also works as dependency injection base and specifies interaction of the task with the external world.
+* **Context encapsulation**. Every DataForge task is executed in some context. The context isolates environment for the task and also works as dependency injection base and specifies interaction of the task with the external world. See [Context](../dataforge-context/docs/context.md) and [Plugin Mechanics](../dataforge-context/docs/plugins.md) for more details.
 
 ### Misc
 
@@ -71,6 +71,12 @@ In this section, we will try to cover DataForge main ideas in the form of questi
 > Context and provider definitions
 >
 > **Maturity**: DEVELOPMENT
+>
+> **Features:**
+> - [context](dataforge-context/docs/context.md) : Context is a single extension point for all applications. It allows to configure application and handle its lifecycle.
+> - [plugins](dataforge-context/docs/plugins.md) : Plugin system is a powerful dependency injection and extensibility mechanism. 
+> - [context-serialization](dataforge-context/docs/context-serialization.md) : Serializer aggregation for context. Each plugin could supply its own serializer module. 
+
 
 ### [dataforge-data](dataforge-data)
 >

dataforge-context/README.md

@@ -2,11 +2,18 @@
 
 Context and provider definitions
 
+## Features
+
+ - [context](docs/context.md) : Context is a single extension point for all applications. It allows to configure application and handle its lifecycle.
+ - [plugins](docs/plugins.md) : Plugin system is a powerful dependency injection and extensibility mechanism. 
+ - [context-serialization](docs/context-serialization.md) : Serializer aggregation for context. Each plugin could supply its own serializer module. 
+
+
 ## Usage
 
 ## Artifact:
 
-The Maven coordinates of this project are `space.kscience:dataforge-context:0.11.0-dev-1`.
+The Maven coordinates of this project are `space.kscience:dataforge-context:0.11.0-dev-2`.
 
 **Gradle Kotlin DSL:**
 ```kotlin
@@ -16,6 +23,6 @@ repositories {
 }
 
 dependencies {
-    implementation("space.kscience:dataforge-context:0.11.0-dev-1")
+    implementation("space.kscience:dataforge-context:0.11.0-dev-2")
 }
 ```

dataforge-context/build.gradle.kts

@@ -25,7 +25,22 @@ kscience {
 readme {
     maturity = space.kscience.gradle.Maturity.DEVELOPMENT
 
-    feature("context-serialization","src/docs/context-serialization.md"){
+    feature("context", ref = "docs/context.md"){
+        """
+            Context is a single extension point for all applications. It allows to configure application and handle its lifecycle.
+        """.trimIndent()
+    }
+
+    feature("plugins", ref = "docs/plugins.md"){
+        """
+            Plugin system is a powerful dependency injection and extensibility mechanism. 
+            It allows to construct required capabilities for specific application in a single space. 
+            Also it provides a capability sharing bus. For example one plugin could provide factories for dynamic 
+            instantiation in another plugin. 
+        """.trimIndent()
+    }
+
+    feature("context-serialization","docs/context-serialization.md"){
         """
             Serializer aggregation for context. Each plugin could supply its own serializer module. 
             Those modules are aggregated into one serializer module in context.

dataforge-context/docs/context.md

@@ -0,0 +1,124 @@
+# Context
+
+Context is a central point in dependency injection and lifecycle management. Its extensibility is based on [plugins](plugins.md). It also serves as a `CoroutineScope` and manages lifecycle of the application. Closing a component context terminates it.
+
+## Global Context
+
+`Global` is a special context that is always available. It should not be used outside of tests. Similar to `GlobalScope` in coroutines, it deos not provide proper lifecycle management. `Global` could be closed, but it usually means termination of whole the application.
+
+## Building a Context
+
+In DataForge, the `Context` represents the environment where operations are executed. It manages plugins, properties, and coroutines. Contexts are organized in a hierarchy, with `Global` at the root.
+
+### Basic Context Creation
+
+The simplest way to create a context is to use the `Context` factory function:
+
+```kotlin
+val myContext = Context("my-context")
+```
+
+This creates a new child context of the `Global` context with the name `my-context`. If the context with the given name already exists, throw an exception.
+
+### The Context Builder 
+
+For more complex configurations, you can use the `ContextBuilder`. This is available through the `Context` factory function or by calling `buildContext` on an existing context.
+
+```kotlin
+val myContext = Context("my-context") {
+    // Set a property
+    properties {
+        "myProperty" put 42
+    }
+
+    // Add a plugin by its factory
+    plugin(MyPluginFactory) {
+        // Plugin-specific configuration
+        "pluginConfig" put "someValue"
+    }
+
+    // Configure the coroutine context
+    coroutineContext(Dispatchers.Default)
+}
+```
+
+### Properties
+
+Properties in a context work like environment variables. They are inherited from the parent context. In the builder, you can set them using the `properties` block:
+
+```kotlin
+properties {
+    "node" put {
+        "value" put "something"
+    }
+}
+```
+
+### Plugins
+
+Plugins provide the actual functionality to the context. You can add plugins to a context during construction:
+
+- **By Factory**: The most common way. Allows providing configuration metadata.
+  ```kotlin
+  plugin(MyPluginFactory) {
+      "key" put "value"
+  }
+  ```
+- **By Tag**: If the factory is available in the parent context's registry.
+  ```kotlin
+  plugin("my-plugin", group = "my-group")
+  ```
+- **Existing Instance**: Add an already created plugin instance.
+  ```kotlin
+  plugin(myPluginInstance)
+  ```
+
+### Coroutine Context
+
+Each `Context` is a `CoroutineScope`. You can customize its `CoroutineContext` during building:
+
+```kotlin
+coroutineContext(SupervisorJob() + Dispatchers.IO)
+```
+
+By default, a new context inherits the coroutine context from its parent and adds a `SupervisorJob`.
+
+## Child Contexts
+
+You can create child contexts from any existing context:
+
+```kotlin
+val child = parentContext.buildContext("child") {
+    // ...
+}
+```
+
+Child context inherits all properties and plugins from their parent but could override them. For example, if a child provides a plugin with the same tag as a parent, a child plugin is used. In some cases child plugins could use some information from parent plugins. Parent context usually does not use child plugin information.
+
+Closing parent context automatically closes all its child contexts.
+
+If a child context with the same name already exists, `buildContext` will return it (potentially deriving it if additional configuration is provided).
+
+## Example: Complex Context Setup
+
+```kotlin
+val analysisContext = Context("analysis") {
+    properties {
+        "threads" put 4
+    }
+    
+    plugin(LogManager)
+    
+    plugin(StoragePlugin) {
+        "path" put "./data"
+    }
+}
+
+val subTaskContext = analysisContext.buildContext("subtask") {
+    properties {
+        "priority" put "high"
+    }
+}
+```
+
+In this example, `subTaskContext` will have access to the `LogManager` and `StoragePlugin` from its parent, as well as the `threads` property.

dataforge-context/docs/plugins.md

@@ -0,0 +1,83 @@
+# DataForge Plugin Mechanics
+
+Plugins are the primary way to extend the functionality of a DataForge `Context`. They provide runtime features, services, and configurations that can be shared across different tasks executed within a context.
+
+## The Plugin Interface
+
+A plugin in DataForge is defined by the `Plugin` interface. Every plugin must have:
+- A `PluginTag`: Contains the `name`, `group`, and `version` of the plugin.
+- A `Meta` configuration: Allows the plugin to be configured.
+- A reference to the `Context` it is attached to (via `ContextAware`).
+
+```kotlin
+public interface Plugin : Named, ContextAware, Provider, MetaRepr {
+    public val tag: PluginTag
+    public val meta: Meta
+    public fun dependsOn(): Map<PluginFactory<*>, Meta>
+    public fun attach(context: Context)
+    public fun detach()
+    public val isAttached: Boolean
+    // ...
+}
+```
+
+## Plugin Lifecycle
+
+The lifecycle of a plugin consists of the following stages:
+
+1. **Create**: The plugin instance is created and configured, usually via a `PluginFactory`.
+2. **Attach**: The plugin is attached to a `Context`. This is when the plugin can register its services in the context and initialize itself. The `attach(context)` method is called by the `PluginManager`.
+3. **Detach**: The plugin is removed from the `Context`. It should clean up any resources and registrations.
+4. **Destroy**: The plugin instance is discarded.
+
+## Plugin Identification (PluginTag)
+
+Plugins are identified by a `PluginTag`, which consists of:
+- `name`: The unique name of the plugin within its group.
+- `group`: (Optional) The organization or project the plugin belongs to.
+- `version`: (Optional) The version of the plugin. Version could be used for remote plugin management and for consistent result storage.
+
+The `PluginManager` uses these tags to find and manage plugins, ensuring that no two plugins with the same tag are loaded into the same context.
+
+## Dependency Management
+
+Plugins can declare dependencies on other plugins. Before a plugin is attached, all its dependencies must be present and attached to the context (or its parents).
+
+### Declaring Dependencies
+In `AbstractPlugin`, dependencies are declared using the `require` function:
+
+```kotlin
+class MyPlugin(meta: Meta) : AbstractPlugin(meta) {
+    // Declares a dependency on AnotherPlugin
+    val anotherPlugin by require(AnotherPluginFactory)
+}
+```
+
+The `require` function returns a delegate that lazily fetches the dependent plugin from the context once the plugin is attached.
+
+## Plugin Manager
+
+Each `Context` has a `PluginManager` that:
+- Stores attached plugins.
+- Manages the plugin lifecycle.
+- Handles plugin lookups (including searching in parent contexts).
+
+### Requesting a Plugin
+You can request a plugin from a context using the `request` extension function:
+
+```kotlin
+val myPlugin = context.request(MyPluginFactory)
+```
+
+If the plugin is already present in the context or its parents, it is returned. Otherwise, a new child context is created with the requested plugin.
+
+## Plugin Factories
+
+Plugins are typically created using a `PluginFactory`. This allows the framework to instantiate plugins dynamically based on metadata or explicit requests. Also it allows using type-safe reference to plugins via their types and dependency injection.
+
+```kotlin
+public interface PluginFactory<T : Plugin> : Factory<T> {
+    public val tag: PluginTag
+    override fun build(context: Context, meta: Meta): T
+}
+```

dataforge-context/src/commonMain/kotlin/space/kscience/dataforge/context/Context.kt

@@ -81,14 +81,15 @@ public open class Context internal constructor(
     private val childrenContexts = HashMap<Name, Context>()
 
     /**
-     * Get and validate existing context or build and register a new child context.
-     * @param name the relative (tail) name of the new context. If null, uses context hash code as a marker.
+     * Build and register a new child context.
+     * @param name the relative (tail) name of the new context. If null, use context hash code as a marker.
      */
     @OptIn(DFExperimental::class)
     @ThreadSafe
     public fun buildContext(name: Name? = null, block: ContextBuilder.() -> Unit = {}): Context {
         val existing = name?.let { childrenContexts[name] }
-        return existing?.deriveContext(block = block) ?: ContextBuilder(this, name).apply(block).build().also {
+        if (existing != null) error("Context $name already exists in $this")
+        return ContextBuilder(this, name).apply(block).build().also {
             childrenContexts[it.name] = it
         }
     }
@@ -109,6 +110,11 @@ public open class Context internal constructor(
         "plugins" putIndexed plugins.map { it.toMeta() }
     }
 
+    override fun toString(): String {
+        val parentString = if(parent == Global) "" else ", parent=$parent"
+        return "Context(name=$name$parentString)"
+    }
+
     /**
      * A lazily initialized property that defines the serialization module for the current context.
      *

docs/README-TEMPLATE.md

@@ -50,7 +50,7 @@ In this section, we will try to cover DataForge main ideas in the form of questi
 
 * **Modularisation**. Contrary to lot other frameworks, DataForge is intrinsically modular. The mandatory part is a rather tiny core module. Everything else could be customized.
 
-* **Context encapsulation**. Every DataForge task is executed in some context. The context isolates environment for the task and also works as dependency injection base and specifies interaction of the task with the external world.
+* **Context encapsulation**. Every DataForge task is executed in some context. The context isolates environment for the task and also works as dependency injection base and specifies interaction of the task with the external world. See [Context](../dataforge-context/docs/context.md) and [Plugin Mechanics](../dataforge-context/docs/plugins.md) for more details.
 
 ### Misc