You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
TODO: Integrate this documentation within the larger MeTTa Book
6
6
7
-
Modules are implementations of free-standing MeTTa functionality that can be imported into other MeTTa modules or programs. Modules may be implemented in MeTTa code itself, but they may also include functionality implemented with, or linked from host languages such as Rust, C, or Python. Modules may include additional files and resources as well.
7
+
Modules are implementations of free-standing MeTTa functionality that can be
8
+
imported into other MeTTa modules or programs. Modules may be implemented in
9
+
MeTTa code itself, but they may also include functionality implemented with, or
10
+
linked from host languages such as Rust, C, or Python. Modules may include
11
+
additional files and resources as well.
8
12
9
-
**_NOTE:_** Importantly, a module can have sub-module dependencies, aka "downward" dependencies, but it cannot have "upward" dependencies, ie. dependencies on the client code importing the module.
13
+
**_NOTE:_** Importantly, a module can have sub-module dependencies, aka
14
+
"downward" dependencies, but it cannot have "upward" dependencies, ie.
15
+
dependencies on the client code importing the module.
10
16
11
17
### What *is* a Module?
12
18
13
-
Fundamentally a module in a persistent encapsulation of a context within which MeTTa code can run. Every module has a unique [Space] (and also a [Tokenizer], for now). For MeTTa code running within the context of a module, the `&self` token will resolve to the module's space.
19
+
Fundamentally a module in a persistent encapsulation of a context within which
20
+
MeTTa code can run. Every module has a unique [Space] (and also a [Tokenizer],
21
+
for now). For MeTTa code running within the context of a module, the `&self`
22
+
token will resolve to the module's space.
14
23
15
-
A loaded module is represented with the [MettaMod] struct. In addition to a [Space] and a [Tokenizer], a module may also contain a filesystem path to the module's resources, the sub-modules imported by the module, a [ModuleDescriptor] object, and a [PkgInfo] struct, the latter of which are documented in the [Package Management](#package-management) section.
24
+
A loaded module is represented with the [MettaMod] struct. In addition to a
25
+
[Space] and a [Tokenizer], a module may also contain a filesystem path to the
26
+
module's resources, the sub-modules imported by the module, a
27
+
[ModuleDescriptor] object, and a [PkgInfo] struct, the latter of which are
28
+
documented in the [Package Management](#package-management) section.
16
29
17
-
To execute code in the context of any loaded module, use the [RunnerState::new_with_module] method.
30
+
To execute code in the context of any loaded module, use the
31
+
[RunnerState::new_with_module] method.
18
32
19
33
### Loading a Module
20
34
21
-
Modules are loaded into a [Metta] runner using one of the module loading methods: [Metta::load_module_direct], [Metta::load_module_at_path], or [RunContext::load_module]. Loaded modules are referred to with a [ModId].
35
+
Modules are loaded into a [Metta] runner using one of the module loading
36
+
methods: [Metta::load_module_direct], [Metta::load_module_at_path], or
37
+
[RunContext::load_module]. Loaded modules are referred to with a [ModId].
22
38
23
-
Fundamentally, all modules are loaded via a loader object that implements the [ModuleLoader] trait. Irrespective of the module's original format or host language, a loader object's [ModuleLoader::load] function ultimately loads the module into the runner.
39
+
Fundamentally, all modules are loaded via a loader object that implements the
40
+
[ModuleLoader] trait. Irrespective of the module's original format or host
41
+
language, a loader object's [ModuleLoader::load] function ultimately loads the
42
+
module into the runner.
24
43
25
44
#### Module Names & Name Paths
26
45
27
-
Each loaded module must have a name. A legal module name is an ascii string, containing only alpha-numeric characters plus `_` and `-`.
46
+
Each loaded module must have a name. A legal module name is an ascii string,
47
+
containing only alpha-numeric characters plus `_` and `-`.
28
48
29
-
If module loading is initiated through the MeTTa `import!` operation or the corresponding [RunContext::load_module] API call, then the module name will be used to identify the module to load, following the logic in the [Module Name Resolution](#module-name-resolution) section.
49
+
If module loading is initiated through the MeTTa `import!` operation or the
50
+
corresponding [RunContext::load_module] API call, then the module name will be
51
+
used to identify the module to load, following the logic in the [Module Name
52
+
Resolution](#module-name-resolution) section.
30
53
31
-
Direct module-loading API calls such as [Metta::load_module_direct], [Metta::load_module_at_path], or [Metta::load_module_alias] all take an explicit module name or name path.
54
+
Direct module-loading API calls such as [Metta::load_module_direct],
55
+
[Metta::load_module_at_path], or [Metta::load_module_alias] all take an
56
+
explicit module name or name path.
32
57
33
-
Upon loading, the module is subsequently placed into the module name path hierarchy, where `top` is always the name for the top-module in the runner and the `':'` character acts as the separator. An example module name path looks like `top:mod1:sub_a`, and an example hierarchy is illustrated below.
58
+
Upon loading, the module is subsequently placed into the module name path
59
+
hierarchy, where `top` is always the name for the top-module in the runner and
60
+
the `':'` character acts as the separator. An example module name path looks
61
+
like `top:mod1:sub_a`, and an example hierarchy is illustrated below.
34
62
35
63
```
36
64
top = 0
@@ -42,19 +70,29 @@ top = 0
42
70
└─sub_b = 6
43
71
```
44
72
45
-
In addition, the `self` token may be used at the beginning of a module name path to refer to the currently running module context. In the context of the top module, `top` and `self` should have an identical meaning.
73
+
In addition, the `self` token may be used at the beginning of a module name
74
+
path to refer to the currently running module context. In the context of the
75
+
top module, `top` and `self` should have an identical meaning.
46
76
47
77
Step-by-step, the MeTTa code:
48
78
```
49
79
!(import! &self some_module)
50
80
```
51
-
will cause the name `some_module` to be resolved into a specific module instance; if that module is not yet loaded then it will be, and finally the module will be imported, in totality, into the currently executing module (context).
81
+
will cause the name `some_module` to be resolved into a specific module
82
+
instance; if that module is not yet loaded then it will be, and finally the
83
+
module will be imported, in totality, into the currently executing module
**NOTE**: The same loaded module (with the same ModId) may appear multiple times in the hierarchy, sometimes with different names. This could be the effect of an "import as" operation or making a module alias.
88
+
**NOTE**: The same loaded module (with the same ModId) may appear multiple
89
+
times in the hierarchy, sometimes with different names. This could be the
90
+
effect of an "import as" operation or making a module alias.
56
91
57
-
**NOTE**: The same module name may occur in multiple places in the hierarchy, and there is no guaranteed a name will always refer to the same module. However, within a given node of the module name hierarchy, a module name will always be unique.
92
+
**NOTE**: The same module name may occur in multiple places in the hierarchy,
93
+
and there is no guaranteed a name will always refer to the same module.
94
+
However, within a given node of the module name hierarchy, a module name will
95
+
always be unique.
58
96
59
97
### Importing a Module
60
98
@@ -63,20 +101,30 @@ A module is imported into another module using one of the import methods:
63
101
-[MettaMod::import_all_from_dependency], corresponding to `import * from module`
64
102
-[MettaMod::import_item_from_dependency_as], corresponding to `import item from module as name`
65
103
66
-
Once imported, a sub-module is accessed via an embedded [Space] atom in the destination module's space, [Tokenizer] entries for accessing the source module's space, tokens, or a combination of the two.
104
+
Once imported, a sub-module is accessed via an embedded [Space] atom in the
105
+
destination module's space, [Tokenizer] entries for accessing the source
106
+
module's space, tokens, or a combination of the two.
67
107
68
108
#### Behavior WIP
69
109
70
-
TODO: The precise semantics of importing (in other words, linking) are still under discussion and development. Specifically we may wish to provide a mechanism to explicitly declare what is exported from a module (and thus available for import). This would be similar to the `export` key words in some languages, or the `pub` visibility qualifiers in Rust.
110
+
TODO: The precise semantics of importing (in other words, linking) are still
111
+
under discussion and development. Specifically we may wish to provide a
112
+
mechanism to explicitly declare what is exported from a module (and thus
113
+
available for import). This would be similar to the `export` key words in some
114
+
languages, or the `pub` visibility qualifiers in Rust.
71
115
72
-
In addition, some changes will be needed so that [Tokenizer] entries can be imported and accessed between modules. Currently Tokenizer entries are only imported using the [MettaMod::import_all_from_dependency] method, and the mechanism at work may lead to unreachable Tokenizer entries.
116
+
In addition, some changes will be needed so that [Tokenizer] entries can be
117
+
imported and accessed between modules. Currently Tokenizer entries are only
118
+
imported using the [MettaMod::import_all_from_dependency] method, and the
119
+
mechanism at work may lead to unreachable Tokenizer entries.
More discussion on these topics is in the section: "Importing / Linking" of the `modules_internal_discussion.md` file.
126
+
More discussion on these topics is in the section: "Importing / Linking" of the
127
+
`modules_internal_discussion.md` file.
80
128
81
129
## Package Management
82
130
@@ -85,38 +133,66 @@ Package Management is the set of features that allow for:
85
133
- expressing version requirements and selecting compatible versions
86
134
- loading modules from files or other locations
87
135
88
-
Modularity is a fundamental and inseparable aspect of the MeTTa runner, but Package Management features could be optional.
136
+
Modularity is a fundamental and inseparable aspect of the MeTTa runner, but
137
+
Package Management features could be optional.
89
138
90
139
### Module File Formats
91
140
92
-
Modules may be loaded from files and other file-system-like resources (for example, a github repo) using the objects that implement the [FsModuleFormat] trait. This trait contains the interface to interpret a file or a directory and instantiate a [ModuleLoader] to load the module(s) contained within.
141
+
Modules may be loaded from files and other file-system-like resources (for
142
+
example, a github repo) using the objects that implement the [FsModuleFormat]
143
+
trait. This trait contains the interface to interpret a file or a directory
144
+
and instantiate a [ModuleLoader] to load the module(s) contained within.
93
145
94
-
The objects [SingleFileModuleFmt] and [DirModuleFmt] are part of the default environment and are capable of loading MeTTa modules from single `.metta` files and directories containing a `module.metta` file respectively. Additionally, the `hyperon` Python module contains a [FsModuleFormat] for loading MeTTa modules from Python modules - both stand-alone `.py` files as well as directories containing an `__init__.py` file.
146
+
The objects [SingleFileModuleFmt] and [DirModuleFmt] are part of the default
147
+
environment and are capable of loading MeTTa modules from single `.metta` files
148
+
and directories containing a `module.metta` file respectively. Additionally,
149
+
the `hyperon` Python module contains a [FsModuleFormat] for loading MeTTa
150
+
modules from Python modules - both stand-alone `.py` files as well as
151
+
directories containing an `__init__.py` file.
95
152
96
-
More information on the individual module file formats is available in the MeTTa usage documentation and MeTTa Python documentation respectively.
153
+
More information on the individual module file formats is available in the
154
+
MeTTa usage documentation and MeTTa Python documentation respectively.
97
155
98
156
### The PkgInfo Structure
99
157
100
-
Each module has an associated [PkgInfo] structure, which provides the module author a place to specify meta-data about the module and express requirements for the module's dependencies. Additionally a [PkgInfo] can provide explicit loading instructions such as file system paths or github URLs for dependent modules. The [PkgInfo] structure is the same concept as the Cargo.toml file used in Cargo/Rust.
158
+
Each module has an associated [PkgInfo] structure, which provides the module
159
+
author a place to specify meta-data about the module and express requirements
160
+
for the module's dependencies. Additionally a [PkgInfo] can provide explicit
161
+
loading instructions such as file system paths or github URLs for dependent
162
+
modules. The [PkgInfo] structure is the same concept as the Cargo.toml file
163
+
used in Cargo/Rust.
101
164
102
-
The [PkgInfo] should be initialized inside the module's loader function. If it is not initialized then default values will be used.
165
+
The [PkgInfo] should be initialized inside the module's loader function. If it
166
+
is not initialized then default values will be used.
103
167
104
-
The fields of the [PkgInfo] struct are documented in the Rust MeTTa documentation [here](TODO link to
105
-
docs on https://docs.rs when Rust crate is published).
168
+
The fields of the [PkgInfo] struct are documented in the Rust MeTTa
TODO: PkgInfo documentation also belongs in user-facing docs. In that section, cover how to specify the pkginfo as a MeTTa structure and/or in a `_pkg-info.metta` or `_pkg-info.json` file as opposed to as a Rust struct.
172
+
TODO: PkgInfo documentation also belongs in user-facing docs. In that section,
173
+
cover how to specify the pkginfo as a MeTTa structure and/or in a
174
+
`_pkg-info.metta` or `_pkg-info.json` file as opposed to as a Rust struct.
108
175
109
176
### Module Name Resolution
110
177
111
-
When MeTTa code executes the `!(import! &space some_module)` operation, the module name needs to be mapped to a loaded or loadable module. This process occurs according to the logic described by the flowchart below.
178
+
When MeTTa code executes the `!(import! &space some_module)` operation, the
179
+
module name needs to be mapped to a loaded or loadable module. This process
180
+
occurs according to the logic described by the flowchart below.
1. First the module name is checked against the modules which are already loaded within the context of the running module. This ensures the same instance of a shared dependency will be loaded everywhere.
184
+
1. First the module name is checked against the modules which are already
185
+
loaded within the context of the running module. This ensures the same
186
+
instance of a shared dependency will be loaded everywhere.
116
187
117
-
2. If a loaded module is not available, the [PkgInfo] will be checked for a corresponding entry. If an entry specifies a specific location in the file system or a remote repository, then the module will be loaded from that location. Additionally, the [PkgInfo] may specify version requirements for use by the catalog in locating and selecting an accaptable module.
188
+
2. If a loaded module is not available, the [PkgInfo] will be checked for a
189
+
corresponding entry. If an entry specifies a specific location in the file
190
+
system or a remote repository, then the module will be loaded from that
191
+
location. Additionally, the [PkgInfo] may specify version requirements for
192
+
use by the catalog in locating and selecting an accaptable module.
118
193
119
-
3. Finally, the Catalogs from the Environment will be queried in priority order. (See the Catalogs section below)
194
+
3. Finally, the Catalogs from the Environment will be queried in priority
195
+
order. (See the Catalogs section below)
120
196
121
197
By default, the built-in search paths / catalogs are:
122
198
@@ -129,23 +205,36 @@ Depending on the host OS, the config directory locations will be:
In the future we may create a centralized module catalog along the lines of `PyPI` or `crates.io`.
208
+
In the future we may create a centralized module catalog along the lines of
209
+
`PyPI` or `crates.io`.
133
210
134
211
### Catalogs
135
212
136
-
An object that implements the [ModuleCatalog] trait exposes an interface to locate modules based on name and version constraints, and create [ModuleLoader] objects to retrieve and load those modules.
213
+
An object that implements the [ModuleCatalog] trait exposes an interface to
214
+
locate modules based on name and version constraints, and create [ModuleLoader]
215
+
objects to retrieve and load those modules.
137
216
138
-
One built-in [ModuleCatalog] object type is the [DirCatalog]. As described in the "Module File Formats" section, a [DirCatalog] uses a collection of [FsModuleFormat] objects to export a catalog of modules contained within its associated directory.
217
+
One built-in [ModuleCatalog] object type is the [DirCatalog]. As described in
218
+
the "Module File Formats" section, a [DirCatalog] uses a collection of
219
+
[FsModuleFormat] objects to export a catalog of modules contained within its
220
+
associated directory.
139
221
140
-
Additional catalogs may be implemented for other module repository formats or protocols - for example a central package service similar to `PyPI` or `crates.io`, as mentioned earlier.
222
+
Additional catalogs may be implemented for other module repository formats or
223
+
protocols - for example a central package service similar to `PyPI` or
224
+
`crates.io`, as mentioned earlier.
141
225
142
226
### Implementing a [ModuleLoader]
143
227
144
-
All modules are ultimately loaded programmatically through the MeTTa API, and it's the role of a [ModuleLoader] to make the necessary API calls.
228
+
All modules are ultimately loaded programmatically through the MeTTa API, and
229
+
it's the role of a [ModuleLoader] to make the necessary API calls.
145
230
146
-
The [ModuleLoader::load] method ultimately sets up the module. Each module has its own [Space] so the space needs to be created first. Then the module must be initialized using the [RunContext::init_self_module] method.
231
+
The [ModuleLoader::load] method ultimately sets up the module. Each module has
232
+
its own [Space] so the space needs to be created first. Then the module must
233
+
be initialized using the [RunContext::init_self_module] method.
147
234
148
-
After `init_self_module` has run, it is now legal to access the module data stricture using [RunContext::module] or [RunContext::module_mut], as well as enqueuing MeTTa code or additional operations to run.
235
+
After `init_self_module` has run, it is now legal to access the module data
236
+
stricture using [RunContext::module] or [RunContext::module_mut], as well as
237
+
enqueuing MeTTa code or additional operations to run.
0 commit comments