Managing Many Modules (M3)

[x87]

M3 is a hypothetical situation where we have hundreds of CLEO5 modules available for use. This triggers some problems, such as:

• inevitable name clashes
• breaking changes
• discoverability

Name Clashing

Example: two authors develop their scripts and modules. They call their modules utils.s. It is now impossible to install both scripts because you can only have one module.

Breaking Changes

Example: large and popular module used by many scripts. Author finds a bug and changes the function definition (e.g. adds an extra param).

Discoverability

How to find a module that is useful for me? What should I know before using it? Modules may or may not come with source code supplied.

Potential Solutions

• Name Clashing. Introduce a new config file called modules.ini where the user can map modules to different filenames. E.g. for scripts a.cs and b.cs each using utils.s this may look as follows:

[a.cs]
utils.s=utils_a.s

[b.cs]
utils.s=utils_b.s

the CLEO will use this mapping to correctly resolve module calls.

• Breaking Changes. Semver, with MAJOR.MINOR versioning. Modules may have an optional version part in their name, e.g. utils@2.0.s. Then you may also have utils@1.0.s, utils@2.1.s.
  Now, lets see what happens, when CLEO encounters a module call:

  • "f@utils.s" - resolves to utils@1.0.s (or any 1.X version) or utils.s in that order
  • "f@utils@1.s" - resolves to utils@1.0.s (most recent 1.X version)
  • "f@utils@2.s" - resolves to utils@2.1.s (most recent 2.X version)
  • "f@utils.s@2.0" - resolves to utils@2.0.s (exact match)
  • "f@utils.s@2.1" - resolves to utils@2.1.s (exact match)
  • "f@utils.s@2.3" - runtime error
  • "f@utils.s@3" - runtime error

• Discoverability. Document modules and their exported functions in SBL. Similar to Native code section for SA.

[MiranDMC]

Hm can not it simply be solved by naming modules properly? Like MiranMiscUtils instead of Utils?
Modules collision resolving will be for sure too much to ask for typical user. Extending simple command limits in INI file already proven to be difficult.
Naming clash can also be solved by placing modules in script's directory, or using ModLoader. Internally module should be identified by full path and module search resolved in similar way ML works.

Breaking changes. When you are creating module for other people to use you should not make breaking changes.
This also can be solved by introducing function overloading or support of default parameters.
Maybe module version should be embedded inside module file and include command would take version as argument? These could defaulting to v1.

[x87]

Hm can not it simply be solved by naming modules properly? Like MiranMiscUtils instead of Utils?

This would help to minimize the clashes, but not eliminate entirely. Imagine it happened, and now you have
scripts a.cs and b.cs each using mod.s. What's next? And no, I'm not using Modloader.

Breaking changes.

There can be many reasons to stick to a particular version of the module, not necessarily causing function interface change.
After some thinking, I come to a conclusion that versioning might be a bit difficult concept for modding community, so first solution (custom mapping) can also help if you want to stick to a particular version.

[a.cs]
utils.s=utils_v1.s

[b.cs]
utils.s=utils_v2.s

[MiranDMC]

Modloader is exact answer to problem of multiple mods trying to use same filenames.
Now I'm convinced that modules loading should work in similar fashion as dll files. First look in your root, then globally.
Even without ML you can place script in sub directory along with it's modules and trigger it from another script.

Regarding the module versions. Why there is new version?

1. adding new functionality
2. fixing critical errors
3. introducing breaking changes

For no.1 version does not matter, for no.2 it is even harmful to use older one, 3 actually requires older version.
Question is: is it even worth to implement whole versioning system to just enable bad practice number 3?
Lets say there is 20 modules. I can not imagine end user renaming and versioning all his module files, then hand picking in some global INI file versions to use by each CLEO script. How will user even know some modules need to be kept in older version? By troubleshooting requiring time, effort and developer's skills?
Even if so INI file exists it should be one peer cs script. Some mods already have their ini files, but having section like [cleo_modules_remap] should cause no problems.

[x87]

modules.ini is a global CLEO config file. It's only purpose is to help to resolve collisions. Most of the time will be empty. But when needed, it will be there.

And no, module resolution has nothing to do with paths. Moving module to its own directory requires code change, this is not the same situation I describe. Again, imagine 2 scripts loading utils.s. You can not change the code. But you want to run them, and only thing that stops you is the module name.

Ignore versioning, I already said it might be too difficult to maintain. Focus on one example that I gave. How to run both scripts?

[MiranDMC]

If there is module like utils.s to live along main cs file it is sign the mod was meant to be used via ModLoader. Then it will not be possible to simply use it as intended.
It again shows if no absolute(virtual) path to module is specified it should be resolved as any other path, then fallback to cleo_modules.
Fixing something through ini files should be last resort.

[x87]

I agree that Modloader might help. My point was that CLEO should provide a way to fix file collisions using CLEO only.

If user is unfamiliar with script code and don't or can't use ML (which is not that uncommon), they could at least be advised to update their modules.ini

[x87]

ML itself has collision resolution with user-defined priorities, and they don't advise using third-party tools for that

[MiranDMC]

I think it need to be checked how modules loading is implemented at this moment. modules:\ virtual path was added for some reason. Modules cooperation with ML was also never tested.

[x87]

modules should be resolved just like any filepath.

import from CLEO\mod.s
import from .\mod.s
import from ..\mod.s
import from modules:\mod.s