PyMeTTa handies [issue for ideas]

[Necr0x0Der]

There can be various automations and syntactic sugar to make Python-MeTTa integration more convenient and pretty.
A few ideas for starters:

1. Introduce @grounded decorator for extensions. Currently, @register_tokens decorates a function, which returns a token-atom map. For simple cases, when we want to have a grounded function, which token and representation in MeTTa correspond to its name in Python, and it is a pure Python function with unwrap=True, we could decorate it directly, i.e.

@grounded
def sum_squares(xs):
  arr = jnp.array(xs)
  return int(jnp.sum(arr*arr))

instead of

def sum_squares(xs):
  arr = jnp.array(xs)
  return int(jnp.sum(arr*arr))


@register_atoms
def def_atoms():
   return {
       'sum_squares': OperationAtom('sum_squares', sum_squares)
   }

Not sure if it is useful only for extensions, or there can be some sense in having a similar decorator in the main Python script.

2. It would be useful to make a backward import of MeTTa functions to Python. Something like

metta = MeTTa()
metta.run('''
(= (my-function $xs) ...)
'''
)
# metta.import2py('my-function')
metta.my_function(...)

That is, importing MeTTa functions to Python can be done by creating methods in metta object. Or, maybe, there are other ways to make calling MeTTa functions from Python more convenient.

3. MyOperationAtom([1, 2, 3]) as a syntactic sugar for E(MyOperationAtom, ValueAtom([1, 2, 3])).

4. Convenient access to grounded atoms defined in metta from Python. Currently, if the atom is not created in a Python script and put into a variable (MyOperationAtom = OperationAtom(...)), it is very inconvenient to call it from MeTTa. It typically requires parsing from textual representation with metta.run or metta.parse*. It would actually be nice to have them readily accessible as Python objects. Maybe, something like metta.stdlib.randint or whatever.

[patham9]

Nice ideas, I particularly like the @grounded decorator to handle the simpler cases of operation atom registrations. When using LLM's for code generation it will be very straightforward this way as well.

[Necr0x0Der]

grounded decorator can work without parameters for extensions, but if one wants to use such decorators in the main Python script, MeTTa object should be passed to them, so this decorators could register the function they decorate. Currently, register_atom decorator creates a wrapper object, which is detected during the module import process (in _priv_register_module_tokens). The same could be done not for extensions, but for the main Python script, so instead of writing something like

m = MeTTa()
@grounded(m)
def my_func(x)
  return x

one could write

@grounded
def my_func(x)
  return x

m = MeTTa()
m.register_context_tokens() # or another name

The latter is not more compact, but more convenient in sense that we don't need to create MeTTa object before declaring grounded functions. Not sure if it's worthy, though

[Necr0x0Der]

A simplistic implementation of grounded is here #981

[Necr0x0Der]

I also got an idea to create a variation of MeTTa, say, PyMeTTa with the following behavior:

def my_func(x):
  return x
m = PyMeTTa()
m.run("! (my_func 3)") # reduced

Basically, you don't need to manually register functions, use decorators, etc. All Python functions are automatically accessible from PyMeTTa runner.

I was thinking about tweaking only the Python runner by substituting symbol atoms with grounded atoms if corresponding Python functions are found in the current context. This cannot be done during evaluation of MeTTa code without deep modifications of the interpreter (because if one MeTTa function constructs an expression with Symbol, which could be replaced with a Python function, it should be done at runtime, and the interpreter should be able to access the Python runtime), unfortunately. But it could be done at parse time. But if we parse an expression and modify its s-expression version in Python with such replacements before adding it to Space or evaluating it, it could be done purely on Python side without adding new tokens. Unfortunately, Python runner is currently using hp.metta_run, so this also requires deeper (but simpler) changes.

I'm not sure if such PyMeTTa makes sense, or it will be too uncontrollable.

As an alternative to PyMeTTa, one could also introduce something like MeTTa.register_all_locals, which would work similar to what _priv_register_module_tokens is doing now, but without checking that the function (or another object like a class) is decorated. Once again, not sure if it could be usable.

[vsbogd]

I'm not sure if such PyMeTTa makes sense, or it will be too uncontrollable.

I think it makes sense and we don't even need PyMeTTa, just modify MeTTa to behave as you described. Originally I tried to keep Python interface similar to Rust. My intention was to simplify support and allow people quickly switch from one language to another if needed. But now Python API is very different and each new helper makes it more different. One the other hand I don't see any demand to switch between languages and having similar API. Thus may be we should just make Python API completely different.

[vsbogd]

This cannot be done during evaluation of MeTTa code without deep modifications of the interpreter (because if one MeTTa function constructs an expression with Symbol, which could be replaced with a Python function, it should be done at runtime, and the interpreter should be able to access the Python runtime), unfortunately.

It is what I am trying to reach in new API. One could have Python runtime atomspace which is called from MeTTa runtime and Python functions in this case are interpreter by Python interpreter.

[Necr0x0Der]

It is what I am trying to reach in new API

Interesting. It would be nice to revisit the possibility of automatic access to Python objects without py-atom / py-dot after this.

Meanwhile, I keep thinking about backward import of metta functions to Python.
Say, in case of

m = MeTTa()
m.run('''
(child Bob Tom)
(man Bob)
(= (is-son $y $x)
   (unify &self (, (child $y $x) (man $y)) True False)
)
''')

instead of

m.run("! (is-son Bob Tom)")

we could have

m.import_func('is-son')
m.is_son('Bob', 'Tom')

There are some ambiguities, though. In the case above, 'Tom' should be converted to Symbol. But what if it should be a grounded string instead, which is passed to a Python function in MeTTa? And when m.is_son('Bob', 'Tom') should be converted to a corresponding Python object, and when it should remain Atom...

One can imagine other options.

• unambiguous, but not convenient

m.is_son(S('Bob'), S('Tom')) # -> ValueAtom('True')

• arguments are parsed: 'Tom' is parsed to Symbol, '"Tom"' is parsed to grounded atom String. (Tom Bob) is parsed to E(S('Tom'), S(''Bob')). This option will have clean semantics, but lacks compositionality (output of one imported function will not be a valid input for another). The latter can be mitigated by smart behavior (non-string and non-atom Python objects are always wrapped into grounded atoms, Atoms always remain the same, strings are parsed by MeTTa runner, while the behavior of lists and tuples can be somehow chosen, say, lists remain grounded objects, and tuples are converted to expression atoms). I'm not sure that parsing strings is a good idea, but not sure how better to distinguish between symbols and strings.
• There can be a few import function (or unwrap) flag with different behavior.

@patham9 , any thoughts or preferences?

[TeamSPoon]

Culd in @grounded include the argument and return types