idea: use of Thoth.Json.Python with plain classes for runtime type validation

[joprice]

In js, I have made use of the fromValue as a form of "runtime type validation" similar to how zod is used in ts using anonymous records. I found that something similar can be done with classes with a dict member and dataclasses by providing a separate instance of IDecoderHelpers toDecode.Advanced.fromValue with a few overrides:

...
   member _.isObject jsonValue =
          pyInstanceof jsonValue pyDict
          || hasattr (jsonValue, "__dict__")
          || hasattr (jsonValue, "__dataclass_fields__")

   member _.hasProperty fieldName jsonValue =
          if hasattr (jsonValue, "__dataclass_fields__") then
            emitPyStatement (jsonValue, fieldName) "return $1 in $0.__dataclass_fields__"
          else
            emitPyStatement (jsonValue, fieldName) "return $1 in $0"

  member _.getProperties jsonValue =
          emitPyStatement
            jsonValue
            """
            if hasattr($0, "__dataclass_fields__"):
                return $0.__dataclass_fields__.keys()
            if hasattr($0, "__dict__"):
                return $0.__dict__.keys()
            return $0.keys()
          """
  
member _.getProperty(fieldName: string, jsonValue: obj) =
          emitPyStatement (jsonValue, fieldName) "return $0.__getattribute__($1)"
...

This could be useful for wrapping interfaces like argparse when using subcommands, where each command has a callback that takes an object, or perhaps some event listening interfaces that respond with similar arbitrary data objects.

[MangelMaxime]

I am not too familiar with Python so it is difficult for me to evaluate this proposition.

Something to note is that Thoth.Json manual API aims to remove "any magic", my vision with this API is for you to be in full control over what happens in the code.

If we can improve the situation without breaking changes or cons, I think I will be ok with adding new stuff.

@joprice Could you please provide an example of what you are trying to do with this code?

@Freymaurer any though on this proposition?

[Freymaurer]

I am afraid I can't really follow without more information 😞

Maybe this is for auto decoder/encoder? But i am really not sure.

[MangelMaxime]

I think this is for the manual API but allows to get properties for a wider range of Python types.

Right now I only standard Python objects which supports .keys() perhaps? But with this changes we could support:

• Dataclass
• Dict
• normal Python objects which supports .keys()

normal Python objects which supports `.keys()` is a really bad name, but I don't know how this types are named in Python 😅

Like I don't know what a Dataclass is

[joprice]

One example is using decoders to parse rows returned from a database. Both psycopg3 (when using the dict_row row factory https://www.psycopg.org/psycopg3/docs/advanced/rows.html) and node-postgres return json-like objects. I imagine this would apply to Mongo and other document stores as well.

Another is doing a "second-pass" of validation on an object pulled from such a store: Say a new nullable column is introduced, but some parts of the code (or maybe after a migration of old data) treat it as non-nullable. You can use Thoth as a library to parse from the model with optional fields to one with non-optional, and get nice errors. (This could also be done with some applicative error handling library, perhaps with a CE or codegen, but so far I've found it nice to re-use Thoth encoders and I plan on writing a Myriad generator for them which can then be reused for such validation).

[MangelMaxime]

@joprice

Would you be ok with sending a PR to implements the changes you are mentioning with tests to cover them ? Python specifics tests can be added to Thoth.Json.Tests.Python directly.

Because I suspect, you will need to create the test values/objects using Python directly in order to control their representation to use dataclass info.