Skip to content

Modernize data validation #1674

Open
Open
@BryceGattis

Description

@BryceGattis

We should consider switching to more modern data validation libraries such as "pydantic".

Motivation
The schema library we are using is pretty outdated these days. We are using a very old version of schema that was released in 2014. We have a couple of complaints with the current system:

  • We currently are relying on "evil" metaclasses that implicitly inject attributes to classes at runtime. This both doesn't work very well with IDEs and also is just hard to make sense of from a developers point of view.
  • Additionally, all of the attributes we are defining in these schemas have no docstring, and therefore their expected usage and data is completely unknown to new users.
  • The current schema library can have some pretty awful output for users, and can be difficult to parse and determine what went wrong. Here's an example output:
  File "D:\Code\rez\venv\lib\site-packages\rez\vendor\schema\schema.py", line 235, in validate
    raise SchemaError([None] + x.autos, [e] + x.errors)
rez.vendor.schema.schema.SchemaError: Or(<class 'rez.utils.sourcecode.SourceCode'>, Schema({Optional(<class 'str'>): Or(Or(<class 'str'>, [<class 'str'>]), {Optional(<class 'str'>): <class 'object'>, 'command': Or(<class 'str'>, [<class 'str'>]), Optional('requires'): [Or(<class 'str'>, And(<class 'rez.util
s.formatting.PackageRequest'>, Use(<class 'str'>)))], Optional('run_on'): Or(<class 'str'>, [<class 'str'>]), Optional('on_variants'): Or(<class 'bool'>, {'type': 'requires', 'value': [Or(<class 'str'>, And(<class 'rez.utils.formatting.PackageRequest'>, Use(<class 'str'>)))]})})})) did not validate 'None'  
'None' should be instance of {Optional(<class 'str'>): Or(Or(<class 'str'>, [<class 'str'>]), {Optional(<class 'str'>): <class 'object'>, 'command': Or(<class 'str'>, [<class 'str'>]), Optional('requires'): [Or(<class 'str'>, And(<class 'rez.utils.formatting.PackageRequest'>, Use(<class 'str'>)))], Optional
('run_on'): Or(<class 'str'>, [<class 'str'>]), Optional('on_variants'): Or(<class 'bool'>, {'type': 'requires', 'value': [Or(<class 'str'>, And(<class 'rez.utils.formatting.PackageRequest'>, Use(<class 'str'>)))]})})}

Related Conversations

#1662 (comment)

Note that if we use pydantic, we must use pydantic < 2.6, as 2.6 introduced a breaking change with Python 3.7 which is our lowest support Python version as of now.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions