Skip to content

Addressing performance burden from #6990 #7565

Description

@edan-bainglass

#6990 introduced a Pydantic model patching system to reduce the burden on plugin developers. However, it has been reported that this inccurs a hit on performance per interpreter (e.g., daemon worker). For example, adding a print statement to Entity.__init_subclass__ (from where all patches trigger) yields the following:

In [1]: from aiida import orm
Initializing subclass User
Initializing subclass Computer
Initializing subclass AuthInfo
Initializing subclass Comment
Initializing subclass Group
Initializing subclass AutoGroup
Initializing subclass ImportGroup
Initializing subclass UpfFamily
Initializing subclass Log
Initializing subclass Node
Initializing subclass Data
Initializing subclass BaseType
Initializing subclass ArrayData
Initializing subclass KpointsData
Initializing subclass BandsData
Initializing subclass OrbitalData
Initializing subclass ProjectionData
Initializing subclass TrajectoryData
Initializing subclass XyData
Initializing subclass Bool
Initializing subclass SinglefileData
Initializing subclass CifData
Initializing subclass AbstractCode
Initializing subclass Code
Initializing subclass InstalledCode
Initializing subclass ContainerizedCode
Initializing subclass PortableCode
Initializing subclass Dict
Initializing subclass EnumData
Initializing subclass NumericType
Initializing subclass Float
Initializing subclass FolderData
Initializing subclass Int
Initializing subclass JsonableData
Initializing subclass List
Initializing subclass RemoteData
Initializing subclass RemoteStashData
Initializing subclass RemoteStashCompressedData
Initializing subclass RemoteStashCustomData
Initializing subclass RemoteStashFolderData
Initializing subclass Str
Initializing subclass StructureData
Initializing subclass UpfData
Initializing subclass ProcessNode
Initializing subclass CalculationNode
Initializing subclass CalcFunctionNode
Initializing subclass CalcJobNode
Initializing subclass WorkflowNode
Initializing subclass WorkChainNode
Initializing subclass WorkFunctionNode

Not ideal to be patching the full ORM on import, as one might be interested in only a single node, or even no intention to use the model system.

A better solution would be to patch things (and only those that absolutely are worth patching - to reduce plugin developer burdens) on first use, i.e., JIT, i.e., lazy loading!

The planned ORM restructure (see #7556) may naturally resolve this, but unclear at the moment.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

Type

No type

Fields

Priority

Urgent

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions