Vectorization of entities member functions to avoid GIL issues

[roigcarlo]

Greetings,

After extensive discussion, testing, and debugging, the vectorization team is thrilled to announce a breakthrough! We've devised a solution to the GIL issue when vectorizing entity member functions.

The key lies in defining an extra Pybind11 attribute alongside our standard definitions. This method offers excellent flexibility, requires minimal additional effort, and shows a very small performance overhead. We'll now focus on comprehensive benchmarking and code refinement to prepare it for general implementation across our entities.

Best regards,

The vectorization team

[roigcarlo]

Under construction

After today's session, we collected some conclusions:

We are inclined to drop the Python approach.

• It is not possible to avoid the GIL while passing the function directly.
• We can use pointers to member functions, but it's cumbersome to program and would pollute the Python interface files.

This led us to think that a good solution would be to create specific interface functions in Python to wrap all operations, like:

• "ContainerA.GetELementIds"
• "ContainerB.CalculateLeftHandSides"
• etc...

This solves the problems with the GIL but shares some of the same problems that the alternative solutions have: It pollutes the interface.
In addition to that, we have identified that the new use of these operations would be too similar to the current state of expressions with some caveats like lazy evaluation.
In light of all this, we have decided to simplify the current implementation of expressions so those can:

• CollectData()
• StoreData()

This allows us to create snapshots of the data that we want to retrieve from the C++ side by creating specific expressions which execute the data collection or calculation in the CollectData() method, which will vary for every different expression and serves as the alternative to the specific functions in the Python interface.

The ownership of the data was another hot topic and we decided to expose that data to pythoin using two different functions:

• MoveData(): Which will transfer the ownership
• ViewData(): Which will expose the data bbut keeping the ownership.

Finally we also decided to add method called GetShape to know the shape of the data.

Final Interface:

template<TDataType T>
class Expression:
public:
    Expression() {}

    void CollectData()
    std::move(mData) MoveData() {}
    std::span(mData) ViewData() {}
    void StoreData() {}
    
    DenseVector<int> Shape() {}
    void PrintObject() {}

Examples use cases

# Get a copy of the ID's of the elements
ids_expr = ExpressionIds(mp.Elements)
ids_expr.CollectData()
ids = ids_expr.MoveData() #ownership is passed to python, so we can safely delete the ids_expr

# Get a view of the Jacobian of the elements
ids_expr = ExpressionIds(mp.Elements)
ids_expr.CollectData()
ids = ids_expr.ViewData() #ownership of data is still in the ids_expr. YOU CAN NOT REMOVE IT

as a one liner, we will add a python function to make it safe to use the data retrieval as a one liner

# Get a view of the Jacobian of the elements
ids = Evaluate(ExpressionIds, Elements) #after this only ids exists, as a numpy array
#ExpressionIds here is the type of the Expression and Elements is the parameter needed to construct it

[rubenzorrilla]

As a brainstorming the following expressions are needed:

BaseExpression - defines the virtual functions

• NodeVariableExpression - retrieves the data that are in the database of nodes
• ElementVariableExpression - similar but with Elements
• ConditionVariableExpression - similar but with conditions
• PropertiesVariableExpression -
• NodeAAAExpression - AAA represents any method of the node to be vectorized, say Ids, Coordinates, InitialCoords, Flags etc
• GeometryAAAExpression - where AAA represents all of the methods to be vectorized, say Areas, Ids, Jacobian, ShapeFunctions, etc.
• ElementAAAExpression
• ConditionAAAExpression
• ConstraintsAAAExpressions

[rubenzorrilla]

about the name of the expression there were some proposals

• TensorView
• TensoryProxy
• Tensor
• Dataset

[sunethwarna]

Finally, we decided to go with the TensorAdapter.