Metahuman DNA Animation Overview

[SkookumArts]

I'm working on creating a series of nodes in Houdini to load, modify/evaluate and save DNA files. So I thought it might be a good idea to write down everything I've figured out so far before I forget! Also, this might be helpful to someone else.

The following explains how animation data is store in a DNA file and how you can retrieve it. I'm sure there are aspects that I forgot to mention, so feel free to comment below and I'll add or update.

The following is NOT included, but it's likely I'll add it later.

• GUI control to Raw control conversion
• PSD (Corrective Expressions)
• Blendshape Deltas
• Animated Maps (Haven't messed with this yet, but I believe they work like the blendshape activation)

Gigantic Spreadsheet of Animation

Imagine that you have a gigantic spreadsheet which contains the animation data for all 870 joints in the face rig. Each joint takes up 9 rows (pos xyz, rot xyz and scale xyz). So joint 0 is at the top of the spreadsheet and joint 869 is at the bottom. Each column in the spreadsheet is an expression - Either raw controls like browDownL, jawOpen or corrective expressions (PSDs). The raw value for any given joint animation is the cell that lies at the intersection of a row and a column (Think battleship). Example below

Prune Unused Cells

The image above is an "uncompressed" version of the animation spreadsheet. The data is then "pruned" so any row that is all zeros (or below a specified tolerance?) is removed. So the scale attributes and joint 4 are removed. Which turns the above data into this.

Note, the "uncompressed" spreadsheet would also contain as many columns as there are expressions, so 0 to number of expressions.

Joint Groups

The rig of 870 joints are grouped together to create 124 "joint groups". The joints are grouped based on their location, so joint group 37 is a cluster of joints on the left eye brow, while joint group 77 is on the right cheek. Then the gigantic spreadsheet of animation data is then chopped up and divvied out based on those joint groups. So each joint group has a smaller spreadsheet that only contains it's joints. The previous examples were joint group 20, which only contains 2 joints.

Joint groups in the image above are the different groupings of colored spheres

Finding the Animation Data

The animation data for a particular joint group can be acquired by getting three arrays from the following functions.

getJointGroupInputIndices(jointGroupIndex) Column indices that the requested joint group contains. The column indices point into the entire, uncompressed joint matrix.
getJointGroupOutputIndices(jointGroupIndex) Row indices that the requested joint group contains. The row indices point into the entire, uncompressed joint matrix.
getJointGroupValues(jointGroupIndex) Values that the requested joint group contains.

These three arrays make up the joint group's "spreadsheet".
"InputIndices" is the columns
"OutputIndices" is the rows
"Values" is the big block of position, rotation and scale values.

Let's say we want to find the pose for the "browRaiseInR" expression. We start with it's "raw control index", which happens to be 5. Then we iterate through all the joint groups (124 of them), looking for index 5 in their columns of expressions. We can see that joint group 20 contains animation data for "browRaiseInR". We can then do something like the follow to get a particular joint's transform.

function float[] GetJointTransform(int jointIndex, rawControlIndex, rowsToEvaluate; int column_indices[], row_indices[]; float values[]) 
{
    // The animation data is in a large matrix
    // Where the coumns are expressions (Raw Control index values) and the rows are the joint attributes (pos, rot, scale)
    // Columns indices is "getJointGroupInputIndices(jointGroupIndex)". Row indices is "getJointGroupOutputIndices(jointGroupIndex)" 
    
    // Joint transform pos, rot, scale
    float jointTransform[] = {0, 0, 0, 0, 0, 0, 0, 0, 0};
    
    // Get the column for the wanted expression
    int expressionColumn = find(column_indices, rawControlIndex);
    int columnCount = len(column_indices);
    
    // Found the wanted raw expression in this joint group's animations
    if(expressionColumn >= 0)
    {
        for(int rowIndex = 0; rowIndex < rowsToEvaluate; rowIndex++)
        {
            int rowAttribIndex = row_indices[rowIndex];
            int rowJointIndex = rowAttribIndex / 9;
            
            if(rowJointIndex == jointIndex)
            {
                // What attribute is in this row, pos (0, 1, 2), rot (3, 4, 5), scale (6, 7, 8)
                int attribute_index = rowAttribIndex % 9;
                // The values array contains the raw data
                jointTransform[attribute_index] = values[rowIndex * columnCount + expressionColumn];
            }
        }
    
    return jointTransform;
}

This is Houdini Vex code

Note for when you're looking at the API reference, the term Epic uses for "spreadsheet" is "Joint Matrix".

Anyhow, feel free to chime in with questions. Also, I'd love to know if anyone from the Metahuman team see anything off.