If the goal is to test spatial understanding wouldn't we want to say something like: A skateboarder rides in a regular stance. What direction does the board rotate when performing a backside flip? What direction does the board flip? Should be clockwise and counter clockwise or right and left.
The way you're doing it just seems to test the model on its knowledge of the trick names.
To go off on a tangent, there was a study awhile ago with mice, children and adults. They disorient them and then put them in a room with some targets and landmarks and see if they can correctly choose the right direction. It turns out the infants did as well as the mice. It wasn't until the children were old enough and learned their right from left that the test became simple.
My theory is if a model has no understanding of basics like it's left from right then it will struggle with everything else. Skate bench would be a unique and challenging way of testing that I think.
It may be too hard at first but that's ok.
If the goal is to test spatial understanding wouldn't we want to say something like: A skateboarder rides in a regular stance. What direction does the board rotate when performing a backside flip? What direction does the board flip? Should be clockwise and counter clockwise or right and left.
The way you're doing it just seems to test the model on its knowledge of the trick names.
To go off on a tangent, there was a study awhile ago with mice, children and adults. They disorient them and then put them in a room with some targets and landmarks and see if they can correctly choose the right direction. It turns out the infants did as well as the mice. It wasn't until the children were old enough and learned their right from left that the test became simple.
My theory is if a model has no understanding of basics like it's left from right then it will struggle with everything else. Skate bench would be a unique and challenging way of testing that I think.
It may be too hard at first but that's ok.