Description
MediaPipe Solution (you are using)
holistic
Programming language
javascript
Are you willing to contribute it
Yes
Describe the feature and the current behaviour/state
I would have a similar request to:
#4116
But to keep the face while disable the hand gesture. Like what was mentioned in this response 2 years ago:
#2028 (comment)
As we can track from my demo website: https://openlive3d.com/
The performance will drop when any hand is detected.
When the user of my website is playing games,
they want better performance and don't necessarily need hand gesture detection.
Having the "Dynamically Disabling Sections for Holistic" will provide better control for our users,
with different levels of hardware.
Will this change the current API? How?
Yes.
Currently, these are the parameters that I use for the holistic model:
hModel.setOptions({
cameraOn: true,
modelComplexity: 0,
useCpuInference: false,
smoothLandmarks: false,
enableSegmentation: false,
smoothSegmentation: false,
refineFaceLandmarks: true,
minDetectionConfidence: 0.5,
minTrackingConfidence: 0.55,
});I am expecting to add enableFaceLandmarks and enableHandGestures parameters,
defaulting to true for both. And the set options could become:
hModel.setOptions({
cameraOn: true,
modelComplexity: 0,
useCpuInference: false,
smoothLandmarks: false,
enableSegmentation: false,
smoothSegmentation: false,
refineFaceLandmarks: true,
minDetectionConfidence: 0.5,
minTrackingConfidence: 0.55,
enableFaceLandmarks: true,
enableHandGestures: false,
});Who will benefit with this feature?
Holistic model users have different use cases, some users only need body pose and hand gesture:
Some people only need body pose and facial landmark:
Having this option will increase the performance for different specific use cases, since they can disable the section that they don't need.
Please specify the use cases for this feature
Similar to the refineFaceLandmarks, we are looking to have similar options for the all facial landmarks and also hand gesture.
As mentioned as Dynamically Disabling Sections for Holistic in this response:
#2028 (comment)
Such feature will help different users to only enable the sections for their specific use cases and disable the sections that they don't need. If we can set options to get real-time enablement and disablement, it will greatly increase the possibilities to dynamically save more computation resources by use case by time.
For my specific use case: https://openlive3d.com/
- When a user is playing other games that require high GPU usage, I would like to provide the option to disable hand gesture.
- When a user selected an emotion that won't require eye and mouth detection (for example angry face), I would like to disable the facial landmark.
So that I can gain some performance to provide smoother experiences.
Any Other info
Thanks for all the work in mediapipe.
This is an amazing model that enables so many possibilities for all of us.