Skip to content

hand_gesture_recognition

Directory actions

More options

Directory actions

More options

Latest commit

 

History

History

hand_gesture_recognition

README.md

Hand Gesture Recognition Models

Model list

Chip MOBILENETV2_0_5_S8_V1
ESP32-S3 alt text
ESP32-P4 alt text

Model Latency

name input(hwc) preprocess(ms) model(ms) postprocess(ms)
mobilenetv2_0_5_s8_v1_s3 128 * 128 * 3 2.4 115.6 0.4
mobilenetv2_0_5_s8_v1_p4 128 * 128 * 3 0.9 28.5 0.2

Model Usage

HandGestureCls accepts a HandGestureCls::model_type_t parameter. It has a default value determined by default model option in menuconfig.

How to New HandGestureCls

HandGestureCls *m_cls = new HandGestureCls();

Configurable Options in Menuconfig

See Kconfig.

Model to Flash

  • CONFIG_FLASH_HAND_GESTURE_CLS_MOBILENETV2_0_5_S8_V1

Whether to flash the model when the model location is set to FLASH rodata or FLASH partition.

Default Model

  • CONFIG_HAND_GESTURE_CLS_MOBILENETV2_0_5_S8_V1

Default model to use if no parameter is passed to HandGestureCls.

Model Location

  • CONFIG_HAND_GESTURE_CLS_MODEL_IN_FLASH_RODATA
  • CONFIG_HAND_GESTURE_CLS_MODEL_IN_FLASH_PARTITION
  • CONFIG_HAND_GESTURE_CLS_MODEL_IN_SDCARD

This component supports to load model from three different locations.

Note

If model location is set to FLASH partition, partition.csv must contain a partition named hand_gesture_cls, and the partition should be big enough to hold the model file.

SDCard Directory

  • CONFIG_HAND_GESTURE_CLS_MODEL_SDCARD_DIR

When model locates in sdcard, you can change the model directory relative to the sdcard mount point.

The default value of this option is models/s3 for ESP32S3 and models/p4 for ESP32P4. When using default value, just copy models folder to sdcard root directory.

Note

Do not change the model name when copy the models to sdcard.

Hand Gesture Recognizer

This component also contains HandGestureRecognizer. If you want to recognize a hand gesture, in addition to HandGestureRecognizer, you also need HandDetect. See How to New HandDetect.

How to New Hand Gesture Recognizer

HandGestureRecognizer *hand_gesture_recognizer = new HandGestureRecognizer("????");

How to Recognize a Hand Gesture

dl::image::img_t img = {.data=DATA, .width=WIDTH, .height=HEIGHT, .pix_type=PIX_TYPE};
std::vector<dl::recognition::result_t> res = hand_gesture_recognizer->recognize(img, hand_detect->run(img));

More details, see dl::image::img_t and dl::recognition::result_t.