How to run MediaPipe real-time gesture recognizer task in C++?

[pollarana]

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)

None

OS Platform and Distribution

Ubuntu 24.04 LTS Linux 6.11.0-19-generic

MediaPipe Tasks SDK version

No response

Task name (e.g. Image classification, Gesture recognition etc.)

Gesture recognition

Programming Language and version (e.g. C++, Python, Java)

C++

Describe the actual behavior

bazel buil is executed but does not run further because targets and pbtxt file are missing

Describe the expected behaviour

running the gesture recognizer example in full, like hand tracking

Standalone code/steps you may have used to try to get what you need

see the "other info" section

Other info / Complete Logs

bazel build -c opt --define MEDIAPIPE_DISABLE_GPU=1 mediapipe/tasks/cc/vision/gesture_recognizer:gesture_recognizer
INFO: Elapsed time: 1716.747s, Critical Path: 1131.64s
INFO: 2577 processes: 285 internal, 2292 processwrapper-sandbox.
INFO: Build completed successfully, 2577 total actions

bazel run  --define MEDIAPIPE_DISABLE_GPU=1 mediapipe/tasks/cc/vision/gesture_recognizer:gesture_recognizer
ERROR: Cannot run target //mediapipe/tasks/cc/vision/gesture_recognizer:gesture_recognizer: Not executable
INFO: Elapsed time: 0.610s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)
ERROR: Build failed. Not running target

[kalyan2789g]

Hi @pollarana,

To facilitate a more comprehensive analysis of the reported issue, kindly provide the following information:

• MediaPipe version being used.

• Bazel version.

Additionally, please share the specific steps you followed to reproduce the issue, enabling us to investigate it effectively on our end.

Thanks,
@kalyan2789g

[pollarana]

Hello, @kalyan2789g. I'm using bazel version 6.5.0 and mediapipe version v0.10.23.
I cloned this repository yesterday, so it's clean (I haven't changed anything inside).
Before that, there was an attempt to throw gesture_recognizer into the hand_tracking example. The files mediapipe/graphs/hand_tracking/hand_tracking_desktop_live.pbtxt and mediapipe/examples/desktop/demo_run_graph_main.cc have been changed.
The following fragment has been added to the hand_tracking_desktop_live.pbtxt

 node {
   calculator:
   "mediapipe.tasks.vision.gesture_recognizer.GestureRecognizerGraph"
   input_stream: "IMAGE:image_in"
   input_stream: "NORM_RECT:norm_rect"
   output_stream: "HAND_GESTURES:hand_gestures"
   output_stream: "LANDMARKS:hand_landmarks"
   output_stream: "WORLD_LANDMARKS:world_hand_landmarks"
   output_stream: "HAND_RECT_NEXT_FRAME:hand_rect_next_frame"
   output_stream: "HANDEDNESS:handedness"
   output_stream: "IMAGE:image_out"
   options {
     [mediapipe.tasks.vision.GestureRecognizerGraphOptions.ext]
     {
       base_options {
         model_asset {
           file_name: "hand_gesture.tflite"
         }
       }
       running_mode: LIVE_STREAM
       hand_landmark_detector_options {
         base_options {
           model_asset {
             file_name: "hand_landmark.tflite"
           }
         }
       }
     }
   }
 }

To the file demo_run_graph_main.cc the dependency #include "mediapipe/framework/formats/classification.pb.h" has been added and the section "Get the graph result packet, or stop if that fails." was changed accordingly:

mediapipe::Packet packet;
    if (!poller.Next(&packet)) break;
    auto& hand_gestures = packet.Get<std::vector<mediapipe::ClassificationList>>();

    for (const auto& classification_list : hand_gestures) {
      for (int i = 0; i < classification_list.classification_size(); i++) {
        const auto& classification = classification_list.classification(i);
        std::cout << classification.display_name() << " ";
      }
    }

As a result of these changes, the build was completed successfully, but an error occurs when trying to run.

[libprotobuf ERROR external/com_google_protobuf/src/google/protobuf/text_format.cc:335] Error parsing text-format mediapipe.CalculatorGraphConfig: 48:6: Extension "mediapipe.tasks.vision.GestureRecognizerGraphOptions.ext" is not defined or is not an extension of "mediapipe.CalculatorOptions".
F0000 00:00:1744097348.342553    5045 parse_text_proto.h:33] Check failed: ParseTextProto(input, &result) 
*** Check failure stack trace: ***
    @     0x5ad6a42e7579  absl::log_internal::LogMessage::SendToLog()
    @     0x5ad6a42e7a1d  absl::log_internal::LogMessageFatal::~LogMessageFatal()
    @     0x5ad6a3ae6205  RunMPPGraph()
    @     0x5ad6a3b1390f  main
    @     0x7efbad02a1ca  (unknown)
    @     0x7efbad02a28b  __libc_start_main
    @     0x5ad6a3b21d05  _start
Aborted (core dumped)

[kalyan2789g]

Hi @pollarana,

We appreciate you providing the steps and supplementary information. We are now in the process of reproducing the issue internally and consulting with our development team. We will keep you updated on our progress.

Thanks,
@kalyan2789g

[whhone]

For comment#1, the mediapipe/tasks/cc/vision/gesture_recognizer:gesture_recognizer is a cc_library instead of a cc_binary. You need to build a binary with the cc_library and run it.

mediapipe/mediapipe/tasks/cc/vision/gesture_recognizer/BUILD

Line 23 in a24e83e

cc_library(

For comment#3, I think it is a valid issue in mediapipe/examples/desktop/demo_run_graph_main.cc.

@pollarana Can you give more details how you ran mediapipe/examples/desktop/simple_run_graph_main.cc. @kalyan2789g might then reproduce and look into the error.

[github-actions]

This issue has been marked stale because it has no recent activity since 7 days. It will be closed if no further activity occurs. Thank you.