Description
Have I written custom code
Yes
OS Platform and Distribution
Ubuntu 22.04
MediaPipe Tasks SDK version
0.10.13 and 0.10.14
Task name
face landmarker
Programming Language and version (e.g. C++, Python, Java)
python 3.11.9
Describe the actual behavior
occasionally and very often the user supplied python callback for a live stream will be called twice with the same timestamp_ms value, in the face landmarks task.
Describe the expected behaviour
each timestamp given to detect_async should trigger only one instance of the result callback.
i.e., ocassionally and within less than one minute of operation at 30 fps, this pattern will occur:
pushing image number 965 having image timestamp 55418390 to mediapipe ...
callback called with timestamp 55418390 ...
pushing image number 966 having image timestamp 55418422 to mediapipe ...
callback called with timestamp 55418390 ...(the log messages are my own).
Other info
The release notes of 0.10.14 mention some Fixed result_callback() argument. Maybe it is related.
It seems to happen circumstantially always after mediapipe has dropped images from its processing, in line with the backpressure feature which extends now to the python api as per the online doc which says:
If the detection function is called when the Face Landmarker task is busy processing another frame, the task will ignore the new input frame
But not always when it drops images, does this double call with the same timestamp happen, so this affinity is only circumstantial.
P.S. I avoid reentrancy of my callback through a python lock object, just to be on the safe side in case mediapipe would concurrently call into the callback, which I doubt it would, but just to be on the safe side. the timestamp is a plain python immutable that doesn't get overwritten by concurrent calls (which mediapipe probably doesn't do anyway). so it looks like two consecutive callback calls using the same timestamp happen.