Skip to content

Dedupe Fails with Tensorflow Error #91

@NarryG

Description

@NarryG

I managed to pull my mediaitems down before API deprecation, but when I run the dedupe, it errors out

2025-04-01 17:42:40 worker-1  | [2025-04-01 21:42:40,092: WARNING/ForkPoolWorker-31] 
  2%|▏         | 1086/70030 [00:10<10:33, 108.77it/s]
2025-04-01 17:42:40 worker-1  | [2025-04-01 21:42:40,193: WARNING/ForkPoolWorker-31] 
  2%|▏         | 1097/70030 [00:10<10:32, 108.92it/s]
2025-04-01 17:42:40 worker-1  | [2025-04-01 21:42:40,300: WARNING/ForkPoolWorker-31] 
  2%|▏         | 1108/70030 [00:10<10:44, 107.02it/s]
2025-04-01 17:42:40 worker-1  | [2025-04-01 21:42:40,410: WARNING/ForkPoolWorker-31] 
  2%|▏         | 1119/70030 [00:10<10:57, 104.86it/s]
2025-04-01 17:42:40 worker-1  | [2025-04-01 21:42:40,516: WARNING/ForkPoolWorker-31] 
  2%|▏         | 1130/70030 [00:10<10:59, 104.40it/s]
2025-04-01 17:42:40 worker-1  | E0000 00:00:1743543760.529261     314 calculator_graph.cc:898] INVALID_ARGUMENT: CalculatorGraph::Run() failed: 
2025-04-01 17:42:40 worker-1  | Calculator::Process() for node "mediapipe_tasks_vision_image_embedder_imageembeddergraph__mediapipe_tasks_core_inferencesubgraph__inferencecalculator__mediapipe_tasks_vision_image_embedder_imageembeddergraph__mediapipe_tasks_core_inferencesubgraph__InferenceCalculator" failed: ; RET_CHECK failure (mediapipe/calculators/tensor/inference_calculator_utils.cc:144) (tflite_tensor.bytes)==(input_tensor.bytes())TfLiteTensor and Tensor sizes do not match. TfLiteTensor dims: [1, 224, 224, 3], type: FLOAT32, bytes: 602112 vs. MP Tensor dims: [1, 224, 224, 1], type: kFloat32, bytes: 200704;  at index 0
2025-04-01 17:42:40 worker-1  | [2025-04-01 21:42:40,533: WARNING/ForkPoolWorker-31] 
  2%|▏         | 1131/70030 [00:10<10:56, 104.99it/s]
2025-04-01 17:42:40 worker-1  | [2025-04-01 21:42:40,546: ERROR/ForkPoolWorker-31] Task app.tasks.process_duplicates[749a2db5-3c95-4a3a-b92f-3f24d552ddc1] raised unexpected: ValueError('CalculatorGraph::Run() failed: \nCalculator::Process() for node "mediapipe_tasks_vision_image_embedder_imageembeddergraph__mediapipe_tasks_core_inferencesubgraph__inferencecalculator__mediapipe_tasks_vision_image_embedder_imageembeddergraph__mediapipe_tasks_core_inferencesubgraph__InferenceCalculator" failed: ; RET_CHECK failure (mediapipe/calculators/tensor/inference_calculator_utils.cc:144) (tflite_tensor.bytes)==(input_tensor.bytes())TfLiteTensor and Tensor sizes do not match. TfLiteTensor dims: [1, 224, 224, 3], type: FLOAT32, bytes: 602112 vs. MP Tensor dims: [1, 224, 224, 1], type: kFloat32, bytes: 200704;  at index 0')
2025-04-01 17:42:40 worker-1  | Traceback (most recent call last):
2025-04-01 17:42:40 worker-1  |   File "/usr/src/app/app/lib/duplicate_image_detector.py", line 125, in _calculate_embeddings
2025-04-01 17:42:40 worker-1  |     embedding_result = embedder.embed(mp_image)
2025-04-01 17:42:40 worker-1  |   File "/usr/local/lib/python3.9/site-packages/mediapipe/tasks/python/vision/image_embedder.py", line 229, in embed
2025-04-01 17:42:40 worker-1  |     output_packets = self._process_image_data({
2025-04-01 17:42:40 worker-1  |   File "/usr/local/lib/python3.9/site-packages/mediapipe/tasks/python/vision/core/base_vision_task_api.py", line 95, in _process_image_data
2025-04-01 17:42:40 worker-1  |     return self._runner.process(inputs)
2025-04-01 17:42:40 worker-1  | ValueError: CalculatorGraph::Run() failed: 
2025-04-01 17:42:40 worker-1  | Calculator::Process() for node "mediapipe_tasks_vision_image_embedder_imageembeddergraph__mediapipe_tasks_core_inferencesubgraph__inferencecalculator__mediapipe_tasks_vision_image_embedder_imageembeddergraph__mediapipe_tasks_core_inferencesubgraph__InferenceCalculator" failed: ; RET_CHECK failure (mediapipe/calculators/tensor/inference_calculator_utils.cc:144) (tflite_tensor.bytes)==(input_tensor.bytes())TfLiteTensor and Tensor sizes do not match. TfLiteTensor dims: [1, 224, 224, 3], type: FLOAT32, bytes: 602112 vs. MP Tensor dims: [1, 224, 224, 1], type: kFloat32, bytes: 200704;  at index 0
2025-04-01 17:42:40 worker-1  | 
2025-04-01 17:42:40 worker-1  | During handling of the above exception, another exception occurred:
2025-04-01 17:42:40 worker-1  | 
2025-04-01 17:42:40 worker-1  | Traceback (most recent call last):
2025-04-01 17:42:40 worker-1  |   File "/usr/local/lib/python3.9/site-packages/celery/app/trace.py", line 453, in trace_task
2025-04-01 17:42:40 worker-1  |     R = retval = fun(*args, **kwargs)
2025-04-01 17:42:40 worker-1  |   File "/usr/src/app/app/__init__.py", line 25, in __call__
2025-04-01 17:42:40 worker-1  |     return self.run(*args, **kwargs)
2025-04-01 17:42:40 worker-1  |   File "/usr/src/app/app/tasks.py", line 111, in process_duplicates
2025-04-01 17:42:40 worker-1  |     results = task_instance.run()
2025-04-01 17:42:40 worker-1  |   File "/usr/src/app/app/lib/process_duplicates_task.py", line 113, in run
2025-04-01 17:42:40 worker-1  |     similarity_map = duplicate_detector.calculate_similarity_map()
2025-04-01 17:42:40 worker-1  |   File "/usr/src/app/app/lib/duplicate_image_detector.py", line 61, in calculate_similarity_map
2025-04-01 17:42:40 worker-1  |     embeddings = self._calculate_embeddings()
2025-04-01 17:42:40 worker-1  |   File "/usr/src/app/app/lib/duplicate_image_detector.py", line 126, in _calculate_embeddings
2025-04-01 17:42:40 worker-1  |     embeddings.append(embedding_result.embeddings[0].embedding)
2025-04-01 17:42:40 worker-1  |   File "/usr/local/lib/python3.9/site-packages/mediapipe/tasks/python/vision/core/base_vision_task_api.py", line 226, in __exit__
2025-04-01 17:42:40 worker-1  |     self.close()
2025-04-01 17:42:40 worker-1  |   File "/usr/local/lib/python3.9/site-packages/mediapipe/tasks/python/vision/core/base_vision_task_api.py", line 209, in close
2025-04-01 17:42:40 worker-1  |     self._runner.close()
2025-04-01 17:42:40 worker-1  | ValueError: CalculatorGraph::Run() failed: 
2025-04-01 17:42:40 worker-1  | Calculator::Process() for node "mediapipe_tasks_vision_image_embedder_imageembeddergraph__mediapipe_tasks_core_inferencesubgraph__inferencecalculator__mediapipe_tasks_vision_image_embedder_imageembeddergraph__mediapipe_tasks_core_inferencesubgraph__InferenceCalculator" failed: ; RET_CHECK failure (mediapipe/calculators/tensor/inference_calculator_utils.cc:144) (tflite_tensor.bytes)==(input_tensor.bytes())TfLiteTensor and Tensor sizes do not match. TfLiteTensor dims: [1, 224, 224, 3], type: FLOAT32, bytes: 602112 vs. MP Tensor dims: [1, 224, 224, 1], type: kFloat32, bytes: 200704;  at index 0

Running on Docker Desktop on WSL2, Win11 24H2. I have GPU passthrough set up but it's falling back to CPU if that matters

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions