I added recently the ability to see how much time Robotoff spend on each step (preprocessing, Triton inference, postprocessing) for each model in #1703 and #1704.
It looks like some steps are severely under-optimized. For example, here is the result when running locally (once the image is loaded in memory), on CPU, the nutriscore object detection model on 10 images:
preprocessing: 724 ms (+/- 43.1 ms), including NMS (non-maximum suppression) 8.77 ms (+/- 0.794 ms)
inference: 101 ms (+/- 0.017 ms)
post-processing: 61.62 ms (+/- 0.005.85 ms)