Improving performance and time

[samuelezoc]

Hello everyone, I'm using the ocr_predictor(pretrained=True) with default values. I would lower the execution time improving the performance. I have a RTX 4070 SUPER.

Which det_bs and reco_bs should i use?
Thanks!

[felixdittrich92]

Hi @samuelezoc 👋,

I'm running on a RTX4080 and a det_bs of 4 / reco_bs of 1024 works best in my case :)
In addition with ocr_predictor(pretrained=True).cuda().half()

Best regards,
Felix

[samuelezoc]

I'm using TensorFlow. I found in the documentation the half-precision model optimization as you mentioned:

import tensorflow as tf
from tensorflow.keras import mixed_precision
mixed_precision.set_global_policy('mixed_float16')
predictor = ocr_predictor(reco_arch="crnn_mobilenet_v3_small", det_arch="linknet_resnet34", pretrained=True)

But it returns some errors: (this is just one of many cases)

Exception Type: InvalidArgumentError
Exception Value: 
Exception encountered when calling layer 'crnn_2' (type CRNN).
Value for attr 'T' of half is not in the list of allowed values: float, double
	; NodeDef: {{node CTCBeamSearchDecoder}}; Op<name=CTCBeamSearchDecoder; signature=inputs:T, sequence_length:int32 -> decoded_indices:top_paths*int64, decoded_values:top_paths*int64, decoded_shape:top_paths*int64, log_probability:T; attr=beam_width:int,min=1; attr=top_paths:int,min=1; attr=merge_repeated:bool,default=true; attr=T:type,default=DT_FLOAT,allowed=[DT_FLOAT, DT_DOUBLE]> [Op:CTCBeamSearchDecoder] name: 

Call arguments received by layer 'crnn_2' (type CRNN):
  • x=tf.Tensor(shape=(128, 32, 128, 3), dtype=float16)
  • target=None
  • return_model_output=False
  • return_preds=True
  • beam_width=1
  • top_paths=1
  • kwargs={'training': 'False'}

Tried with different reco_arch and det_arch aswell. And I'm not using det_bs=4 and reco_bs=1024 yet.
Thanks!

[felixdittrich92]

Hey @samuelezoc thanks for the feedback i will try to have a look next week 👍
Would it be an option to use pytorch ftm instead ?

[samuelezoc]

Just built with pytorch and works very well!

[felixdittrich92]

sounds good 🤗