Hello! Our static bug checker has found a performance issue in onnxruntime/python/tools/transformers/benchmark.py: run_with_tf_optimizations (1)(2)(3) is repeatedly called in a for loop, but there are tf.function decorated functions run_in_eager_mode and run_in_graph_mode defined and called in run_with_tf_optimizations.

In that case, when run_with_optimizations is called in a loop, the functions run_in_eager_mode and run_in_graph_mode will create a new graph every time, and that can trigger tf.function retracing warning.

Here is the tensorflow document to support it.

Briefly, for better efficiency, it's better to use:

@tf.function
def inner():
    pass

def outer():
    inner()  

than:

def outer():
    @tf.function
    def inner():
        pass
    inner()

Looking forward to your reply.