We should add some benchmarking w.r.t. the python library.
For instance, benchmark embedding a big batch of strings with mini LM and compare performance with the default python pipeline, to ensure we are in the same ballpark.
Could probably combine this work with the work on testing directly against the python implementation.
We should add some benchmarking w.r.t. the python library.
For instance, benchmark embedding a big batch of strings with mini LM and compare performance with the default python pipeline, to ensure we are in the same ballpark.
Could probably combine this work with the work on testing directly against the python implementation.