This project is no longer actively maintained. While existing releases remain available, there are no planned updates, bug fixes, new features, or security patches. Users should be aware that vulnerabilities may not be addressed.
In this example we will use a pretrained Wav2Vec2 model for Speech2Text using the transformers library: https://huggingface.co/docs/transformers/model_doc/wav2vec2 and serve it using torchserve.
Apart from the usual dependencies as shown here: https://github.com/pytorch/serve/blob/master/docs/getting_started.md, we need to install torchaudio and transformers.
You can install these into your current environment or follow these steps which should give you all necessary prerequisites from scratch:
- Install miniconda: https://docs.conda.io/en/latest/miniconda.html
- run
python ../../ts_scripts/install_dependencies.pyto install binary dependencies - Install all needed packages with
conda env create -f environment.yml - Activate conda environment:
conda activate wav2vec2env
Next, we need to download our wav2vec2 model and archive it for use by torchserve:
./download_wav2vec2.py # Downloads model and creates folder `./model` with all necessary files
./archive_model.sh # Creates .mar archive using torch-model-archiver and moves it to folder `./model_store`Now let's start the server and try it out with our example file!
torchserve --start --model-store model_store --models Wav2Vec2=Wav2Vec2.mar --ncs --disable-token-auth --enable-model-api
# Once the server is running, let's try it with:
curl -X POST http://127.0.0.1:8080/predictions/Wav2Vec2 --data-binary '@./sample.wav' -H "Content-Type: audio/basic"
# Which will happily return:
I HAD THAT CURIOSITY BESIDE ME AT THIS MOMENT%