v1.8.3: Add an experimental REST API
What's Changed
- Add a FastAPI app by @juliendenize in #113
We released an experimental REST API leveraging Fast API to handle requests from tokenization, through generation via calls to an engine, to detokenization.
For a detailed documentation see [https://mistralai.github.io/mistral-common/usage/experimental/].
Here is how to launch the server:
pip install mistral-common[server]
mistral_common serve mistralai/Magistral-Small-2507 \
--host 127.0.0.1 --port 8000 \
--engine-url http://127.0.0.1:8080 --engine-backend llama_cpp \
--timeout 60Then you can see the Swagger at: http://localhost:8000.
Full Changelog: v1.8.2...v1.8.3