Skip to content

v1.8.3: Add an experimental REST API

Choose a tag to compare

@juliendenize juliendenize released this 25 Jul 16:16
· 24 commits to main since this release
f47fb16

What's Changed

We released an experimental REST API leveraging Fast API to handle requests from tokenization, through generation via calls to an engine, to detokenization.

For a detailed documentation see [https://mistralai.github.io/mistral-common/usage/experimental/].

Here is how to launch the server:

pip install mistral-common[server]

mistral_common serve mistralai/Magistral-Small-2507 \
--host 127.0.0.1 --port 8000 \
--engine-url http://127.0.0.1:8080 --engine-backend llama_cpp \
--timeout 60

Then you can see the Swagger at: http://localhost:8000.

Full Changelog: v1.8.2...v1.8.3