You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Aphrodite is an inference engine that optimizes the serving of HuggingFace-compatible models at scale. Built on vLLM's Paged Attention technology, it delivers high-performance model inference for multiple concurrent users. Aphrodite serves as the backend engine powering [PygmalionAI](https://pygmalion.chat)'s chat platforms and API infrastructure.
7
+
Sonar (formerly Aphrodite Engine) is an inference engine that optimizes the serving of HuggingFace-compatible models at scale. Built on vLLM's Paged Attention technology, it delivers high-performance model inference for multiple concurrent users. Sonar serves as the backend engine powering [PygmalionAI](https://pygmalion.chat)'s chat platforms and API infrastructure.
8
8
9
-
Aphrodite builds upon and integrates the exceptional work from [various projects](#acknowledgements), primarily [vLLM](https://vllm.ai).
9
+
Sonar builds upon and integrates the exceptional work from [various projects](#acknowledgements), primarily [vLLM](https://vllm.ai).
10
10
11
11
## Features
12
12
@@ -24,7 +24,7 @@ Aphrodite builds upon and integrates the exceptional work from [various projects
24
24
25
25
## Quickstart
26
26
27
-
Install the engine:
27
+
Install the engine (the Python package and CLI keep the historical `aphrodite` name for now):
28
28
29
29
```sh
30
30
pip install -U aphrodite-engine
@@ -49,13 +49,13 @@ This will create a [OpenAI](https://platform.openai.com/docs/api-reference/)-com
49
49
50
50
### Notes
51
51
52
-
1. By design, Aphrodite takes up 92% of your GPU's VRAM. If you're not serving an LLM at scale, you may want to limit the amount of memory it takes up. You can do this in the API example by launching the server with the `--gpu-memory-utilization 0.6` (0.6 means 60%).
52
+
1. By design, Sonar takes up 92% of your GPU's VRAM. If you're not serving an LLM at scale, you may want to limit the amount of memory it takes up. You can do this in the API example by launching the server with the `--gpu-memory-utilization 0.6` (0.6 means 60%).
53
53
54
54
2. You can view the full list of commands by running `aphrodite run --help`.
55
55
56
56
## Acknowledgements
57
57
58
-
Aphrodite Engine would have not been possible without the phenomenal work of other open-source projects. A (non-exhaustive) list:
58
+
Sonar would have not been possible without the phenomenal work of other open-source projects. A (non-exhaustive) list:
0 commit comments