Skip to content

Commit dae0727

Browse files
jalumajaluma
committed
fix(deploy): improve Docker-Compose and quickstart on Docker (#2037)
* chore: update docker-compose with profiles * docs: add quick start doc
1 parent 6674b46 commit dae0727

File tree

4 files changed

+194
-4
lines changed

4 files changed

+194
-4
lines changed

‎.docker/router.yml

+16
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
http:
2+
services:
3+
ollama:
4+
loadBalancer:
5+
healthCheck:
6+
interval: 5s
7+
path: /
8+
servers:
9+
- url: http://ollama-cpu:11434
10+
- url: http://ollama-cuda:11434
11+
- url: http://host.docker.internal:11434
12+
13+
routers:
14+
ollama-router:
15+
rule: "PathPrefix(`/`)"
16+
service: ollama

‎docker-compose.yaml

+84-4
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,99 @@
11
services:
2-
private-gpt:
2+
3+
#-----------------------------------
4+
#---- Private-GPT services ---------
5+
#-----------------------------------
6+
7+
# Private-GPT service for the Ollama CPU and GPU modes
8+
# This service builds from an external Dockerfile and runs the Ollama mode.
9+
private-gpt-ollama:
310
build:
11+
context: .
412
dockerfile: Dockerfile.external
513
volumes:
614
- ./local_data/:/home/worker/app/local_data
715
ports:
8-
- 8001:8001
16+
- "8001:8001"
917
environment:
1018
PORT: 8001
1119
PGPT_PROFILES: docker
1220
PGPT_MODE: ollama
1321
PGPT_EMBED_MODE: ollama
22+
PGPT_OLLAMA_API_BASE: http://ollama:11434
23+
profiles:
24+
- ""
25+
- ollama
26+
- ollama-cuda
27+
- ollama-host
28+
29+
# Private-GPT service for the local mode
30+
# This service builds from a local Dockerfile and runs the application in local mode.
31+
private-gpt-local:
32+
build:
33+
context: .
34+
dockerfile: Dockerfile.local
35+
volumes:
36+
- ./local_data/:/home/worker/app/local_data
37+
- ./models/:/home/worker/app/models
38+
entrypoint: sh -c ".venv/bin/python scripts/setup && .venv/bin/python -m private_gpt"
39+
ports:
40+
- "8001:8001"
41+
environment:
42+
PORT: 8001
43+
PGPT_PROFILES: local
44+
HF_TOKEN: ${HF_TOKEN}
45+
profiles:
46+
- local
47+
48+
#-----------------------------------
49+
#---- Ollama services --------------
50+
#-----------------------------------
51+
52+
# Traefik reverse proxy for the Ollama service
53+
# This will route requests to the Ollama service based on the profile.
1454
ollama:
15-
image: ollama/ollama:latest
55+
image: traefik:v2.10
1656
ports:
17-
- 11434:11434
57+
- "11435:11434"
58+
- "8081:8080"
59+
command:
60+
- "--providers.file.filename=/etc/router.yml"
61+
- "--log.level=ERROR"
62+
- "--api.insecure=true"
63+
- "--providers.docker=true"
64+
- "--providers.docker.exposedbydefault=false"
65+
- "--entrypoints.web.address=:11434"
66+
volumes:
67+
- /var/run/docker.sock:/var/run/docker.sock:ro
68+
- ./.docker/router.yml:/etc/router.yml:ro
69+
extra_hosts:
70+
- "host.docker.internal:host-gateway"
71+
profiles:
72+
- ""
73+
- ollama
74+
- ollama-cuda
75+
- ollama-host
76+
77+
# Ollama service for the CPU mode
78+
ollama-cpu:
79+
image: ollama/ollama:latest
80+
volumes:
81+
- ./models:/root/.ollama
82+
profiles:
83+
- ""
84+
- ollama
85+
86+
# Ollama service for the CUDA mode
87+
ollama-cuda:
88+
image: ollama/ollama:latest
1889
volumes:
1990
- ./models:/root/.ollama
91+
deploy:
92+
resources:
93+
reservations:
94+
devices:
95+
- driver: nvidia
96+
count: 1
97+
capabilities: [gpu]
98+
profiles:
99+
- ollama-cuda

‎fern/docs.yml

+9
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,9 @@ tabs:
1010
overview:
1111
display-name: Overview
1212
icon: "fa-solid fa-home"
13+
quickstart:
14+
display-name: Quickstart
15+
icon: "fa-solid fa-rocket"
1316
installation:
1417
display-name: Installation
1518
icon: "fa-solid fa-download"
@@ -32,6 +35,12 @@ navigation:
3235
contents:
3336
- page: Introduction
3437
path: ./docs/pages/overview/welcome.mdx
38+
- tab: quickstart
39+
layout:
40+
- section: Getting started
41+
contents:
42+
- page: Quickstart
43+
path: ./docs/pages/quickstart/quickstart.mdx
3544
# How to install PrivateGPT, with FAQ and troubleshooting
3645
- tab: installation
3746
layout:

‎fern/docs/pages/quickstart/quickstart.mdx

+85
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
This guide provides a quick start for running different profiles of PrivateGPT using Docker Compose.
2+
The profiles cater to various environments, including Ollama setups (CPU, CUDA, MacOS) and fully Local setup.
3+
4+
If you want to run PrivateGPT locally without Docker, refer to the [Local Installation Guide](/installation).
5+
6+
#### Prerequisites
7+
- **Docker and Docker Compose:** Ensure both are installed on your system.
8+
[Installation Guide for Docker](https://docs.docker.com/get-docker/), [Installation Guide for Docker Compose](https://docs.docker.com/compose/install/).
9+
- **Clone PrivateGPT Repository:** Clone the PrivateGPT repository to your machine and navigate to the directory:
10+
```sh
11+
git clone https://github.com/zylon-ai/private-gpt.git
12+
cd private-gpt
13+
```
14+
15+
---
16+
17+
## Ollama Setups (Recommended)
18+
19+
Ollama setups are recommended for their ease of use and optimized configurations. Ollama offers different profiles depending on your hardware capabilities and operating system.
20+
21+
### 1. Default/Ollama CPU
22+
23+
**Description:**
24+
This profile runs the Ollama service using CPU resources. It is the standard configuration for running Ollama-based Private-GPT services without GPU acceleration.
25+
26+
**Run:**
27+
To start the services, use either of the following commands:
28+
```sh
29+
docker-compose up
30+
```
31+
or
32+
```sh
33+
docker-compose --profile ollama up
34+
```
35+
36+
### 2. Ollama Nvidia CUDA
37+
38+
**Description:**
39+
This profile leverages GPU acceleration with CUDA support, suitable for computationally intensive tasks that benefit from GPU resources.
40+
41+
**Requirements:**
42+
- Ensure that your system has compatible GPU hardware and the necessary NVIDIA drivers installed. The installation process is detailed [here](https://docs.nvidia.com/cuda/cuda-installation-guide-microsoft-windows/index.html).
43+
44+
**Run:**
45+
To start the services with CUDA support, use:
46+
```sh
47+
docker-compose --profile ollama-cuda up
48+
```
49+
50+
### 3. Ollama Host
51+
52+
**Description:**
53+
This profile is designed for running PrivateGPT using Ollama installed on the host machine. This setup is particularly useful for MacOS users, as Docker does not yet support Metal GPU.
54+
55+
**Requirements:**
56+
- Install Ollama on your machine by following the instructions at [ollama.ai](https://ollama.ai/).
57+
- Start the Ollama service with the command:
58+
```sh
59+
OLLAMA_HOST=0.0.0.0 ollama serve
60+
```
61+
62+
**Run:**
63+
To start the services with the host configuration, use:
64+
```sh
65+
docker-compose --profile ollama-host up
66+
```
67+
68+
---
69+
70+
## Fully Local Setups
71+
72+
### LlamaCPP + HuggingFace Embeddings
73+
74+
**Description:**
75+
This profile runs the Private-GPT services locally using `llama-cpp` and Hugging Face models.
76+
77+
**Requirements:**
78+
- **Hugging Face Token (HF_TOKEN):** Required for accessing Hugging Face models. Obtain your token following [this guide](/installation/getting-started/troubleshooting#downloading-gated-and-private-models).
79+
80+
**Run:**
81+
Start the services with your Hugging Face token:
82+
```sh
83+
HF_TOKEN=<your_hf_token> docker-compose up --profile local
84+
```
85+
Replace `<your_hf_token>` with your actual Hugging Face token.

0 commit comments

Comments
 (0)