README.md

ollama

Containers running Ollama

Usage with docker

• Ensure that GPU support is enabled in docker (or adapt docker-compose.yaml) :

docker run --gpus all nvcr.io/nvidia/k8s/cuda-sample:nbody nbody -gpu -benchmark

• Start : docker compose up -d
• To use Ollama CLI :

# pull models from https://ollama.com/library
docker compose exec ollama ollama pull llama3
docker compose exec ollama ollama pull gemma2
# interactive model
docker compose exec ollama ollama run llama3.1

• To use Ollama API :

# list models
curl -sS http://localhost:11434/api/tags | jq -r '.models[].name'

# pull model from https://ollama.com/library
curl http://localhost:11434/api/pull -d '{
  "name": "llama3"
}'

# use model
curl http://localhost:11434/api/generate -d '{
  "model": "llama3.2",
  "prompt": "Why is the sky blue?"
}'

• To create custom model from OLLAMA Modelfile, a sample models/geoassistant is available :

docker compose exec ollama /bin/bash
ollama create geoassistant -f /models/geoassistant/Modelfile
ollama run geoassistant
# Do you know the most visited museums in Paris?

Ressources

• github.com - ollama/ollama
• hub.docker.com - ollama/ollama
• ollama - API
• mborne.github.io/outils/cuda-toolkit (french)

Clients :

• Open WebUI - Starting With Ollama / mborne/docker-devbox - open-webui
• langchain - OllamaChat (Python) / langchain - OllamaChat (JS)