Simple step-by-step installation for Ubuntu 22.04/24.04
This guide will help you install LLM-MS (LLM Meta Search) on a fresh Ubuntu server.
For a fully automated installation, use our installation script:
# Clone the repository
cd ~
git clone https://github.com/dmsl/llmms.git
cd llmms
# Make the installer executable
chmod +x install.sh
# Run the installer
./install.shThe script will automatically:
- ✅ Check system requirements
- ✅ Install all dependencies (Ollama, ChromaDB, Python, Apache)
- ✅ Download AI models
- ✅ Configure all services
- ✅ Verify the installation
Installation takes 20-30 minutes.
After installation completes, access LLM-MS at: http://your-server-ip
If you prefer to install manually or the automated script doesn't work, follow these steps:
- Ollama - Runs AI models locally
- ChromaDB - Stores document embeddings for search
- Python & FastAPI - Backend server
- Apache2 - Web server
- Ubuntu 22.04 or 24.04 LTS
- At least 16GB RAM (32GB recommended)
- 50GB free disk space
- Internet connection
- sudo privileges
sudo apt update && sudo apt upgrade -yInstall Ollama using the official script:
curl -fsSL https://ollama.com/install.sh | shVerify installation:
ollama --versionStart Ollama and enable it to run at startup:
sudo systemctl start ollama
sudo systemctl enable ollama
sudo systemctl status ollamaDownload required AI models (this will take 10-15 minutes):
# Embedding model (required for search)
ollama pull nomic-embed-text
# AI models
ollama pull llama3.1:8b
ollama pull mistral:7b
ollama pull qwen2.5:7bInstall Python 3:
sudo apt install -y python3 python3-pip python3-venvInstall ChromaDB:
pip3 install chromadbCreate a directory for ChromaDB data:
mkdir -p ~/chromadb_dataCreate a systemd service to run ChromaDB automatically:
sudo tee /etc/systemd/system/chromadb.service > /dev/null <<EOF
[Unit]
Description=ChromaDB Service
After=network.target
[Service]
Type=simple
User=$USER
WorkingDirectory=$HOME
ExecStart=/usr/local/bin/chroma run --path $HOME/chromadb_data --host 127.0.0.1 --port 8000
Restart=always
RestartSec=3
[Install]
WantedBy=multi-user.target
EOFStart ChromaDB:
sudo systemctl daemon-reload
sudo systemctl start chromadb
sudo systemctl enable chromadb
sudo systemctl status chromadbDownload the LLM-MS application from GitHub:
cd ~
git clone https://github.com/dmsl/llmms.git
cd llmms/backendCreate a Python virtual environment:
python3 -m venv venvActivate the virtual environment:
source venv/bin/activateInstall Python packages:
pip install --upgrade pip
# Core packages
pip install fastapi uvicorn[standard]
pip install ollama chromadb
pip install python-multipart werkzeug
# Additional packages
pip install numpy pandas pydantic
pip install pypdf PyPDF2 python-docx pillow openpyxlCreate the uploads directory:
mkdir -p /tmp/uploadsCreate a systemd service to run the FastAPI backend:
sudo tee /etc/systemd/system/llmms-api.service > /dev/null <<EOF
[Unit]
Description=LLM-MS FastAPI Service
After=network.target ollama.service chromadb.service
[Service]
Type=simple
User=$USER
WorkingDirectory=$HOME/llmms/backend
Environment="PYTHONPATH=$HOME/llmms/backend"
ExecStart=$HOME/llmms/backend/venv/bin/uvicorn app.main:app --host 127.0.0.1 --port 62828
Restart=always
RestartSec=3
[Install]
WantedBy=multi-user.target
EOFStart the API service:
sudo systemctl daemon-reload
sudo systemctl start llmms-api
sudo systemctl enable llmms-api
sudo systemctl status llmms-apiInstall Apache2:
sudo apt install -y apache2Enable required Apache modules:
sudo a2enmod proxy proxy_http proxy_wstunnel rewrite headers sslCreate Apache configuration for LLM-MS:
sudo tee /etc/apache2/sites-available/llmms.conf > /dev/null <<EOF
<VirtualHost *:80>
ServerName localhost
DocumentRoot $HOME/llmms/frontend
<Directory $HOME/llmms/frontend>
Options -Indexes +FollowSymLinks
AllowOverride All
Require all granted
</Directory>
# Proxy API requests to FastAPI
ProxyPreserveHost On
ProxyPass /api/ http://127.0.0.1:62828/
ProxyPassReverse /api/ http://127.0.0.1:62828/
ErrorLog \${APACHE_LOG_DIR}/llmms_error.log
CustomLog \${APACHE_LOG_DIR}/llmms_access.log combined
</VirtualHost>
EOFEnable the site and restart Apache:
sudo a2dissite 000-default.conf
sudo a2ensite llmms.conf
sudo apache2ctl configtest
sudo systemctl restart apache2Check that all services are running:
# Check Ollama
systemctl status ollama
# Check ChromaDB
systemctl status chromadb
# Check FastAPI
systemctl status llmms-api
# Check Apache
systemctl status apache2Test connectivity:
# Test Ollama
curl http://localhost:11434
# Test ChromaDB
curl http://localhost:8000/api/v1/heartbeat
# Test FastAPI
curl http://localhost:62828/get_modelsOpen your web browser and go to:
http://your-server-ip
Replace your-server-ip with:
localhostif accessing from the same machine- Your server's IP address if accessing remotely
GPU acceleration makes AI models 10-50x faster. Choose your GPU type:
Step 1: Install NVIDIA Driver
sudo add-apt-repository ppa:graphics-drivers/ppa -y
sudo apt update
sudo ubuntu-drivers autoinstall
sudo rebootStep 2: Verify Installation
After reboot:
nvidia-smiYou should see your GPU listed.
Step 3: Configure Ollama for GPU
sudo systemctl edit ollama.serviceAdd these lines:
[Service]
Environment="CUDA_VISIBLE_DEVICES=0"
Save and restart:
sudo systemctl daemon-reload
sudo systemctl restart ollamaStep 4: Test GPU
# Run a model and watch GPU usage
nvidia-smi -l 1In another terminal:
ollama run llama3.1:8b "Hello"You should see GPU usage spike.
Step 1: Install ROCm
# Download installer
wget https://repo.radeon.com/amdgpu-install/6.2.4/ubuntu/jammy/amdgpu-install_6.2.60204-1_all.deb
# Install
sudo apt install ./amdgpu-install_6.2.60204-1_all.deb
# Install ROCm
sudo amdgpu-install --no-dkms --usecase=rocm
# Add user to groups
sudo usermod -a -G render,video $USER
sudo rebootStep 2: Install Ollama ROCm Version
# Download ROCm version
curl -L https://ollama.com/download/ollama-linux-amd64-rocm.tgz -o ollama-rocm.tgz
# Extract
sudo tar -C /usr -xzf ollama-rocm.tgzStep 3: Configure for AMD GPU
sudo systemctl edit ollama.serviceAdd these lines (adjust gfx version for your GPU):
[Service]
Environment="HSA_OVERRIDE_GFX_VERSION=10.3.0"
Environment="HIP_VISIBLE_DEVICES=0"
Common gfx versions:
- RX 6600/6700:
10.3.0 - RX 7700/7900:
11.0.0
Save and restart:
sudo systemctl daemon-reload
sudo systemctl restart ollamaStep 4: Test GPU
# Monitor GPU
watch -n 1 rocm-smiIn another terminal:
ollama run llama3.1:8b "Hello"Check logs:
sudo journalctl -u llmms-api -n 50
sudo journalctl -u chromadb -n 50
sudo journalctl -u ollama -n 50Check Apache logs:
sudo tail -f /var/log/apache2/llmms_error.logCheck if port 80 is open:
sudo netstat -tlnp | grep :80Install GPU acceleration (see Step 9 above)
Check what's using the port:
sudo lsof -i :62828
sudo lsof -i :8000
sudo lsof -i :11434# Restart all services
sudo systemctl restart ollama chromadb llmms-api apache2
# Check all services
sudo systemctl status ollama chromadb llmms-api apache2
# View logs
sudo journalctl -u llmms-api -f
# Pull new AI model
ollama pull model-name
# List installed models
ollama list- Secure your installation - Set up firewall and SSL certificate
- Add more models - Use
ollama pullto add more AI models - Configure backups - Back up your ChromaDB data regularly
- Check logs:
sudo journalctl -u llmms-api -f - Ollama docs: https://ollama.com/docs
- FastAPI docs: https://fastapi.tiangolo.com
- 11434 - Ollama
- 8000 - ChromaDB
- 62828 - FastAPI Backend
- 80 - Apache Web Server
Installation Complete! 🎉
Your LLM-MS system is now running at http://your-server-ip