Automated deployment script to run your private, self-hosted LLM inference server on Akamai Cloud GPU instances. Pre-configured with OpenAI's gpt-oss-20b (20B parameter open-source model) optimized for instruction following. Get vLLM and Open-WebUI up and running in minutes with a single command.
gpt-oss-20b is OpenAI's open-source 20B parameter model that delivers exceptional performance-per-parameter efficiency. Despite its compact size, it achieves an Intelligence Index score of 52 (high reasoning) on the Artificial Analysis benchmark, ranking alongside much larger models like MiniMax-M2 and outperforming many 70B+ parameter alternatives.
Key advantages:
- Memory efficient: 20B parameters fit comfortably on a single RTX 4000 Ada GPU with room for context
- High-throughput inference: Optimized architecture enables fast token generation, ideal for real-time applications
- Cost-effective: Smaller model size means lower GPU requirements and reduced operational costs
- Production-ready: Instruction-tuned for reliable, high-quality responses out of the box
Check out these other quickstart repositories:
| Model | Parameters | Description | Repository |
|---|---|---|---|
| GPT-OSS-20B | 20B | Large-scale open-source GPT model | ai-quickstart-gpt-oss-20b |
| Qwen3-14B-FP8 | 14B | Qwen3 with FP8 quantization (this repo) | ai-quickstart-qwen3-14b-fp8 |
| NVIDIA Nemotron Nano 9B v2 | 9B | NVIDIA's efficient Nemotron model | ai-quickstart-nvidia-nemotron-nano-9b-v2 |
Just run this single command:
curl -fsSL https://raw.githubusercontent.com/linode/ai-quickstart-gpt-oss-20b/main/deploy.sh | bashThat's it! The script will download required files and guide you through the interactive deployment process.
- Fully Automated Deployment: handles instance creation with real-time progress tracking
- Basic AI Stack: vLLM for LLM inference with pre-loaded model and Open-WebUI for chat interface
- Cross-Platform Support: Works on macOS and Windows (Git Bash/WSL)
- Ubuntu 24.04 LTS with NVIDIA drivers
- Docker & NVIDIA Container Toolkit
- Systemd service for automatic startup on reboot
| Service | Description | |
|---|---|---|
 |
Caddy | Reverse proxy with automatic HTTPS (port 80/443) |
 |
vLLM | High-throughput LLM inference engine with OpenAI-compatible API (port 8000) |
 |
Open-WebUI | Feature-rich web interface for AI chat interactions (port 8080) |
- Active Linode account with GPU access enabled
- Required: bash, curl, ssh, jq
- Note: jq will be auto-installed if missing
No installation required - just run:
curl -fsSL https://raw.githubusercontent.com/linode/ai-quickstart-gpt-oss-20b/main/deploy.sh | bashDownload the script and run locally:
curl -fsSLO https://raw.githubusercontent.com/linode/ai-quickstart-gpt-oss-20b/main/deploy.sh
bash deploy.shIf you prefer to inspect or customize the scripts:
git clone https://github.com/linode/ai-quickstart-gpt-oss-20b
cd ai-quickstart-gpt-oss-20b
./deploy.shNote
if you like to add more services check out docker compose template file
vi /template/docker-compose.yml
The script will ask you to:
- Choose a region (e.g., us-east, eu-west)
- Select GPU instance type
- Provide instance label
- Select or generate SSH keys
- Confirm deployment
The script automatically:
- Creates GPU instance in your linode account
- Monitors cloud-init installation progress
- Waits for Open-WebUI health check
- Waits for vLLM model loading
Once complete, you'll see:
🎉 Setup Complete!
✅ Your AI LLM instance is now running!
🌐 Access URLs:
Open-WebUI: https://<ip-label>.ip.linodeusercontent.com
🔐 Access Credentials:
SSH: ssh -i /path/to/your/key root@<instance-ip>
# Install script called by cloud-init service
/opt/ai-quickstart-gpt-oss-20b/install.sh
# docker compose file called by systemctl at startup
/opt/ai-quickstart-gpt-oss-20b/docker-compose.yml
# Caddy reverse proxy configuration
/opt/ai-quickstart-gpt-oss-20b/Caddyfile
# service definition
/etc/systemd/system/ai-quickstart-gpt-oss-20b.service
To delete a deployed instance:
# Remote execution
curl -fsSL https://raw.githubusercontent.com/linode/ai-quickstart-gpt-oss-20b/main/delete.sh | bash -s -- <instance_id>
# Or download script and run
curl -fsSLO https://raw.githubusercontent.com/linode/ai-quickstart-gpt-oss-20b/main/delete.sh
bash delete.sh <instance_id>The script will show instance details and ask for confirmation before deletion.
ai-quickstart-gpt-oss-20b/
├── deploy.sh # Main deployment script
├── delete.sh # Instance deletion script
├── script/
│ └── quickstart_tools.sh # Shared functions (API, OAuth, utilities)
└── template/
├── cloud-init.yaml # Cloud-init configuration
├── docker-compose.yml # Docker Compose configuration
├── Caddyfile # Caddy reverse proxy configuration
└── install.sh # Post-boot installation script
-
Configure Cloud Firewall (Recommended)
- Create Linode Cloud Firewall
- Restrict access to ports 80/443 by source IP
- Allow SSH (port 22) from trusted IPs only
-
SSH Security
- SSH key authentication required
- Root password provided for emergency console access only
# SSH into your instance
ssh -i /path/to/your/key root@<instance-ip>
# Check container status
docker ps -a
# Check Docker containers log
cd /opt/ai-quickstart-gpt-oss-20b && docker compose logs -f
# Check systemd service status
systemctl status ai-quickstart-gpt-oss-20b.service
# View systemd service logs
journalctl -u ai-quickstart-gpt-oss-20b.service -n 100
# Check cloud-init logs
tail -f /var/log/cloud-init-output.log -n 100
# Restart all services
systemctl restart ai-quickstart-gpt-oss-20b.service
# Check NVIDIA GPU status
nvidia-smi
# Check vLLM loaded models
curl http://localhost:8000/v1/models
# Check Open-WebUI health
curl http://localhost:8080/health
# Check vLLM container logs
docker logs vllmSuccessful deployments register anonymous statistics (project name, region, instance type) to help improve the service. To opt out, remove the deployment_complete call from deploy.sh.
Issues and pull requests are welcome! For major changes, please open an issue first to discuss what you would like to change.
This project is licensed under the Apache License 2.0.