Skip to content

Commit c0b0f07

Browse files
committed
feat(cli): implement VSR CLI tool for semantic router management
This PR introduces the VSR (vLLM Semantic Router) CLI tool - a comprehensive command-line interface that reduces setup time from hours to minutes and provides a unified interface for deployment, monitoring, and troubleshooting across multiple environments. ## Key Features ### Core Commands - **deploy**: Multi-environment deployment (Local, Docker Compose, Kubernetes, Helm) - **config**: Configuration management with validation and templates - **model**: Model lifecycle management (download, list, validate, remove, inspect) - **status**: Health monitoring and status checks - **debug**: Interactive debugging and diagnostic tools - **dashboard**: Dashboard management and access - **test**: Prompt testing and validation - **upgrade**: Seamless version upgrades - **get**: Resource inspection (logs, pods, services) ### Implementation Details - Built with Go and Cobra framework for robust CLI experience - Comprehensive test coverage with 3,400+ lines of test code - Support for multiple deployment targets with environment detection - Process lifecycle management with orphan process prevention - Integrated health checking and diagnostics - Shell completion support (bash, zsh, fish, powershell) ### Documentation - Complete CLI documentation with quickstart guide - Command reference with examples - Troubleshooting guide - Integration with existing semantic-router documentation ### Files Changed - 49 files changed, 11,531 insertions(+), 16 deletions(-) - New CLI implementation in src/semantic-router/cmd/vsr/ - New CLI packages in src/semantic-router/pkg/cli/ - Documentation in website/docs/cli/ - Build system integration in tools/make/build-run-test.mk Resolves #234 Signed-off-by: Srinivas A <[email protected]>
1 parent a03202b commit c0b0f07

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

53 files changed

+11579
-36
lines changed

README.md

Lines changed: 52 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -93,7 +93,58 @@ Watch the quick demo of the dashboard below:
9393

9494
## Quick Start 🚀
9595

96-
Get up and running in seconds with our interactive setup script:
96+
### Using VSR CLI (Recommended)
97+
98+
The `vsr` CLI tool provides a unified interface for managing the vLLM Semantic Router across all environments. It reduces setup time from hours to minutes with intelligent auto-detection, comprehensive diagnostics, and beautiful CLI output.
99+
100+
#### Installation
101+
102+
```bash
103+
# Clone and build
104+
cd semantic-router/src/semantic-router
105+
make build-cli
106+
export PATH=$PATH:$(pwd)/bin
107+
108+
# Verify installation
109+
vsr --version
110+
```
111+
112+
#### Get Started in 4 Commands
113+
114+
```bash
115+
vsr init # Initialize configuration
116+
make download-models # Download AI models
117+
vsr config validate # Validate setup
118+
vsr deploy docker # Deploy with Docker Compose
119+
```
120+
121+
#### Key Features
122+
123+
- **Multi-Environment Support**: Deploy to Local, Docker, Kubernetes, or Helm
124+
- **Model Management**: Download, validate, list, and inspect models
125+
- **Health Monitoring**: Status checks, diagnostics, and health reports
126+
- **Debug Tools**: Interactive debugging and troubleshooting
127+
- **Dashboard Integration**: Auto-detect and open dashboard in browser
128+
- **Enhanced Logging**: Multi-environment log fetching with filtering
129+
130+
#### Common Commands
131+
132+
```bash
133+
vsr status # Check deployment status
134+
vsr logs --follow # View logs in real-time
135+
vsr health # Quick health check
136+
vsr dashboard # Open dashboard
137+
vsr model list # List available models
138+
vsr debug # Run diagnostics
139+
vsr upgrade docker # Upgrade deployment
140+
vsr undeploy docker # Stop deployment
141+
```
142+
143+
For complete CLI documentation, see [src/semantic-router/cmd/vsr/README.md](src/semantic-router/cmd/vsr/README.md) or [Quick Start Guide](src/semantic-router/cmd/vsr/QUICKSTART.md).
144+
145+
### Using Quickstart Script
146+
147+
Alternatively, get up and running in seconds with our interactive setup script:
97148

98149
```bash
99150
bash ./scripts/quickstart.sh
Lines changed: 165 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,165 @@
1+
# VSR Quick Start Guide
2+
3+
Get the vLLM Semantic Router up and running in minutes.
4+
5+
## Prerequisites
6+
7+
- **Go 1.21+** (for building)
8+
- **Docker** (for Docker deployments)
9+
- **kubectl** (for Kubernetes deployments)
10+
- **Helm** (for Helm deployments)
11+
12+
## 1. Build VSR
13+
14+
```bash
15+
cd semantic-router/src/semantic-router
16+
make build-cli
17+
export PATH=$PATH:$(pwd)/bin
18+
```
19+
20+
## 2. Initialize Configuration
21+
22+
```bash
23+
vsr init
24+
```
25+
26+
This creates `config/config.yaml`. Edit it to configure your model and endpoints.
27+
28+
## 3. Download Models
29+
30+
```bash
31+
make download-models
32+
```
33+
34+
## 4. Validate Configuration
35+
36+
```bash
37+
vsr config validate
38+
```
39+
40+
Fix any errors reported before proceeding.
41+
42+
## 5. Deploy
43+
44+
Choose your deployment environment:
45+
46+
### Local (Development)
47+
48+
```bash
49+
vsr deploy local
50+
```
51+
52+
### Docker Compose (Recommended)
53+
54+
```bash
55+
vsr deploy docker
56+
```
57+
58+
### Kubernetes
59+
60+
```bash
61+
vsr deploy kubernetes --namespace default
62+
```
63+
64+
### Helm
65+
66+
```bash
67+
vsr deploy helm --namespace default
68+
```
69+
70+
## 6. Check Status
71+
72+
```bash
73+
vsr status
74+
```
75+
76+
## 7. Test the Router
77+
78+
```bash
79+
vsr test-prompt "What is the weather today?"
80+
```
81+
82+
## 8. View Logs
83+
84+
```bash
85+
vsr logs --follow
86+
```
87+
88+
## Common Commands
89+
90+
| Command | Purpose |
91+
|---------|---------|
92+
| `vsr status` | Check deployment status |
93+
| `vsr logs` | View logs |
94+
| `vsr health` | Quick health check |
95+
| `vsr dashboard` | Open dashboard in browser |
96+
| `vsr model list` | List available models |
97+
| `vsr undeploy [env]` | Stop deployment |
98+
| `vsr upgrade [env]` | Upgrade to latest version |
99+
| `vsr debug` | Run diagnostics |
100+
101+
## Troubleshooting
102+
103+
### Configuration Issues
104+
105+
```bash
106+
vsr config validate --verbose
107+
```
108+
109+
### Deployment Issues
110+
111+
```bash
112+
vsr debug
113+
```
114+
115+
### Port Conflicts
116+
117+
Check which ports are in use:
118+
119+
```bash
120+
vsr debug
121+
```
122+
123+
### Can't Connect to Dashboard
124+
125+
```bash
126+
# For Docker/Local
127+
vsr dashboard
128+
129+
# For Kubernetes/Helm
130+
vsr dashboard --namespace [your-namespace]
131+
```
132+
133+
## Next Steps
134+
135+
- Read the [full documentation](README.md) for advanced features
136+
- Learn about [model management](README.md#model-commands)
137+
- Explore [deployment options](README.md#-deployment)
138+
- Set up [monitoring and metrics](README.md#monitoring-commands)
139+
140+
## Getting Help
141+
142+
```bash
143+
# General help
144+
vsr --help
145+
146+
# Command-specific help
147+
vsr deploy --help
148+
vsr model --help
149+
```
150+
151+
## Quick Reference
152+
153+
```bash
154+
# Full workflow
155+
vsr init # Initialize config
156+
make download-models # Download models
157+
vsr config validate # Validate
158+
vsr deploy docker # Deploy
159+
vsr status # Check status
160+
vsr test-prompt "hello" # Test
161+
vsr logs --follow # Monitor
162+
vsr undeploy docker # Clean up
163+
```
164+
165+
For complete documentation, see [README.md](README.md).

0 commit comments

Comments
 (0)