Skip to content

GRPC.md

Latest commit

 

History

History
131 lines (93 loc) · 3.04 KB

GRPC.md

File metadata and controls

131 lines (93 loc) · 3.04 KB
Raw

gRPC Live Request Stream

The gateway exposes a gRPC server that streams live request/response information to connected clients in real time.

Configuration

Add grpc_addr to your config.yaml to enable the gRPC stream:

listen_addr: "127.0.0.1:4180"
grpc_addr: "127.0.0.1:4181"

The gRPC server only starts if grpc_addr is explicitly set. If omitted, the gateway runs without the streaming interface.

Service Definition

service RequestStreamService {
  rpc StreamRequests(StreamFilter) returns (stream RequestEvent);
}

message StreamFilter {
  string proxy_key = 1;   // optional: filter by proxy key (e.g. "openai")
  string path_prefix = 2; // optional: filter by request path prefix
}

message RequestEvent {
  google.protobuf.Timestamp timestamp = 1;
  string method = 2;
  string path = 3;
  int32 status_code = 4;
  bytes request_body = 5;
  bytes response_body = 6;
  string client_ip = 7;
  string proxy_key = 8;
}

Request and response bodies are capped at 64 KB per event.

TUI Client

A built-in TUI client renders live request events with colored output.

Usage

# Stream all requests
go run ./cmd/streamclient --addr 127.0.0.1:4181

# Filter by proxy key
go run ./cmd/streamclient --addr 127.0.0.1:4181 --proxy-key openai

# Filter by path prefix
go run ./cmd/streamclient --addr 127.0.0.1:4181 --path-prefix /v1/chat

Flags

Flag Default Description
--addr 127.0.0.1:4181 gRPC server address
--proxy-key (none) Only show events for this proxy
--path-prefix (none) Only show events matching prefix

Controls

  • ↑ / ↓ — Scroll through events
  • q / Ctrl+C — Quit

Development

Prerequisites

# Install protobuf compiler
sudo apt-get install -y protobuf-compiler

# Install Go protoc plugins
go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest

Regenerate Proto

mkdir -p gen/acg/v1
protoc \
  --proto_path=proto \
  --go_out=gen --go_opt=paths=source_relative \
  --go-grpc_out=gen --go-grpc_opt=paths=source_relative \
  acg/v1/stream.proto

Build

go build ./...

Run Tests

go test ./...

Quick Test (end-to-end)

Terminal 1 — start the gateway:

export OPENAI_CREDENTIAL=sk-your-key
go run ./cmd/gateway --config config.yaml

Terminal 2 — start the stream client:

go run ./cmd/streamclient --addr 127.0.0.1:4181

Terminal 3 — send a request through the gateway:

curl http://127.0.0.1:4180/openai/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4","messages":[{"role":"user","content":"Hello"}]}'

The TUI client will display the request in real time with colored status codes (green 2xx, yellow 3xx, red 4xx/5xx), method, path, proxy key, client IP, and truncated request/response bodies.