Skip to content

Per-service restart and lifecycle management #3

Description

@raamsri

Problem

vctl service up/down manages all services as a single unit via lifecycle.start/stop. There's no way to restart an individual service (e.g., restart a crashed pipecat worker without restarting the inference server).

When a service crashes (e.g., pipecat worker dies from a GStreamer bug), the only recovery is to restart everything or manually SSH in and run the start command.

Proposed

Add per-service lifecycle commands:

vctl service restart <service-name>
vctl service logs <service-name>

This requires velocity.yml services to declare their own start/stop commands:

services:
  atoms-worker:
    path: ./atoms
    start: $VENV/python main.py --worker-mode warm
    stop: pkill -f "main.py --worker-mode"
  inference:
    path: ./inference
    port: 8100
    start: $VENV/uvicorn app.main:app --host 0.0.0.0 --port 8100 --workers 2

Currently services only declares path, port, and routes — the actual process management is embedded in the monolithic lifecycle.start shell script.

Context

  • Pipecat worker crashed during audio mixing (GStreamer stack smash), leaving all subsequent calls with no available worker
  • Recovery required manually SSH-ing and running the exact start command with correct paths
  • atoms-platform uses turbo which manages child processes, but pipecat uses nohup background processes with no supervision

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions