SRE over the shoulder assistant

Prototype of an over the shoulder SRE AI assistant, this prototype is helping engineer make sense of graph they are watching.

It could help SRE with what ever tasks they are doing:

• Analyze on-screen graphs to instantly find anomalies, trends, and correlations with other visible metrics.
• Automatically document every step taken during an outage for simplified incident reporting or to train other AI systems.
• Act as a real-time guardrail to ensure commands and actions align with security and compliance policies.
• Provide on-demand, context-aware help, like remembering command flags or suggesting relevant code improvements.

Getting Started

git clone git@github.com:Rootly-AI-Labs/SRE-screen-sherpa.git
cd SRE-screen-sherpa
python -m venv venv
source venv/bin/activate
pip install -r requirements.txt
python sre-screen-sherpa.py

To generate your own video

• Record your graph video, feed it to Gemini along with the following prompt.
• Paste the output in the video-annotations.json file
• Insert your video in this directory with the name input-video.mp4
• Run python sre-screen-sherpa.py

Output

• The processed video will be saved as annoted_video.mp4.