-
Notifications
You must be signed in to change notification settings - Fork 1
128 lines (116 loc) · 6.89 KB
/
Copy pathscheduled-runs.yml
File metadata and controls
128 lines (116 loc) · 6.89 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
# Run due, unattended scheduled agent runs (lik-ui/scripts/run_scheduled.py). Runs on a cadence
# with no internet-facing endpoint and no long-lived GitHub secret: GitHub OIDC assumes the
# SSM-read role, fetches the shared Anthropic key + a table-SCOPED DB credential from SSM, and
# connects to the public Lightsail Postgres directly. Mirrors prune-sessions.yml; see
# docs/plans/2026-07-28-002-feat-scheduled-unattended-agent-runs-plan.md.
#
# Least privilege (R18/U9): this job connects with a dedicated DB role granted only on the
# scheduled_runs + sessions tables — NOT the master credential — so a compromise of the CI
# credential cannot read/write the whole database over the public endpoint. The scoped role's
# user + password live in SSM ($SSM_PREFIX/shared/SCHEDULED_RUNS_DB_{USER,PASSWORD}); U9 provisions
# the role and grants the SSM-read OIDC role access to those params.
#
# Prerequisites (set once in repo Settings, on the `prod` environment), same as prune-sessions.yml:
# - Variable AWS_SSM_READ_ROLE_ARN, AWS_REGION, SSM_PREFIX.
name: Scheduled agent runs
on:
schedule:
- cron: "23 7 * * *" # daily at 07:23 UTC — the scan cadence bounds scheduling granularity (tunable)
workflow_dispatch: {} # manual run for verification / catch-up
permissions:
id-token: write # required for OIDC
contents: read
issues: write # a failed scan opens a tracking issue (ops-level alerting; owner-level is in-app)
# Shared, repo-wide serialization group (see deploy-images.yml, which uses the SAME group). It does
# two jobs at once:
# 1. Never let two scans overlap (a slow run + the next tick) — the atomic claim also guards this,
# but serializing avoids racing on the same due rows at all.
# 2. Never let a container deploy run WHILE a scan is running. A scheduled run drives an agent that
# holds a long-lived MCP session to lik-mcp; a deploy replaces that container (a rolling cutover
# Lightsail can't drain), which severs the session mid-run and fails the sync. Sharing the group
# makes a deploy queue behind an in-flight scan (and vice versa) instead of colliding.
# Trade-off: an urgent deploy waits up to one scan (~tens of minutes). Acceptable — runs are
# idempotent/resumable, so the cost of a collision would only be a wasted re-run, not data loss.
concurrency:
group: lik-prod-mutations
cancel-in-progress: false
jobs:
run-scheduled:
runs-on: ubuntu-latest
# Fail fast instead of hanging: a stalled scan otherwise holds the concurrency group and
# starves the next tick. Must exceed one run's per-agent hard backstop (max_runtime_s + the
# scanner's 120s margin) so the scanner records the outcome itself rather than being killed
# mid-run by GitHub (a kill leaves the row unrecorded). Today's largest budget is 3000s, so
# 55 min clears 3000+120s with room for setup. If many schedules are due at once they run
# sequentially within the job, so raise this or cap the scan if that grows.
timeout-minutes: 55
# The prod environment scopes the vars/role below and its `main` branch policy; scheduled runs
# execute on the default branch, which satisfies it.
environment: prod
env:
SSM_PREFIX: ${{ vars.SSM_PREFIX }}
LIK_UI_ENV: prod
LIK_UI_DB_SSLMODE: require
# Only used to log each run's owner-facing chat page in the job output (this job serves no
# HTTP and builds no callbacks). The custom domain is stable and already hardcoded in
# deploy-images.yml; SSM has no such param (terraform computes it from the Lightsail URL).
LIK_UI_APP_BASE_URL: https://ui.lik.navapbc.com
steps:
- uses: actions/checkout@v6
- name: Install uv
uses: astral-sh/setup-uv@v7
- name: Configure AWS credentials (OIDC)
uses: aws-actions/configure-aws-credentials@v6
with:
role-to-assume: ${{ vars.AWS_SSM_READ_ROLE_ARN }}
aws-region: ${{ vars.AWS_REGION }}
- name: Fetch config + secrets from SSM
run: |
# Non-secret config (String params) — the DB target, one source shared with infra.
inst=$(aws ssm get-parameter --name "$SSM_PREFIX/config/DB_INSTANCE" \
--query Parameter.Value --output text)
echo "DB_INSTANCE=$inst" >> "$GITHUB_ENV"
uidb=$(aws ssm get-parameter --name "$SSM_PREFIX/config/LIK_UI_DB_NAME" \
--query Parameter.Value --output text)
echo "LIK_UI_DB_NAME=$uidb" >> "$GITHUB_ENV"
# Secrets (SecureString) — masked before they touch the log.
key=$(aws ssm get-parameter --name "$SSM_PREFIX/shared/ANTHROPIC_API_KEY" \
--with-decryption --query Parameter.Value --output text)
echo "::add-mask::$key"
echo "LIK_UI_ANTHROPIC_API_KEY=$key" >> "$GITHUB_ENV"
# Table-scoped DB role (NOT the master credential) — least privilege (R18/U9).
dbuser=$(aws ssm get-parameter --name "$SSM_PREFIX/shared/SCHEDULED_RUNS_DB_USER" \
--query Parameter.Value --output text)
echo "LIK_UI_DB_USER=$dbuser" >> "$GITHUB_ENV"
pw=$(aws ssm get-parameter --name "$SSM_PREFIX/shared/SCHEDULED_RUNS_DB_PASSWORD" \
--with-decryption --query Parameter.Value --output text)
echo "::add-mask::$pw"
echo "LIK_UI_DB_PASSWORD=$pw" >> "$GITHUB_ENV"
- name: Discover DB host/port from Lightsail
run: |
read -r host port < <(aws lightsail get-relational-database \
--relational-database-name "$DB_INSTANCE" \
--query 'relationalDatabase.[masterEndpoint.address,masterEndpoint.port]' \
--output text)
if [ -z "$host" ] || [ "$host" = "None" ]; then
echo "::error::could not resolve Lightsail endpoint for $DB_INSTANCE"; exit 1
fi
{ echo "LIK_UI_DB_HOST=$host"; echo "LIK_UI_DB_PORT=$port"; } >> "$GITHUB_ENV"
- name: Run due scheduled agent runs
working-directory: lik-ui
run: uv run --frozen python scripts/run_scheduled.py
# Ops-level alerting for an otherwise-invisible unattended job (owner-level failure/skip
# surfaces in the Settings "Scheduled runs" health badge). A non-zero exit means at least one
# run failed/timed-out/lapsed; open a tracking issue so a silent stop doesn't go unnoticed.
- name: Open a tracking issue on failure
if: failure()
env:
GH_TOKEN: ${{ github.token }}
run: |
run_url="${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
gh issue create \
--title "scheduled-runs cron failed" \
--body "The scheduled agent-runs workflow failed or a run errored: $run_url
Investigate before the next scheduled tick. Owner-facing failures also show in each
schedule's health badge on the lik-ui Settings page." \
|| echo "::warning::scheduled-runs failed and the tracking issue could not be created"