-
Notifications
You must be signed in to change notification settings - Fork 16
169 lines (155 loc) · 8.35 KB
/
Copy pathdeploy.yml
File metadata and controls
169 lines (155 loc) · 8.35 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
name: Deploy jaseci-blogs with jac-scale
on:
push:
branches:
- main
workflow_dispatch:
permissions:
id-token: write
contents: read
jobs:
deploy-jaseci-blogs:
runs-on: ubuntu-latest
permissions:
id-token: write
contents: read
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v4.2.0
with:
aws-region: us-east-2
role-to-assume: arn:aws:iam::776241927220:role/GitHubActionsJaseciDeployRole
role-session-name: GitHubActions-JaseciBlogs
audience: sts.amazonaws.com
- name: Update kubeconfig for EKS
run: |
aws eks update-kubeconfig --region us-east-2 --name jaseci-cluster
kubectl config current-context
- name: Set up Python 3.12
uses: actions/setup-python@v5
with:
python-version: '3.12'
- name: Install Jac toolchain and project dependencies
run: |
pip install --upgrade pip
# jaclang core ships the `jac` CLI and is pip-installable. The Jac
# plugins (jac-scale, jac-client) no longer carry a pyproject.toml
# (jaseci-labs/jaseci#6322), so they install via `jac install`, which
# reads [dependencies] in jac.toml into the project's .jac/venv.
# `jac start` adds that venv to sys.path automatically.
# Pin jaclang: the unpinned `pip install jaclang` pulls the latest,
# whose newer in-pod binary bootstrap ("[jac] Fetching latest version
# … Could not determine latest version from GitHub Releases") crash-
# loops the pod. Pinned to the last pre-breakage release. Bump in
# lockstep with the jac-scale/jac-client pins in jac.toml.
pip install jaclang==0.16.0
jac install
- name: Free the code PVC before syncing (scale down code-server)
# ROOT CAUSE (jac-scale 0.2.23): the shared code PVC jaseci-blogs-code-pvc
# is ReadWriteOnce on EBS (attaches to one node at a time). The long-lived
# `code-server` pod holds it, and jac-scale runs the `code-sync` pod — which
# needs the SAME RWO volume — BEFORE it recreates code-server. So during
# sync, code-sync competes with the still-running code-server pod; if it
# lands on a different node it wedges in Multi-Attach and the deploy times
# out at "Syncing application code to PVC". Plain retry does NOT converge —
# the scheduler keeps re-placing code-sync on the same off-node (observed
# 5/5 failures in run 29217559535).
#
# DETERMINISTIC FIX: scale code-server to 0 first, detaching the volume, so
# code-sync can attach on ANY node. jac-scale delete+recreates code-server
# at the end of the deploy (kubernetes_target.jac:547-556), so this only
# moves that teardown earlier. The APP pod serves from an emptyDir (it wgets
# code from code-server over HTTP; it does NOT mount this PVC), so the live
# site stays UP throughout. Permanent fix (RWX/EFS): .github/CODE_SYNC_RWO_FIX.md.
run: |
NS=jaseci-blogs
if kubectl get deploy/jaseci-blogs-code-server -n "$NS" >/dev/null 2>&1; then
SEL=$(kubectl get deploy/jaseci-blogs-code-server -n "$NS" \
-o go-template='{{range $k,$v := .spec.selector.matchLabels}}{{$k}}={{$v}},{{end}}' | sed 's/,$//')
kubectl scale deploy/jaseci-blogs-code-server -n "$NS" --replicas=0
echo "Waiting for code-server pod to terminate (EBS volume detach)…"
kubectl wait --for=delete pod -l "$SEL" -n "$NS" --timeout=120s || true
fi
# Clear any code-sync pod wedged by a prior run so jac-scale makes a fresh one.
kubectl delete pod jaseci-blogs-code-sync -n "$NS" \
--ignore-not-found --force --grace-period=0 2>/dev/null || true
- name: Deploy jaseci-blogs with jac-scale
working-directory: .
env:
# Submission-portal secrets. jac.toml's [plugins.scale.secrets]
# interpolates these ${...} env vars at deploy time into the
# jaseci-blogs-secrets k8s Secret. Unset values deploy as empty,
# which gracefully disables /submit + /reviewer. Only an OAuth App is
# needed — every GitHub action runs as the signed-in user (fork + PR).
# See SUBMIT_FLOW.md.
GH_OAUTH_CLIENT_ID: ${{ vars.BLOG_OAUTH_CLIENT_ID }}
GH_OAUTH_CLIENT_SECRET: ${{ secrets.BLOG_OAUTH_CLIENT_SECRET }}
# One random value backs both: jac-scale signs the session JWT with
# JWT_SECRET, and SUBMIT_SESSION_SECRET is the Fernet key for the GitHub
# token at rest. Reused so there's no extra secret to manage (and so an
# unset JWT_SECRET can't silently fall back to jac-scale's default).
JWT_SECRET: ${{ secrets.BLOG_SESSION_SECRET }}
SUBMIT_SESSION_SECRET: ${{ secrets.BLOG_SESSION_SECRET }}
run: |
# `jac install` put jaclang + jac-scale + jac-client into the project
# venv (.jac/venv). Run THAT venv's jac so the jac-scale plugin loads
# at startup and registers the `--scale` flag — the global `jac` (from
# `pip install jaclang`) builds its arg parser before the venv is on
# the path, so it does not recognize `--scale`.
#
# The "Free the code PVC" step above detached the RWO code volume, so the
# code-sync pod can now attach on any node and this should succeed on the
# first try. The short retry is belt-and-suspenders for transient EBS
# detach latency; it force-deletes any wedged code-sync pod between tries.
# Permanent fix (RWX/EFS): .github/CODE_SYNC_RWO_FIX.md.
NS=jaseci-blogs
for attempt in 1 2 3; do
echo "::group::Deploy attempt ${attempt}/3"
kubectl delete pod jaseci-blogs-code-sync -n "$NS" \
--ignore-not-found --force --grace-period=0 2>/dev/null || true
kubectl wait --for=delete pod/jaseci-blogs-code-sync -n "$NS" \
--timeout=60s 2>/dev/null || true
if .jac/venv/bin/jac start main.jac --scale; then
echo "::endgroup::"
echo "Deployed successfully on attempt ${attempt}."
exit 0
fi
echo "::endgroup::"
echo "Attempt ${attempt} failed; retrying…"
done
echo "All attempts failed. The live app pod was never touched, so the site is"
echo "still serving the last-synced code. See .github/CODE_SYNC_RWO_FIX.md."
exit 1
- name: Ensure code-server is running
if: always()
# Safety net: jac-scale recreates code-server (at replicas=1) only after a
# successful sync. If the deploy failed before that, code-server may be left
# at 0 replicas from the "Free the code PVC" step — the site stays up (app
# pod serves from emptyDir), but restore code-server so new app-pod boots and
# the next sync aren't blocked. No-op once jac-scale has recreated it at 1.
run: |
NS=jaseci-blogs
kubectl scale deploy/jaseci-blogs-code-server -n "$NS" --replicas=1 2>/dev/null || true
- name: Verify deployment
run: |
echo "Checking deployment status..."
kubectl get deployments -n jaseci-blogs
kubectl get services -n jaseci-blogs
kubectl get pods -n jaseci-blogs
- name: Deployment Summary
if: always()
run: |
echo "## jaseci-blogs Deployment Summary" >> $GITHUB_STEP_SUMMARY
echo "### Configuration" >> $GITHUB_STEP_SUMMARY
echo "- **App Name**: jaseci-blogs" >> $GITHUB_STEP_SUMMARY
echo "- **Namespace**: jaseci-blogs" >> $GITHUB_STEP_SUMMARY
echo "- **Cluster**: jaseci-cluster" >> $GITHUB_STEP_SUMMARY
echo "- **Region**: us-east-2" >> $GITHUB_STEP_SUMMARY
echo "- **Triggered By**: ${{ github.event_name }}" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "### Deployment Status" >> $GITHUB_STEP_SUMMARY
echo "\`\`\`" >> $GITHUB_STEP_SUMMARY
kubectl get all -n jaseci-blogs 2>&1 >> $GITHUB_STEP_SUMMARY || echo "Unable to fetch deployment status" >> $GITHUB_STEP_SUMMARY
echo "\`\`\`" >> $GITHUB_STEP_SUMMARY