Skip to content

Commit 3ce0a2b

Browse files
Lutherwavesclaude
andauthored
ci: run the integration tests against a real gVisor daemon (#21)
* ci: run the integration tests against a real gVisor daemon Closes #3. CI compiled the integration suite and never ran it, so a green run graded pkg/docker at 17.7% unit coverage while the engine's actual verification — containment, blocked egress, the reaper, the resource caps under attack — ran only when someone remembered to do it locally. The blocker was assumed to be gVisor needing hardware a hosted runner does not have. It does not: the systrap platform uses seccomp and needs neither KVM nor nested virtualisation, so ubuntu-latest can register runsc. That also keeps the suite off a self-hosted runner, which on a public repo would mean executing a stranger's pull request on our own hardware. Registration is asserted in its own step, and asserted by booting a guest rather than by reading the runtime list: openblox refuses to fall back to runc, so an absent or broken runtime would otherwise surface as fifty identical ErrRuntimeUnavailable failures instead of one legible message. Per the issue, it fails loudly rather than skipping — a silent skip would rebuild exactly the false confidence being removed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * ci: register runsc explicitly instead of trusting the postinst The gVisor package's postinst registers the runtime on some hosts and did not on the ubuntu-latest runner image, so docker info never listed runsc and the verify step failed. Run runsc install directly — it is idempotent and merges into daemon.json — and restart rather than reload, since a CI runner has no containers worth preserving. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent a032502 commit 3ce0a2b

2 files changed

Lines changed: 72 additions & 10 deletions

File tree

.github/workflows/ci.yml

Lines changed: 66 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -81,9 +81,6 @@ jobs:
8181
CGO_ENABLED: '1'
8282
run: go test -race -cover -v ./...
8383

84-
# Integration tests need a gVisor-capable host, which hosted runners are not.
85-
# Compiling them on every PR is what stops them rotting between the rare runs
86-
# on a self-hosted box — a test that no longer builds is a test nobody runs.
8784
vuln:
8885
name: Vulnerabilities
8986
runs-on: ubuntu-latest
@@ -124,3 +121,69 @@ jobs:
124121
env:
125122
CGO_ENABLED: '0'
126123
run: go build -tags integration ./... && go vet -tags integration ./...
124+
125+
# The engine is only actually verified by the integration suite: containment on
126+
# a live container, egress blocked, the reaper, the resource caps under attack.
127+
# Everything else in this file grades a package whose unit coverage is 17.7%.
128+
#
129+
# This was long assumed to need a self-hosted box. It does not: gVisor's systrap
130+
# platform uses seccomp and does not require KVM or nested virtualisation, so a
131+
# standard hosted runner can register runsc. That also keeps the suite off a
132+
# self-hosted runner, which for a public repo would mean executing a stranger's
133+
# PR on our own hardware.
134+
integration:
135+
name: Integration Tests (gVisor)
136+
runs-on: ubuntu-latest
137+
needs: lint
138+
timeout-minutes: 25
139+
steps:
140+
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
141+
with:
142+
fetch-depth: 1
143+
persist-credentials: false
144+
- uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
145+
with:
146+
go-version-file: go.mod
147+
cache: true
148+
149+
- name: Install gVisor
150+
run: |
151+
set -euo pipefail
152+
curl -fsSL https://gvisor.dev/archive.key \
153+
| sudo gpg --dearmor -o /usr/share/keyrings/gvisor-archive-keyring.gpg
154+
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/gvisor-archive-keyring.gpg] https://storage.googleapis.com/gvisor/releases release main" \
155+
| sudo tee /etc/apt/sources.list.d/gvisor.list > /dev/null
156+
sudo apt-get update -qq
157+
sudo apt-get install -y -qq runsc
158+
# Register explicitly rather than trusting the package's postinst. It
159+
# does register on some hosts (it logs "Runtime runsc not found:
160+
# adding") but did not on this runner image, and a silently
161+
# unregistered runtime turns the whole suite red one step later.
162+
# `runsc install` merges into /etc/docker/daemon.json and is idempotent.
163+
sudo runsc install
164+
sudo systemctl restart docker
165+
runsc --version
166+
167+
# openblox refuses to fall back to runc, so without this the whole suite
168+
# would fail on ErrRuntimeUnavailable rather than on anything it tests.
169+
# Asserted as its own step so that failure is legible instead of arriving
170+
# as fifty identical test errors.
171+
- name: Verify runsc is registered
172+
run: |
173+
set -euo pipefail
174+
docker info --format '{{range $k, $v := .Runtimes}}{{$k}} {{end}}' | tr ' ' '\n' | grep -qx runsc
175+
# Prove it actually boots a guest, not merely that it is listed: a
176+
# registered-but-broken runtime is the failure this guards against.
177+
kernel=$(docker run --rm --runtime=runsc --network=none alpine:3.20 uname -r)
178+
echo "guest kernel: $kernel"
179+
case "$kernel" in
180+
*gvisor*) ;;
181+
*) echo "::error::runsc ran but the guest is not gVisor ($kernel)"; exit 1 ;;
182+
esac
183+
184+
# -count=1 because these assert against live host state (process counts,
185+
# free memory) that a cached PASS would silently stand in for.
186+
- name: Run integration tests
187+
env:
188+
CGO_ENABLED: '0'
189+
run: go test -tags integration -count=1 -v -timeout 20m ./...

README.md

Lines changed: 6 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@
77

88
[![CI](https://github.com/blox-eng/openblox/actions/workflows/ci.yml/badge.svg)](https://github.com/blox-eng/openblox/actions/workflows/ci.yml)
99
[![Tests](https://img.shields.io/endpoint?url=https://openblox.sh/badges/tests.json)](https://github.com/blox-eng/openblox/actions/workflows/ci.yml)
10-
[![Integration tests](https://img.shields.io/endpoint?url=https://openblox.sh/badges/integration.json)](https://github.com/blox-eng/openblox/issues/3)
10+
[![Integration tests](https://img.shields.io/endpoint?url=https://openblox.sh/badges/integration.json)](https://github.com/blox-eng/openblox/actions/workflows/ci.yml)
1111
[![Coverage](https://img.shields.io/endpoint?url=https://openblox.sh/badges/coverage.json)](https://github.com/blox-eng/openblox/issues/10)
1212
[![Lines of Go](https://img.shields.io/endpoint?url=https://openblox.sh/badges/loc.json)](ARCHITECTURE.md)
1313
[![Go Reference](https://pkg.go.dev/badge/github.com/blox-eng/openblox.svg)](https://pkg.go.dev/github.com/blox-eng/openblox)
@@ -118,13 +118,12 @@ size, test counts, and coverage cannot drift from the code that produced them.
118118
Two direct dependencies (`docker/docker` and `containerd/errdefs`). Every release
119119
publishes a multi-architecture sandbox image with an SBOM and build provenance.
120120
CI runs lint, tests, and `govulncheck`, gating on newly reachable
121-
vulnerabilities.
121+
vulnerabilities. The integration suite runs there too, against a real gVisor
122+
daemon on a hosted runner — including the adversarial cases that try to break
123+
the resource caps.
122124

123-
Honest about the gaps: the integration suite compiles in CI but does not run
124-
there — it needs a host with Docker and `runsc`, so it runs locally
125-
([#3](https://github.com/blox-eng/openblox/issues/3)). Using openblox today also
126-
means giving your service access to the Docker socket, which is root-equivalent
127-
on the host; closing that is
125+
Honest about the gaps: using openblox today means giving your service access to
126+
the Docker socket, which is root-equivalent on the host; closing that is
128127
[#2](https://github.com/blox-eng/openblox/issues/2), the most consequential open
129128
issue. The [open issues](https://github.com/blox-eng/openblox/issues) are the
130129
honest roadmap.

0 commit comments

Comments
 (0)