Skip to content

Commit c0d49e3

Browse files
Lutherwavesclaude
andcommitted
ci: run the integration tests against a real gVisor daemon
Closes #3. CI compiled the integration suite and never ran it, so a green run graded pkg/docker at 17.7% unit coverage while the engine's actual verification — containment, blocked egress, the reaper, the resource caps under attack — ran only when someone remembered to do it locally. The blocker was assumed to be gVisor needing hardware a hosted runner does not have. It does not: the systrap platform uses seccomp and needs neither KVM nor nested virtualisation, so ubuntu-latest can register runsc. That also keeps the suite off a self-hosted runner, which on a public repo would mean executing a stranger's pull request on our own hardware. Registration is asserted in its own step, and asserted by booting a guest rather than by reading the runtime list: openblox refuses to fall back to runc, so an absent or broken runtime would otherwise surface as fifty identical ErrRuntimeUnavailable failures instead of one legible message. Per the issue, it fails loudly rather than skipping — a silent skip would rebuild exactly the false confidence being removed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent f89bdc0 commit c0d49e3

2 files changed

Lines changed: 68 additions & 10 deletions

File tree

.github/workflows/ci.yml

Lines changed: 62 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -81,9 +81,6 @@ jobs:
8181
CGO_ENABLED: '1'
8282
run: go test -race -cover -v ./...
8383

84-
# Integration tests need a gVisor-capable host, which hosted runners are not.
85-
# Compiling them on every PR is what stops them rotting between the rare runs
86-
# on a self-hosted box — a test that no longer builds is a test nobody runs.
8784
vuln:
8885
name: Vulnerabilities
8986
runs-on: ubuntu-latest
@@ -124,3 +121,65 @@ jobs:
124121
env:
125122
CGO_ENABLED: '0'
126123
run: go build -tags integration ./... && go vet -tags integration ./...
124+
125+
# The engine is only actually verified by the integration suite: containment on
126+
# a live container, egress blocked, the reaper, the resource caps under attack.
127+
# Everything else in this file grades a package whose unit coverage is 17.7%.
128+
#
129+
# This was long assumed to need a self-hosted box. It does not: gVisor's systrap
130+
# platform uses seccomp and does not require KVM or nested virtualisation, so a
131+
# standard hosted runner can register runsc. That also keeps the suite off a
132+
# self-hosted runner, which for a public repo would mean executing a stranger's
133+
# PR on our own hardware.
134+
integration:
135+
name: Integration Tests (gVisor)
136+
runs-on: ubuntu-latest
137+
needs: lint
138+
timeout-minutes: 25
139+
steps:
140+
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
141+
with:
142+
fetch-depth: 1
143+
persist-credentials: false
144+
- uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
145+
with:
146+
go-version-file: go.mod
147+
cache: true
148+
149+
- name: Install gVisor
150+
run: |
151+
set -euo pipefail
152+
curl -fsSL https://gvisor.dev/archive.key \
153+
| sudo gpg --dearmor -o /usr/share/keyrings/gvisor-archive-keyring.gpg
154+
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/gvisor-archive-keyring.gpg] https://storage.googleapis.com/gvisor/releases release main" \
155+
| sudo tee /etc/apt/sources.list.d/gvisor.list > /dev/null
156+
sudo apt-get update -qq
157+
sudo apt-get install -y -qq runsc
158+
# The package's postinst registers the runtime with dockerd itself;
159+
# runtimes are reloadable, so this needs no daemon restart.
160+
sudo systemctl reload docker
161+
runsc --version
162+
163+
# openblox refuses to fall back to runc, so without this the whole suite
164+
# would fail on ErrRuntimeUnavailable rather than on anything it tests.
165+
# Asserted as its own step so that failure is legible instead of arriving
166+
# as fifty identical test errors.
167+
- name: Verify runsc is registered
168+
run: |
169+
set -euo pipefail
170+
docker info --format '{{range $k, $v := .Runtimes}}{{$k}} {{end}}' | tr ' ' '\n' | grep -qx runsc
171+
# Prove it actually boots a guest, not merely that it is listed: a
172+
# registered-but-broken runtime is the failure this guards against.
173+
kernel=$(docker run --rm --runtime=runsc --network=none alpine:3.20 uname -r)
174+
echo "guest kernel: $kernel"
175+
case "$kernel" in
176+
*gvisor*) ;;
177+
*) echo "::error::runsc ran but the guest is not gVisor ($kernel)"; exit 1 ;;
178+
esac
179+
180+
# -count=1 because these assert against live host state (process counts,
181+
# free memory) that a cached PASS would silently stand in for.
182+
- name: Run integration tests
183+
env:
184+
CGO_ENABLED: '0'
185+
run: go test -tags integration -count=1 -v -timeout 20m ./...

README.md

Lines changed: 6 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@
77

88
[![CI](https://github.com/blox-eng/openblox/actions/workflows/ci.yml/badge.svg)](https://github.com/blox-eng/openblox/actions/workflows/ci.yml)
99
[![Tests](https://img.shields.io/endpoint?url=https://openblox.sh/badges/tests.json)](https://github.com/blox-eng/openblox/actions/workflows/ci.yml)
10-
[![Integration tests](https://img.shields.io/endpoint?url=https://openblox.sh/badges/integration.json)](https://github.com/blox-eng/openblox/issues/3)
10+
[![Integration tests](https://img.shields.io/endpoint?url=https://openblox.sh/badges/integration.json)](https://github.com/blox-eng/openblox/actions/workflows/ci.yml)
1111
[![Coverage](https://img.shields.io/endpoint?url=https://openblox.sh/badges/coverage.json)](https://github.com/blox-eng/openblox/issues/10)
1212
[![Lines of Go](https://img.shields.io/endpoint?url=https://openblox.sh/badges/loc.json)](ARCHITECTURE.md)
1313
[![Go Reference](https://pkg.go.dev/badge/github.com/blox-eng/openblox.svg)](https://pkg.go.dev/github.com/blox-eng/openblox)
@@ -118,13 +118,12 @@ size, test counts, and coverage cannot drift from the code that produced them.
118118
Two direct dependencies (`docker/docker` and `containerd/errdefs`). Every release
119119
publishes a multi-architecture sandbox image with an SBOM and build provenance.
120120
CI runs lint, tests, and `govulncheck`, gating on newly reachable
121-
vulnerabilities.
121+
vulnerabilities. The integration suite runs there too, against a real gVisor
122+
daemon on a hosted runner — including the adversarial cases that try to break
123+
the resource caps.
122124

123-
Honest about the gaps: the integration suite compiles in CI but does not run
124-
there — it needs a host with Docker and `runsc`, so it runs locally
125-
([#3](https://github.com/blox-eng/openblox/issues/3)). Using openblox today also
126-
means giving your service access to the Docker socket, which is root-equivalent
127-
on the host; closing that is
125+
Honest about the gaps: using openblox today means giving your service access to
126+
the Docker socket, which is root-equivalent on the host; closing that is
128127
[#2](https://github.com/blox-eng/openblox/issues/2), the most consequential open
129128
issue. The [open issues](https://github.com/blox-eng/openblox/issues) are the
130129
honest roadmap.

0 commit comments

Comments
 (0)