Skip to content

Commit c4cd16a

Browse files
xgboost: add serving-container longevity/resource-impact test (#6366)
* xgboost: add serving-container longevity/resource-impact test Adds test/xgboost/container/test_longevity.py: drives the XGBoost serving container under sustained /invocations load for a fixed duration while sampling memory and CPU via docker stats. Asserts no memory leak (bounded growth from a post-warmup baseline), bounded peak memory, all requests succeed, and the container stays healthy after load. Wires a lightweight longevity-test job into xgboost.pipeline.yml gated on run-integ-test so it runs on PRs (new reusable xgboost.tests-longevity.yml), adds a stats() sampler to ServingContainer, and documents the new tier. Addresses ORR action item: test container impact on resources (CPU, memory). * xgboost: reformat README table (mdformat) * TEMP: touch Dockerfile to force CI build (validate longevity test; revert before merge) * Revert "TEMP: touch Dockerfile to force CI build (validate longevity test; revert before merge)" This reverts commit 494efa2.
1 parent c49cf2e commit c4cd16a

5 files changed

Lines changed: 272 additions & 7 deletions

File tree

.github/workflows/xgboost.pipeline.yml

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -131,6 +131,19 @@ jobs:
131131
config-file: ${{ inputs.config-file }}
132132
image-uri: ${{ needs.build.outputs.image-uri || '' }}
133133

134+
# Longevity test — sustained-load resource-impact check on the serving
135+
# container. Lightweight; runs on PRs alongside the local integ tests.
136+
longevity-test:
137+
if: ${{ always() && !failure() && !cancelled() && inputs.run-integ-test }}
138+
needs: [build]
139+
concurrency:
140+
group: ${{ github.workflow }}-longevity-test-${{ inputs.config-file }}-${{ github.ref }}
141+
cancel-in-progress: true
142+
uses: ./.github/workflows/xgboost.tests-longevity.yml
143+
with:
144+
config-file: ${{ inputs.config-file }}
145+
image-uri: ${{ needs.build.outputs.image-uri || '' }}
146+
134147
# Full integration suite — GPU container tests + SageMaker e2e + benchmarks.
135148
# Heavy; only runs on the release pipeline, not on PRs.
136149
integ-test-full:
@@ -152,7 +165,7 @@ jobs:
152165
inputs.release &&
153166
!failure() && !cancelled() &&
154167
needs.build.result == 'success'
155-
needs: [build, sanity-test, security-test, unit-test, integ-test, integ-test-full]
168+
needs: [build, sanity-test, security-test, unit-test, integ-test, longevity-test, integ-test-full]
156169
runs-on: ubuntu-latest
157170
outputs:
158171
environment: ${{ steps.check.outputs.environment }}
Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
name: Reusable XGBoost Longevity Test
2+
3+
# Sustained-load resource-impact test for the XGBoost serving container.
4+
# Lightweight (a few minutes) so it can run on PRs alongside the local
5+
# integration tests, unlike the full container/e2e suite which is release-only.
6+
7+
on:
8+
workflow_call:
9+
inputs:
10+
config-file:
11+
description: "Path to image config YAML"
12+
required: true
13+
type: string
14+
image-uri:
15+
description: "Image URI to test. If empty, derives prod URI from config."
16+
required: false
17+
type: string
18+
default: ""
19+
20+
permissions:
21+
contents: read
22+
23+
jobs:
24+
longevity-test:
25+
runs-on:
26+
- codebuild-runner-${{ github.run_id }}-${{ github.run_attempt }}
27+
fleet:x86-g6xl-runner
28+
buildspec-override:true
29+
timeout-minutes: 30
30+
steps:
31+
- uses: actions/checkout@v6
32+
33+
- name: Resolve image URI
34+
id: image
35+
uses: ./.github/actions/resolve-image-uri
36+
with:
37+
image-uri: ${{ inputs.image-uri }}
38+
ci-aws-account-id: ${{ vars.CI_AWS_ACCOUNT_ID }}
39+
prod-aws-account-id: ${{ vars.PROD_AWS_ACCOUNT_ID }}
40+
aws-region: ${{ vars.AWS_REGION }}
41+
config-file: ${{ inputs.config-file }}
42+
43+
- name: Check image exists
44+
id: check
45+
uses: ./.github/actions/check-image-exists
46+
with:
47+
image-uri: ${{ steps.image.outputs.image-uri }}
48+
49+
- name: ECR login
50+
if: steps.check.outputs.exists == 'true'
51+
uses: ./.github/actions/ecr-authenticate
52+
with:
53+
aws-account-id: ${{ steps.image.outputs.aws-account-id }}
54+
aws-region: ${{ vars.AWS_REGION }}
55+
image-uri: ${{ steps.image.outputs.image-uri }}
56+
57+
- name: Install test dependencies
58+
if: steps.check.outputs.exists == 'true'
59+
run: |
60+
uv venv --python 3.12
61+
source .venv/bin/activate
62+
uv pip install -r test/requirements.txt docker pytest boto3 requests
63+
64+
- name: Run longevity test
65+
if: steps.check.outputs.exists == 'true'
66+
run: |
67+
source .venv/bin/activate
68+
cd test/
69+
python3 -m pytest -v --tb=short -rA --log-cli-level=INFO \
70+
--image ${{ steps.image.outputs.image-uri }} \
71+
xgboost/container/test_longevity.py
72+
73+
- name: Skip notice
74+
if: steps.check.outputs.exists != 'true'
75+
run: echo "Image not found in ECR — skipping longevity test (first release scenario)"

test/xgboost/README.md

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -17,15 +17,16 @@ test/xgboost/
1717
Runs the XGBoost container locally via docker-py. The container is mounted with
1818
`/opt/ml/` directory structures and exercised directly — no SageMaker jobs are created.
1919

20-
| File | What it tests |
21-
| ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ |
22-
| `test_training.py` | Algorithm-mode training: libsvm/csv, single/multi-file, weights, HPO metrics, objectives, verbosity, checkpoint/reload, distributed, invalid hyperparameters |
23-
| `test_scoring.py` | Inference: csv/libsvm/protobuf payloads, execution parameters, 20 MB payload, content type validation |
24-
| `test_batch_transform.py` | Batch transform with `SAGEMAKER_BATCH=True` |
20+
| File | What it tests |
21+
| ------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
22+
| `test_training.py` | Algorithm-mode training: libsvm/csv, single/multi-file, weights, HPO metrics, objectives, verbosity, checkpoint/reload, distributed, invalid hyperparameters |
23+
| `test_scoring.py` | Inference: csv/libsvm/protobuf payloads, execution parameters, 20 MB payload, content type validation |
24+
| `test_batch_transform.py` | Batch transform with `SAGEMAKER_BATCH=True` |
25+
| `test_longevity.py` | Resource-impact longevity: serving container under sustained `/invocations` load, sampling memory/CPU via `docker stats`; asserts no memory leak, bounded peak usage, and liveness after load |
2526

2627
Supporting files:
2728

28-
- `container_helper.py``run_training()` and `ServingContainer` context manager
29+
- `container_helper.py``run_training()` and `ServingContainer` context manager (with `stats()` for resource sampling)
2930
- `generate_models.py` — generates XGBoost 3.0.5-compatible inference models
3031

3132
### Tier 2: E2E Tests (`e2e/`)

test/xgboost/container/container_helper.py

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -313,6 +313,37 @@ def invocations(self, data, content_type, accept=None):
313313
def execution_parameters(self):
314314
return requests.get(self._url("/execution-parameters"), timeout=5)
315315

316+
# -- resource sampling ---------------------------------------------------
317+
318+
def stats(self):
319+
"""Return a single resource sample: {mem_bytes, mem_mb, cpu_percent}.
320+
321+
Reads one non-streaming ``docker stats`` snapshot for the container and
322+
computes CPU% the same way the docker CLI does (delta of container CPU
323+
usage over delta of system CPU usage, scaled by online CPUs).
324+
"""
325+
raw = self._container.stats(stream=False)
326+
327+
mem_bytes = raw.get("memory_stats", {}).get("usage", 0)
328+
329+
cpu_percent = 0.0
330+
try:
331+
cpu = raw["cpu_stats"]
332+
precpu = raw["precpu_stats"]
333+
cpu_delta = cpu["cpu_usage"]["total_usage"] - precpu["cpu_usage"]["total_usage"]
334+
system_delta = cpu["system_cpu_usage"] - precpu["system_cpu_usage"]
335+
online_cpus = cpu.get("online_cpus") or len(cpu["cpu_usage"].get("percpu_usage", []))
336+
if system_delta > 0 and cpu_delta > 0:
337+
cpu_percent = (cpu_delta / system_delta) * online_cpus * 100.0
338+
except (KeyError, TypeError, ZeroDivisionError):
339+
cpu_percent = 0.0
340+
341+
return {
342+
"mem_bytes": mem_bytes,
343+
"mem_mb": mem_bytes / (1024 * 1024),
344+
"cpu_percent": cpu_percent,
345+
}
346+
316347
def get_logs(self):
317348
if self._container:
318349
return self._container.logs().decode("utf-8", errors="replace")
Lines changed: 145 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,145 @@
1+
"""Longevity / resource-impact container test.
2+
3+
Exercises the XGBoost serving container under sustained inference load and
4+
tracks its resource footprint (memory + CPU) over time. The goal is to catch
5+
container-level regressions that unit / functional tests miss:
6+
7+
- memory leaks — RSS that climbs steadily and never plateaus under repeated
8+
``/invocations`` requests
9+
- runaway resource use — CPU or memory that blows past a sane bound
10+
- loss of liveness — the container becoming unresponsive after prolonged load
11+
12+
This is intentionally lightweight (a couple of minutes) so it can run on every
13+
PR alongside the other local container tests, not just on the release pipeline.
14+
"""
15+
16+
import logging
17+
import os
18+
import time
19+
20+
from .container_helper import ServingContainer
21+
22+
LOGGER = logging.getLogger(__name__)
23+
24+
25+
# ---------------------------------------------------------------------------
26+
# Tunables
27+
# ---------------------------------------------------------------------------
28+
29+
# Total wall-clock duration to keep hammering the container.
30+
LOAD_DURATION_SECONDS = int(os.environ.get("XGB_LONGEVITY_DURATION", "150"))
31+
# How often to sample docker stats (memory / cpu).
32+
SAMPLE_INTERVAL_SECONDS = 5
33+
# Warm-up window ignored when computing the memory-growth baseline — lets the
34+
# process finish lazy allocation / model load before we start trusting numbers.
35+
WARMUP_SECONDS = 20
36+
37+
# Memory growth from the post-warmup baseline to the end of the run that we
38+
# consider indicative of a leak. XGBoost inference on a fixed model should be
39+
# flat; a steadily climbing RSS is the signal we want to catch.
40+
MEM_GROWTH_LIMIT_MB = 150
41+
# Absolute ceiling — the small mnist model should never need anywhere near this.
42+
MEM_ABSOLUTE_LIMIT_MB = 4096
43+
44+
MODEL_NAME = "mnist-xgb-model"
45+
INPUT_FILE = "mnist-1.csv"
46+
CONTENT_TYPE = "text/csv"
47+
48+
49+
def _model_path(resources, model_name):
50+
return os.path.join(resources, "models", model_name)
51+
52+
53+
def _input_path(resources, filename):
54+
return os.path.join(resources, "input", filename)
55+
56+
57+
def test_serving_longevity_no_memory_leak(docker_client, image_uri, inference_resources):
58+
"""Sustained-load longevity check on the serving container.
59+
60+
Sends inference requests continuously for ``LOAD_DURATION_SECONDS`` while
61+
sampling memory and CPU. Asserts the container stays healthy, every request
62+
succeeds, and memory does not grow past the leak threshold.
63+
"""
64+
model_dir = _model_path(inference_resources, MODEL_NAME)
65+
with open(_input_path(inference_resources, INPUT_FILE), "rb") as f:
66+
payload = f.read()
67+
68+
request_count = 0
69+
failures = 0
70+
samples = [] # list of (elapsed_s, mem_mb, cpu_percent)
71+
72+
with ServingContainer(docker_client, image_uri, model_dir) as ctx:
73+
start = time.time()
74+
next_sample = 0.0
75+
76+
while True:
77+
elapsed = time.time() - start
78+
if elapsed >= LOAD_DURATION_SECONDS:
79+
break
80+
81+
# Drive load.
82+
try:
83+
resp = ctx.invocations(data=payload, content_type=CONTENT_TYPE)
84+
request_count += 1
85+
if resp.status_code != 200:
86+
failures += 1
87+
LOGGER.warning("Non-200 response: %s %s", resp.status_code, resp.text[:200])
88+
except Exception as exc: # connection dropped == container in trouble
89+
failures += 1
90+
LOGGER.warning("Invocation raised: %s", exc)
91+
92+
# Sample resources on a fixed cadence.
93+
if elapsed >= next_sample:
94+
sample = ctx.stats()
95+
samples.append((elapsed, sample["mem_mb"], sample["cpu_percent"]))
96+
LOGGER.info(
97+
"t=%5.1fs reqs=%d mem=%.1fMB cpu=%.1f%%",
98+
elapsed,
99+
request_count,
100+
sample["mem_mb"],
101+
sample["cpu_percent"],
102+
)
103+
next_sample = elapsed + SAMPLE_INTERVAL_SECONDS
104+
105+
# Container must still answer a health check after the load.
106+
assert ctx.ping().status_code == 200, "container unhealthy after sustained load"
107+
108+
# -- assertions ----------------------------------------------------------
109+
110+
assert request_count > 0, "no requests were sent"
111+
assert samples, "no resource samples collected"
112+
assert failures == 0, f"{failures}/{request_count} invocations failed under sustained load"
113+
114+
mem_series = [mem for _, mem, _ in samples]
115+
peak_mem = max(mem_series)
116+
cpu_series = [cpu for _, _, cpu in samples]
117+
peak_cpu = max(cpu_series)
118+
119+
# Baseline = first sample taken after the warm-up window (fall back to the
120+
# first sample if the run is shorter than warm-up for any reason).
121+
post_warmup = [mem for elapsed, mem, _ in samples if elapsed >= WARMUP_SECONDS]
122+
baseline_mem = post_warmup[0] if post_warmup else mem_series[0]
123+
final_mem = mem_series[-1]
124+
mem_growth = final_mem - baseline_mem
125+
126+
LOGGER.info(
127+
"longevity summary: requests=%d duration=%ds peak_mem=%.1fMB "
128+
"baseline_mem=%.1fMB final_mem=%.1fMB growth=%.1fMB peak_cpu=%.1f%%",
129+
request_count,
130+
LOAD_DURATION_SECONDS,
131+
peak_mem,
132+
baseline_mem,
133+
final_mem,
134+
mem_growth,
135+
peak_cpu,
136+
)
137+
138+
assert peak_mem < MEM_ABSOLUTE_LIMIT_MB, (
139+
f"peak memory {peak_mem:.1f}MB exceeded absolute ceiling {MEM_ABSOLUTE_LIMIT_MB}MB"
140+
)
141+
assert mem_growth < MEM_GROWTH_LIMIT_MB, (
142+
f"memory grew {mem_growth:.1f}MB (baseline {baseline_mem:.1f}MB -> "
143+
f"final {final_mem:.1f}MB) over {LOAD_DURATION_SECONDS}s, exceeding leak "
144+
f"threshold {MEM_GROWTH_LIMIT_MB}MB"
145+
)

0 commit comments

Comments
 (0)