[Feature] Functional test enablement in CI

[Gregory-Pereira]

Functional tests in CI

Part of: CI testing automation epic (#12)
Depends on: reusable build-image workflow (#13), repo restructure (#11)

Summary

We have ~25 fast tests in tests/test_dual_server_client.py that check health endpoints, prediction requests, input validation, data ingestion, model sync, and server config. None of them run in CI. This issue adds a workflow that calls the build-image workflow on every code-touching PR and on merges to main, starts the containers via docker compose, runs the functional test subset, and blocks merge on failure.

Design

New file: .github/workflows/ci-functional-tests.yaml, triggers on PRs against main and on pushes to main.

How it runs:

1. Call build-image.yaml to build and push images (tagged per the build workflow's tagging strategy -- pr-<number> + sha-<commit> on PRs, sha-<commit> + latest on main).
2. Pull the images using output refs from the build workflow. On PRs, containers whose source didn't change fall back to latest from main.
3. Start training and prediction servers via docker-compose.ci.yaml.
4. Wait for readiness via health check polling.
5. Run pytest -m functional against the running services.
6. Tear everything down.

Tests to mark as functional

These are the fast tests from the existing suite:

• Health and readiness: test_prediction_server_healthz, test_training_server_healthz, test_prediction_server_readyz, test_training_server_readyz
• Single and bulk predictions: test_prediction_via_prediction_server, test_bulk_prediction_strict, test_bulk_prediction_all_valid
• Input validation: test_prediction_missing_prefix_cache_score, test_token_in_flight_negative_rejected, test_bulk_prediction_with_validation_errors, test_invalid_pod_type
• Feature handling: test_prediction_with_pod_type_prefill, test_prediction_with_pod_type_decode, test_prediction_with_token_in_flight_features, test_backward_compat_without_token_in_flight_features
• Data ingestion: test_add_training_data_to_training_server, test_training_data_with_pod_type, test_training_data_with_token_in_flight_features, test_training_data_backward_compat_without_tif
• Server status and config: test_prediction_server_status, test_training_server_model_info, test_training_server_models_list, test_server_configuration
• Model sync and reload: test_prediction_server_model_sync, test_model_consistency_between_servers
• Flush API: test_training_server_flush_api, test_training_server_flush_error_handling
• Metrics: test_training_server_metrics

Pytest markers

Add @pytest.mark.functional to each of the tests above. Register the marker in pyproject.toml:

[tool.pytest.ini_options]
markers = [
    "functional: fast correctness checks that run on every PR",
    "regression: expensive tests that train models and verify statistical properties",
]

(Register both markers here even though the regression workflow is a separate issue, so we don't have to touch this config twice.)

Docker compose for CI

New file: docker-compose.ci.yaml

services:
  training:
    image: ${TRAINING_IMAGE:-ghcr.io/llm-d/llm-d-latency-predictor-training:latest}
    ports:
      - "8000:8000"
    environment:
      - MODEL_TYPE=lightgbm
      - OBJECTIVE_TYPE=quantile

  prediction:
    image: ${PREDICTION_IMAGE:-ghcr.io/llm-d/llm-d-latency-predictor-prediction:latest}
    ports:
      - "8001:8001"
    environment:
      - TRAINING_SERVER_URL=http://training:8000
    depends_on:
      training:
        condition: service_healthy

TRAINING_IMAGE and PREDICTION_IMAGE get set from the build workflow outputs. Defaults point to latest for local dev.

Changes to existing files

• tests/test_dual_server_client.py -- add @pytest.mark.functional to each functional test. Update PREDICTION_SERVER_URL and TRAINING_SERVER_URL defaults to localhost ports matching docker compose instead of the current placeholder IPs.
• pyproject.toml -- register functional and regression markers.

Open questions

• Whether the test container runs as a compose service that exits on completion, or as a separate docker compose exec / docker run step after the servers are up.

Acceptance criteria

• Functional tests run on every code-touching PR and on merges to main, calling the build workflow
• Docker compose starts training + prediction servers from the built images
• Test functions are marked @pytest.mark.functional
• Functional test failures block merge
• PREDICTION_SERVER_URL and TRAINING_SERVER_URL default to values that work in both CI (docker compose) and manual Kubernetes deployment
• functional and regression markers registered in pyproject.toml