ci: run FC tests with a running AC by DavSanchez · Pull Request #2412 · newrelic/newrelic-agent-control

DavSanchez · 2026-04-14T10:02:32Z

Add Fleet Control E2E Tests for Linux and Kubernetes

This PR adds automated end-to-end testing for Fleet Control integration with Agent Control on both Linux hosts and Kubernetes clusters.

Overview

These tests validate the integration by:

Deploying Agent Control with Fleet Control enabled
Triggering test suites via Fleet Control's external API (staging)
Polling for completion and validating results

The tests are ran in our available GitHub larger runners.

What's New

Linux Host Tests (fleet_control_host_linux_e2e.yml)

Rust-based implementation in test/e2e-runner
Installs AC packages and runs e2e-runner binary
Manual dispatch only (scheduled runs disabled for now until Fleet exposes the means to trigger their tests)

Kubernetes Tests (fleet_control_k8s_e2e.yml)

Bash script implementation (fleet_control_api.sh)
Deploys AC to Minikube via Tilt in the same way as other E2E tests
Scheduled to run at 4 AM weekdays

Technical Details

Fleet Control API Integration:

Endpoint: https://fleet-management-e2e-test-runner.staging-service.newrelic.com
Auth: System Identity JWT tokens via newrelic-auth-rs
Flow: POST to trigger tests → Poll status endpoint → Handle 404/204/200/450 responses
Timing: 5-min initial wait, 30-sec polling, 10-min timeout

Key Differences from Standard K8s E2E Tests:

K8s tests do NOT use newrelic-integration-e2e-action
No e2e-*.yml spec file (external API orchestration vs NRQL validation)
Separate bash script instead of Rust (simpler, no extra build dependencies)

Testing

# Trigger manually
gh workflow run fleet_control_host_linux_e2e.yml
gh workflow run fleet_control_k8s_e2e.yml

# With custom fleet
gh workflow run fleet_control_k8s_e2e.yml -f fleet_id="YOUR_FLEET_ID"

Default Fleets (Test Automation account, staging):

Linux: MTIyMTA0NzV8TkdFUHxGTEVFVHwwMTlkOGNkOC0xNTc1LTdjZjAtYjBlNi02ZThlYzM2YjAyODU (inactive for now)
K8s: MTIyMTA0NzV8TkdFUHxGTEVFVHwwMTliZTAzZC03MDYwLTcxMDctOTUwYS04YTFiODc3YjJiN2Q

Observability

Results reported to NR as AgentControlE2ETest events:

Scenario: fleet-control
Environment: linux-fleet-control or k8s-fleet-control
Account: "Agent Control Canaries (staging)"

sigilioso · 2026-04-16T12:47:30Z

Are we duplicating the logic to query FC api here and in .github/workflows/scripts/fleet_control_api.sh? Should we consider just calling the script from on-host's e2e rust code? (Or the other way around if we exposed it as some sort of command)

Hmm If I'm between calling a shell script from Rust code or calling Rust code from the GH Actions I think I'll go with the second alternative. I'll check how to do that.

sigilioso · 2026-04-16T12:49:46Z

+
+          # Use input fleet_id if provided, otherwise use default
+          # Fleet `AC-FC E2E tests (linux)` on staging using FC's `Test Automation` account.
+          FLEET_ID="${{ inputs.fleet_id || 'MTIyMTA0NzV8TkdFUHxGTEVFVHwwMTlkOGNkOC0xNTc1LTdjZjAtYjBlNi02ZThlYzM2YjAyODU' }}"


Do we need the || thing here? Isn't the field default value handling it already?

Not for the scheduled case I think, but might be a leftover from an earlier format of the workflow, let me check.

sigilioso · 2026-04-16T12:51:07Z

+      - name: Upload e2e result summary
+        if: always()
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
+        with:
+          name: e2e-result-linux-fleet-control-joint
+          path: e2e-result-*-linux-fleet-control-joint.txt
+          if-no-files-found: warn


I think this is for canaries to report the AC's e2e summary, do we need it here? Are we using the uploaded result anywhere?

That's what I wanted to check with you regarding the scripts that end up pushing something to Slack. I'll ping you.

sigilioso · 2026-04-16T12:53:17Z

+  on-host-fleet-control-tests-check:
+    name: OnHost Fleet Control E2E Check
+    if: always()
+    runs-on: ubuntu-latest
+    needs: [on-host-linux-fleet-control-e2e]
+    steps:
+      - name: Check if needed jobs succeeded
+        uses: re-actors/alls-green@05ac9388f0aebcb5727afa17fcccfecd6f8ec5fe
+        with:
+          jobs: ${{ toJSON(needs) }}


Do we need the all-green check here? We don't need to deal with GitHub required checks for the periodic / manual-dispatch workflow, do we?

Yeah you're right, this is C&P leftovers. Removing!

sigilioso · 2026-04-16T12:54:49Z

Should we add some sort of comment to easily distinguish these e2e from AC's regular e2e? After all the purpose is pretty different, isn't it?

🤔 Maybe it is already covered in the k8s-e2e/fleet-control-joint-tests README, but it seems pretty k8s-specific.

Hmm yeah I can add a header comment. I had a README.md in the test scenario so I'll reuse the wording here.

alvarocabanas · 2026-04-20T14:04:36Z

    write(config_path, updated_content);
 }

+#[cfg(target_os = "windows")]


Why wasn't this before? Is clippy complaining now?
Same for the other new annotations.

I modified the tests for all platforms so I got back to conditional compilation. It's optional though so if you prefer to not have it juts let me know and I can remove it!

Just removed them as it will be easier to work without them. Thanks for the heads up!

also add logic for running the fleet control API trigger (and tests) on all supported platforms

DavSanchez · 2026-04-21T14:35:29Z

Check this message for an example of alerts from these tests!

DavSanchez marked this pull request as ready for review April 15, 2026 21:10

DavSanchez requested a review from a team as a code owner April 15, 2026 21:10

DavSanchez force-pushed the ci/fc-tests-with-running-ac branch 2 times, most recently from 99fa9fe to 2cd8a0e Compare April 16, 2026 12:46

sigilioso reviewed Apr 16, 2026

View reviewed changes

alvarocabanas self-requested a review April 16, 2026 13:47

alvarocabanas reviewed Apr 16, 2026

View reviewed changes

Comment thread test/k8s-e2e/fleet-control-joint-tests/ac-values-fleet-control.yml Outdated

Comment thread .github/workflows/fleet_control_k8s_e2e.yml Outdated

DavSanchez force-pushed the ci/fc-tests-with-running-ac branch 3 times, most recently from accb3b0 to a444989 Compare April 17, 2026 19:01

DavSanchez added k8s-extended-e2e Trigger extended k8s e2e on a PR onhost-extended-e2e Execution of on host e2e in the current branch labels Apr 17, 2026

DavSanchez force-pushed the ci/fc-tests-with-running-ac branch 8 times, most recently from fa7d814 to ffc9581 Compare April 20, 2026 08:37

ci: run FC tests with a running AC

79c56b9

DavSanchez force-pushed the ci/fc-tests-with-running-ac branch from d0da580 to cd64721 Compare April 20, 2026 15:31

alvarocabanas reviewed Apr 21, 2026

View reviewed changes

DavSanchez added 3 commits April 21, 2026 12:21

docs: new workflows usage

d501280

fix: redundant default fleet ids

80f4e3d

build: use unique version name

683f0fc

DavSanchez added 22 commits April 21, 2026 12:21

ci: remove alls green usage

c93e281

fix: NBSP character

f4ac91d

refactor: reuse E2E runner code for k8s tests

89a63fd

ci: upload test result to slack

a7fac0d

ci: parameterize test table headers

51b2574

ci: rename tests

e38df5a

ci: allow account name override

b1fbc1b

ci: pass fleet id via env vars to tiltfile

01d433f

ci: required data for isolated fleet control test trigger

4930bf4

chore: naming

f451d77

refactor: move fleet control API utilities to common module

df0f5d7

also add logic for running the fleet control API trigger (and tests) on all supported platforms

style: gate logic on its used platforms

472ad5c

style: debug log for req payload

101557a

ci: pass debug flag to root CLI

f72f402

fix: trailing slashes for test urls

72110e7

feat: parameterize over the test suite

29fc794

feat: parameterize fleet test suite on CLI

f7bae20

ci: richer slack reports

4685725

build: remove conditional compilation flags

c7e9292

feat: parse time fleet data

9d6c921

fix: data report

184a92d

style: alert format

a108e74

DavSanchez force-pushed the ci/fc-tests-with-running-ac branch from ecc63d2 to a108e74 Compare April 21, 2026 11:23

docs: add more info

c1ebbdb

Conversation

DavSanchez commented Apr 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Add Fleet Control E2E Tests for Linux and Kubernetes

Overview

What's New

Technical Details

Testing

Default Fleets (Test Automation account, staging):

Observability

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

DavSanchez commented Apr 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

DavSanchez commented Apr 14, 2026 •

edited

Loading