feat: replace mcp-server-tester with RTF smoke tests

[DaleSeo]

This replaces the mcp-server-tester NPM package with an RTF-based smoke test for e2e testing. The old setup depended on a pinned third-party NPM tool and a live external API (theSpaceDevs), both of which are fragile in CI. The new test plan references shared scenario and environment configs from apollographql/rtf-mcp via GitHub file providers, runs the MCP server and a subgraph mock in Docker Compose, and validates 14 tools across 8 test cases (introspect, search, validate with valid/invalid queries, and execute). CI now builds a Docker image from the current code and tests it with rtf run, removing the Node.js dependency entirely.

Depends on https://github.com/apollographql/rtf-mcp/pull/33 (branch feat/mcp-server-smoke-test) which adds the shared mcp-server environment config and mcp-tool-smoke scenario. There's a TODO in the test plan to remove the git_ref references once that PR merges.

Testing

Run the smoke test (requires Docker):

❯ GITHUB_TOKEN=$(gh auth token) rtf run e2e/rtf/smoke/test-plan.yaml -v --force
 INFO loading and resolving test plan
 INFO checking if templating will work
 INFO creating output directory
 INFO executing test plan
 INFO templating test plan
 INFO checking test plan
 INFO executing environment setup
[+] Running 3/3
 ✔ Network rtf-mcp-smoke_default  Created                                                  0.0s 
 ✔ Container subgraph-mock        Healthy                                                  0.7s 
 ✔ Container apollo-mcp-server    Healthy                                                  0.7s 
 INFO executing scenario

Checking health at http://apollo-mcp-server:8000/health...
Health check passed

Connecting to MCP server at http://apollo-mcp-server:8000/mcp
Session established: 96db3a62-918b-44c8-9e9f-dd756ef935ab

=== tools/list ===
Server returned 13 tool(s):
  - GetCustomerWithOrders
  - GetFeaturedProducts
  - GetOrderDetails
  - GetProductById
  - GetProductBySku
  - GetProductReviews
  - GetRecentReviews
  - GetStoreStats
  - ListProducts
  - SearchProducts
  - execute
  - introspect
  - search
  - validate
  PASS: tool 'GetCustomerWithOrders' present
  PASS: tool 'GetFeaturedProducts' present
  PASS: tool 'GetOrderDetails' present
  PASS: tool 'GetProductById' present
  PASS: tool 'GetProductBySku' present
  PASS: tool 'GetProductReviews' present
  PASS: tool 'GetRecentReviews' present
  PASS: tool 'GetStoreStats' present
  PASS: tool 'ListProducts' present
  PASS: tool 'SearchProducts' present
  PASS: tool 'execute' present
  PASS: tool 'introspect' present
  PASS: tool 'search' present
  PASS: tool 'validate' present

=== Introspect Product type ===
  calling: introspect
  PASS: call succeeded
  PASS: response contains 'Product'

=== Search for product queries ===
  calling: search
  PASS: call succeeded
  PASS: response contains 'product'

=== Validate a valid query ===
  calling: validate
  PASS: call succeeded
  PASS: response contains 'Operation is valid'

=== Validate an invalid query (unknown field) ===
  calling: validate
  PASS: call returned expected error
  PASS: response contains 'does not have a field `nonExistentField`'

=== Validate query with invalid nested field ===
  calling: validate
  PASS: call returned expected error
  PASS: response contains 'does not have a field `bogusField`'

=== Validate query with missing required argument ===
  calling: validate
  PASS: call returned expected error
  PASS: response contains 'required argument'

=== Execute GetStoreStats ===
  calling: GetStoreStats
  PASS: call succeeded

=== Execute GetFeaturedProducts ===
  calling: GetFeaturedProducts
  PASS: call succeeded

===============================
Results: 28/28 passed
All tests passed
 INFO executing environment teardown
[+] Running 3/3
 ✔ Container apollo-mcp-server    Removed                                                  0.1s 
 ✔ Container subgraph-mock        Removed                                                  0.1s 
 ✔ Network rtf-mcp-smoke_default  Removed                                                  0.1s 
 INFO writing out resolved test plan and variables
 INFO done

[apollo-librarian]

✅ Docs preview has no changes

The preview was not built because there were no changes.

Build ID: a42fac1d00cc6c6c31f95f1d
Build Logs: View logs

---

✅ AI Style Review — No Changes Detected

No MDX files were changed in this pull request.

Review Log: View detailed log

This review is AI-generated. Please use common sense when accepting these suggestions, as they may not always be accurate or appropriate for your specific context.

[sminez]

You should use the action we provide from release-tools for running the test plan rather than attempting to clone the repo like this. RTF is an internal repo still which is why you're getting that error in your CI run.

[DaleSeo]

That's cool, @sminez! Updated the workflow to use it. :)

[DaleSeo]

Any ideas about this permission issue in CI?

https://github.com/apollographql/apollo-mcp-server/actions/runs/24463431474/job/71483772955?pr=727

[github-actions]

⏭️ Changeset check skipped via label