Using Locust for macro benchmarking and CI/CD integration of API performance

[user1500177]

Question

I’m exploring whether Locust can be used for macro-benchmarking of an API, and I’d like guidance on best practices and real-world usage.

What I want to achieve

• Macro benchmarking / load benchmarking of an API
• Benchmark 2 endpoints
• Get stable and comparable results
• Run Locust as part of CI/CD
• Export results as CSV or HTML reports
• Detect performance regressions over time

Questions

1. Macro benchmarking with Locust

• Is Locust commonly used for macro benchmarking (system-level performance) of APIs?
• Is it used in production environments / industry for this purpose, or mainly for load testing?

2. CI/CD integration

• Is it considered a good practice to integrate Locust into CI/CD pipelines?
• Are there recommended patterns for running Locust in CI and collecting benchmark results for comparison?

3. Test server vs real server

When running Locust:

• Should the real test server already be running before executing the Locust file?
• Or is it acceptable to create some kind of test client inside Locust itself and send requests to that?

4. Starting the API server from Locust

I’m considering the following approach for CI/CD simplicity:

• Use Locust start/stop hooks
• Start the API server (for example a Uvicorn server) using a command like:

  uv run filename

• Run the Locust test against it
• Stop the server automatically at the end of the Locust run

Questions related to this approach:

• Is it acceptable to start the API server as a subprocess from the Locust file?
• If so, what is the recommended way to:
  • Start the subprocess on Locust start
  • Cleanly stop it on Locust shutdown?
• Are there any known pitfalls with this approach?

5. TestClient vs subprocess

• Is it possible to start an in-memory TestClient-style app (instead of a real HTTP server) and benchmark it using Locust?
• Or is Locust fundamentally designed to benchmark only real running HTTP services?

My goal is to have CI/CD simply run:

locust -f locustfile.py

…and produce CSV / HTML reports without requiring manual server orchestration.

Any guidance, best practices, or real-world examples would be greatly appreciated.

[cyberw]

You used AI to write this, right? Why didnt you just ask the AI those questions? Or read the documentation? Feels like you're asking for someone else to do your work for you...

I'll bite on one of the questions though:

Is it possible to start an in-memory TestClient-style app (instead of a real HTTP server) and benchmark it using Locust?

No, testing something in-memory/in-process is not a good use case for Locust. Always run Locust (or any other load testing tool) as a separate process, ideally on a separate machine.