Backend Profiling: Ability to trace the server and benchmark scripts

[freddyaboulton]

Description

Modifies the server so that traces corresponding to the different parts of the request lifecycle are collected if the GRADIO_PROFILING environment variable is set. If the environment variable is not set, the tracing is a no-op so it should not impact the performance of production applications.

I also added some scripts in scripts/benchmark to automatically profile any gradio application under different levels of concurrent requests and concurrency_limits. The primary script is scripts/benchmark/runner.py. You can call it like so:

python scripts/benchmark/runner.py\
--app scripts/benchmark/apps/echo_text.py --tiers 10,100\
 --requests-per-user 30 --mode burst\
--concurrency-limit 100\
 --output-dir benchmark_results/echo_text_wave_100

• The app can theoretically be any gradio app though the runner uses a heuristic to determine the value of the input data to send with the request that needs to be hardened to truly handle any application. This is something I'm still thinking about.
• The --tiers are the numbers of concurrent users tested.
• The --mode is either burst or wave. If burst all the requests hit the server at the same time. With wave some random noise is added to stagger them.

This will output two json files along with some summary statistics to the --output-dir specified.

I also added scripts/benchmark/generate_charts.py to automatically generate some charts for the developer. However, I am not sure these are very valuable in general. Should be straightforward to point a coding agent at the output directory and have it create a pandas + matplotlib script to plot the data however you like.

AI Disclosure

We encourage the use of AI tooling in creating PRs, but the any non-trivial use of AI needs be disclosed. E.g. if you used Claude to write a first draft, you should mention that. Trivial tab-completion doesn't need to be disclosed. You should self-review all PRs, especially if they were generated with AI.

• I used AI to write the benchmarking scripts and generate plots.
• I did not use AI

🎯 PRs Should Target Issues

Before your create a PR, please check to see if there is an existing issue for this change. If not, please create an issue before you create this PR, unless the fix is very small.

Not adhering to this guideline will result in the PR being closed.

Testing and Formatting Your Code

1. PRs will only be merged if tests pass on CI. We recommend at least running the backend tests locally, please set up your Gradio environment locally and run the backed tests: bash scripts/run_backend_tests.sh

2. Please run these bash scripts to automatically format your code: bash scripts/format_backend.sh, and (if you made any changes to non-Python files) bash scripts/format_frontend.sh

[gradio-pr-bot]

🪼 branch checks and previews

•	Name	Status	URL
	Spaces	ready!	Spaces preview
	Website	ready!	Website preview
🦄	Changes	detected!	Details

---

Install Gradio from this PR

pip install https://gradio-pypi-previews.s3.amazonaws.com/345dd78de407de3c3fa38848295edb49c42184e9/gradio-6.9.0-py3-none-any.whl

Install Gradio Python Client from this PR

pip install "gradio-client @ git+https://github.com/gradio-app/gradio@345dd78de407de3c3fa38848295edb49c42184e9#subdirectory=client/python"

Install Gradio JS Client from this PR

npm install https://gradio-npm-previews.s3.amazonaws.com/345dd78de407de3c3fa38848295edb49c42184e9/gradio-client-2.1.0.tgz

[gradio-pr-bot]

🦄 change detected

This Pull Request includes changes to the following packages.

Package	Version
gradio	minor

---

• Backend Profiling: Ability to trace the server and benchmark scripts

---

✅ Changeset approved by @freddyaboulton

• Maintainers can remove approval by unchecking this checkbox.

Something isn't right?

• Maintainers can change the version label to modify the version bump.
• If the bot has failed to detect any changes, or if this pull request needs to update multiple packages to different versions or requires a more comprehensive changelog entry, maintainers can update the changelog file directly.