Add API to expose toolbar data

[robhudson]

Summary:

Enhance the Django Debug Toolbar by providing an API to expose collected debug data. This API will enable new UIs to interact with the debug information programmatically, especially useful for API-only Django projects or cases where injecting HTML interferes with the application's frontend.

---

Proposal:

Add two initial API endpoints, building upon the serializable branch, which already stores collected debug stats as JSON in pluggable storage.

1. Get All Request IDs and Basic Info

   • Endpoint to retrieve a list of all available request_ids along with some basic metadata about each request, similar to the current history panel.
   • GET /__debug__/api/requests/

2. Get Full Data for a Specific Request

   • Endpoint to retrieve all debug data collected for a given request_id.
   • GET /__debug__/api/requests/<str:request_id>/

---

Motivation:

Currently, the Django Debug Toolbar injects an HTML interface into pages for debugging. However:

• For API-only Django projects, there's no user interface to display debug data.
• In projects with custom or complex frontends, injecting HTML can interfere with functionality or styling.
• A programmatic API enables developers to build custom debugging UIs tailored to their needs.

This API will provide greater flexibility and usability for a wider range of projects while leveraging the existing serializable storage architecture.

---

Implementation Details:

1. Base the API on the serializable branch:
   • The debug stats are already stored as JSON in pluggable storage, making it straightforward to expose this data via an API.
2. Consider Pagination:
   • For the /requests/ endpoint, if there are many requests, pagination can be implemented to improve performance and usability.
   • Pagination can be implemented within the pluggable storage classes from the serializable branch, as the method of pagination will likely depend on the specific storage backend being used.

[tim-schilling]

Thank you for writing this up. I'm generally onboard with this. I'm curious if we should ask for a sample client app to consume and render this. @robhudson you mentioned a TUI interface. Is that something that could be open-sourced and be mentioned as a third-party plugin?

[robhudson]

I've got the start of a TUI app based on Textual that I'd be happy to release/share once it's updated to use this API.

[robhudson]

To ensure that the JSON web API is secure and is not accidentally exposed in production, we can consider this 3-layer approach:

1. Enable API via settings.DEBUG_TOOLBAR_CONFIG

Use the settings.DEBUG_TOOLBAR_CONFIG dictionary to control access to the API. Add API_ENABLED to manage whether the API is accessible. This would default to False. When False this would not add the URLs resulting in a 404.

2. Use @require_show_toolbar decorator

Leverage the existing @require_show_toolbar decorator to enforce standard debug toolbar access checks, which, by default, validate that the toolbar is enabled based on the DEBUG setting and the client’s REMOTE_ADDR.

This ensures the API is available only in development environments and to trusted IPs, just like the debug toolbar itself.

3. Require a token for additional security

Introduce a token-based authentication mechanism for the API. The token should be retrieved from settings.DEBUG_TOOLBAR_CONFIG['API_TOKEN'] and passed in the request headers (e.g., X-Debug-Token).

This adds an additional layer of security by ensuring that even if the environment or IP checks are bypassed, access to the API still requires a valid token.

4. Document the above as part of the installation steps

The docs can walk the user through these optional steps. To hopefully increase the chances of a strong token, we can provide an example of generating a secure token or include a management command that outputs one.

---

The hope is that this layered approach significantly reduces the risk of accidental exposure or unauthorized access, making the API safe by default.

[tim-schilling]

@robhudson this all sounds good. I believe this should be enough such that a developer won't have enabled this unless they really mean to.

I also want to call out that the settings page should sanitize the settings.DEBUG_TOOLBAR_CONFIG['API_TOKEN'] because of https://github.com/django/django/blob/main/django/views/debug.py#L115-L130. So at least if someone is running with that mode, a person can't find the key to the Debug API from an 500 debug page.

[tim-schilling]

@robhudson I've got the serializable branch updated with main in #2072.