Skip to content

Feature/docs 2986 new api discovery #1580

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

MaximBashurov
Copy link
Collaborator

No description provided.

Copy link

netlify bot commented Apr 22, 2025

Deploy Preview for pensive-dubinsky-5f7a00 ready!

Name Link
🔨 Latest commit b0290a9
🔍 Latest deploy log https://app.netlify.com/projects/pensive-dubinsky-5f7a00/deploys/68400dcac174fa000815701d
😎 Deploy Preview https://deploy-preview-1580--pensive-dubinsky-5f7a00.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

* Required and optional parameters and headers of requests and responses including:

* [Type/format](./exploring.md#format-and-data-type) of data sent in each parameter
* Date and time when parameter information was last updated
Copy link

@akadymov-wallarm akadymov-wallarm May 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MaximBashurov Why don't we mention things like sensitive data type, risk score, number of requests, application and sensitive business flow? Not sure we need to add them: just want to understand why there are not listed along with other attributes of API entries and parameters.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This part lists major objects: hosts, endpoints, and parameters of requests and responses. The remaining things are mentioned further on the same page.


* Have a full visibility into the whole API estate.
* See what data ([REST](exploring.md#rest-endpoint-details), [GraphQL](exploring.md#graphql-operation-details)) is going into and out of the APIs.
* Get a list of endpoints with the open vulnerabilities.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MaximBashurov List remove "vulnerabilities" for now. Yes, API Discovery will display any vuln related to a specific API endpoint (there are few of them) in the expanded "Risk score" section, but no specific filter or column for vulns. We are going to introduce more vulns and integrate API D with new product - Security issues (AASM) within https://wallarm.atlassian.net/browse/PROD-2798 and https://wallarm.atlassian.net/browse/DISC-10

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed


### Noise detection

The API Discovery module bases noise detection on the two major traffic parameters:

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MaximBashurov We are not sure what will be the default stability threshold values (probably same - 5 requests with 200 response over 300 seconds), but we'll choose them within the task: https://wallarm.atlassian.net/browse/IO-2612

No edits here for now, just FYI

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it, thanks.

* Only those requests to which the server responded in the 2xx range are processed.
* Requests that do not conform to the design principles of the REST or GraphQL API are not processed.

For REST, this is done by controlling the `Content-Type` header of responses: if it does not contain `application/json` (like `Content-Type: application/json;charset=utf-8`), the request is considered to be a non-REST API and is not analyzed. If the header does not exist, API Discovery analyzes the request.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MaximBashurov let's modify it:

An entry should NOT be classified as a valid API call and not displayed in API Discovery if both following conditions is met:

  1. The request path contains a file extension (i.e., the last path segment does not match the pattern .*.[a-zA-Z0-9]+).
  2. The response Content-Type header
  • is not present or
  • is present and it is not starting with application/ and does not contain json after slash (case-insensitive match, without charset suffix).

https://wallarm.atlassian.net/issues/IO-2485

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed


### Sensitive data detection

API Discovery [detects and highlights](sensitive-data.md) sensitive data consumed and carried by your APIs:

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MaximBashurov Let's add AI prompts as a separate sensitive data group as well

Copy link
Collaborator Author

@MaximBashurov MaximBashurov Jun 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added, but cannot actually see them in Sensitive data filter, let's discuss.


In addition to automatic identification, you can manually adjust the assigned sensitive business flow tags and manually set tags for the endpoints of your choice.

Once endpoints are assigned with the sensitive business flow tags, it becomes possible to filter all discovered endpoint by a specific business flow which makes it easier on protecting the most critical business capabilities.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MaximBashurov in new API Discovery users will operate with smth further than endpoints - API entries. We call them this way since for non-REST protocols one API endpoint represents multiple API entries with different schema, parameters and behaviour.

Since one record in API Discovery correlates with one API entry rather with endpoint, I suggest that we replace API endpoints with this new term or smth similar.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's discuss.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed and decided to leave as it is for now.

@@ -0,0 +1,48 @@
# Sensitive Business Flows <a href="../../about-wallarm/subscription-plans/#waap-and-advanced-api-security"><img src="../../images/api-security-tag.svg" style="border: none;"></a>

With the sensitive business flow capability, Wallarm's [API Discovery](overview.md) can automatically identify endpoints that are critical to specific business flows and functions, such as authentication, account management, billing, and similar critical capabilities. Learn from this article how to use the sensitive business flow functionality.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MaximBashurov I believe adding AI agents to the list would be nice in terms of our market positioning

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed


![API Discovery - Sensitive business flows](../images/about-wallarm-waf/api-discovery-2.0/api-discovery-sbf.png)

You can do the same in the endpoint details.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MaximBashurov for now, you cannot. We decided not to implement this capability since it is duplicating the same button in the list (which is visible even when the API entry is expanded). Let's skip this part.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed


In the **Status** column for endpoints and parameters, API Discovery provides data about changes in your API for the last week:

* **New** for the endpoints discovered within a week.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MaximBashurov I recommend replacing endpoint term with smth like API entry or route within the whole section.


![API Discovery - track changes](../images/about-wallarm-waf/api-discovery-2.0/api-discovery-changes.png)

To change the time period, redefine dates in the **Changes since** filter.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MaximBashurov Not relevant here. Changes since doesn't affect status column (opposed to old API Discovery). We believe it improves clarity of the user flow.

Copy link
Collaborator Author

@MaximBashurov MaximBashurov Jun 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. Suggest to rename filter from Changes since to Changed since, let's discuss along with other filters and UI changes ("endpoint" in UI).


![API Discovery - built API inventory](../images/about-wallarm-waf/api-discovery-2.0/api-discovery-built-inventory.png)

By default, endpoints and operations are sorted by host/endpoint or operation name. Also, **Group by host** is on. With grouping by host disabled, you can sort endpoints by risk.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MaximBashurov we improved UI for sorting by risk score. Even when the grouping is on, user is able to apply sorting and disable the grouping at the same time.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With that in mind, text is still correct: sorting is only possible with grouping off.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants