Commit c1bceb7
committed
aws/eni: Use default AWS API page size
When constructing the `input` [`DescribeNetworkInterfacesInput`](https://pkg.go.dev/github.com/aws/aws-sdk-go-v2/service/ec2#DescribeNetworkInterfacesInput), its `MaxResults` field is set to the `maxResults` parameter of `GetDetachedNetworkInterfaces`. But despite having the same name, those two parameters are very different:
* `maxResults` is the [maximum number of ENIs](https://github.com/cilium/cilium/blob/6d1aa98670fe046fe3ac5a2b4167cfece416603e/pkg/aws/eni/eni_gc.go#L29-L31) we want returned by `GetDetachedNetworkInterfaces`
* `input.MaxResults` is actually the (max) page size of paginated AWS API calls, see [pagination doc](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/Query-Requests.html#api-pagination)
The way the AWS API constructs a `DescribeNetworkInterfaces` call response is:
1. First gather all ENIs,
2. Then break them into pages based on the requested max page size
3. Then apply filters to remove ENIs that don't match the filters
4. Finally send all pages with whatever is left in them.
This means that for example in an account with 1000 ENIs, with 5 ENIs that have a specific tag, if we make a `DescribeNetworkInterfaces` with a filter on this tag and request a page size of 25, the AWS API will start by gathering all 1000 ENIs. Then group them in 40 pages of 25 ENIs each. Then apply the filter which will leave most pages empty, only at most five pages will have at least one ENI depending on whether the 5 ENIs matching the tag filter were in the same pages or not. Then the API will send all 40 pages one by one even if most of those are empty. this means that we make 40 individual API calls to get 5 ENIs.
To avoid this situation, we set the AWS API response max page size to [its maximum value of 1000](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeNetworkInterfaces.html#API_DescribeNetworkInterfaces_RequestParameters). using the existing `defaults.ENIMaxResultsPerApiCall`.
Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>1 parent 57e907f commit c1bceb7
1 file changed
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
212 | 212 | | |
213 | 213 | | |
214 | 214 | | |
215 | | - | |
| 215 | + | |
216 | 216 | | |
217 | 217 | | |
218 | 218 | | |
| |||
0 commit comments