fix: fetch federated trust bundles concurrently #6491
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
markgoddard
requested review from
MarcosDY,
amartinezfayo,
evan2645,
rturner3 and
sorindumitru
as code owners
SPIRE agent periodically syncs entries, federated bundles and SVIDs from the SPIRE server. This happens every 5 seconds with a backoff. There is a 30 second timeout for this process. If there are many federations for the entries allowed for the agent, this timeout can be exceeded. This change adds a concurrent fetch for federated bundles to reduce the chance of hitting the timeout. Fixes: spiffe#6490
markgoddard
force-pushed
the
issues/6490
branch
from
4142365 to
a11fc24
Compare
Collaborator
sorindumitru
left a comment
sorindumitru
left a comment
There was a problem hiding this comment.
Thanks @markgoddard for providing a fix to this issue. This is looking good. I think we have some tests for fetching the federated bundles, but could you add one with more bundles than the number of workers to make sure that keeps working?
| } | ||
|
|
||
| // fetchBundle fetches a single federated bundle from SPIRE server. | ||
| func (c *client) fetchBundle(ctx context.Context, bundleClient bundlev1.BundleClient, trustDomain string) (*types.Bundle, error) { |
Collaborator
There was a problem hiding this comment.
Could we include federated in the names of these functions since they can only deal with federated bundles.
Suggested change
| func (c *client) fetchBundle(ctx context.Context, bundleClient bundlev1.BundleClient, trustDomain string) (*types.Bundle, error) { | |
| func (c *client) fetchFederatedBundle(ctx context.Context, bundleClient bundlev1.BundleClient, trustDomain string) (*types.Bundle, error) { |
| // fetchBundlesConcurrently fetches federated bundles concurrently. | ||
| // This is done to improve sync times when there are many federations. This should ensure that the | ||
| // sync does not exceed rpcTimeout. | ||
| func (c *client) fetchBundlesConcurrently(ctx context.Context, bundleClient bundlev1.BundleClient, trustDomains []string, bundles []*types.Bundle) ([]*types.Bundle, error) { |
Collaborator
There was a problem hiding this comment.
Suggested change
| func (c *client) fetchBundlesConcurrently(ctx context.Context, bundleClient bundlev1.BundleClient, trustDomains []string, bundles []*types.Bundle) ([]*types.Bundle, error) { | |
| func (c *client) fetchFederatedBundlesConcurrently(ctx context.Context, bundleClient bundlev1.BundleClient, trustDomains []string, bundles []*types.Bundle) ([]*types.Bundle, error) { |
Comment on lines
+642
to
+644
| wg.Add(1) | ||
| go func() { | ||
| defer wg.Done() |
Collaborator
There was a problem hiding this comment.
I think you can now do something like this to avoid the Add() and defer:
Suggested change
| wg.Add(1) | |
| go func() { | |
| defer wg.Done() | |
| wg.Go(func() { |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
SPIRE agent periodically syncs entries, federated bundles and SVIDs from
the SPIRE server. This happens every 5 seconds with a backoff. There is
a 30 second timeout for this process. If there are many federations for
the entries allowed for the agent, this timeout can be exceeded.
This change adds a concurrent fetch for federated bundles to reduce the
chance of hitting the timeout.
Fixes: #6490