Limiter extension API interfaces (draft 4) #12953

jmacd · 2025-04-30T22:29:25Z

Description

See #12603. This follows discussion in #12700 and has been updated extensively based on feedback. This PR is too large to merge as-is but can be taken as a model for a series of smaller PRs.

Link to tracking issue

Part of #9591.

Testing

TODO

Documentation

Done.

bogdandrutu · 2025-05-01T03:21:44Z

extension/extensionlimiter/README.md

+All limiters feature a `MustDeny` method which is made available for
+applications to test when a limit is fully saturated. This special
+limit request is defined as the equivalent of passing a zero value to
+the limiter.


This special limit request is defined as the equivalent of passing a zero value to
the limiter.

What is the need of this requirement to be this way instead of simply be a standalone API? Here is an example, if a limiter has providers for num requests and items, and only items is fully saturated, I should not call both requests which passes and also the items provider to figure this out.

Since there are multiple types of limits, I would prefer the "MustDeny" to be independent of the size type and call it only once at the earliest stage as possible, otherwise I have to call 4 time (for rate limiter) at the beginning with 0.

I don't see any strong reasons not to use a standalone API. The user will have to handle separate objects anyway, in most cases, since they call limiters for individual weights but ask MustDeny for all of the un-checked limits in general, I think.

I will change this to use a standalone API.

bogdandrutu · 2025-05-01T03:29:07Z

extension/extensionlimiter/README.md

+        metadata.Type,
+        createDefaultConfig,
+        xreceiver.WithMetrics(createMetrics, metadata.MetricsStability),
+        xreceiver.WithLimiters(getLimiters),


Should the limiters be part of the receiver instance? A factory should be agnostic of the config, and per "getLimiters is a function to get the effective[]configmiddleware.Config" this comes from a config.

I am not sure, this is a nuanced question because some receivers like OTLP support different middleware settings for each protocol.

If we are going to configure request_items and memory_size via receivers, while configuring request_count and network_bytes in middleware, then I see two choices for where the limiters are configured:

All limiters are in a middlewares list (consistently; even if all you are using are limiters, you can use middlewares, i.e., even for non-grpc and non-http you could use middlewares just as limiters). This way users just set middlewares. If receivers have different protocols, they use different limiters. (I prefer this option.)

Receivers have a separate config section, similar to the various exporterhelper configs which is embedded in receivers. Then users have to understand where limiters are configured, which is IMO more complicated. Some limiters are in the receiver-level config, some limiters are in the middleware-level config. (I prefer this other option.)

I'll do what the group would prefer, of course!

bogdandrutu · 2025-05-01T03:40:14Z

extension/extensionlimiter/rate.go

+	// Limiter includes MustDeny().
+	Limiter
+


here is my confusion, I think the RateLimiterProvider should implement the Limiter, because I want to call the "saturation" only once asap. I know I can still do that by checking all the weight type, but for a limiter that supports only items I don't think you expect to call this in the "middleware".

RateLimiterProvider is called in Start() typically (for static limiter configurations). Are we on the same page?

Re: only once: I created an adapter for combining multiple limiters (which may be called at different times) by their MustDeny(). That's how I call MustDeny once, it's the MultiLimiter in limiterhelper/consumer.go. When a wrapper limiter constructs >1 limiter in the wrapper, there is a return value limiter which combines all of them for MustDeny calls.

I think I sorted this out. Thank you for pointing to this. I have modified the RateLimiter and ResourceLimiter interfaces to include GetChecker() methods which allow obtaining a Checker, having the MustDeny method. The MustDeny method and Checker interface apply to all weight keys. I have documented that MustDeny may be called more than once per request since middleware, receiver, and wrappers can all reasonably call this in various circumstances.

bogdandrutu · 2025-05-01T03:44:57Z

extension/extensionlimiter/rate.go

+// RateLimiterFunc is an easy way to construct RateLimiters.
+type RateLimiterFunc func(ctx context.Context, value uint64) error
+
+var _ RateLimiter = RateLimiterFunc(nil)
+
+// MustDeny implements RateLimiter.
+func (f RateLimiterFunc) MustDeny(ctx context.Context) error {
+	return f.Limit(ctx, 0)
+}
+
+// Limit implements RateLimiter.
+func (f RateLimiterFunc) Limit(ctx context.Context, value uint64) error {
+	if f == nil {
+		return nil
+	}
+	return f(ctx, value)
+}


The pattern we use is here you have a RateLimiterLimitFunc that only implements the Limit, then the implementation will embed 2 funcs.

Also, per your slack message "I tried to follow your style for what I will call extensive use of function-combinators and narrow interfaces" -> this pattern is based on the https://pkg.go.dev/net/http#HandlerFunc

I see what you mean. Would you prefer to have MustDeny be a completely separate interface, and limiters have two functions? I can go that way. Initially I had a single interface, just Acquire(x) where x==0 can be interpreted as MustDeny. I think that's what @axw was asking for here.

Re: http.HandlerFunc.

I like the pattern! No complaints. I wouldn't call net/http an extensive-user of this pattern, however. In the middleware work, extensionmiddlewaretest has its own copy of what we might call http.RoundTripperFunc (which does not exist).

I think that's what @axw was asking for #12953 (comment).

I understand that, but I go back to my point which is MustDeny is an interface that can be called as early as possible and is independent of the WeightKey. Your proposal and @axw supported version requires me to call Acquire(0) for 4 different keys (https://github.com/open-telemetry/opentelemetry-collector/pull/12953/files#diff-4c7245f5f721ab2af95d5dd48178e834ffb261e04c3e63f4e9dc844ddb284a04R19).

In my understanding: In general, every "limiter" has 2 thresholds, one that is a soft limit and decision can be to wait "space is available" or drop after some "timeout/deadline", and one when immediately will reject the request to free resources. My suggestion, and understanding is to make the MustDeny the hard limit, when I need to immediately cancel the request.

Separate topic:

I see what you mean. Would you prefer to have MustDeny be a completely separate interface, and limiters have two functions? I can go that way. Initially I had a single interface, just Acquire(x) where x==0 can be interpreted as MustDeny.

I don't think I am trying to suggest to have just Acquire(x) where x==0. I am suggesting a helper func to implement only a func in the interface, but interface still have 2 functions MustDeny and Acquire.

I have greatly misunderstood this detail. I will need to do more work on this, thank you for this comment.

I'm not sure if I've understood this correctly, but I've changed the code structure based on this discussion.

There is now a separate interface named Checker with a MustDeny function. There is a MustDenyFunc adapter named after the function, not after the interface.

The rate limiter now appears as an interface that embeds the Checker and a Limit function:

type RateLimiter interface { Checker Limit(ctx context.Context, value uint64) error }

and there is a LimitFunc again named after a function. To construct an Error or a Nop implementation of RateLimiter, I build a struct with two functions, like:

// Verify that a rate limiter is constructed of two functions. var _ RateLimiter = struct { MustDenyFunc LimitFunc }{}

extension/extensionlimiter/wrapper.go

bogdandrutu · 2025-05-01T03:56:49Z

extension/extensionlimiter/rate.go

+// middleware (e.g., grpc.StatsHandler).
+//
+// See the README for more recommendations.
+type RateLimiter interface {


Maybe I missed somewhere this discussion, but it feels to me that RateLimiter == ResourceLimiter where the ReleaseFunc is a no-op func. Is that true? If yes, do we need this?

I do not support this idea.

The the meaning behind these limits is different. I liken this to the distinction between OTel metrics UpDownCounter and Counter. The admission controller in contrib/internal/otelarrow wouldn't do anything if you called it in a gRPC statshandler for network bytes, just be meaningless overhead, because it thinks that the release function means something (i.e., it only supports resource limits) and the gRPC stats handler has no way to call the release func after the network bytes are no longer used (i.e., it only supports rate limits).

I have updated the README with all of the open questions that I see.

I refer to a hypothetical case involving data-dependent limiting, in which we could imagine a non-blocking way to filter data that passes a rate limit and drop data that cannot be accepted. If there is a non-blocking use-case of that sort, then it makes sense to have a wrapper that would abstract the detail. I called this the TryLimitOrAcquire function which would return a no-op release function in case of the rate limiter. hth

axw · 2025-05-02T03:34:32Z

extension/extensionlimiter/README.md

+risks wasting memory. In general, an overloaded limiter that is
+saturated SHOULD fail requests immediately.
+
+Limiter implementations SHOULD consider the context deadline when


Should SHOULD be MUST? I tend to think considering context deadline should be mandatory. But maybe I'm being pedantic - did you just mean here that limiters MAY choose not to block if they can anticipate that the context deadline will come before the limiter is no longer saturated?

Following @bogdandrutu's suggestion, I've separated the MustDeny method (which takes only context) from the Acquire/Limit methods which are weight-dependent.

extension/extensionlimiter/README.md

axw · 2025-05-02T03:39:56Z

extension/extensionlimiter/README.md

+Limiters implementations MAY block the request or fail immediately,
+subject to internal logic. A limiter aims to avoid waste, which


What is the internal logic? Are you saying this is a property of the limiter, rather than the caller?

If we made it caller-defined, then I think we could get rid of the MustDeny call: instead you would make a non-blocking request for 0 items. If the limiter is saturated, that would return an error; otherwise it would return success without affecting the capacity.

I have raised a question about non-blocking methods in the open-questions section of the README.

…tor into jmacd/limiter_v4

jmacd · 2025-05-06T22:20:10Z

Reviewers, please see the open questions:

https://github.com/open-telemetry/opentelemetry-collector/pull/12953/files#diff-50c94a038eae5ba5747b0b2c502b0753f2af3664671c4aaec46d4916b828dc25

bogdandrutu

The public API looks good, some naming suggestions and some go implementation in the helper.

bogdandrutu · 2025-05-08T23:14:12Z

extension/extensionlimiter/extensionlimiter.go

+
+// Checker is for checking when a limit is saturated.  This can be
+// called prior to the start of work to check for limiter saturation.
+type Checker interface {


Up for discussion (not sure): Should this be the "basic" limiter, then call it "Limiter" instead?

bogdandrutu · 2025-05-08T23:15:55Z

extension/extensionlimiter/limiterhelper/checker.go

+	var err error
+	for _, lim := range ls {
+		if lim == nil {
+			continue
+		}
+		err = errors.Join(err, lim.MustDeny(ctx))
+	}
+	return err


errors.Join -> is a bit inefficient when lim.MustDeny(ctx) returns nil. I prefer the multierr.Append which does a better job.

bogdandrutu · 2025-05-08T23:17:09Z

extension/extensionlimiter/extensionlimiter.go

+// CheckerProvider is an interface to obtain checkers for a group of
+// weight keys.
+type CheckerProvider interface {
+	// GetChecker returns a checker for a group of weight keys.
+	GetChecker(...Option) (Checker, error)
+}


What is the point of having the extra layer of "Provider" for the "Checker"?

bogdandrutu · 2025-05-08T23:21:59Z

extension/extensionlimiter/weight.go

+// checked at a certain stage.  The receiver and middleware can both
+// be responsible for applying limits, and this type helps ensure
+// limits are applied only across cooperating sub-components.
+type WeightSet []WeightKey


Can you please add this later when the first usage comes? I don't want to argue more about this PR, and I am unable to understand where this is used.

bogdandrutu · 2025-05-08T23:22:54Z

extension/extensionlimiter/weight.go

+// StandardNotMiddlewareKeys methods return the list of middleware
+// keys that can be automatically configured through middleware and
+// not.
+type WeightKey string


Not sure how important this is, do you see a big usage of this on the critical path? If yes, should we use an enum as int instead, since that will be a bit faster for type checks?

bogdandrutu · 2025-05-08T23:24:53Z

extension/extensionlimiter/rate.go

+	CheckerProvider
+
+	// GetRateLimiter returns a rate limiter for a weight key.
+	GetRateLimiter(WeightKey, ...Option) (RateLimiter, error)


Do you think we should have different Options for different providers (different types so we can expand them independently)?

bogdandrutu · 2025-05-08T23:27:26Z

extension/extensionlimiter/limiterhelper/wrapper.go

+// configmiddleware or limiterhelper is responsible for constructing
+// the correct wrapper from these two kinds of limiter; users will use
+// this interface consistently.
+type LimiterWrapper interface {


Do you expect others to implement this interface? Otherwise we should have this as a struct.

bogdandrutu · 2025-05-08T23:28:07Z

extension/extensionlimiter/limiterhelper/wrapper.go

+// the appropriate interface for callers that can easily wrap a
+// function call, because for wrapped calls there is no distinction
+// between rate limiters and resource limiters.
+type LimiterWrapperProvider interface {


Do you expect others to implement this interface? Otherwise we should have this as a struct.

bogdandrutu · 2025-05-08T23:31:20Z

extension/extensionlimiter/README.md

+- The protocol name
+- The signal kind
+- The caller's component ID


I am not convinced about protocol, but I see the other 2 being useful.

bogdandrutu · 2025-05-08T23:36:08Z

extension/extensionlimiter/README.md

+functions. No examples are provided. How will limiters configure, for
+example, tenant-specific limits?
+
+##### Data-dependent limits


If #39199 is accepted, do you still need support here?

jmacd added 7 commits April 30, 2025 13:20

lint

2eb77f5

close

04380f4

readme

7444153

move multi-limiter

e597fa1

data-dep example

e6675c8

lint

ac0d1ec

lint

b3e4554

jmacd requested a review from a team as a code owner April 30, 2025 22:29

jmacd requested a review from dmitryax April 30, 2025 22:29

github-actions bot added the receiver/otlp label Apr 30, 2025

jmacd requested a review from axw April 30, 2025 22:30

jmacd marked this pull request as draft April 30, 2025 22:33

bogdandrutu reviewed May 1, 2025

View reviewed changes

axw reviewed May 2, 2025

View reviewed changes

jmacd added 8 commits May 5, 2025 15:30

Merge branch 'main' of github.com:open-telemetry/opentelemetry-collec…

ffad28a

…tor into jmacd/limiter_v4

wip split Checker (was Limiter)

b8ed41d

rename

d07da61

move wrapper into limiterhelper

f3a2f2f

style

8b31cd6

call checker once

40aee98

checker all keys

137be27

readme

4a44264

bogdandrutu approved these changes May 8, 2025

View reviewed changes

bogdandrutu reviewed May 8, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Limiter extension API interfaces (draft 4) #12953

Limiter extension API interfaces (draft 4) #12953

jmacd commented Apr 30, 2025 •

edited

Loading

bogdandrutu May 1, 2025

jmacd May 3, 2025

bogdandrutu May 1, 2025

jmacd May 3, 2025

bogdandrutu May 1, 2025

jmacd May 3, 2025

jmacd May 6, 2025

bogdandrutu May 1, 2025

jmacd May 3, 2025

bogdandrutu May 3, 2025 •

edited

Loading

bogdandrutu May 3, 2025

jmacd May 6, 2025

jmacd May 6, 2025

bogdandrutu May 1, 2025

jmacd May 3, 2025

jmacd May 6, 2025

axw May 2, 2025

jmacd May 6, 2025

axw May 2, 2025

jmacd May 6, 2025

jmacd commented May 6, 2025

bogdandrutu left a comment

bogdandrutu May 8, 2025

bogdandrutu May 8, 2025

bogdandrutu May 8, 2025

bogdandrutu May 8, 2025

bogdandrutu May 8, 2025

bogdandrutu May 8, 2025

bogdandrutu May 8, 2025

bogdandrutu May 8, 2025

bogdandrutu May 8, 2025

bogdandrutu May 8, 2025

		Limiters implementations MAY block the request or fail immediately,
		subject to internal logic. A limiter aims to avoid waste, which

Limiter extension API interfaces (**draft 4**) #12953

Are you sure you want to change the base?

Limiter extension API interfaces (**draft 4**) #12953

Conversation

jmacd commented Apr 30, 2025 • edited Loading

Description

Link to tracking issue

Testing

Documentation

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bogdandrutu May 3, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jmacd commented May 6, 2025

bogdandrutu left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Limiter extension API interfaces (draft 4) #12953

Limiter extension API interfaces (draft 4) #12953

jmacd commented Apr 30, 2025 •

edited

Loading

bogdandrutu May 3, 2025 •

edited

Loading