Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[pkg/ottl] Add ParseSeverity function #37280

Open
wants to merge 32 commits into
base: main
Choose a base branch
from

Conversation

bacherfl
Copy link
Contributor

@bacherfl bacherfl commented Jan 17, 2025

Description

This PR adds the ParseSeverity function, as discussed in the linked ticket. I also had to make a minor change to the
internal mapGetter, handling the map literals to return a raw map[string]any, instead of a pcommon.Map. This is because if there is a map literal within a slice, the pcommon.Slice.FromRaw cannot handle the pcommon.Map, as it only works with raw data types.

This change is however transparent, and the behavior to the outside of this package does not change.
EDIT: After merging main with the support for value expressions, introduced in #36883, this would affect the type of values returned by ParseValueExpression - previously this could potentially return map[string]any/[]any, but with the changes introduced in this PR, this would return a pcommon.Map/pcommon.Slice.
Please let me know if I should do this change in a separate PR though.

Link to tracking issue

Fixes #35079

Testing

Added unit and e2e tests

Documentation

Describe new function in the readme

Signed-off-by: Florian Bacher <[email protected]>
Signed-off-by: Florian Bacher <[email protected]>
Signed-off-by: Florian Bacher <[email protected]>
Signed-off-by: Florian Bacher <[email protected]>
Signed-off-by: Florian Bacher <[email protected]>
Signed-off-by: Florian Bacher <[email protected]>
Signed-off-by: Florian Bacher <[email protected]>
Signed-off-by: Florian Bacher <[email protected]>
Copy link
Contributor

This PR was marked stale due to lack of activity. It will be closed in 14 days.

@github-actions github-actions bot added the Stale label Feb 18, 2025
@TylerHelmuth
Copy link
Member

@bacherfl is this still draft?

@github-actions github-actions bot removed the Stale label Feb 19, 2025
@bacherfl
Copy link
Contributor Author

@bacherfl is this still draft?

@TylerHelmuth Right now this is blocked by #37408 so I left it as draft for now. Once that other PR has been merged this should be ready though

TylerHelmuth pushed a commit that referenced this pull request Mar 12, 2025
…#37408)

<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description

This PR fixes the limitations described in #37405. The recently
introduced `ValueExpression` had to be adapted, and will now only return
raw types, instead of their `pcommon.Map`/`pcommon.Slice` eqivalents, as
suggested in
#37280 (comment)

<!-- Issue number (e.g. #1234) or full URL to issue, if applicable. -->
#### Link to tracking issue
Fixes #37405 

<!--Describe what testing was performed and which tests were added.-->
#### Testing

Adapted and extended the unit and e2e tests

---------

Signed-off-by: Florian Bacher <[email protected]>
# Conflicts:
#	pkg/ottl/e2e/e2e_test.go
#	pkg/ottl/expression.go
#	pkg/ottl/parser.go
@bacherfl bacherfl marked this pull request as ready for review March 13, 2025 10:20
@bacherfl bacherfl requested a review from a team as a code owner March 13, 2025 10:20
@atoulme
Copy link
Contributor

atoulme commented Mar 16, 2025

Ready for review by codeowners: @TylerHelmuth, @kentquirk, @bogdandrutu, @evan-bradley, @edmocosta please review.

@edmocosta
Copy link
Contributor

@bacherfl is it ready for review? description says it's still in progress.

@bacherfl
Copy link
Contributor Author

@bacherfl is it ready for review? description says it's still in progress.

yes - i forgot to update the description, but it is ready

Comment on lines +76 to +78
if err := mapstructure.Decode(raw, &s); err != nil {
return nil, fmt.Errorf("cannot decode severity mapping: %w", err)
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dont understand how this is validating.

Copy link
Contributor Author

@bacherfl bacherfl Mar 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes the name of the function was not accurate, I renamed it now. But speaking of validation - do you think it would be better to validate the full mapping at the beginning of the function, or rather return an error when we run into an error during the evaluation of the log level?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can't validate the map statically during startup, so I think returning errors when we encounter something unexpected is the right thing to do since it is more efficient.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, why do we need to decode? I think we can check the type of the value when we try to use it and if it isn't a []any then we error.

Also would we need to check of other slice types? if the underlying value is a []string than it wont be considered a []any.

@edmocosta
Copy link
Contributor

TBH, I found this function's usage a bit confusing, and I'm not sure how it would scale if we decide to add new options. I think it would be clear (and more scalable) if we named the conditions types, for example

The current mappings:

ParseSeverity(severity_number, {
       "error":["err", { "min": 3, "max": 4 }],
       "info":[{ "min": 1, "max": 2 }],
     }
))

Could be expressed as:

PaseSeverity(severity_number, {
     "error": [{"equals": "err"}, {"range":[3,4]}],
     "info":  [{"range":[1,2]}],
})

With condition names, it's clear for me that the loose "err" string is an "equals" condition, and that the map with min and max key is a "range" condition (even if expressed as [2]int). It also makes it easy for introducing other conditions types in the future, such as "contains", "regex", etc.

Another future possibility would be supporting the AND operator by grouping conditions in the same map, being each key a condition, for example:

PaseSeverity(severity_number, {
     "error": [{"equals": "err", "range":[3,4]},  {"range":[5,6]}],
})

Just sharing my thoughts and opening it for discussion. If we don't have further plan for this function, it would be fine as it's (although naming the conditions would simplify the code, I guess).

@bacherfl
Copy link
Contributor Author

Just sharing my thoughts and opening it for discussion. If we don't have further plan for this function, it would be fine as it's (although naming the conditions would simplify the code, I guess).

@edmocosta the goal here was to use the same structure as the severity parser in stanza: #35079 (comment)

@edmocosta
Copy link
Contributor

Just sharing my thoughts and opening it for discussion. If we don't have further plan for this function, it would be fine as it's (although naming the conditions would simplify the code, I guess).

@edmocosta the goal here was to use the same structure as the severity parser in stanza: #35079 (comment)

I'm sorry for the noise, I probably misunderstood it and I thought the goal was having the same stanza functionality, and not exactly replicate the same configuration structure. I'm still don't understand why that should be a requirement, but if you folks agreed on that, I'm fine. Thanks for clarifying.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[pkg/ottl] ParseSeverity function
6 participants