Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add LogPatternTool #413
Add LogPatternTool #413
Changes from all commits
f539402
0ba9f8e
470d83b
1605288
c18663a
6b71d21
File filter
Filter by extension
Conversations
Jump to
There are no files selected for viewing
Check warning on line 75 in src/main/java/org/opensearch/agent/ToolPlugin.java
Codecov / codecov/patch
src/main/java/org/opensearch/agent/ToolPlugin.java#L75
Check warning on line 93 in src/main/java/org/opensearch/agent/ToolPlugin.java
Codecov / codecov/patch
src/main/java/org/opensearch/agent/ToolPlugin.java#L92-L93
Check warning on line 90 in src/main/java/org/opensearch/agent/tools/LogPatternTool.java
Codecov / codecov/patch
src/main/java/org/opensearch/agent/tools/LogPatternTool.java#L90
Check warning on line 111 in src/main/java/org/opensearch/agent/tools/LogPatternTool.java
Codecov / codecov/patch
src/main/java/org/opensearch/agent/tools/LogPatternTool.java#L106-L111
Check warning on line 114 in src/main/java/org/opensearch/agent/tools/LogPatternTool.java
Codecov / codecov/patch
src/main/java/org/opensearch/agent/tools/LogPatternTool.java#L113-L114
Check warning on line 119 in src/main/java/org/opensearch/agent/tools/LogPatternTool.java
Codecov / codecov/patch
src/main/java/org/opensearch/agent/tools/LogPatternTool.java#L118-L119
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we stop if longestField is fewer than maybe 30 chars or continue to search for 10 more records in case the first one has outlier data?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It just imitates the logic of Discover log pattern.
It sounds good to avoid outliner, but it could also make the logic more complex and inconsistent with Discover.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The above suggestion is not ideal, just want to trigger some discussion. It's ok to leave it as it is if we don't have better idea.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What if the longest field is missing in
hits[0]
? Shall we iterate though all the hits?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, that's possible. And also, in some cases, the longest field in the first log maybe not the most longest overall.
But I think firstly we just imitate what current log pattern feature do, and do enhancement later. Better to discuss with Observability team as well. You can check the current implementation here Code Ref
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we add one more check to see whether parseField is one of the columns inside the index? Sometimes customer could type wrong parse field name, in the PPL, it will raise an error to show the parse field doesn't exist. We could do the same logic.
Check warning on line 122 in src/main/java/org/opensearch/agent/tools/LogPatternTool.java
Codecov / codecov/patch
src/main/java/org/opensearch/agent/tools/LogPatternTool.java#L121-L122
Check warning on line 124 in src/main/java/org/opensearch/agent/tools/LogPatternTool.java
Codecov / codecov/patch
src/main/java/org/opensearch/agent/tools/LogPatternTool.java#L124
Check warning on line 129 in src/main/java/org/opensearch/agent/tools/LogPatternTool.java
Codecov / codecov/patch
src/main/java/org/opensearch/agent/tools/LogPatternTool.java#L126-L129
Check warning on line 138 in src/main/java/org/opensearch/agent/tools/LogPatternTool.java
Codecov / codecov/patch
src/main/java/org/opensearch/agent/tools/LogPatternTool.java#L131-L138
Check warning on line 140 in src/main/java/org/opensearch/agent/tools/LogPatternTool.java
Codecov / codecov/patch
src/main/java/org/opensearch/agent/tools/LogPatternTool.java#L140
Check warning on line 142 in src/main/java/org/opensearch/agent/tools/LogPatternTool.java
Codecov / codecov/patch
src/main/java/org/opensearch/agent/tools/LogPatternTool.java#L142
Check warning on line 144 in src/main/java/org/opensearch/agent/tools/LogPatternTool.java
Codecov / codecov/patch
src/main/java/org/opensearch/agent/tools/LogPatternTool.java#L144
Check warning on line 147 in src/main/java/org/opensearch/agent/tools/LogPatternTool.java
Codecov / codecov/patch
src/main/java/org/opensearch/agent/tools/LogPatternTool.java#L147
Check warning on line 152 in src/main/java/org/opensearch/agent/tools/LogPatternTool.java
Codecov / codecov/patch
src/main/java/org/opensearch/agent/tools/LogPatternTool.java#L149-L152
Check warning on line 159 in src/main/java/org/opensearch/agent/tools/LogPatternTool.java
Codecov / codecov/patch
src/main/java/org/opensearch/agent/tools/LogPatternTool.java#L154-L159
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For better readability, please try regex:
[a-zA-Z0-9]
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For this version, I think we just need to copy the code from sql plugin without doing too much change. So developer can better track the source and diff.
What you suggested could be an enhancement in a separate PR.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure
Check warning on line 223 in src/main/java/org/opensearch/agent/tools/LogPatternTool.java
Codecov / codecov/patch
src/main/java/org/opensearch/agent/tools/LogPatternTool.java#L221-L223
Check warning on line 232 in src/main/java/org/opensearch/agent/tools/LogPatternTool.java
Codecov / codecov/patch
src/main/java/org/opensearch/agent/tools/LogPatternTool.java#L229-L232
Check warning on line 235 in src/main/java/org/opensearch/agent/tools/LogPatternTool.java
Codecov / codecov/patch
src/main/java/org/opensearch/agent/tools/LogPatternTool.java#L234-L235
Check warning on line 241 in src/main/java/org/opensearch/agent/tools/LogPatternTool.java
Codecov / codecov/patch
src/main/java/org/opensearch/agent/tools/LogPatternTool.java#L240-L241
Check warning on line 255 in src/main/java/org/opensearch/agent/tools/LogPatternTool.java
Codecov / codecov/patch
src/main/java/org/opensearch/agent/tools/LogPatternTool.java#L255
Check warning on line 281 in src/main/java/org/opensearch/agent/tools/LogPatternTool.java
Codecov / codecov/patch
src/main/java/org/opensearch/agent/tools/LogPatternTool.java#L281
Check warning on line 286 in src/main/java/org/opensearch/agent/tools/LogPatternTool.java
Codecov / codecov/patch
src/main/java/org/opensearch/agent/tools/LogPatternTool.java#L286