Skip to content

Conversation

sarub0b0
Copy link
Owner

@sarub0b0 sarub0b0 commented Oct 18, 2025

Summary

This PR adds JMESPath filter syntax (jmespath:) to the log query feature as an alternative to the jq: filter. JMESPath provides a different JSON query language that some users may find more intuitive for certain use cases.

Motivation

  • Issue Enhance JSON logging #809 requests support for both jq and JMESPath filter syntaxes
  • JMESPath offers a simpler, more declarative syntax for common JSON queries
  • Provides users with choice between two complementary JSON query languages

Changes

New Query Syntax

  • jmespath:<expression> - Filter JSON logs using JMESPath expressions
  • Aliases: jmes:, jm:
  • Example: pod:api log:error jmespath:message or pod:api jm:data.userId

Implementation Details

  • Extends the existing JsonFilter enum with JMESPath variant alongside Jq
  • Adds JMESPathProgram struct to hold compiled expressions with 'static lifetime
  • Adds jmespath_expression parser to handle the new syntax with three aliases
  • JMESPath expressions are compiled at parse time using the jmespath crate (v0.4.0 with sync feature)
  • Mutually exclusive with jq: filter (last one specified wins, consistent with other single-value filters)
  • Adds FilterError::JMESPathCompile for compilation errors (holds JmespathError directly)

Error Handling

  • Parse errors show helpful messages with syntax guidance
  • Runtime evaluation errors display the error and fall back to showing the original log line
  • Serialization errors are caught and handled gracefully
  • Compilation errors are caught early during query parsing

Query Compatibility

  • Works alongside existing filters: log: / !log:, pod:, container:, label selectors, etc.
  • Only one JSON filter (jq or jmespath) can be active at a time

Testing

Added 22 comprehensive tests covering:

Parser tests (9 tests):

  • jmespath:, jmes:, and jm: syntax variants (5 test cases)
  • Integration with attribute parser (3 test cases)
  • Integration with parse_attributes (1 test case)

Filter tests (8 tests):

  • Valid JMESPath expression compilation
  • Complex expressions (array projections, object restructuring)
  • Invalid syntax handling
  • Combination with other filters (pod, log, container)
  • Multiple JMESPath filters (last one wins)
  • Display trait formatting
  • Mutual exclusivity with jq filter

LogCollector tests (6 tests):

  • Field extraction from JSON logs
  • Data restructuring with object literals
  • Non-JSON input handling (returns raw log)
  • Pretty print integration
  • Error handling (nonexistent fields return null)
  • Array projection queries

All 491 tests passing (469 existing + 22 new)

Documentation

  • README updated with JMESPath query syntax, aliases, and usage examples
  • Help dialog updated to include jmespath:<expr> with aliases in the query reference

Commits

  1. feat: add jmespath dependency for JSON log filtering
  2. feat: implement jmespath filter for JSON log processing
  3. docs: add jmespath filter to log query help dialog
  4. docs: add jmespath filter documentation to README

Related: #809
Follows: #841 (jq filter implementation)

@sarub0b0 sarub0b0 linked an issue Oct 18, 2025 that may be closed by this pull request
@sarub0b0 sarub0b0 marked this pull request as ready for review October 19, 2025 04:39
sarub0b0 and others added 4 commits October 19, 2025 13:40
Add jmespath crate (v0.4.0) with sync feature to support JMESPath
query syntax as an alternative to jq for filtering JSON logs.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
Add JMESPath filter support with three syntax aliases:
- jmespath:<expression>
- jmes:<expression> (short form)
- jm:<expression> (shortest form)

Implementation details:
- Add JMESPathProgram struct to hold compiled expressions
- Extend JsonFilter enum with JMESPath variant
- Add FilterError::JMESPathCompile for compilation errors
- Parse and compile JMESPath expressions at filter parse time
- Apply JMESPath filters in LogCollector with error handling
- Mutually exclusive with jq filter (last one wins)

Error handling:
- Parse errors show compilation failure messages
- Runtime errors display error and fall back to original log
- Non-JSON input returns raw log without filtering

Tests added:
- Parser tests for all three aliases (jmespath, jmes, jm)
- Filter compilation tests (valid, invalid, complex expressions)
- Filter combination tests (with other filters, jq/jmespath precedence)
- LogCollector tests (field extraction, restructuring, error cases)
- All 491 tests passing

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
Add jmespath:<expr> with aliases (jmes, jm) to the Queries section
of the log query help dialog, maintaining consistency with other
query syntax documentation.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
Add JMESPath filter to the Supported Queries table with:
- Syntax: jmespath:<expr>
- Aliases: jmes, jm
- Description and usage examples
- Example query: pod:api log:error jmespath:message

JMESPath provides a simpler, more declarative syntax for common
JSON queries compared to jq.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
@sarub0b0 sarub0b0 force-pushed the 809-log-query-jmespath-filter branch from e53322d to e466a69 Compare October 19, 2025 04:40
@sarub0b0 sarub0b0 merged commit fd16352 into main Oct 19, 2025
1 check passed
@sarub0b0 sarub0b0 deleted the 809-log-query-jmespath-filter branch October 19, 2025 04:46
sarub0b0 added a commit that referenced this pull request Oct 19, 2025
Add missing JMESPATH grammar definition to the Query Syntax section in README.md.
This completes the documentation for the jmespath filter feature added in PR #852.

Changes:
- Add JMESPATH to QUERY definition
- Add JMESPATH grammar rule with aliases (jmespath, jmes, jm)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
sarub0b0 added a commit that referenced this pull request Oct 19, 2025
Add missing JMESPATH grammar definition to the Query Syntax section in README.md.
This completes the documentation for the jmespath filter feature added in PR #852.

Changes:
- Add JMESPATH to QUERY definition
- Add JMESPATH grammar rule with aliases (jmespath, jmes, jm)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Enhance JSON logging

1 participant