Skip to content

partitioningprocessor: implement logs partitioning#1172

Open
axw wants to merge 1 commit intoelastic:mainfrom
axw:partitioningprocessor-logspartitioner
Open

partitioningprocessor: implement logs partitioning#1172
axw wants to merge 1 commit intoelastic:mainfrom
axw:partitioningprocessor-logspartitioner

Conversation

@axw
Copy link
Copy Markdown
Member

@axw axw commented Apr 20, 2026

This PR implements the OTTL-based partitioning logic for logs. It is not wired up to the processor yet.

Relates to open-telemetry/opentelemetry-collector-contrib#39199

@axw axw marked this pull request as ready for review April 20, 2026 12:52
@axw axw requested a review from a team as a code owner April 20, 2026 12:52
This implements the internal partitioning logic for
logs. It is not wired up to the processor yet.
@axw axw force-pushed the partitioningprocessor-logspartitioner branch from ac20366 to 3b3e009 Compare April 20, 2026 12:53
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Apr 20, 2026

📝 Walkthrough

Walkthrough

The PR implements log partitioning by OTTL expressions: Config was changed to include a TODO (and Validate() removed), and a new PartitionKeyConfig type was added. A parser (parser.go) builds OTTL parsers for resource/scope/log contexts and constructs a logs partitioner. A partitioner implementation (partitioner.go) groups plog.Logs by evaluated string keys. Extensive tests were added for parser, partitioning behavior, key formatting/order, deduplication, schema preservation, and processor capabilities. go.mod was updated to add an OTTL dependency and related indirect modules.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • 🛠️ Update Documentation: Commit on current branch
  • 🛠️ Update Documentation: Create PR

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 golangci-lint (2.11.4)

level=error msg="[linters_context] typechecking error: pattern ./...: directory prefix . does not contain main module or its selected dependencies"


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@axw
Copy link
Copy Markdown
Member Author

axw commented Apr 21, 2026

@CodeRabbit review

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Apr 21, 2026

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@processor/partitioningprocessor/partitioner.go`:
- Around line 120-123: The evaluateScope method in scopeLogsPartitioner uses
ottlscope.NewTransformContextPtr(sl.Scope(), rl.Resource(), sl), which passes
the scope as the single schema URL item so GetScopeSchemaURLItem() and
GetResourceSchemaURLItem() return the same schemaURLItem causing
resource.schema_url to read the scope schema URL; update evaluateScope (or the
context creation) to provide both resource and scope schema URL items separately
(or detect and reject expressions that reference both schema URLs) so
resource.schema_url resolves to the resource's schema URL, and add a unit test
exercising differing scope vs resource schema URLs to ensure
evaluateStringExpressions (called from evaluateScope) behaves correctly;
reference functions/types: evaluateScope, scopeLogsPartitioner,
ottlscope.NewTransformContextPtr, evaluateStringExpressions, and the
resource.schema_url / scope.schema_url getters.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: fa45cd84-8909-41eb-bba4-6d35fa22365d

📥 Commits

Reviewing files that changed from the base of the PR and between ac20366 and 3b3e009.

⛔ Files ignored due to path filters (1)
  • processor/partitioningprocessor/go.sum is excluded by !**/*.sum
📒 Files selected for processing (7)
  • processor/partitioningprocessor/config.go
  • processor/partitioningprocessor/go.mod
  • processor/partitioningprocessor/parser.go
  • processor/partitioningprocessor/parser_test.go
  • processor/partitioningprocessor/partitioner.go
  • processor/partitioningprocessor/partitioner_test.go
  • processor/partitioningprocessor/processor_test.go
✅ Files skipped from review due to trivial changes (2)
  • processor/partitioningprocessor/processor_test.go
  • processor/partitioningprocessor/parser.go
🚧 Files skipped from review as they are similar to previous changes (1)
  • processor/partitioningprocessor/go.mod

Comment thread processor/partitioningprocessor/partitioner.go
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant