Skip to content
Draft
Show file tree
Hide file tree
Changes from 55 commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
e5f5b34
fix
lalitb Apr 12, 2026
c224520
fix
lalitb Apr 14, 2026
797f51c
Make the userevents receiver viable for VM demos
lalitb Apr 14, 2026
255ca85
Move the Linux userevents receiver into contrib
lalitb Apr 14, 2026
ec0ecc6
Add multi-core orchestration to the userevents VM demo
lalitb Apr 14, 2026
00ff4a5
Keep userevents timestamps and tracefs failures usable on Linux VMs
lalitb Apr 14, 2026
67c3bd4
Share one_collect as the userevents collection backend
lalitb Apr 15, 2026
3f69d41
Preserve record time without unsafe code in the one_collect adapter
lalitb Apr 15, 2026
1f1a8e8
Co-locate userevents receiver tests with their owning crate
lalitb Apr 15, 2026
e3be486
Drop stale userevents dependencies from unrelated crates
lalitb Apr 15, 2026
f05d39a
Remove personal userevents demo scripts from the repo
lalitb Apr 15, 2026
f7fed98
Improve userevents receiver decode path and gating
lalitb Apr 17, 2026
ecdaad3
Refactor user-events collection modules
lalitb Apr 17, 2026
473aa7b
Gate tracepoint_decode behind userevents feature
lalitb Apr 17, 2026
bd74167
Clean up userevents receiver docs and deps
lalitb Apr 17, 2026
f9280d5
Drain perf ring during receiver ingress drain
lalitb Apr 17, 2026
39eb3bc
Fix rustfmt ordering in userevents modules
lalitb Apr 17, 2026
206e077
Fix userevents receiver warnings and test gating
lalitb Apr 17, 2026
0653e42
Allow missing docs on userevents e2e test
lalitb Apr 17, 2026
9f87603
Clarify userevents batching loss note
lalitb Apr 17, 2026
4d6dbe9
Revert collection common module split
lalitb Apr 17, 2026
dd0ff8e
Bound one-collect adapter drain budget
lalitb Apr 17, 2026
394ba67
Restrict userevents receiver to supported decode mode
lalitb Apr 17, 2026
5f52c8c
Move one_collect adapter under userevents receiver
lalitb Apr 17, 2026
7868da1
Clean up userevents receiver output schema
lalitb Apr 19, 2026
696aed3
Refine userevents receiver hot path and docs
lalitb Apr 19, 2026
7130695
Clarify userevents encoder attribute comment
lalitb Apr 19, 2026
a3458fc
Simplify userevents CPU coverage docs
lalitb Apr 19, 2026
9c8ec3f
Clarify userevents platform and locality docs
lalitb Apr 19, 2026
28edfb7
Fix userevents review follow-ups
lalitb Apr 19, 2026
e84473a
Remove unused one-collect submodule wiring
lalitb Apr 20, 2026
c7f7903
Clamp userevents numeric conversions
lalitb Apr 20, 2026
e39f15e
Restore contrib receivers feature
lalitb Apr 20, 2026
c333782
Restore optional contrib dependencies
lalitb Apr 20, 2026
b7d2951
fmt
lalitb Apr 20, 2026
d534cfb
Use microsoft URN for userevents receiver
lalitb Apr 20, 2026
46fa52b
Remove stale userevents decoder helpers
lalitb Apr 20, 2026
3e9abe6
Remove unused decoded attr helper
lalitb Apr 20, 2026
bfb95d9
Make userevents receiver URN vendor neutral
lalitb Apr 21, 2026
627bdf9
Merge branch 'main' into user-events-receiver
lalitb Apr 21, 2026
42cb598
Merge branch 'main' into user-events-receiver
lalitb Apr 21, 2026
ac916af
fix
lalitb Apr 22, 2026
ac20b3e
Merge branch 'main' into user-events-receiver
lalitb Apr 22, 2026
8ef54de
Merge branch 'main' into user-events-receiver
lalitb Apr 23, 2026
e8c2c98
Anchor user events timestamp conversion
lalitb Apr 23, 2026
079efca
Anchor user events timestamp conversion
lalitb Apr 23, 2026
15e42c2
Document user events receiver follow-ups
lalitb Apr 23, 2026
483e660
Document user events receiver follow-ups
lalitb Apr 23, 2026
a87d54e
Merge branch 'user-events-receiver' of github.com:lalitb/otel-arrow i…
lalitb Apr 23, 2026
22e4e64
Update ingestion backend comment wording
lalitb Apr 23, 2026
cd43173
Optimize userevents PartC decode staging
lalitb Apr 23, 2026
c1fff64
Clarify userevents decode staging TODO
lalitb Apr 23, 2026
920c83c
fmt
lalitb Apr 24, 2026
a419d32
Merge branch 'main' into user-events-receiver
lalitb Apr 24, 2026
2a045b2
Document kernel/distro requirements for userevents receiver
lalitb Apr 24, 2026
99ed372
Merge branch 'main' into user-events-receiver
lalitb Apr 24, 2026
2018652
review comments
lalitb Apr 25, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions rust/otap-dataflow/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -279,6 +279,9 @@ crypto-openssl = ["otap-df-otap/crypto-openssl"]
contrib-exporters = ["otap-df-contrib-nodes/contrib-exporters"]
geneva-exporter = ["otap-df-contrib-nodes/geneva-exporter"]
azure-monitor-exporter = ["otap-df-contrib-nodes/azure-monitor-exporter"]
# Contrib receivers (opt-in) - now in contrib-nodes
contrib-receivers = ["otap-df-contrib-nodes/contrib-receivers"]
userevents-receiver = ["otap-df-contrib-nodes/userevents-receiver"]
# Contrib processors (opt-in) - now in contrib-nodes
contrib-processors = ["otap-df-contrib-nodes/contrib-processors"]
condense-attributes-processor = ["otap-df-contrib-nodes/condense-attributes-processor"]
Expand Down
32 changes: 27 additions & 5 deletions rust/otap-dataflow/crates/contrib-nodes/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -14,30 +14,33 @@ workspace = true

[dependencies]
# Workspace crates (likely used by most/all features)
otap-df-channel = { workspace = true }
otap-df-config = { workspace = true }
otap-df-engine = { workspace = true }
otap-df-engine-macros = { workspace = true }
otap-df-otap = { workspace = true }
otap-df-pdata = { workspace = true }
otap-df-pdata-views = { workspace = true }
otap-df-telemetry = { workspace = true }
otap-df-channel = { workspace = true }
otap-df-telemetry-macros = { workspace = true }

arrow.workspace = true
async-trait.workspace = true
futures.workspace = true
tokio.workspace = true
tracing.workspace = true
bytes.workspace = true
prost.workspace = true
serde.workspace = true
serde_json.workspace = true
thiserror.workspace = true
linkme.workspace = true
tokio.workspace = true
tracing.workspace = true
base64 = { workspace = true, optional = true }
chrono = { workspace = true, optional = true }
geneva-uploader = { workspace = true, optional = true }
humantime-serde = { workspace = true, optional = true }
tracepoint_decode = { version = "0.5.0", optional = true }
itoa = { workspace = true, optional = true }
geneva-uploader = { workspace = true, optional = true }
opentelemetry-proto = { workspace = true, optional = true }
data_engine_recordset = { workspace = true, optional = true }
data_engine_recordset_otlp_bridge = { workspace = true, optional = true }
Expand All @@ -46,14 +49,28 @@ azure_core = { workspace = true, optional = true, features = ["reqwest"] }
azure_identity = { workspace = true, optional = true }
flate2 = { workspace = true, optional = true }
http = { workspace = true, optional = true }
itoa = { workspace = true, optional = true }
rand = { workspace = true, optional = true }
ryu = { workspace = true, optional = true }
reqwest = { workspace = true, optional = true, features = ["rustls-no-provider"] }
sysinfo = { workspace = true, optional = true }
urlencoding = { workspace = true, optional = true }

[target.'cfg(target_os = "linux")'.dependencies]
# TODO: Remove this pinned git dependency once one-collect is upstream in the
# normal dependency graph and no longer needs to be pulled directly here.
one_collect = { git = "https://github.com/microsoft/one-collect.git", rev = "2d33e674348168f4a19bd3ab40d92f1c556407ec", features = ["scripting"], optional = true }
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: put the date of the commit here in comment, so it is to tell which date's version we are consuming here.


[features]
contrib-receivers = [
"userevents-receiver",
]
userevents-receiver = [
"dep:base64",
"dep:chrono",
"dep:humantime-serde",
"dep:one_collect",
"dep:tracepoint_decode",
]
contrib-exporters = [
"geneva-exporter",
"azure-monitor-exporter",
Expand Down Expand Up @@ -97,11 +114,16 @@ axum.workspace = true
clap.workspace = true
criterion = { workspace = true, features = ["html_reports"] }
humantime.workspace = true
core_affinity.workspace = true
eventheader_dynamic = "0.5.0"
otap-df-channel = { workspace = true }
otap-df-engine = { workspace = true, features = ["test-utils"] }
otap-df-otap = { workspace = true, features = ["test-utils"] }
otap-df-pdata = { workspace = true, features = ["testing"] }
pretty_assertions.workspace = true
rand.workspace = true
serde_yaml.workspace = true
tracing-subscriber = { workspace = true, features = ["fmt", "std", "env-filter"] }
wiremock.workspace = true

[[bench]]
Expand Down
18 changes: 17 additions & 1 deletion rust/otap-dataflow/crates/contrib-nodes/README.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,15 @@
<!-- markdownlint-disable MD013 -->

# Contrib Nodes

This crate contains optional (feature-gated) contrib processors and exporters.
This crate contains contrib receivers, processors, and exporters.

## Folder Layout

- `src/exporters/`
- Contrib exporters
- `src/receivers/`
- Contrib receivers
- `src/processors/`
- Contrib processors

Expand All @@ -14,6 +18,18 @@ This crate contains optional (feature-gated) contrib processors and exporters.
Feature flags are grouped into aggregate categories and individual node flags.
Aggregate flags enable all nodes in their category.

### Receivers

| Node | URN | Module |
| ---- | --- | ------ |
| userevents_receiver | `urn:otel:receiver:userevents` | `src/receivers/userevents_receiver/` |

#### userevents_receiver

- Reads Linux `user_events` tracepoints through per-CPU perf sessions
- Supports single-tracepoint and multi-tracepoint configuration
- Supports the `common_schema_otel_logs` decode format for supported log payloads
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- Supports the `common_schema_otel_logs` decode format for supported log payloads
- Supports the `microsoft_common_schema_logs` decode format for supported log payloads
  1. I think it's best to quality CS with Microsoft, as it is hard for users to known what is "common" about it.
    2.CommonLogs is just CS Logs - its inspired by OTel Logs data model, but not tied to it, so I suggest avoiding the term OTel Logs, Just "Logs" is good.


### Exporters

- `contrib-exporters` (enables all contrib exporters)
Expand Down
3 changes: 3 additions & 0 deletions rust/otap-dataflow/crates/contrib-nodes/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,5 +6,8 @@
/// Exporter implementations for contrib nodes.
pub mod exporters;

/// Receiver implementations for contrib nodes.
pub mod receivers;

/// Processor implementations for contrib nodes.
pub mod processors;
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

/// Linux userevents receiver.
#[cfg(feature = "userevents-receiver")]
pub mod userevents_receiver;
Loading
Loading