Skip to content

Commit 0945fe6

Browse files
Universal Profilingagsecresamannoccicauemarcondeschristos68k
committed
Initial commit
Pre-OSS Optimyze/Elastic contributors: Co-Authored-By: Adam Gowdiak <[email protected]> Co-Authored-By: Adrien Mannocci <[email protected]> Co-Authored-By: Caue Marcondes <[email protected]> Co-Authored-By: Christos Kalkanis <[email protected]> Co-Authored-By: Damien Mathieu <[email protected]> Co-Authored-By: Daniel Mitterdorfer <[email protected]> Co-Authored-By: Florian Lehner <[email protected]> Co-Authored-By: Francesco Gualazzi <[email protected]> Co-Authored-By: Israel Ogbole <[email protected]> Co-Authored-By: Jan Calanog <[email protected]> Co-Authored-By: Joel Höner <[email protected]> Co-Authored-By: Joe Rowell <[email protected]> Co-Authored-By: Joseph Crail <[email protected]> Co-Authored-By: Joseph Kruskal <[email protected]> Co-Authored-By: M.J. Fieggen (Joni) <[email protected]> Co-Authored-By: Sean Heelan <[email protected]> Co-Authored-By: Stephanie Boomsma <[email protected]> Co-Authored-By: Thomas Dullien <[email protected]> Co-Authored-By: Timo Teräs <[email protected]> Co-Authored-By: Tim Rühsen <[email protected]> Co-Authored-By: Victor Martinez <[email protected]> Co-Authored-By: Victor Michel <[email protected]>
0 parents  commit 0945fe6

File tree

445 files changed

+67603
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

445 files changed

+67603
-0
lines changed

‎.github/workflows/unit-test-on-pull-request.yml

Lines changed: 120 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,120 @@
1+
name: otel-profiling-agent
2+
3+
on:
4+
push:
5+
branches: [main]
6+
pull_request:
7+
branches: ["**"]
8+
9+
jobs:
10+
lint:
11+
name: Lint
12+
runs-on: ubuntu-latest
13+
strategy:
14+
fail-fast: true
15+
max-parallel: 2
16+
matrix:
17+
go: ["stable"]
18+
steps:
19+
- name: Install dependencies
20+
run: sudo apt-get install -y llvm clang dwz cmake curl unzip
21+
- name: Install Zydis
22+
shell: bash
23+
run: |
24+
cd /tmp
25+
git clone --depth 1 --branch v3.1.0 --recursive https://github.com/zyantific/zydis.git
26+
cd zydis
27+
rm -rf build
28+
mkdir build
29+
cd build
30+
cmake -DZYDIS_BUILD_EXAMPLES=OFF ..
31+
make -j$(nproc)
32+
sudo make install
33+
cd zycore
34+
sudo make install
35+
- name: Set up Go ${{matrix.go}}
36+
uses: actions/setup-go@v5
37+
with:
38+
go-version: ${{matrix.go}}
39+
check-latest: true
40+
cache-dependency-path: |
41+
go.sum
42+
id: go
43+
- name: Install gRPC dependencies
44+
env:
45+
PB_URL: "https://github.com/protocolbuffers/protobuf/releases/download/v24.4/"
46+
PB_FILE: "protoc-24.4-linux-x86_64.zip"
47+
INSTALL_DIR: "/usr/local"
48+
run: |
49+
go install google.golang.org/protobuf/cmd/[email protected]
50+
go install google.golang.org/grpc/cmd/[email protected]
51+
curl -LO "$PB_URL/$PB_FILE"
52+
sudo unzip "$PB_FILE" -d "$INSTALL_DIR" 'bin/*' 'include/*'
53+
sudo chmod +xr "$INSTALL_DIR/bin/protoc"
54+
sudo find "$INSTALL_DIR/include" -type d -exec chmod +x {} \;
55+
sudo find "$INSTALL_DIR/include" -type f -exec chmod +r {} \;
56+
rm "$PB_FILE"
57+
- name: Check out
58+
uses: actions/checkout@v4
59+
- name: Linter
60+
run: |
61+
go version
62+
curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(go env GOPATH)/bin v1.54.2
63+
make lint
64+
65+
test:
66+
name: Test
67+
runs-on: ubuntu-latest
68+
strategy:
69+
fail-fast: true
70+
max-parallel: 2
71+
matrix:
72+
go: ["stable"]
73+
steps:
74+
- name: Install dependencies
75+
run: sudo apt-get install -y llvm clang dwz cmake curl unzip
76+
- name: Install Zydis
77+
shell: bash
78+
run: |
79+
cd /tmp
80+
git clone --depth 1 --branch v3.1.0 --recursive https://github.com/zyantific/zydis.git
81+
cd zydis
82+
rm -rf build
83+
mkdir build
84+
cd build
85+
cmake -DZYDIS_BUILD_EXAMPLES=OFF ..
86+
make -j$(nproc)
87+
sudo make install
88+
cd zycore
89+
sudo make install
90+
- name: Set up Go ${{matrix.go}}
91+
uses: actions/setup-go@v5
92+
with:
93+
go-version: ${{matrix.go}}
94+
check-latest: true
95+
cache-dependency-path: |
96+
go.sum
97+
id: go
98+
- name: Install gRPC dependencies
99+
env:
100+
PB_URL: "https://github.com/protocolbuffers/protobuf/releases/download/v24.4/"
101+
PB_FILE: "protoc-24.4-linux-x86_64.zip"
102+
INSTALL_DIR: "/usr/local"
103+
run: |
104+
go install google.golang.org/protobuf/cmd/[email protected]
105+
go install google.golang.org/grpc/cmd/[email protected]
106+
curl -LO "$PB_URL/$PB_FILE"
107+
sudo unzip "$PB_FILE" -d "$INSTALL_DIR" 'bin/*' 'include/*'
108+
sudo chmod +xr "$INSTALL_DIR/bin/protoc"
109+
sudo find "$INSTALL_DIR/include" -type d -exec chmod +x {} \;
110+
sudo find "$INSTALL_DIR/include" -type f -exec chmod +r {} \;
111+
rm "$PB_FILE"
112+
- name: Check out
113+
uses: actions/checkout@v4
114+
- name: Build
115+
run: |
116+
echo $PATH
117+
make test
118+
- name: Tests
119+
run: |
120+
make test

‎.gitignore

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
*.o
2+
*.pb.go
3+
.idea
4+
otel-profiling-agent
5+
tracer.ebpf
6+
tracer.ebpf.arm64
7+
tracer.ebpf.x86

‎.golangci.yml

Lines changed: 150 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,150 @@
1+
service:
2+
golangci-lint-version: 1.54.x
3+
4+
run:
5+
skip-dirs:
6+
- artifacts
7+
- build-targets
8+
- design
9+
- docker-images
10+
- docs
11+
- etc
12+
- experiments
13+
- infrastructure
14+
- legal
15+
- libpf-rs
16+
- mocks
17+
- pf-code-indexing-service/cibackend/gomock_*
18+
- pf-debug-metadata-service/dmsbackend/gomock_*
19+
- pf-host-agent/support/ci-kernels
20+
- pf-storage-backend/storagebackend/gomock_*
21+
- scratch
22+
- systemtests/benchmarks/_outdata
23+
- target
24+
- virt-tests
25+
- vm-images
26+
27+
linters:
28+
enable-all: true
29+
disable:
30+
# Disabled because of
31+
# - too many non-sensical warnings
32+
# - not relevant for us
33+
# - false positives
34+
#
35+
# "might be worth fixing" means we should investigate/fix in the mid term
36+
- containedctx # might be worth fixing
37+
- contextcheck # might be worth fixing
38+
- cyclop
39+
- depguard
40+
- dupword
41+
- durationcheck # might be worth fixing
42+
- errname # might be worth fixing
43+
- errorlint # might be worth fixing
44+
- exhaustive
45+
- exhaustivestruct
46+
- exhaustruct
47+
- forcetypeassert # might be worth fixing
48+
- funlen
49+
- gci # might be worth fixing
50+
- gochecknoglobals
51+
- gochecknoinits
52+
- gocognit
53+
- gocyclo
54+
- godot
55+
- godox # complains about TODO etc
56+
- goerr113
57+
- gofumpt
58+
- goimports # might be worth fixing
59+
- golint # might be worth fixing
60+
- gomnd
61+
- gomoddirectives
62+
- ifshort
63+
- interfacebloat
64+
- ireturn
65+
- maintidx
66+
- makezero
67+
- nestif
68+
- nilerr # might be worth fixing
69+
- nilnil
70+
- nlreturn
71+
- noctx # might be worth fixing
72+
- nolintlint
73+
- nonamedreturns
74+
- nosnakecase
75+
- paralleltest
76+
- scopelint # might be worth fixing
77+
- sqlclosecheck # might be worth fixing
78+
- tagalign
79+
- tagliatelle
80+
- testableexamples # might be worth fixing
81+
- testpackage
82+
- tparallel # might be worth fixing
83+
- thelper
84+
- varnamelen
85+
- wastedassign
86+
- wsl
87+
- wrapcheck
88+
- forbidigo
89+
# the following linters are deprecated
90+
- exhaustivestruct
91+
- scopelint
92+
- nosnakecase
93+
- interfacer
94+
- maligned
95+
- ifshort
96+
- structcheck # might be worth fixing
97+
- deadcode
98+
- golint
99+
- varcheck
100+
101+
linters-settings:
102+
goconst:
103+
min-len: 2
104+
min-occurrences: 2
105+
gocritic:
106+
enabled-tags:
107+
- diagnostic
108+
- experimental
109+
- opinionated
110+
- performance
111+
- style
112+
disabled-checks:
113+
- dupImport # https://github.com/go-critic/go-critic/issues/845
114+
- ifElseChain
115+
- octalLiteral
116+
- whyNoLint
117+
- wrapperFunc
118+
- sloppyTestFuncName
119+
- sloppyReassign
120+
- uncheckedInlineErr # Experimental rule with high false positive rate.
121+
122+
# Broken with Go 1.18 feature (https://github.com/golangci/golangci-lint/issues/2649):
123+
- hugeParam
124+
- rangeValCopy
125+
- typeDefFirst
126+
- paramTypeCombine
127+
gocyclo:
128+
min-complexity: 15
129+
govet:
130+
enable-all: true
131+
check-shadowing: true
132+
disable:
133+
- fieldalignment
134+
settings:
135+
printf: # analyzer name, run `go tool vet help` to see all analyzers
136+
funcs: # run `go tool vet help printf` to see available settings for `printf` analyzer
137+
- debug,debugf,debugln
138+
- error,errorf,errorln
139+
- fatal,fatalf,fataln
140+
- info,infof,infoln
141+
- log,logf,logln
142+
- warn,warnf,warnln
143+
- print,printf,println,sprint,sprintf,sprintln,fprint,fprintf,fprintln
144+
lll:
145+
line-length: 100
146+
tab-width: 4
147+
maligned:
148+
suggest-new: true
149+
misspell:
150+
locale: US

‎AUTHORS

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
## Open-Source contributors
2+
3+
For contributors statistics that occurred after the profiling agent was open sourced, please refer to [the GitHub statistics](https://github.com/elastic/otel-profiling-agent/pulse).
4+
5+
## Pre-OSS Optimyze/Elastic contributors
6+
7+
Adam Gowdiak <[email protected]>
8+
Adrien Mannocci <[email protected]>
9+
Caue Marcondes <[email protected]>
10+
Christos Kalkanis <[email protected]>
11+
Damien Mathieu <[email protected]>
12+
Daniel Mitterdorfer <[email protected]>
13+
Florian Lehner <[email protected]>
14+
Francesco Gualazzi <[email protected]>
15+
Israel Ogbole <[email protected]>
16+
Jan Calanog <[email protected]>
17+
Joel Höner <[email protected]>
18+
Joe Rowell <[email protected]>
19+
Joseph Crail <[email protected]>
20+
Joseph Kruskal <[email protected]>
21+
M.J. Fieggen (Joni) <[email protected]>
22+
Sean Heelan <[email protected]>
23+
Stephanie Boomsma <[email protected]>
24+
Thomas Dullien <[email protected]>
25+
Timo Teräs <[email protected]>
26+
Tim Rühsen <[email protected]>
27+
Victor Martinez <[email protected]>
28+
Victor Michel <[email protected]>

‎Dockerfile

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
FROM debian:testing
2+
3+
WORKDIR /agent
4+
5+
ARG arch=amd64
6+
7+
RUN apt-get update -y && apt-get dist-upgrade -y && apt-get install -y \
8+
curl wget cmake dwz lsb-release software-properties-common gnupg git clang llvm \
9+
golang linux-headers-$arch unzip
10+
11+
RUN git clone --depth 1 --branch v3.1.0 --recursive https://github.com/zyantific/zydis.git && \
12+
cd zydis && mkdir build && cd build && \
13+
cmake -DZYDIS_BUILD_EXAMPLES=OFF .. && make -j$(nproc) && make install && \
14+
cd zycore && make install && \
15+
cd ../../.. && rm -rf zydis
16+
17+
RUN wget -qO- https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(go env GOPATH)/bin v1.54.2
18+
19+
20+
# gRPC dependencies
21+
RUN go install google.golang.org/protobuf/cmd/[email protected]
22+
RUN go install google.golang.org/grpc/cmd/[email protected]
23+
24+
RUN \
25+
PB_URL="https://github.com/protocolbuffers/protobuf/releases/download/v24.4/"; \
26+
PB_FILE="protoc-24.4-linux-x86_64.zip"; \
27+
INSTALL_DIR="/usr/local"; \
28+
\
29+
curl -LO "$PB_URL/$PB_FILE" \
30+
&& unzip "$PB_FILE" -d "$INSTALL_DIR" 'bin/*' 'include/*' \
31+
&& chmod +xr "$INSTALL_DIR/bin/protoc" \
32+
&& find "$INSTALL_DIR/include" -type d -exec chmod +x {} \; \
33+
&& find "$INSTALL_DIR/include" -type f -exec chmod +r {} \; \
34+
&& rm "$PB_FILE"
35+
36+
RUN echo "export PATH=\"\$PATH:\$(go env GOPATH)/bin\"\nexport KERNEL_HEADERS=\"/lib/modules/$(ls /lib/modules)\"" >> ~/.bashrc
37+
38+
ENTRYPOINT ["/bin/bash", "-l", "-c"]

‎KNOWN_KERNEL_LIMITATIONS.md

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
Known limitations
2+
=================
3+
The Linux kernel is constantly evolving and so is eBPF. To be able to load our eBPF code with older kernel versions we have to write code to avoid some limitations. This file documents the restrictions we ran into while writing the code.
4+
5+
Number of tracepoints
6+
---------------------
7+
Affects kernel < 4.15.
8+
9+
There was a limit of 1 eBPF program per tracepoint/kprobe.
10+
This limit no longer holds and was removed with commit [e87c6bc3852b981e71c757be20771546ce9f76f3](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=e87c6bc3852b981e71c757be20771546ce9f76f3).
11+
12+
13+
Obtaining Kernel backtrace
14+
--------------------------
15+
Affects kernel < 4.18
16+
17+
It is not possible to get individual backtraces from kernel mode stack with bpf_get_stackid(). It returns hash of the backtrace, and if it collides with another backtrace before the agent has collected it, we might report wrong kernel backtracec.
18+
A more suitable helper bpf_get_stack() was added in commit [c195651e565ae7f41a68acb7d4aa7390ad215de1](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=c195651e565ae7f41a68acb7d4aa7390ad215de1).
19+
20+
21+
Kernel version check
22+
--------------------
23+
Affects kernel < 5.0.
24+
25+
As part of the verification of eBPF programs, the `kern_version` attribute was checked and it needed to match with the currently running kernel version.
26+
This check was removed with commit [6c4fc209fcf9d27efbaa48368773e4d2bfbd59aa](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=6c4fc209fcf9d27efbaa48368773e4d2bfbd59aa).
27+
28+
29+
eBPF instruction limit
30+
----------------------
31+
Affects kernel < 5.2.
32+
33+
The number of eBPF instructions per program was limited to 4096 instructions.
34+
This limit was raised to 1 million eBPF instructions with commit [c04c0d2b968ac45d6ef020316808ef6c82325a82](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=c04c0d2b968ac45d6ef020316808ef6c82325a82).
35+
36+
37+
eBPF inner arrays (map-in-map) must be of same size
38+
---------------------------------------------------
39+
Affects kernel < 5.10.
40+
41+
This restriction was removed with commit[4a8f87e60f6db40e640f1db555d063b2c4dea5f1](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=4a8f87e60f6db40e640f1db555d063b2c4dea5f1).

0 commit comments

Comments
 (0)