Commit e7ea34a
authored
feat(pyroscope): Replace Parca gRPC debuginfo upload with Pyroscope Connect API (#5891)
Replace the old Parca gRPC-based debuginfo/symbol upload with the new
Pyroscope Connect bidirectional streaming API. This simplifies the
upload protocol from a 4-step gRPC flow (ShouldInitiateUpload →
InitiateUpload → Upload chunks → MarkUploadFinished) to a single bidi
stream.
**Protocol change**
- New `PyroscopeSymbolUploader` replaces `ParcaSymbolUploader` using a
single Connect bidi stream for the full upload lifecycle
- Removed signed-URL upload strategy and gRPC upload client
(`grpc_upload_client.go`)
- Skips vdso uploads (no backing file, always fails)
- Empty buildID is passed as empty `GnuBuildId` (fileID goes into
`OtelFileId` only)
**Interface change: `Client()` → `DebugInfoClients()`**
- `Appender.Client()` returned a single gRPC client (first match).
`Appender.DebugInfoClients()` returns ALL Connect clients, enabling
proper fan-out in the proxy
- Updated across all implementations (Fanout, AppenderMock, enrich,
relabel, write, java/loop_test)
**receive_http proxy rewrite**
- Old: thin gRPC proxy forwarding to one upstream + separate RPC
endpoints for each step
- New: Connect bidi stream handler that fans out to all downstream
clients concurrently; only accepting endpoints receive chunks
- Removed `grpc.go` (gRPC server setup) — uses
`debuginfov1alpha1connect.RegisterDebuginfoServiceHandler` on the HTTP
router
**debuginfo.Client simplified**
- Removed `grpc.ClientConn` management and `debuginfo_client.go` (gRPC
dial + TLS + basic-auth)
- Takes a pre-built Connect client directly; reuses the h2c HTTP client
from `write.go`
**New config in `pyroscope.ebpf`**
- `debug_info` block with `upload_enabled` (default false),
`cache_size`, `queue_size`, `worker_num`
- `symb_cache_enabled` attr (default true) — allows disabling local
symbolization for testing server-side symbolization
**Dependencies**
- Removed all `buf.build/gen/go/parca-dev/parca` deps
- Bumped `github.com/grafana/pyroscope/api` to v1.3.2
- OTel SDK pinned to v1.39.0 (avoids schema URL conflict)
**Tests**
- 7 uploader integration tests (success, decline, in-progress, empty
buildID, multi-chunk, dedup, worker queue)
- 5 proxy fan-out tests (single accept, multi accept, multi decline,
mixed, no endpoints)1 parent 8ca028f commit e7ea34a
28 files changed
Lines changed: 1449 additions & 715 deletions
File tree
- collector
- extension/alloyengine
- internal/component/pyroscope
- ebpf
- reporter/parca/reporter
- enrich
- java
- receive_http
- relabel
- write
- debuginfo
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
99 | 99 | | |
100 | 100 | | |
101 | 101 | | |
102 | | - | |
103 | | - | |
104 | 102 | | |
105 | 103 | | |
106 | 104 | | |
| |||
539 | 537 | | |
540 | 538 | | |
541 | 539 | | |
542 | | - | |
| 540 | + | |
543 | 541 | | |
544 | 542 | | |
545 | 543 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | | - | |
2 | | - | |
3 | | - | |
4 | | - | |
5 | 1 | | |
6 | 2 | | |
7 | 3 | | |
| |||
1235 | 1231 | | |
1236 | 1232 | | |
1237 | 1233 | | |
1238 | | - | |
1239 | | - | |
| 1234 | + | |
| 1235 | + | |
1240 | 1236 | | |
1241 | 1237 | | |
1242 | 1238 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
20 | | - | |
21 | | - | |
22 | 20 | | |
23 | 21 | | |
24 | 22 | | |
| |||
456 | 454 | | |
457 | 455 | | |
458 | 456 | | |
459 | | - | |
| 457 | + | |
460 | 458 | | |
461 | 459 | | |
462 | 460 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | | - | |
2 | | - | |
3 | | - | |
4 | | - | |
5 | 1 | | |
6 | 2 | | |
7 | 3 | | |
| |||
1253 | 1249 | | |
1254 | 1250 | | |
1255 | 1251 | | |
1256 | | - | |
1257 | | - | |
| 1252 | + | |
| 1253 | + | |
1258 | 1254 | | |
1259 | 1255 | | |
1260 | 1256 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
78 | 78 | | |
79 | 79 | | |
80 | 80 | | |
81 | | - | |
| 81 | + | |
82 | 82 | | |
83 | 83 | | |
84 | 84 | | |
| |||
1001 | 1001 | | |
1002 | 1002 | | |
1003 | 1003 | | |
1004 | | - | |
1005 | | - | |
1006 | 1004 | | |
1007 | 1005 | | |
1008 | 1006 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | | - | |
2 | | - | |
3 | | - | |
4 | | - | |
5 | 1 | | |
6 | 2 | | |
7 | 3 | | |
| |||
1263 | 1259 | | |
1264 | 1260 | | |
1265 | 1261 | | |
1266 | | - | |
1267 | | - | |
| 1262 | + | |
| 1263 | + | |
1268 | 1264 | | |
1269 | 1265 | | |
1270 | 1266 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
9 | | - | |
10 | 9 | | |
| 10 | + | |
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
| |||
61 | 61 | | |
62 | 62 | | |
63 | 63 | | |
64 | | - | |
| 64 | + | |
65 | 65 | | |
66 | 66 | | |
| 67 | + | |
67 | 68 | | |
68 | | - | |
69 | | - | |
70 | | - | |
| 69 | + | |
71 | 70 | | |
72 | | - | |
| 71 | + | |
73 | 72 | | |
74 | 73 | | |
75 | 74 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
3 | 3 | | |
4 | 4 | | |
5 | 5 | | |
6 | | - | |
7 | 6 | | |
| 7 | + | |
8 | 8 | | |
9 | 9 | | |
10 | 10 | | |
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
14 | | - | |
15 | | - | |
16 | | - | |
17 | | - | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
18 | 18 | | |
19 | 19 | | |
20 | 20 | | |
| |||
29 | 29 | | |
30 | 30 | | |
31 | 31 | | |
32 | | - | |
33 | | - | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
34 | 37 | | |
35 | 38 | | |
36 | 39 | | |
37 | | - | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
38 | 43 | | |
39 | 44 | | |
40 | 45 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
| 9 | + | |
9 | 10 | | |
10 | 11 | | |
11 | 12 | | |
| |||
37 | 38 | | |
38 | 39 | | |
39 | 40 | | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
40 | 44 | | |
41 | 45 | | |
42 | 46 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
96 | 96 | | |
97 | 97 | | |
98 | 98 | | |
99 | | - | |
100 | | - | |
101 | | - | |
102 | | - | |
103 | | - | |
104 | | - | |
105 | | - | |
106 | | - | |
107 | | - | |
108 | | - | |
109 | | - | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
110 | 113 | | |
111 | 114 | | |
112 | 115 | | |
| |||
311 | 314 | | |
312 | 315 | | |
313 | 316 | | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
314 | 320 | | |
315 | 321 | | |
316 | | - | |
317 | 322 | | |
318 | 323 | | |
319 | 324 | | |
| |||
336 | 341 | | |
337 | 342 | | |
338 | 343 | | |
| 344 | + | |
339 | 345 | | |
340 | 346 | | |
341 | 347 | | |
| |||
366 | 372 | | |
367 | 373 | | |
368 | 374 | | |
| 375 | + | |
| 376 | + | |
| 377 | + | |
| 378 | + | |
| 379 | + | |
| 380 | + | |
| 381 | + | |
369 | 382 | | |
370 | 383 | | |
371 | 384 | | |
| |||
0 commit comments