Unable to Perform eBPF Profiling in Showcase Demo with Latest OAP, UI, and Rover Image #13161

KHADEPALLAVI975 · 2025-04-08T11:21:17Z

KHADEPALLAVI975
Apr 8, 2025

I'm deploying the Apache SkyWalking Showcase demo using the following setup command:

export FEATURE_FLAGS=mesh-with-agent,single-node,elasticsearch,rover,events,als,cilium

This brings up the environment with mesh observability, Cilium, ALS, and Rover enabled. The deployment is successful and I’m seeing normal service mesh metrics like:

Load , Success rate , Latency

All Kubernetes pods are running, and the OAP, UI, and Rover components are using the latest images.

❌ What's not working:

No processes are listed under the eBPF profiling section.
Flame graphs do not appear in the UI.

My Setup:

OAP TAG : e820c75
UI TAG : e820c75
Rover TAG : 67622c352b98fd32782a3e7afc7d3fbd6d6ec8e3
Kubernetes environment: KIND
Kernel version: 5.15.167-0515167-generic
Rover deployment: DaemonSet

Questions:

Are there extra configuration steps needed for eBPF profiling to work in the showcase?
Has anyone managed to get flame graphs showing up with the latest Rover + OAP combo?
Could this be a known issue or compatibility mismatch between components?

Happy to provide logs or config snippets if needed — any help or pointers would be much appreciated!

mrproliu · 2025-04-08T12:19:31Z

mrproliu
Apr 8, 2025
Collaborator

Are there extra configuration steps needed for eBPF profiling to work in the showcase?

Are you using the Cilium services in your service? If not, please remove it. I'm not sure the Envoy will be installed when using Cilium.

Has anyone managed to get flame graphs showing up with the latest Rover + OAP combo?

It has already be used in the skywalking demo: https://demo.skywalking.apache.org/dashboard/MESH/Service/bWVzaC1zdnI6OnNvbmdzLnNhbXBsZS1zZXJ2aWNlcw==.1/Mesh-Service

16 replies

KHADEPALLAVI975 Apr 11, 2025
Author

time="2025-04-11T06:48:43Z" level=info msg="ready to starting profiling task: 36dcce484408c9b9a3513f94cd80a6b3b875c583503a3fcfcf653531157fb212" module=profiling.task
time="2025-04-11T06:48:43Z" level=info msg="ready to starting profiling task: 04b064f92110b2cb679b16972c2f1db9fda7ce74c3eb8bbaa7bf4f360112e68c" module=profiling.task
time="2025-04-11T06:48:43Z" level=info msg="ready to starting profiling task: 9aa175e4b98e5eaa6e77ebf193a0e09d8c1b8ab0fc6ea4992a971341fafcc712" module=profiling.task
time="2025-04-11T06:48:43Z" level=info msg="ready to starting profiling task: f03aabeb167eb05d569a771b3a551402b6d5f473d1734731206ba75778ff9bfd" module=profiling.task
time="2025-04-11T06:48:43Z" level=info msg="ready to starting profiling task: 75b553c99a5663f01c07533b070cbecee6e0a0ce48cee4d97f084ce92e854af7" module=profiling.task
time="2025-04-11T06:48:43Z" level=info msg="ready to starting profiling task: b49f7dc9c0b3c3e156c74cf71fc02e9cc528c531c6b2660990c1a88354ad2cc4" module=profiling.task
time="2025-04-11T06:48:43Z" level=debug msg="success running on cpu profiling, perf event fds: [1192 1229 1230 1234 1235 1239 1240 1244 1245 1249 1252 1256]" module=profiling.task.oncpu
time="2025-04-11T06:48:43Z" level=info msg="profiling task has been started. taskId: 75b553c99a5663f01c07533b070cbecee6e0a0ce48cee4d97f084ce92e854af7, pid: [69122]" module=profiling.task
time="2025-04-11T06:48:43Z" level=debug msg="success running on cpu profiling, perf event fds: [1166 1214 1217 1221 1223 1226 1231 1236 1241 1246 1253 1257]" module=profiling.task.oncpu
time="2025-04-11T06:48:43Z" level=info msg="profiling task has been started. taskId: 36dcce484408c9b9a3513f94cd80a6b3b875c583503a3fcfcf653531157fb212, pid: [65894]" module=profiling.task
time="2025-04-11T06:48:43Z" level=debug msg="success running on cpu profiling, perf event fds: [1201 1215 1216 1220 1224 1228 1233 1238 1243 1248 1251 1254]" module=profiling.task.oncpu
time="2025-04-11T06:48:43Z" level=info msg="profiling task has been started. taskId: 04b064f92110b2cb679b16972c2f1db9fda7ce74c3eb8bbaa7bf4f360112e68c, pid: [54074]" module=profiling.task
time="2025-04-11T06:48:43Z" level=debug msg="success running on cpu profiling, perf event fds: [1142 1218 1219 1222 1225 1227 1232 1237 1242 1247 1250 1255]" module=profiling.task.oncpu
time="2025-04-11T06:48:43Z" level=info msg="profiling task has been started. taskId: 9aa175e4b98e5eaa6e77ebf193a0e09d8c1b8ab0fc6ea4992a971341fafcc712, pid: [68820]" module=profiling.task
time="2025-04-11T06:48:43Z" level=debug msg="total found stacks: 0" module=profiling.task.oncpu
time="2025-04-11T06:48:43Z" level=debug msg="total found stacks: 0" module=profiling.task.oncpu
time="2025-04-11T06:48:43Z" level=debug msg="total found stacks: 0" module=profiling.task.oncpu
time="2025-04-11T06:48:43Z" level=debug msg="total found stacks: 0" module=profiling.task.offcpu
time="2025-04-11T06:48:43Z" level=debug msg="total found stacks: 0" module=profiling.task.oncpu
time="2025-04-11T06:48:44Z" level=debug msg="total found 1 off cpu symbols: [finish_task_switch.isra.0]" module=profiling.task.offcpu
time="2025-04-11T06:48:44Z" level=debug msg="update the process 164179 monitor status to false" module=access_log.common
time="2025-04-11T06:48:44Z" level=debug msg="update the process 179086 monitor status to false" module=access_log.common
time="2025-04-11T06:48:44Z" level=debug msg="update the process 179086 monitor status to false" module=access_log.common
time="2025-04-11T06:48:44Z" level=debug msg="update the process 179086 monitor status to false" module=access_log.common
time="2025-04-11T06:48:44Z" level=debug msg="update the process 179086 monitor status to false" module=access_log.common
time="2025-04-11T06:48:44Z" level=debug msg="update the process 179086 monitor status to false" module=access_log.common
time="2025-04-11T06:48:44Z" level=debug msg="update the process 164179 monitor status to false" module=access_log.common
time="2025-04-11T06:48:44Z" level=debug msg="update the process 164179 monitor status to false" module=access_log.common
time="2025-04-11T06:48:44Z" level=debug msg="attach to the kprobe: finish_task_switch.isra.0" module=tools.btf
time="2025-04-11T06:48:44Z" level=info msg="profiling task has been started. taskId: f03aabeb167eb05d569a771b3a551402b6d5f473d1734731206ba75778ff9bfd, pid: [68820]" module=profiling.task
time="2025-04-11T06:48:44Z" level=debug msg="update the process 164179 monitor status to false" module=access_log.common
time="2025-04-11T06:48:44Z" level=debug msg="update the process 164179 monitor status to false" module=access_log.common
time="2025-04-11T06:48:44Z" level=debug msg="update the process 395711 monitor status to false" module=access_log.common
time="2025-04-11T06:48:44Z" level=debug msg="update the process 211070 monitor status to false" module=access_log.common
time="2025-04-11T06:48:44Z" level=debug msg="update the process 158318 monitor status to false" module=access_log.common
time="2025-04-11T06:48:44Z" level=debug msg="update the process 395742 monitor status to false" module=access_log.common
time="2025-04-11T06:48:44Z" level=debug msg="update the process 395744 monitor status to false" module=access_log.common
time="2025-04-11T06:48:44Z" level=debug msg="update the process 395742 monitor status to false" module=access_log.common
time="2025-04-11T06:48:44Z" level=debug msg="update the process 395742 monitor status to false" module=access_log.common
time="2025-04-11T06:48:44Z" level=debug msg="update the process 395745 monitor status to false" module=access_log.common
time="2025-04-11T06:48:44Z" level=debug msg="update the process 395747 monitor status to false" module=access_log.common
time="2025-04-11T06:48:44Z" level=debug msg="update the process 395745 monitor status to false" module=access_log.common
time="2025-04-11T06:48:44Z" level=debug msg="update the process 165930 monitor status to false" module=access_log.common
time="2025-04-11T06:48:44Z" level=debug msg="update the process 395742 monitor status to false" module=access_log.common
time="2025-04-11T06:48:44Z" level=debug msg="update the process 395753 monitor status to false" module=access_log.common
time="2025-04-11T06:48:44Z" level=debug msg="receive connect event, connection ID: 1699698947653640, randomID: 3907818998, pid: 395742, fd: 8, role: unknown: func: Write, family: 1, success: 1, conntrack exist: false" module=access_log.collector.connection
time="2025-04-11T06:48:44Z" level=debug msg="cannot found the socket paire from connect event, connection ID: 1699698947653640, randomID: 3907818998" module=access_log.collector.connection
time="2025-04-11T06:48:44Z" level=debug msg="receive connect event, connection ID: 1699750487261191, randomID: 2946201840, pid: 395754, fd: 7, role: unknown: func: Write, family: 1, success: 1, conntrack exist: false" module=access_log.collector.connection
time="2025-04-11T06:48:44Z" level=debug msg="cannot found the socket paire from connect event, connection ID: 1699750487261191, randomID: 2946201840" module=access_log.collector.connection
time="2025-04-11T06:48:44Z" level=debug msg="receive connect event, connection ID: 1699746192293896, randomID: 1993532738, pid: 395753, fd: 8, role: server: func: Read, family: 1, success: 1, conntrack exist: false" module=access_log.collector.connection
time="2025-04-11T06:48:44Z" level=debug msg="cannot found the socket paire from connect event, connection ID: 1699746192293896, randomID: 1993532738" module=access_log.collector.connection
time="2025-04-11T06:48:44Z" level=debug msg="update the process 395751 monitor status to false" module=access_log.common
time="2025-04-11T06:48:44Z" level=debug msg="update the process 395754 monitor status to false" module=access_log.common
time="2025-04-11T06:48:44Z" level=debug msg="update the process 165930 monitor status to false" module=access_log.common
time="2025-04-11T06:48:44Z" level=debug msg="update the process 395755 monitor status to false" module=access_log.common
time="2025-04-11T06:48:44Z" level=debug msg="update the process 395755 monitor status to false" module=access_log.common
time="2025-04-11T06:48:44Z" level=debug msg="update the process 395759 monitor status to false" module=access_log.common
time="2025-04-11T06:48:44Z" level=debug msg="update the process 395755 monitor status to false" module=access_log.common
time="2025-04-11T06:48:44Z" level=debug msg="update the process 395757 monitor status to false" module=access_log.common
time="2025-04-11T06:48:44Z" level=debug msg="update the process 165930 monitor status to false" module=access_log.common
time="2025-04-11T06:48:44Z" level=debug msg="update the process 395764 monitor status to false" module=access_log.common
time="2025-04-11T06:48:44Z" level=debug msg="update the process 165930 monitor status to false" module=access_log.common
time="2025-04-11T06:48:44Z" level=debug msg="update the process 395755 monitor status to false" module=access_log.common
time="2025-04-11T06:48:44Z" level=debug msg="update the process 395767 monitor status to false" module=access_log.common
time="2025-04-11T06:48:44Z" level=debug msg="update the process 211075 monitor status to false" module=access_log.common
time="2025-04-11T06:48:44Z" level=debug msg="update the process 163601 monitor status to false" module=access_log.common
time="2025-04-11T06:48:45Z" level=debug msg="update the process 179086 monitor status to false" module=access_log.common
time="2025-04-11T06:48:45Z" level=debug msg="update the process 179086 monitor status to false" module=access_log.common
time="2025-04-11T06:48:45Z" level=debug msg="update the process 179086 monitor status to false" module=access_log.common
time="2025-04-11T06:48:45Z" level=debug msg="update the process 179086 monitor status to false" module=access_log.common
time="2025-04-11T06:48:45Z" level=debug msg="update the process 179086 monitor status to false" module=access_log.common
time="2025-04-11T06:48:45Z" level=debug msg="update the process 191220 monitor status to false" module=access_log.common
time="2025-04-11T06:48:45Z" level=debug msg="update the process 141929 monitor status to false" module=access_log.common
time="2025-04-11T06:48:45Z" level=debug msg="update the process 172090 monitor status to false" module=access_log.common
time="2025-04-11T06:48:45Z" level=debug msg="update the process 395778 monitor status to false" module=access_log.common
time="2025-04-11T06:48:45Z" level=debug msg="update the process 395778 monitor status to false" module=access_log.common
time="2025-04-11T06:48:45Z" level=debug msg="update the process 395778 monitor status to false" module=access_log.common
time="2025-04-11T06:48:45Z" level=debug msg="update the process 395780 monitor status to false" module=access_log.common
time="2025-04-11T06:48:45Z" level=debug msg="update the process 395778 monitor status to false" module=access_log.common
time="2025-04-11T06:48:45Z" level=debug msg="update the process 395782 monitor status to false" module=access_log.common
time="2025-04-11T06:48:45Z" level=debug msg="update the process 395780 monitor status to false" module=access_log.common
time="2025-04-11T06:48:45Z" level=debug msg="update the process 395785 monitor status to false" module=access_log.common
time="2025-04-11T06:48:45Z" level=debug msg="update the process 395778 monitor status to false" module=access_log.common
time="2025-04-11T06:48:45Z" level=debug msg="update the process 395778 monitor status to false" module=access_log.common
time="2025-04-11T06:48:45Z" level=debug msg="receive connect event, connection ID: 1699853566476296, randomID: 2846823001, pid: 395778, fd: 8, role: unknown: func: Write, family: 1, success: 1, conntrack exist: false" module=access_log.collector.connection
time="2025-04-11T06:48:45Z" level=debug msg="cannot found the socket paire from connect event, connection ID: 1699853566476296, randomID: 2846823001" module=access_log.collector.connection
time="2025-04-11T06:48:45Z" level=debug msg="update the process 395788 monitor status to false" module=access_log.common
time="2025-04-11T06:48:45Z" level=debug msg="receive connect event, connection ID: 1699900811116551, randomID: 66034808, pid: 395789, fd: 7, role: unknown: func: Write, family: 1, success: 1, conntrack exist: false" module=access_log.collector.connection
time="2025-04-11T06:48:45Z" level=debug msg="cannot found the socket paire from connect event, connection ID: 1699900811116551, randomID: 66034808" module=access_log.collector.connection
time="2025-04-11T06:48:45Z" level=debug msg="update the process 395789 monitor status to false" module=access_log.common
time="2025-04-11T06:48:45Z" level=debug msg="receive connect event, connection ID: 1699896516149256, randomID: 724867649, pid: 395788, fd: 8, role: server: func: Read, family: 1, success: 1, conntrack exist: false" module=access_log.collector.connection
time="2025-04-11T06:48:45Z" level=debug msg="cannot found the socket paire from connect event, connection ID: 1699896516149256, randomID: 724867649" module=access_log.collector.connection
time="2025-04-11T06:48:45Z" level=debug msg="update the process 395790 monitor status to false" module=access_log.common
time="2025-04-11T06:48:45Z" level=debug msg="update the process 395790 monitor status to false" module=access_log.common
time="2025-04-11T06:48:45Z" level=debug msg="update the process 395790 monitor status to false" module=access_log.common
time="2025-04-11T06:48:45Z" level=debug msg="update the process 395792 monitor status to false" module=access_log.common
time="2025-04-11T06:48:45Z" level=debug msg="update the process 395794 monitor status to false" module=access_log.common
time="2025-04-11T06:48:45Z" level=debug msg="update the process 395790 monitor status to false" module=access_log.common
time="2025-04-11T06:48:45Z" level=debug msg="update the process 395790 monitor status to false" module=access_log.common
time="2025-04-11T06:48:45Z" level=debug msg="update the process 395797 monitor status to false" module=access_log.common
time="2025-04-11T06:48:45Z" level=debug msg="update the process 395797 monitor status to false" module=access_log.common
time="2025-04-11T06:48:45Z" level=debug msg="update the process 395797 monitor status to false" module=access_log.common
time="2025-04-11T06:48:45Z" level=debug msg="update the process 211070 monitor status to false" module=access_log.common
time="2025-04-11T06:48:45Z" level=debug msg="update the process 395797 monitor status to false" module=access_log.common
time="2025-04-11T06:48:45Z" level=debug msg="update the process 165930 monitor status to false" module=access_log.common
time="2025-04-11T06:48:45Z" level=debug msg="receive connect event, connection ID: 1699986710462471, randomID: 3970982506, pid: 395809, fd: 7, role: client: func: Connect, family: 2, success: 1, conntrack exist: false" module=access_log.collector.connection
time="2025-04-11T06:48:45Z" level=debug msg="found the connection from the connect event is valid, connection ID: 1699986710462471, randomID: 3970982506" module=access_log.collector.connection
time="2025-04-11T06:48:45Z" level=debug msg="build socket pair success, connection ID: 1699986710462471, randomID: 3970982506, role: client, local: 127.0.0.1:45524, remote: 127.0.0.1:65535" module=access_log.collector.connection
time="2025-04-11T06:48:45Z" level=debug msg="receive connect event, connection ID: 1699986710462471, randomID: 885857337, pid: 395809, fd: 7, role: client: func: Connect, family: 10, success: 1, conntrack exist: false" module=access_log.collector.connection
time="2025-04-11T06:48:45Z" level=debug msg="found the connection from the connect event is valid, connection ID: 1699986710462471, randomID: 885857337" module=access_log.collector.connection
time="2025-04-11T06:48:45Z" level=debug msg="build socket pair success, connection ID: 1699986710462471, randomID: 885857337, role: client, local: ::1:39521, remote: ::1:65535" module=access_log.collector.connection
time="2025-04-11T06:48:45Z" level=debug msg="receive connect event, connection ID: 1699986710462470, randomID: 1363879759, pid: 395809, fd: 6, role: unknown: func: Sendto, family: 1, success: 1, conntrack exist: false" module=access_log.collector.connection
time="2025-04-11T06:48:45Z" level=debug msg="cannot found the socket paire from connect event, connection ID: 1699986710462470, randomID: 1363879759" module=access_log.collector.connection
time="2025-04-11T06:48:45Z" level=debug msg="receive close event, connection ID: 1699986710462471, randomID: 3970982506, pid: 395809, fd: 7" module=access_log.collector.connection
time="2025-04-11T06:48:45Z" level=debug msg="receive close event, connection ID: 1699986710462471, randomID: 885857337, pid: 395809, fd: 7" module=access_log.collector.connection
time="2025-04-11T06:48:45Z" level=debug msg="receive close event, connection ID: 1699986710462470, randomID: 1363879759, pid: 395809, fd: 6" module=access_log.collector.connection
time="2025-04-11T06:48:45Z" level=debug msg="receive connect event, connection ID: 1699986710462469, randomID: 1575978298, pid: 395809, fd: 5, role: client: func: Connect, family: 10, success: 1, conntrack exist: false" module=access_log.collector.connection
time="2025-04-11T06:48:45Z" level=debug msg="the connection from the connect event is only the local port is empty, connection ID: 1699986710462469, randomID: 1575978298" module=access_log.collector.connection
time="2025-04-11T06:48:45Z" level=debug msg="build socket pair success, connection ID: 1699986710462469, randomID: 1575978298, role: client, local: ::1:0, remote: ::1:8080" module=access_log.collector.connection
time="2025-04-11T06:48:45Z" level=debug msg="receive close event, connection ID: 1699986710462469, randomID: 1575978298, pid: 395809, fd: 5" module=access_log.collector.connection
time="2025-04-11T06:48:45Z" level=debug msg="receive connect event, connection ID: 1699986710462470, randomID: 2911376569, pid: 395809, fd: 6, role: client: func: Connect, family: 2, success: 1, conntrack exist: false" module=access_log.collector.connection
time="2025-04-11T06:48:45Z" level=debug msg="found the connection from the connect event is valid, connection ID: 1699986710462470, randomID: 2911376569" module=access_log.collector.connection
time="2025-04-11T06:48:45Z" level=debug msg="build socket pair success, connection ID: 1699986710462470, randomID: 2911376569, role: client, local: 127.0.0.1:58330, remote: 127.0.0.1:8080" module=access_log.collector.connection
time="2025-04-11T06:48:45Z" level=debug msg="receive the socket data event, connection ID: 1699986710462470, random ID: 2911376569, pid: 395809, prev data id: 0, data id: 1, sequence: 0, protocol: 1" module=accesslog.collector.protocols
time="2025-04-11T06:48:45Z" level=debug msg="receive the socket detail event, connection ID: 1699986710462470, random ID: 2911376569, pid: 395809, data id: 1, function name: Sendto, package count: 2, package size: 82, ssl: 0, protocol: 1" module=accesslog.collector.protocols
time="2025-04-11T06:48:45Z" level=debug msg="receive the socket data event, connection ID: 1699986710462470, random ID: 2911376569, pid: 395809, prev data id: 1, data id: 2, sequence: 0, protocol: 1" module=accesslog.collector.protocols
time="2025-04-11T06:48:45Z" level=debug msg="receive the socket detail event, connection ID: 1699986710462470, random ID: 2911376569, pid: 395809, data id: 2, function name: Recvfrom, package count: 1, package size: 195, ssl: 0, protocol: 1" module=accesslog.collector.protocols
time="2025-04-11T06:48:45Z" level=debug msg="receive close event, connection ID: 1699986710462470, randomID: 2911376569, pid: 395809, fd: 6" module=access_log.collector.connection
time="2025-04-11T06:48:45Z" level=debug msg="update the process 165930 monitor status to false" module=access_log.common
time="2025-04-11T06:48:45Z" level=debug msg="update the process 395809 monitor status to false" module=access_log.common
time="2025-04-11T06:48:45Z" level=debug msg="update the process 395811 monitor status to false" module=access_log.common
time="2025-04-11T06:48:45Z" level=debug msg="receive connect event, connection ID: 1699995300397063, randomID: 2671169369, pid: 395811, fd: 7, role: client: func: Connect, family: 2, success: 1, conntrack exist: false" module=access_log.collector.connection
time="2025-04-11T06:48:45Z" level=debug msg="found the connection from the connect event is valid, connection ID: 1699995300397063, randomID: 2671169369" module=access_log.collector.connection
time="2025-04-11T06:48:45Z" level=debug msg="build socket pair success, connection ID: 1699995300397063, randomID: 2671169369, role: client, local: 127.0.0.1:37582, remote: 127.0.0.1:65535" module=access_log.collector.connection
time="2025-04-11T06:48:45Z" level=debug msg="update the process 165930 monitor status to false" module=access_log.common
time="2025-04-11T06:48:45Z" level=debug msg="update the process 395813 monitor status to false" module=access_log.common
time="2025-04-11T06:48:45Z" level=debug msg="update the process 165930 monitor status to false" module=access_log.common
time="2025-04-11T06:48:45Z" level=debug msg="update the process 174094 monitor status to false" module=access_log.common
time="2025-04-11T06:48:45Z" level=debug msg="update the process 174094 monitor status to false" module=access_log.common
time="2025-04-11T06:48:45Z" level=debug msg="update the process 174094 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="update the process 179086 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="update the process 179086 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="update the process 179086 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="update the process 179086 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="update the process 179086 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="update the process 395797 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="update the process 211075 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="update the process 211070 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="update the process 158458 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="update the process 395828 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="update the process 395830 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="update the process 395833 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="update the process 395830 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="update the process 395835 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="update the process 395828 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="update the process 395828 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="update the process 395838 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="update the process 395839 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="receive connect event, connection ID: 1700115559481351, randomID: 2481809479, pid: 395839, fd: 7, role: unknown: func: Write, family: 1, success: 1, conntrack exist: false" module=access_log.collector.connection
time="2025-04-11T06:48:46Z" level=debug msg="cannot found the socket paire from connect event, connection ID: 1700115559481351, randomID: 2481809479" module=access_log.collector.connection
time="2025-04-11T06:48:46Z" level=debug msg="receive connect event, connection ID: 1700111264514056, randomID: 650539604, pid: 395838, fd: 8, role: server: func: Read, family: 1, success: 1, conntrack exist: false" module=access_log.collector.connection
time="2025-04-11T06:48:46Z" level=debug msg="cannot found the socket paire from connect event, connection ID: 1700111264514056, randomID: 650539604" module=access_log.collector.connection
time="2025-04-11T06:48:46Z" level=debug msg="update the process 395840 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="update the process 395840 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="update the process 395842 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="update the process 395840 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="update the process 395840 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="update the process 395846 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="update the process 165930 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="update the process 157188 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="update the process 395848 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="receive close event, connection ID: 1700154214187015, randomID: 3212730294, pid: 395848, fd: 7" module=access_log.collector.connection
time="2025-04-11T06:48:46Z" level=debug msg="receive close event, connection ID: 1700154214187015, randomID: 2515019959, pid: 395848, fd: 7" module=access_log.collector.connection
time="2025-04-11T06:48:46Z" level=debug msg="receive connect event, connection ID: 1700154214187015, randomID: 3212730294, pid: 395848, fd: 7, role: client: func: Connect, family: 2, success: 1, conntrack exist: false" module=access_log.collector.connection
time="2025-04-11T06:48:46Z" level=debug msg="found the connection from the connect event is valid, connection ID: 1700154214187015, randomID: 3212730294" module=access_log.collector.connection
time="2025-04-11T06:48:46Z" level=debug msg="build socket pair success, connection ID: 1700154214187015, randomID: 3212730294, role: client, local: 127.0.0.1:57756, remote: 127.0.0.1:65535" module=access_log.collector.connection
time="2025-04-11T06:48:46Z" level=debug msg="receive connect event, connection ID: 1700154214187015, randomID: 2515019959, pid: 395848, fd: 7, role: client: func: Connect, family: 10, success: 1, conntrack exist: false" module=access_log.collector.connection
time="2025-04-11T06:48:46Z" level=debug msg="found the connection from the connect event is valid, connection ID: 1700154214187015, randomID: 2515019959" module=access_log.collector.connection
time="2025-04-11T06:48:46Z" level=debug msg="build socket pair success, connection ID: 1700154214187015, randomID: 2515019959, role: client, local: ::1:41109, remote: ::1:65535" module=access_log.collector.connection
time="2025-04-11T06:48:46Z" level=debug msg="receive connect event, connection ID: 1700154214187014, randomID: 2679474587, pid: 395848, fd: 6, role: unknown: func: Sendto, family: 1, success: 1, conntrack exist: false" module=access_log.collector.connection
time="2025-04-11T06:48:46Z" level=debug msg="cannot found the socket paire from connect event, connection ID: 1700154214187014, randomID: 2679474587" module=access_log.collector.connection
time="2025-04-11T06:48:46Z" level=debug msg="update the process 165930 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="update the process 395849 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="update the process 395849 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="receive close event, connection ID: 1700167099088903, randomID: 1709478490, pid: 395851, fd: 7" module=access_log.collector.connection
time="2025-04-11T06:48:46Z" level=debug msg="update the process 395851 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="update the process 395853 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="receive close event, connection ID: 1700167099088903, randomID: 2638963452, pid: 395851, fd: 7" module=access_log.collector.connection
time="2025-04-11T06:48:46Z" level=debug msg="update the process 395849 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="receive close event, connection ID: 1700167099088902, randomID: 3737474276, pid: 395851, fd: 6" module=access_log.collector.connection
time="2025-04-11T06:48:46Z" level=debug msg="receive connect event, connection ID: 1700167099088903, randomID: 1709478490, pid: 395851, fd: 7, role: client: func: Connect, family: 2, success: 1, conntrack exist: false" module=access_log.collector.connection
time="2025-04-11T06:48:46Z" level=debug msg="found the connection from the connect event is valid, connection ID: 1700167099088903, randomID: 1709478490" module=access_log.collector.connection
time="2025-04-11T06:48:46Z" level=debug msg="build socket pair success, connection ID: 1700167099088903, randomID: 1709478490, role: client, local: 127.0.0.1:59783, remote: 127.0.0.1:65535" module=access_log.collector.connection
time="2025-04-11T06:48:46Z" level=debug msg="receive connect event, connection ID: 1700167099088903, randomID: 2638963452, pid: 395851, fd: 7, role: client: func: Connect, family: 10, success: 1, conntrack exist: false" module=access_log.collector.connection
time="2025-04-11T06:48:46Z" level=debug msg="found the connection from the connect event is valid, connection ID: 1700167099088903, randomID: 2638963452" module=access_log.collector.connection
time="2025-04-11T06:48:46Z" level=debug msg="build socket pair success, connection ID: 1700167099088903, randomID: 2638963452, role: client, local: ::1:34604, remote: ::1:65535" module=access_log.collector.connection
time="2025-04-11T06:48:46Z" level=debug msg="receive connect event, connection ID: 1700167099088902, randomID: 3737474276, pid: 395851, fd: 6, role: unknown: func: Sendto, family: 1, success: 1, conntrack exist: false" module=access_log.collector.connection
time="2025-04-11T06:48:46Z" level=debug msg="cannot found the socket paire from connect event, connection ID: 1700167099088902, randomID: 3737474276" module=access_log.collector.connection
time="2025-04-11T06:48:46Z" level=debug msg="update the process 165930 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="receive connect event, connection ID: 1700167099088901, randomID: 1133114952, pid: 395851, fd: 5, role: client: func: Connect, family: 10, success: 1, conntrack exist: false" module=access_log.collector.connection
time="2025-04-11T06:48:46Z" level=debug msg="the connection from the connect event is only the local port is empty, connection ID: 1700167099088901, randomID: 1133114952" module=access_log.collector.connection
time="2025-04-11T06:48:46Z" level=debug msg="build socket pair success, connection ID: 1700167099088901, randomID: 1133114952, role: client, local: ::1:0, remote: ::1:8080" module=access_log.collector.connection
time="2025-04-11T06:48:46Z" level=debug msg="receive close event, connection ID: 1700167099088901, randomID: 1133114952, pid: 395851, fd: 5" module=access_log.collector.connection
time="2025-04-11T06:48:46Z" level=debug msg="receive connect event, connection ID: 1700167099088902, randomID: 3802425244, pid: 395851, fd: 6, role: client: func: Connect, family: 2, success: 1, conntrack exist: false" module=access_log.collector.connection
time="2025-04-11T06:48:46Z" level=debug msg="found the connection from the connect event is valid, connection ID: 1700167099088902, randomID: 3802425244" module=access_log.collector.connection
time="2025-04-11T06:48:46Z" level=debug msg="build socket pair success, connection ID: 1700167099088902, randomID: 3802425244, role: client, local: 127.0.0.1:58390, remote: 127.0.0.1:8080" module=access_log.collector.connection
time="2025-04-11T06:48:46Z" level=debug msg="update the process 395857 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="update the process 165930 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="update the process 395853 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="update the process 395860 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="update the process 395861 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="update the process 395862 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="update the process 395849 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="update the process 395861 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="update the process 395862 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="update the process 395866 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="update the process 395849 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="update the process 395849 monitor status to false" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="start to consume kernel logs: 93, protocol logs: 0" module=access_log.common
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395040" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1696683880611847, random ID: 2178186211, exist connection: false, delay: false" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395040" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1696683880611847, random ID: 3181420365, exist connection: false, delay: false" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395040" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1696683880611847, random ID: 2178186211, exist connection: false, delay: false" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395040" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1696683880611847, random ID: 3181420365, exist connection: false, delay: false" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395066" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1696795549761543, random ID: 234766575, exist connection: false, delay: false" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395070" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1696812729630727, random ID: 2695690801, exist connection: false, delay: false" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395094" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1696915808845831, random ID: 3411664113, exist connection: false, delay: false" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395096" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1696924398780423, random ID: 2634746035, exist connection: false, delay: false" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395098" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1696932988715015, random ID: 673731378, exist connection: false, delay: false" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395265" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1697650248253447, random ID: 3443980789, exist connection: false, delay: false" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395327" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1697916536225799, random ID: 2286843578, exist connection: false, delay: false" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395329" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1697925126160391, random ID: 3523159474, exist connection: false, delay: false" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395329" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1697925126160391, random ID: 3523159474, exist connection: false, delay: false" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395331" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1697933716094983, random ID: 1479426751, exist connection: false, delay: false" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395331" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1697933716094983, random ID: 1479426751, exist connection: false, delay: false" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395361" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1698062565113863, random ID: 1013982779, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395361" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1698062565113863, random ID: 4654287, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395361" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1698062565113863, random ID: 1013982779, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395361" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1698062565113863, random ID: 4654287, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395361" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1698062565113862, random ID: 103406484, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395363" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1698071155048455, random ID: 3263047701, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395363" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1698071155048455, random ID: 3263047701, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395365" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1698079744983047, random ID: 4057935837, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395365" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1698079744983047, random ID: 506961440, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395365" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1698079744983047, random ID: 4057935837, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395365" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1698079744983047, random ID: 506961440, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395426" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1698341737988103, random ID: 4115251082, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395428" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1698350327922695, random ID: 1673148160, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395522" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1698754054848519, random ID: 4098585119, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395522" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1698754054848519, random ID: 1650477146, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395522" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1698754054848519, random ID: 4098585119, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395522" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1698754054848519, random ID: 1650477146, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395613" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699144896872455, random ID: 886276775, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395613" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699144896872455, random ID: 886276775, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395613" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699144896872455, random ID: 3064389798, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395613" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699144896872455, random ID: 3064389798, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395613" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699144896872454, random ID: 1371521423, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395613" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699144896872453, random ID: 875234137, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395613" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699144896872453, random ID: 875234137, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395613" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699144896872454, random ID: 545917397, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395675" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699411184844807, random ID: 3721804366, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395675" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699411184844807, random ID: 426560955, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395675" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699411184844807, random ID: 3721804366, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395675" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699411184844807, random ID: 426560955, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395675" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699411184844805, random ID: 864866883, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395675" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699411184844806, random ID: 2475661529, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395675" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699411184844806, random ID: 1428068679, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395675" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699411184844805, random ID: 864866883, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395677" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699419774779399, random ID: 411917791, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395677" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699419774779399, random ID: 1363863527, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395677" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699419774779399, random ID: 411917791, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395677" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699419774779399, random ID: 1363863527, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395677" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699419774779398, random ID: 969337914, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395677" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699419774779397, random ID: 647412578, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395677" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699419774779398, random ID: 3112570836, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395677" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699419774779397, random ID: 647412578, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395679" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699428364713991, random ID: 2608040221, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395719" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699600163405831, random ID: 1495017873, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395719" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699600163405831, random ID: 1111503242, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395719" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699600163405830, random ID: 467113459, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395719" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699600163405831, random ID: 1495017873, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395719" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699600163405831, random ID: 1111503242, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395719" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699600163405829, random ID: 957629739, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395719" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699600163405830, random ID: 1984342163, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395719" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699600163405829, random ID: 957629739, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395719" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699600163405830, random ID: 1984342163, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395721" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699608753340423, random ID: 2960638782, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395723" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699617343275015, random ID: 3580091313, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395723" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699617343275015, random ID: 4200317348, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395723" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699617343275015, random ID: 3580091313, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395723" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699617343275015, random ID: 4200317348, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395809" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699986710462471, random ID: 3970982506, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395809" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699986710462471, random ID: 885857337, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395809" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699986710462471, random ID: 3970982506, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395809" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699986710462471, random ID: 885857337, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395809" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699986710462470, random ID: 1363879759, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395809" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699986710462469, random ID: 1575978298, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395809" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699986710462469, random ID: 1575978298, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395809" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699986710462470, random ID: 2911376569, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395809" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699986710462470, random ID: 2911376569, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395811" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1699995300397063, random ID: 2671169369, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395848" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1700154214187015, random ID: 3212730294, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395848" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1700154214187015, random ID: 2515019959, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395848" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1700154214187015, random ID: 3212730294, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395848" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1700154214187015, random ID: 2515019959, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395851" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1700167099088903, random ID: 1709478490, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395851" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1700167099088903, random ID: 2638963452, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395851" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1700167099088902, random ID: 3737474276, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395851" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1700167099088903, random ID: 1709478490, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395851" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1700167099088903, random ID: 2638963452, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395851" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1700167099088901, random ID: 1133114952, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395851" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1700167099088901, random ID: 1133114952, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="the log should be also uploaded because the process quick shutdown but the log exist, pid: 395851" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="building kernel log result, connetaion ID: 1700167099088902, random ID: 3802425244, exist connection: false, delay: true" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="ready to send access log, connection count: 0" module=access_log.runner
time="2025-04-11T06:48:46Z" level=debug msg="update the process 395867 monitor status to false" module=access_log.common
time="2025-04-11T06:48:47Z" level=debug msg="update the process 179086 monitor status to false" module=access_log.common
time="2025-04-11T06:48:47Z" level=debug msg="update the process 179086 monitor status to false" module=access_log.common
time="2025-04-11T06:48:47Z" level=debug msg="update the process 179086 monitor status to false" module=access_log.common
time="2025-04-11T06:48:47Z" level=debug msg="update the process 179086 monitor status to false" module=access_log.common
time="2025-04-11T06:48:47Z" level=debug msg="update the process 179086 monitor status to false" module=access_log.common
time="2025-04-11T06:48:47Z" level=debug msg="update the process 395849 monitor status to false" module=access_log.common
time="2025-04-11T06:48:47Z" level=debug msg="update the process 211070 monitor status to false" module=access_log.common
time="2025-04-11T06:48:47Z" level=debug msg="update the process 211075 monitor status to false" module=access_log.common
time="2025-04-11T06:48:47Z" level=debug msg="update the process 165930 monitor status to false" module=access_log.common
time="2025-04-11T06:48:47Z" level=debug msg="update the process 395879 monitor status to false" module=access_log.common
time="2025-04-11T06:48:47Z" level=debug msg="receive connect event, connection ID: 1700287358173191, randomID: 3807926414, pid: 395879, fd: 7, role: client: func: Connect, family: 2, success: 1, conntrack exist: false" module=access_log.collector.connection
time="2025-04-11T06:48:47Z" level=debug msg="found the connection from the connect event is valid, connection ID: 1700287358173191, randomID: 3807926414" module=access_log.collector.connection
time="2025-04-11T06:48:47Z" level=debug msg="build socket pair success, connection ID: 1700287358173191, randomID: 3807926414, role: client, local: 127.0.0.1:50795, remote: 127.0.0.1:65535" module=access_log.collector.connection
time="2025-04-11T06:48:47Z" level=debug msg="receive close event, connection ID: 1700287358173191, randomID: 3807926414, pid: 395879, fd: 7" module=access_log.collector.connection
time="2025-04-11T06:48:47Z" level=debug msg="receive connect event, connection ID: 1700287358173191, randomID: 1358260269, pid: 395879, fd: 7, role: client: func: Connect, family: 10, success: 1, conntrack exist: false" module=access_log.collector.connection
time="2025-04-11T06:48:47Z" level=debug msg="found the connection from the connect event is valid, connection ID: 1700287358173191, randomID: 1358260269" module=access_log.collector.connection
time="2025-04-11T06:48:47Z" level=debug msg="build socket pair success, connection ID: 1700287358173191, randomID: 1358260269, role: client, local: ::1:40637, remote: ::1:65535" module=access_log.collector.connection
time="2025-04-11T06:48:47Z" level=debug msg="update the process 165930 monitor status to false" module=access_log.common
time="2025-04-11T06:48:47Z" level=debug msg="update the process 395881 monitor status to false" module=access_log.common
time="2025-04-11T06:48:47Z" level=debug msg="receive connect event, connection ID: 1700295948107783, randomID: 3382066951, pid: 395881, fd: 7, role: client: func: Connect, family: 2, success: 1, conntrack exist: false" module=access_log.collector.connection
time="2025-04-11T06:48:47Z" level=debug msg="found the connection from the connect event is valid, connection ID: 1700295948107783, randomID: 3382066951" module=access_log.collector.connection
time="2025-04-11T06:48:47Z" level=debug msg="build socket pair success, connection ID: 1700295948107783, randomID: 3382066951, role: client, local: 127.0.0.1:34315, remote: 127.0.0.1:65535" module=access_log.collector.connection
time="2025-04-11T06:48:47Z" level=debug msg="update the process 165930 monitor status to false" module=access_log.common
time="2025-04-11T06:48:47Z" level=debug msg="update the process 395883 monitor status to false" module=access_log.common
time="2025-04-11T06:48:47Z" level=debug msg="update the process 165930 monitor status to false" module=access_log.common
time="2025-04-11T06:48:47Z" level=debug msg="ready to analyze HTTP/1 protocol data, connection ID: 1696503491985414, random ID: 2462051097, data len: 0" module=accesslog.collector.protocols.http1
time="2025-04-11T06:48:47Z" level=debug msg="ready to analyze HTTP/1 protocol data, connection ID: 1699600163405830, random ID: 1984342163, data len: 2" module=accesslog.collector.protocols.http1
time="2025-04-11T06:48:47Z" level=debug msg="readed message, messageType: 0, buf: 0xc06a2d1f20, data id: 1, connection ID: 1699600163405830, random ID: 1984342163, metrics : 0xc16befb980, handle result: 0" module=accesslog.collector.protocols.http1
time="2025-04-11T06:48:47Z" level=debug msg="found fully HTTP1 request and response, contains 2 detail events, connection ID: 1699600163405830, random ID: 1984342163, data range: 1-2(true)" module=accesslog.collector.protocols.http1
time="2025-04-11T06:48:47Z" level=debug msg="delete detail event from buffer, data id: 1, ref: 0xc06a2d1f20, range: 1-2(true)" module=tools.buffer
time="2025-04-11T06:48:47Z" level=debug msg="delete detail event from buffer, data id: 2, ref: 0xc06a2d1f20, range: 1-2(true)" module=tools.buffer
time="2025-04-11T06:48:47Z" level=debug msg="readed message, messageType: 1, buf: 0xc06a2d1f20, data id: 2, connection ID: 1699600163405830, random ID: 1984342163, metrics : 0xc16befb980, handle result: 0" module=accesslog.collector.protocols.http1
time="2025-04-11T06:48:47Z" level=debug msg="ready to analyze HTTP/1 protocol data, connection ID: 1699411184844806, random ID: 2475661529, data len: 0" module=accesslog.collector.protocols.http1
time="2025-04-11T06:48:47Z" level=debug msg="ready to analyze HTTP/1 protocol data, connection ID: 1699144896872454, random ID: 545917397, data len: 0" module=accesslog.collector.protocols.http1
time="2025-04-11T06:48:47Z" level=debug msg="ready to analyze HTTP/1 protocol data, connection ID: 1699986710462470, random ID: 2911376569, data len: 2" module=accesslog.collector.protocols.http1
time="2025-04-11T06:48:47Z" level=debug msg="readed message, messageType: 0, buf: 0xc018ced080, data id: 1, connection ID: 1699986710462470, random ID: 2911376569, metrics : 0xc0e84e6b60, handle result: 0" module=accesslog.collector.protocols.http1
time="2025-04-11T06:48:47Z" level=debug msg="found fully HTTP1 request and response, contains 2 detail events, connection ID: 1699986710462470, random ID: 2911376569, data range: 1-2(true)" module=accesslog.collector.protocols.http1
time="2025-04-11T06:48:47Z" level=debug msg="delete detail event from buffer, data id: 1, ref: 0xc018ced080, range: 1-2(true)" module=tools.buffer
time="2025-04-11T06:48:47Z" level=debug msg="delete detail event from buffer, data id: 2, ref: 0xc018ced080, range: 1-2(true)" module=tools.buffer
time="2025-04-11T06:48:47Z" level=debug msg="readed message, messageType: 1, buf: 0xc018ced080, data id: 2, connection ID: 1699986710462470, random ID: 2911376569, metrics : 0xc0e84e6b60, handle result: 0" module=accesslog.collector.protocols.http1
time="2025-04-11T06:48:47Z" level=debug msg="ready to analyze HTTP/1 protocol data, connection ID: 1695451224997894, random ID: 3369217491, data len: 0" module=accesslog.collector.protocols.http1
time="2025-04-11T06:48:47Z" level=debug msg="ready to analyze HTTP/1 protocol data, connection ID: 906384128344077, random ID: 3540445791, data len: 0" module=accesslog.collector.protocols.http1
time="2025-04-11T06:48:47Z" level=debug msg="ready to analyze HTTP/1 protocol data, connection ID: 906384128344077, random ID: 1130417748, data len: 0" module=accesslog.collector.protocols.http1
time="2025-04-11T06:48:48Z" level=debug msg="update the process 174776 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395886 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395886 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395886 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395888 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395886 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395892 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395891 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395886 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395886 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395895 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395896 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="receive connect event, connection ID: 1700360372617223, randomID: 471921498, pid: 395896, fd: 7, role: unknown: func: Write, family: 1, success: 1, conntrack exist: false" module=access_log.collector.connection
time="2025-04-11T06:48:48Z" level=debug msg="cannot found the socket paire from connect event, connection ID: 1700360372617223, randomID: 471921498" module=access_log.collector.connection
time="2025-04-11T06:48:48Z" level=debug msg="receive connect event, connection ID: 1700356077649928, randomID: 4097047854, pid: 395895, fd: 8, role: server: func: Read, family: 1, success: 1, conntrack exist: false" module=access_log.collector.connection
time="2025-04-11T06:48:48Z" level=debug msg="cannot found the socket paire from connect event, connection ID: 1700356077649928, randomID: 4097047854" module=access_log.collector.connection
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395897 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395897 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395897 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395899 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395897 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395892 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 179086 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 179086 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 179086 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 179086 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 179086 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395897 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395909 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395897 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395897 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395897 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395897 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395897 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395915 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395915 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395915 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395915 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="total found stacks: 0" module=profiling.task.oncpu
time="2025-04-11T06:48:48Z" level=debug msg="total found stacks: 0" module=profiling.task.oncpu
time="2025-04-11T06:48:48Z" level=debug msg="total found stacks: 0" module=profiling.task.oncpu
time="2025-04-11T06:48:48Z" level=debug msg="total found stacks: 0" module=profiling.task.oncpu
time="2025-04-11T06:48:48Z" level=debug msg="total found stacks: 0" module=profiling.task.offcpu
time="2025-04-11T06:48:48Z" level=info msg="detected new process by sync all: pid: 249919, entity: {\"Layer\":\"K8S_SERVICE\",\"ServiceName\":\"skywalking-showcase::nginx.skywalking-showcase\",\"InstanceName\":\"nginx-deployment-b758dcbf8-99vvp\",\"ProcessName\":\"sleep\",\"Labels\":[\"k8s-service\"]}" module=process.finder
time="2025-04-11T06:48:48Z" level=info msg="detected new process by sync all: pid: 249931, entity: {\"Layer\":\"K8S_SERVICE\",\"ServiceName\":\"skywalking-showcase::demo-mongodb-metrics.skywalking-showcase\",\"InstanceName\":\"demo-mongodb-0\",\"ProcessName\":\"/bin/bash\",\"Labels\":[\"k8s-service\"]}" module=process.finder
time="2025-04-11T06:48:48Z" level=info msg="detected new process by sync all: pid: 249949, entity: {\"Layer\":\"K8S_SERVICE\",\"ServiceName\":\"skywalking-showcase::demo-mongodb-metrics.skywalking-showcase\",\"InstanceName\":\"demo-mongodb-0\",\"ProcessName\":\"mongosh\",\"Labels\":[\"k8s-service\"]}" module=process.finder
time="2025-04-11T06:48:48Z" level=info msg="detected new process by sync all: pid: 249950, entity: {\"Layer\":\"K8S_SERVICE\",\"ServiceName\":\"skywalking-showcase::demo-mongodb-metrics.skywalking-showcase\",\"InstanceName\":\"demo-mongodb-0\",\"ProcessName\":\"grep\",\"Labels\":[\"k8s-service\"]}" module=process.finder
time="2025-04-11T06:48:48Z" level=info msg="the process has been recognized as dead, so deleted. pid: 249735, entity: {\"Layer\":\"K8S_SERVICE\",\"ServiceName\":\"skywalking-showcase::nginx.skywalking-showcase\",\"InstanceName\":\"nginx-deployment-b758dcbf8-99vvp\",\"ProcessName\":\"sleep\",\"Labels\":[\"k8s-service\"]}, id: " module=process.finder
time="2025-04-11T06:48:48Z" level=debug msg="remove monitoring process, print all local address with pid" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg=---------------------------- module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="total local address with pid: 7" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="local address: 10.244.0.65, pid: 69069" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="local address: 10.244.0.61, pid: 64010" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="local address: 10.244.0.64, pid: 68820" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="local address: 10.244.0.62, pid: 65821" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="local address: 10.244.0.58, pid: 56304" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="local address: 10.244.0.57, pid: 48861" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="local address: 10.244.0.56, pid: 54004" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg=---------------------------- module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="remove monitoring process, print all local address with pid" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg=---------------------------- module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="total local address with pid: 7" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="local address: 10.244.0.61, pid: 64054" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="local address: 10.244.0.62, pid: 65894" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="local address: 10.244.0.58, pid: 56420" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="local address: 10.244.0.57, pid: 48824" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="local address: 10.244.0.65, pid: 69122" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="local address: 10.244.0.56, pid: 54074" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="local address: 10.244.0.64, pid: 68820" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg=---------------------------- module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="remove monitoring process, print all local address with pid" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg=---------------------------- module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="total local address with pid: 7" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="local address: 10.244.0.61, pid: 64054" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="local address: 10.244.0.62, pid: 65894" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="local address: 10.244.0.57, pid: 48861" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="local address: 10.244.0.65, pid: 69069" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="local address: 10.244.0.64, pid: 68724" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="local address: 10.244.0.56, pid: 53973" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="local address: 10.244.0.58, pid: 56416" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg=---------------------------- module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="remove monitoring process, print all local address with pid" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg=---------------------------- module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="total local address with pid: 7" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="local address: 10.244.0.61, pid: 63936" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="local address: 10.244.0.65, pid: 69069" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="local address: 10.244.0.57, pid: 55622" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="local address: 10.244.0.58, pid: 56419" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="local address: 10.244.0.56, pid: 54074" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="local address: 10.244.0.64, pid: 68820" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="local address: 10.244.0.62, pid: 65894" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg=---------------------------- module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="remove monitoring process, print all local address with pid" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg=---------------------------- module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="total local address with pid: 7" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="local address: 10.244.0.62, pid: 65894" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="local address: 10.244.0.65, pid: 69069" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="local address: 10.244.0.57, pid: 55622" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="local address: 10.244.0.58, pid: 56419" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="local address: 10.244.0.61, pid: 63936" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="local address: 10.244.0.56, pid: 54074" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="local address: 10.244.0.64, pid: 68820" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg=---------------------------- module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395915 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 211070 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 160240 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395928 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395928 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395930 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395932 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395931 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395934 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395934 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395928 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395928 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395939 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395940 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="receive connect event, connection ID: 1700545056210952, randomID: 2856124802, pid: 395939, fd: 8, role: server: func: Read, family: 1, success: 1, conntrack exist: false" module=access_log.collector.connection
time="2025-04-11T06:48:48Z" level=debug msg="cannot found the socket paire from connect event, connection ID: 1700545056210952, randomID: 2856124802" module=access_log.collector.connection
time="2025-04-11T06:48:48Z" level=debug msg="receive connect event, connection ID: 1700549351178247, randomID: 2488587764, pid: 395940, fd: 7, role: unknown: func: Write, family: 1, success: 1, conntrack exist: false" module=access_log.collector.connection
time="2025-04-11T06:48:48Z" level=debug msg="cannot found the socket paire from connect event, connection ID: 1700549351178247, randomID: 2488587764" module=access_log.collector.connection
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395941 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395941 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395941 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395943 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395941 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 164179 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395941 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395948 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 164179 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 164179 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 165930 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395953 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="receive connect event, connection ID: 1700605185753095, randomID: 250838424, pid: 395953, fd: 7, role: client: func: Connect, family: 2, success: 1, conntrack exist: false" module=access_log.collector.connection
time="2025-04-11T06:48:48Z" level=debug msg="found the connection from the connect event is valid, connection ID: 1700605185753095, randomID: 250838424" module=access_log.collector.connection
time="2025-04-11T06:48:48Z" level=debug msg="build socket pair success, connection ID: 1700605185753095, randomID: 250838424, role: client, local: 127.0.0.1:55063, remote: 127.0.0.1:65535" module=access_log.collector.connection
time="2025-04-11T06:48:48Z" level=debug msg="update the process 165930 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="receive close event, connection ID: 1700613775687687, randomID: 123905190, pid: 395955, fd: 7" module=access_log.collector.connection
time="2025-04-11T06:48:48Z" level=debug msg="receive connect event, connection ID: 1700613775687687, randomID: 123905190, pid: 395955, fd: 7, role: client: func: Connect, family: 2, success: 1, conntrack exist: false" module=access_log.collector.connection
time="2025-04-11T06:48:48Z" level=debug msg="found the connection from the connect event is valid, connection ID: 1700613775687687, randomID: 123905190" module=access_log.collector.connection
time="2025-04-11T06:48:48Z" level=debug msg="build socket pair success, connection ID: 1700613775687687, randomID: 123905190, role: client, local: 127.0.0.1:33478, remote: 127.0.0.1:65535" module=access_log.collector.connection
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395955 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="update the process 165930 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="receive connect event, connection ID: 1700622365622279, randomID: 2662381015, pid: 395957, fd: 7, role: client: func: Connect, family: 2, success: 1, conntrack exist: false" module=access_log.collector.connection
time="2025-04-11T06:48:48Z" level=debug msg="found the connection from the connect event is valid, connection ID: 1700622365622279, randomID: 2662381015" module=access_log.collector.connection
time="2025-04-11T06:48:48Z" level=debug msg="build socket pair success, connection ID: 1700622365622279, randomID: 2662381015, role: client, local: 127.0.0.1:56393, remote: 127.0.0.1:65535" module=access_log.collector.connection
time="2025-04-11T06:48:48Z" level=debug msg="receive connect event, connection ID: 1700622365622279, randomID: 4119958662, pid: 395957, fd: 7, role: client: func: Connect, family: 10, success: 1, conntrack exist: false" module=access_log.collector.connection
time="2025-04-11T06:48:48Z" level=debug msg="found the connection from the connect event is valid, connection ID: 1700622365622279, randomID: 4119958662" module=access_log.collector.connection
time="2025-04-11T06:48:48Z" level=debug msg="build socket pair success, connection ID: 1700622365622279, randomID: 4119958662, role: client, local: ::1:49547, remote: ::1:65535" module=access_log.collector.connection
time="2025-04-11T06:48:48Z" level=debug msg="receive connect event, connection ID: 1700622365622278, randomID: 3476194954, pid: 395957, fd: 6, role: unknown: func: Sendto, family: 1, success: 1, conntrack exist: false" module=access_log.collector.connection
time="2025-04-11T06:48:48Z" level=debug msg="cannot found the socket paire from connect event, connection ID: 1700622365622278, randomID: 3476194954" module=access_log.collector.connection
time="2025-04-11T06:48:48Z" level=debug msg="receive close event, connection ID: 1700622365622279, randomID: 2662381015, pid: 395957, fd: 7" module=access_log.collector.connection
time="2025-04-11T06:48:48Z" level=debug msg="receive close event, connection ID: 1700622365622279, randomID: 4119958662, pid: 395957, fd: 7" module=access_log.collector.connection
time="2025-04-11T06:48:48Z" level=debug msg="receive close event, connection ID: 1700622365622278, randomID: 3476194954, pid: 395957, fd: 6" module=access_log.collector.connection
time="2025-04-11T06:48:48Z" level=debug msg="update the process 395957 monitor status to false" module=access_log.common
time="2025-04-11T06:48:48Z" level=debug msg="receive close event, connection ID: 1700622365622277, randomID: 1839991623, pid: 395957, fd: 5" module=access_log.collector.connection
time="2025-04-11T06:48:48Z" level=debug msg="receive connect event, connection ID: 1700622365622277, randomID: 1839991623, pid: 395957, fd: 5, role: client: func: Connect, family: 10, success: 1, conntrack exist: false" module=access_log.collector.connection
time="2025-04-11T06:48:48Z" level=debug msg="the connection from the connect event is only the local port is empty, connection ID: 1700622365622277, randomID: 1839991623" module=access_log.collector.connection
time="2025-04-11T06:48:48Z" level=debug msg="build socket pair success, connection ID: 1700622365622277, randomID: 1839991623, role: client, local: ::1:0, remote: ::1:8080" module=access_log.collector.connection
time="2025-04-11T06:48:48Z" level=debug msg="receive connect event, connection ID: 1700622365622278, randomID: 1213485375, pid: 395957, fd: 6, role: client: func: Connect, family: 2, success: 1, conntrack exist: false" module=access_log.collector.connection
time="2025-04-11T06:48:48Z" level=debug msg="found the connection from the connect event is valid, connection ID: 1700622365622278, randomID: 1213485375" module=access_log.collector.connection
time="2025-04-11T06:48:48Z" level=debug msg="build socket pair success, connection ID: 1700622365622278, randomID: 1213485375, role: client, local: 127.0.0.1:59236, remote: 127.0.0.1:8080" module=access_log.collector.connection

Here is the debug logs of rover

mrproliu Apr 11, 2025
Collaborator

total found stacks: 0 This could mean the rover cannot find any stack executing in the program.
That's strange. You could check the following ways:

checking the program running status in the host that which rover belongs to: cat /proc/{rover pid}/fdinfo/{program fd}. The program fd should be one of the fd from success running on cpu profiling, perf event fds log.
Check the CPU lock is available in the host that which rover belongs to(for On CPU Profiling): perf stat -e cpu-clock -a sleep 1.
Check the program attachment success or not by the command: bpftool prog show
Add some logs into the BPF program to check the program's running status in this file(for on CPU profiling): https://github.com/apache/skywalking-rover/blob/main/bpf/profiling/oncpu.c

wu-sheng Apr 11, 2025
Collaborator

A tip, don't try to profile a one-time-off program/application.
It should be a server and keep running.

KHADEPALLAVI975 Apr 14, 2025
Author

I have attached an image where we can see program attachment success for perf profiling

KHADEPALLAVI975 Apr 14, 2025
Author

time="2025-04-14T17:57:24Z" level=debug msg="no profiling task need to flush" module=profiling.task
time="2025-04-14T17:57:30Z" level=info msg="received 6 profiling task: [3ee0eb32d225f5616cbb2d713c9270b50095b2b1bd133419fb7f71746195a3b7 975e43dcaf9130b94c24ba1e880501e8825c830fb76d8d982e11d2282f83752d d32140861457a657f3a8c953ae65253c1954414eb6a06ff97346e0ff975a1454 e4464133608583fdad2cb10c42e33d9f79b9505d0b7ad3abb05ddb88e867c8e2 093f37a192274e22c039894b545d537407413bd6f27227423113d57d9c8189ba 3b3a7dfa005c8841797996522002055a8ff4fb2bf743f7af3a7831e130c92e65]" module=profiling.task
time="2025-04-14T17:57:30Z" level=info msg="ready to starting profiling task: 3ee0eb32d225f5616cbb2d713c9270b50095b2b1bd133419fb7f71746195a3b7" module=profiling.task
time="2025-04-14T17:57:30Z" level=info msg="ready to starting profiling task: 975e43dcaf9130b94c24ba1e880501e8825c830fb76d8d982e11d2282f83752d" module=profiling.task
time="2025-04-14T17:57:30Z" level=info msg="ready to starting profiling task: d32140861457a657f3a8c953ae65253c1954414eb6a06ff97346e0ff975a1454" module=profiling.task
time="2025-04-14T17:57:30Z" level=info msg="ready to starting profiling task: e4464133608583fdad2cb10c42e33d9f79b9505d0b7ad3abb05ddb88e867c8e2" module=profiling.task
time="2025-04-14T17:57:30Z" level=info msg="ready to starting profiling task: 093f37a192274e22c039894b545d537407413bd6f27227423113d57d9c8189ba" module=profiling.task
time="2025-04-14T17:57:30Z" level=info msg="ready to starting profiling task: 3b3a7dfa005c8841797996522002055a8ff4fb2bf743f7af3a7831e130c92e65" module=profiling.task
time="2025-04-14T17:57:30Z" level=debug msg="success running on cpu profiling, perf event fds: [616 620 621 622 623 624 625 626 627 628 629 630]" module=profiling.task.oncpu
time="2025-04-14T17:57:30Z" level=info msg="profiling task has been started. taskId: 3ee0eb32d225f5616cbb2d713c9270b50095b2b1bd133419fb7f71746195a3b7, pid: [84510]" module=profiling.task
time="2025-04-14T17:57:30Z" level=debug msg="success running on cpu profiling, perf event fds: [634 639 640 641 642 643 644 645 646 647 648 649]" module=profiling.task.oncpu
time="2025-04-14T17:57:30Z" level=info msg="profiling task has been started. taskId: e4464133608583fdad2cb10c42e33d9f79b9505d0b7ad3abb05ddb88e867c8e2, pid: [83860]" module=profiling.task
time="2025-04-14T17:57:30Z" level=debug msg="success running on cpu profiling, perf event fds: [636 652 653 654 655 656 657 658 659 660 661 662]" module=profiling.task.oncpu
time="2025-04-14T17:57:30Z" level=info msg="profiling task has been started. taskId: d32140861457a657f3a8c953ae65253c1954414eb6a06ff97346e0ff975a1454, pid: [83152]" module=profiling.task
time="2025-04-14T17:57:30Z" level=debug msg="total found stacks: 0" module=profiling.task.oncpu
time="2025-04-14T17:57:30Z" level=debug msg="total found stacks: 0" module=profiling.task.oncpu
time="2025-04-14T17:57:30Z" level=debug msg="total found stacks: 0" module=profiling.task.oncpu
time="2025-04-14T17:57:30Z" level=debug msg="success running on cpu profiling, perf event fds: [664 667 668 669 670 671 672 673 674 675 676 677]" module=profiling.task.oncpu
time="2025-04-14T17:57:30Z" level=info msg="profiling task has been started. taskId: 3b3a7dfa005c8841797996522002055a8ff4fb2bf743f7af3a7831e130c92e65, pid: [84447]" module=profiling.task
time="2025-04-14T17:57:30Z" level=debug msg="total found 1 off cpu symbols: [finish_task_switch]" module=profiling.task.offcpu
time="2025-04-14T17:57:30Z" level=info msg="profiling task has been started. taskId: 093f37a192274e22c039894b545d537407413bd6f27227423113d57d9c8189ba, pid: [84510]" module=profiling.task
time="2025-04-14T17:57:34Z" level=debug msg="total found stacks: 0" module=profiling.task.oncpu
time="2025-04-14T17:57:34Z" level=debug msg="total found stacks: 0" module=profiling.task.oncpu
time="2025-04-14T17:57:34Z" level=debug msg="total found stacks: 0" module=profiling.task.oncpu
time="2025-04-14T17:57:34Z" level=debug msg="total found stacks: 0" module=profiling.task.oncpu
time="2025-04-14T17:57:34Z" level=debug msg="total found stacks: 0" module=profiling.task.offcpu
time="2025-04-14T17:57:39Z" level=debug msg="no profiling task found need to execute" module=profiling.task
time="2025-04-14T17:57:39Z" level=debug msg="total found stacks: 0" module=profiling.task.oncpu
time="2025-04-14T17:57:39Z" level=debug msg="total found stacks: 0" module=profiling.task.offcpu
time="2025-04-14T17:57:39Z" level=debug msg="total found stacks: 0" module=profiling.task.oncpu
time="2025-04-14T17:57:39Z" level=debug msg="total found stacks: 0" module=profiling.task.oncpu
time="2025-04-14T17:57:39Z" level=debug msg="total found stacks: 0" module=profiling.task.oncpu
time="2025-04-14T17:57:44Z" level=debug msg="total found stacks: 0" module=profiling.task.oncpu
time="2025-04-14T17:57:44Z" level=debug msg="total found stacks: 0" module=profiling.task.oncpu
time="2025-04-14T17:57:44Z" level=debug msg="total found stacks: 0" module=profiling.task.oncpu
time="2025-04-14T17:57:44Z" level=debug msg="total found stacks: 0" module=profiling.task.offcpu
time="2025-04-14T17:57:44Z" level=debug msg="total found stacks: 0" module=profiling.task.oncpu
time="2025-04-14T17:57:49Z" level=debug msg="no profiling task found need to execute" module=profiling.task
time="2025-04-14T17:57:49Z" level=debug msg="total found stacks: 0" module=profiling.task.oncpu
time="2025-04-14T17:57:49Z" level=debug msg="total found stacks: 0" module=profiling.task.oncpu
time="2025-04-14T17:57:49Z" level=debug msg="total found stacks: 0" module=profiling.task.offcpu
time="2025-04-14T17:57:49Z" level=debug msg="total found stacks: 0" module=profiling.task.oncpu
time="2025-04-14T17:57:49Z" level=debug msg="total found stacks: 0" module=profiling.task.oncpu
time="2025-04-14T17:57:54Z" level=debug msg="total found stacks: 0" module=profiling.task.oncpu
time="2025-04-14T17:57:54Z" level=debug msg="total found stacks: 0" module=profiling.task.oncpu
time="2025-04-14T17:57:54Z" level=debug msg="total found stacks: 0" module=profiling.task.oncpu
time="2025-04-14T17:57:54Z" level=debug msg="total found stacks: 0" module=profiling.task.offcpu
time="2025-04-14T17:57:54Z" level=debug msg="total found stacks: 0" module=profiling.task.oncpu
time="2025-04-14T17:57:58Z" level=warning msg="executing profiling task failure, taskId: 975e43dcaf9130b94c24ba1e880501e8825c830fb76d8d982e11d2282f83752d, reason: field OpensslReadRet: program openssl_read_ret: load program: argument list too long: BPF program is too large. Processed 1000001 insn (2811 line(s) omitted)" module=profiling.task
time="2025-04-14T17:57:59Z" level=debug msg="no profiling task found need to execute" module=profiling.task
time="2025-04-14T17:57:59Z" level=debug msg="total found stacks: 0" module=profiling.task.oncpu
time="2025-04-14T17:57:59Z" level=debug msg="total found stacks: 0" module=profiling.task.oncpu
time="2025-04-14T17:57:59Z" level=debug msg="total found stacks: 0" module=profiling.task.oncpu
time="2025-04-14T17:57:59Z" level=debug msg="total found stacks: 0" module=profiling.task.offcpu
time="2025-04-14T17:57:59Z" level=debug msg="total found stacks: 0" module=profiling.task.oncpu
time="2025-04-14T17:58:04Z" level=debug msg="total found stacks: 0" module=profiling.task.offcpu

I am facing an issue argument list too long: BPF program is too large
Kernel - 5.15.0-136-generic

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unable to Perform eBPF Profiling in Showcase Demo with Latest OAP, UI, and Rover Image #13161

{{title}}

Replies: 1 comment 16 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Unable to Perform eBPF Profiling in Showcase Demo with Latest OAP, UI, and Rover Image #13161

KHADEPALLAVI975 Apr 8, 2025

Replies: 1 comment · 16 replies

mrproliu Apr 8, 2025 Collaborator

KHADEPALLAVI975 Apr 11, 2025 Author

mrproliu Apr 11, 2025 Collaborator

wu-sheng Apr 11, 2025 Collaborator

KHADEPALLAVI975 Apr 14, 2025 Author

KHADEPALLAVI975 Apr 14, 2025 Author

KHADEPALLAVI975
Apr 8, 2025

Replies: 1 comment 16 replies

mrproliu
Apr 8, 2025
Collaborator

KHADEPALLAVI975 Apr 11, 2025
Author

mrproliu Apr 11, 2025
Collaborator

wu-sheng Apr 11, 2025
Collaborator

KHADEPALLAVI975 Apr 14, 2025
Author

KHADEPALLAVI975 Apr 14, 2025
Author