Commit abe7838
authored
[crashtracker/profiler] Notify profiler the application is crashing (#7657)
## Summary of changes
Nth attempt to prevent the profiler from collecting callstacks when the
application is crashing.
## Reason for change
We have this issue for a while and this appeared in some crash reports
lately. The profiler is trying to collect callstacks while the app is
crashing. This can lead to a crash on a crash and appearance of zombie
processes.
```
syscall (sysdeps/unix/sysv/linux/x86_64/syscall.S:38)
_write_validate (/project/obj/libunwind-prefix/src/libunwind/src/mi/Gaddress_validator.c:140)
access_mem (/project/obj/libunwind-prefix/src/libunwind/src/x86_64/Ginit.c:89)
dwarf_get (/project/obj/libunwind-prefix/src/libunwind/include/tdep-x86_64/libunwind_i.h:205)
unw_backtrace2 (/project/obj/libunwind-prefix/src/libunwind/src/mi/backtrace.c:111)
LinuxStackFramesCollector::CollectStackWithBacktrace2(void*) (/project/profiler/src/ProfilerEngine/Datadog.Profiler.Native.Linux/LinuxStackFramesCollector.cpp:285)
LinuxStackFramesCollector::CollectStackSampleSignalHandler(int, siginfo_t*, void*) (/project/profiler/src/ProfilerEngine/Datadog.Profiler.Native.Linux/LinuxStackFramesCollector.cpp)
ProfilerSignalManager::SignalHandler(int, siginfo_t*, void*) (/project/profiler/src/ProfilerEngine/Datadog.Profiler.Native.Linux/ProfilerSignalManager.cpp:183)
__restore_rt
__GI___wait4 (sysdeps/unix/sysv/linux/wait4.c:30)
PROCCreateCrashDump(std::vector<char const*, std::allocator<char const*> >&, char*, int, bool) (/__w/1/s/src/coreclr/pal/src/thread/process.cpp:2545)
PROCCreateCrashDumpIfEnabled (/__w/1/s/src/coreclr/pal/src/thread/process.cpp)
PROCAbort (/__w/1/s/src/coreclr/pal/src/thread/process.cpp:2793)
PROCEndProcess(void*, unsigned int, int) (/__w/1/s/src/coreclr/pal/src/thread/process.cpp:1355)
UnwindManagedExceptionPass1(PAL_SEHException&, _CONTEXT*) (/__w/1/s/src/coreclr/vm/exceptionhandling.cpp)
DispatchManagedException(PAL_SEHException&, bool) (/__w/1/s/src/coreclr/vm/exceptionhandling.cpp)
IL_Throw(Object*) (/__w/1/s/src/coreclr/vm/jithelpers.cpp)
```
When the application is crashing, the .NET runtime calls `fork()`.
Calling this function leads to the creation of a new process (child
process). This process will have a copy of the parent address space.
Then the .NET runtime, in the child process, calls `execve()` to run
`createdump`. We wrapped `execve()` in order to call our own tool which
will create a crash report and send it to our backend.
In our implementation of `execve()`, we set a flag to notify the
profiler to not collect callstack because the application is crashing.
The current implementation does not work because we use an `atomic_int`
which will be copied and when the child process set flag, it will be its
own version of the flag. The parent process won't be able to see it
changed. We need a shared memory so the parent and child processes can
communicate.
## Implementation details
- create a shared memory region (using mmap)
## Test coverage
We should not see issue caused by the profiler in our CI when crashes
happen1 parent ad0a291 commit abe7838
File tree
1 file changed
+30
-5
lines changed- profiler/src/ProfilerEngine/Datadog.Linux.ApiWrapper
1 file changed
+30
-5
lines changedLines changed: 30 additions & 5 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
| 4 | + | |
4 | 5 | | |
5 | 6 | | |
6 | 7 | | |
7 | 8 | | |
| 9 | + | |
8 | 10 | | |
| 11 | + | |
9 | 12 | | |
10 | | - | |
11 | | - | |
12 | 13 | | |
13 | 14 | | |
14 | 15 | | |
| |||
58 | 59 | | |
59 | 60 | | |
60 | 61 | | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
61 | 73 | | |
62 | | - | |
| 74 | + | |
63 | 75 | | |
64 | 76 | | |
65 | 77 | | |
66 | 78 | | |
67 | | - | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
68 | 84 | | |
69 | 85 | | |
70 | 86 | | |
| |||
467 | 483 | | |
468 | 484 | | |
469 | 485 | | |
470 | | - | |
| 486 | + | |
| 487 | + | |
| 488 | + | |
471 | 489 | | |
472 | 490 | | |
473 | 491 | | |
| |||
671 | 689 | | |
672 | 690 | | |
673 | 691 | | |
| 692 | + | |
| 693 | + | |
| 694 | + | |
| 695 | + | |
| 696 | + | |
| 697 | + | |
| 698 | + | |
674 | 699 | | |
675 | 700 | | |
676 | 701 | | |
| |||
0 commit comments