Skip to content

Commit b525e65

Browse files
GranddaveGranddave
committed
Add detached-debug-symbols post
1 parent ea9a2b6 commit b525e65

File tree

1 file changed

+222
-0
lines changed

1 file changed

+222
-0
lines changed

‎content/posts/detached-debug-symbols.md‎

Lines changed: 222 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,222 @@
1+
---
2+
title: "Debug stripped executables with detached symbols in GDB"
3+
date: 2025-04-27T14:11:02+02:00
4+
tags: ["C", "GCC", "GDB", "Debugging", "ELF"]
5+
ShowCodeCopyButtons: true
6+
showToc: true
7+
TocOpen: false
8+
---
9+
10+
Say you have an executable that will be shipped to target machines and you don't want to include any debug symbols, but at the same time if an annoying bug occurs you'd like to debug the executable with symbols. Is it possible to have both ways? As a matter of fact, yes!
11+
12+
[GNU Debugger (GDB)](https://www.sourceware.org/gdb/) has a feature where it can load a separate symbols file to make it easier to debug stripped executables. This would allow you to ship stripped binaries, archive symbols files and when needed load them both into a debug session.
13+
14+
This post got a bit longer than I first expected it was going to be. It was an interesting journey, but I didn't include all the rabbit holes. I left some parts for the reader to dig into [at the end](#further-reading).
15+
16+
## TL;DR
17+
18+
```bash
19+
# Build executable
20+
$ cat > app.c << EOF
21+
#include <stdio.h>
22+
int main() {
23+
printf("Hello, World!\n");
24+
return 0;
25+
}
26+
EOF
27+
$ gcc -g -O0 -o app app.c
28+
```
29+
```bash
30+
# Separate debug symbols from the executable
31+
$ objcopy --only-keep-debug app app.debug
32+
$ strip --strip-debug app
33+
$ ls
34+
app app.c app.debug
35+
```
36+
```bash
37+
# Debug with GDB
38+
$ gdb --exec=app --symbols=app.debug
39+
```
40+
41+
## Deeper explanation
42+
43+
### Compiling, copying and stripping
44+
45+
Let's use the same code as in the example as above.
46+
47+
```c
48+
#include <stdio.h>
49+
int main() {
50+
printf("Hello, World!\n");
51+
return 0;
52+
}
53+
```
54+
55+
56+
First we compile `app.c` without any optimizations (`-O0`) into an executable called `app` which includes debug symbols (`-g`).
57+
58+
```bash
59+
# Compile app
60+
$ gcc -g -O0 -o app app.c
61+
```
62+
63+
Compiling without optimizations is not really necessary in this case, but it makes it easier to follow the code when stepping through instructions.
64+
65+
On Linux the executable is of the [ELF](https://en.wikipedia.org/wiki/Executable_and_Linkable_Format) file format. This file format contains different *sections*, such as `.text` containing executable code, `.data` containing initialized writable data and `.bss` containing uninitialized (zeroed) data, among others. A family of sections we're interested in today are the [debug info sections](https://en.wikipedia.org/wiki/.debug_info). They hold symbols like function names, variable names, filenames and line numbers etc. To list what sections an ELF file contain you can run `readelf --sections $elf_file`.
66+
67+
To separate the debug symbols from the executable we can utilize `objcopy`'s `--only-keep-debug` flag. This copies debug sections from `app` to `app.debug`, which itself also is an ELF file, just like `app`. The naming of this file doesn't seem to be standardized, but the GDB documentation uses the `.debug` suffix, so let's use that. This file can be referred to as either the "*debug file*", "*debug info*" or just "*symbols file*"
68+
69+
```bash
70+
# Copy debug symbols to app.debug
71+
$ objcopy --only-keep-debug app app.debug
72+
```
73+
74+
Next we remove, or *strip* sections from `app` that we neither need, nor want in a deployable target executable. This process makes the executable binary smaller, often a fraction of the size of an executable containing debug symbols, especially for larger programs. We can either strip all sections using `strip $executable`. This removes all sections that are not required for the executable to run. Or specifically just strip the debug symbols, which can be achieved by appending `--strip-debug` flag. Let's strip all unnecesary sections.
75+
76+
```bash
77+
# Strip unnecesary sections from app
78+
$ strip app
79+
```
80+
81+
We now have all the files ready!
82+
83+
```bash
84+
$ ls
85+
app app.c app.debug
86+
```
87+
88+
### Debugging with detached symbols
89+
90+
The next step is to use GDB to load the stripped executable and provide the symbols file for easier debugging.
91+
92+
Let's first try to debug our test executable *without* providing any symbols file.
93+
94+
```bash
95+
$ gdb app
96+
...
97+
Reading symbols from app...
98+
(No debugging symbols found in app)
99+
(gdb) info functions # List available functions
100+
All defined functions:
101+
102+
Non-debugging symbols:
103+
0x0000000000401030 puts@plt
104+
```
105+
106+
Here we started passed our `app` executable to GDB and saw as it initialized that it couldn't find any symbols;
107+
108+
> ```bash
109+
> (No debugging symbols found in app)
110+
> ```
111+
112+
We can also see from `info functions` that no main function was found. Nothing about this is surprising since this is the executable we stripped in a previous step.
113+
114+
Let's now provide the `.debug` file to GDB. I've found three (plus a bonus) separate ways of getting GDB to load symbols from it.
115+
116+
1. `--symbols` flag
117+
2. `symbol-file` command
118+
3. Debug link (Bonus: *build ID* [^1])
119+
120+
[^1]: This is similar to the debug link method. The [documentation](https://sourceware.org/gdb/current/onlinedocs/gdb.html/Separate-Debug-Files.html) mentioned that this was only supported on some OS's, so I didn't dig too deep into it. But it might be a topic for another time.
121+
122+
123+
I'm gonna try to explain how they work here below...
124+
125+
#### 1. Symbols flag
126+
127+
The first two options listed above are explicit instructions to GDB to load the symbols file.
128+
129+
First option uses the `--symbols` flag. This is also the option used in the TL;DR.
130+
131+
Here we specify that `app` is our executable file to be debugged using the `--exec` flag:
132+
133+
```bash
134+
$ gdb --exec=app --symbols=app.debug # Load both executable and symbols file
135+
...
136+
Reading symbols from app.debug...
137+
(gdb) info functions # List available functions
138+
All defined functions:
139+
140+
File app.c:
141+
2: int main();
142+
```
143+
144+
The symbols were loaded!
145+
146+
> ```bash
147+
> Reading symbols from app.debug...
148+
> ...
149+
> File app.c:
150+
> 2: int main();
151+
> ```
152+
153+
#### 2. Symbol file command
154+
155+
Second option is to use the `symbol-file` command.
156+
157+
Load `app` with GDB and pass `app.debug` to execute the `symbol-file` command:
158+
```bash
159+
# Load executable
160+
$ gdb app
161+
...
162+
Reading symbols from app...
163+
(No debugging symbols found in app)
164+
(gdb) symbol-file app.debug # Load symbol file
165+
Reading symbols from app.debug...
166+
(gdb) info functions # List available functions
167+
All defined functions:
168+
169+
File app.c:
170+
2: int main();
171+
```
172+
173+
Again, symbols were loaded successfully.
174+
175+
176+
#### 3. Debug link: Linking symbols file to executable
177+
178+
The third option is to modify the executable itself by adding a special `.gnu_debuglink` section to the binary. This section holds just the basename of the symbols file, in our case it would be `app.debug` (note, not a full path) as well as a [CRC checksum](https://en.wikipedia.org/wiki/Cyclic_redundancy_check) of the symbols file's full content. This means that this specific symbols file is linked to in this specific executable.
179+
180+
```bash
181+
# Link to app.debug in app
182+
$ objcopy --add-gnu-debuglink=app.debug app
183+
```
184+
185+
When running GDB on an executable that has a `.gnu_debuglink` section, GDB will look for the symbol file in a few different places. First, in the same directory as the executable, secondly in a subdirectory called `.debug` and finally through directories set via the `debug-file-directory`. The `debug-file-directory` property is a colon separated string of paths defaulted at GDB's compile-time and can be overridden in runtime.
186+
187+
Say our executable is located in `/home/david/Dev/detached-symbols`, then GDB will search for the filename specified in `.gnu_debuglink` in
188+
189+
- `/home/david/Dev/detached-symbols`
190+
- `/home/david/Dev/detached-symbols/.debug`
191+
- `/usr/lib/debug` (or whatever `debug-file-directory` is set to [^2])
192+
193+
[^2]: You can see what default value your GDB executable was built with by running `gdb --configuration | grep -e '--with-separate-debug-dir'`
194+
195+
Let's try it out:
196+
197+
```bash
198+
$ objcopy --add-gnu-debuglink=app.debug app
199+
$ gdb app
200+
...
201+
Reading symbols from app...
202+
Reading symbols from /home/david/Dev/detached-symbols/app.debug...
203+
```
204+
205+
Notice that I didn't specify any debug file. GDB used the `.gnu_debuglink` section find the symbol file in the same directory as the executable automatically. Neat!
206+
207+
## Closing words
208+
209+
Now we know a few ways of how we can separate the debug information from our executable and when needed debug with the comfort of symbols.
210+
211+
Below are some further reading and resources that might be of use.
212+
213+
### Further reading
214+
215+
- Marking and finding executable and debug info using the [build ID method](https://sourceware.org/gdb/current/onlinedocs/gdb.html/Separate-Debug-Files.html)
216+
- Using [debuginfod](https://sourceware.org/gdb/current/onlinedocs/gdb.html/Debuginfod.html#Debuginfod) to fetch debug info from remotes on-demand using build IDs
217+
218+
### Resources
219+
220+
- [Stack Overflow: extract debug symbol info from ELF binary](https://stackoverflow.com/questions/45659150/extract-debug-symbol-info-from-elf-binary)
221+
- [GDB documentation for separate debug files](https://sourceware.org/gdb/current/onlinedocs/gdb.html/Separate-Debug-Files.html)
222+
- [GDB documentation for `symbol-file` command](https://sourceware.org/gdb/current/onlinedocs/gdb.html/Files.html)

0 commit comments

Comments
 (0)