Skip to content

Commit 9dba5c0

Browse files
[Blog] add blog post on debugging docker container (#163)
* add blog post on debugging docker container * addressed reviews
1 parent 0c1007c commit 9dba5c0

File tree

2 files changed

+378
-0
lines changed

2 files changed

+378
-0
lines changed

src/assets/Debugging/debugging.md

Lines changed: 378 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,378 @@
1+
---
2+
title: Advanced Docker Container Debugging - A Comprehensive Guide
3+
authorName: Prajwol Amatya
4+
authorAvatar: https://1.gravatar.com/avatar/de64e53c0e2cb393dd0d14ffdd53058ee9c607b35e366dd392425bd1b95a034c?size=256
5+
authorLink: https://github.com/prajwolamatya
6+
createdAt: May 07, 2025
7+
tags: debugging, docker, docker container
8+
banner: https://blog.jankaritech.com/src/assets/Debugging/images/Debugging.png
9+
---
10+
11+
Docker has revolutionized modern software development by enabling lightweight, portable, and scalable containerized applications. However, as deployments grow in complexity, so do the challenges in debugging and optimizing containers. In this blog, we'll explore powerful debugging techniques using a sample project with Node.js, Nginx, and Redis. You can get the sample project [here](https://github.com/prajwolamatya/debug-docker).
12+
13+
Our setup consists of:
14+
- A Node.js application (port 3000)
15+
- Nginx as a reverse proxy (port 80)
16+
- Redis for caching (port 6379)
17+
18+
## 1. Container Inspection
19+
### Viewing Running Containers
20+
First, let's check whether our containers are actually running:
21+
22+
```bash
23+
docker compose ps
24+
```
25+
26+
**Example Output:**
27+
```console
28+
Name Command State Ports
29+
debug-docker_nginx_1 /docker-entrypoint.sh ngin ... Up 0.0.0.0:80->80/tcp
30+
debug-docker_nodejs-app_1 docker-entrypoint.sh node ... Up 0.0.0.0:3000->3000/tcp
31+
debug-docker_redis_1 docker-entrypoint.sh redis ... Up 0.0.0.0:6379->6379/tcp
32+
```
33+
34+
- **State:** Shows if container is running/stopped
35+
- **Ports:** Reveals port mappings (host:container)
36+
37+
### Inspecting Container Details
38+
For deeper inspection of a specific container:
39+
40+
```bash
41+
docker inspect debug-docker_nodejs-app_1
42+
```
43+
44+
This returns a JSON with all container details including:
45+
- Network settings
46+
- Mounts
47+
- Environment variables
48+
- IP addresses
49+
50+
**Filter specific information:**
51+
52+
```bash
53+
docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' debug-docker_nodejs-app_1
54+
```
55+
56+
Let's breakdown the command:
57+
- `-f` or `--format` activates Go template formatting
58+
- `{{range .NetworkSettings.Networks}}` iterates through networks
59+
- `{{.IPAddress}}` extracts the IP for each network
60+
- `{{end}}` closes the loop
61+
62+
**Example Output:**
63+
64+
(*The container's internal IP in Docker's network*)
65+
```console
66+
172.19.0.3
67+
```
68+
69+
Docker uses **Go Templates** for the `--format` filtering in `docker inspect`. You can get more details on how to construct the filter in Go's [text/template](https://pkg.go.dev/text/template) package.
70+
Also, you can learn more on formatting output in [Format command and log output](https://docs.docker.com/engine/cli/formatting/) section.
71+
72+
## 2. Log Analysis
73+
### Viewing Container Logs
74+
75+
```bash
76+
docker compose logs
77+
```
78+
79+
This will return the logs of containers that are part of the docker compose setup. You can also get the container specific logs using the following command.
80+
81+
```bash
82+
docker compose logs nodejs-app
83+
```
84+
85+
**Example Output (when accessing the service):**
86+
87+
```console
88+
debug-docker_nodejs-app_1 | Node.js server running on port 3000
89+
```
90+
91+
This confirms your Node.js application launched successfully.
92+
93+
### Generating Request Logs
94+
Make a test request to generate logs:
95+
96+
```bash
97+
curl -v http://localhost:3000
98+
```
99+
100+
After the request, check the logs again to see:
101+
102+
```console
103+
debug-docker_nodejs-app_1 | Node.js server running on port 3000
104+
debug-docker_nodejs-app_1 | GET / 200 7.001 ms - 19
105+
```
106+
107+
To view only GET requests from the last 5 minutes
108+
```bash
109+
docker compose logs --since 5m nodejs-app | grep "GET"
110+
```
111+
112+
**Example Output:**
113+
114+
```console
115+
GET / 200 7.001 ms - 19
116+
```
117+
118+
## 3. Network Troubleshooting
119+
### Checking Container Connectivity
120+
121+
Test if Nginx can reach Node.js:
122+
123+
```bash
124+
docker exec debug-docker_nginx_1 ping nodejs-app
125+
```
126+
127+
- `docker exec`: Executes a command inside a running container (`debug-docker_nginx_1`)
128+
- `ping nodejs-app`: Calls the Linux `ping` utility to test network reachability to the hostname `nodejs-app`
129+
130+
**Example Output:**
131+
132+
```console
133+
PING nodejs-app (172.19.0.3): 56 data bytes
134+
64 bytes from 172.19.0.3: seq=0 ttl=64 time=0.060 ms
135+
```
136+
137+
- **Success:** <1ms response confirms network connectivity
138+
- **Failure:** Would show "unknown host" or timeout
139+
140+
Docker Compose automatically assigns hostnames to containers based on the service names defined in `docker-compose.yml`.
141+
142+
Example:
143+
144+
```yaml
145+
services:
146+
nodejs-app: # This becomes the hostname
147+
image: node:alpine
148+
```
149+
150+
Run `docker compose ps` to see the exact service/container names:
151+
152+
```bash
153+
docker compose ps --format "table {{.Name}}\t{{.Service}}"
154+
```
155+
Output:
156+
157+
```console
158+
NAME SERVICE
159+
debug-docker_nginx_1 nginx
160+
debug-docker_nodejs-app_1 nodejs-app
161+
```
162+
163+
Alternatively, you can use service names directly from `docker-compose.yml` instead of full container name.
164+
165+
```bash
166+
docker compose exec nginx ping nodejs-app
167+
```
168+
169+
**Example Output:**
170+
171+
```console
172+
PING nodejs-app (172.19.0.3): 56 data bytes
173+
64 bytes from 172.19.0.3: seq=0 ttl=64 time=0.042 ms
174+
```
175+
176+
### Examining Post Accessibility
177+
Check if Node.js is listening on port 3000 inside its container:
178+
179+
```bash
180+
docker exec nodejs-app netstat -tuln
181+
```
182+
183+
- `docker exec`: Runs a command inside a specific container (`nodejs-app`).
184+
- `netstat -tuln`: A Linux utility to list all listening network ports with the flags:
185+
- `-t`: Show TCP ports
186+
- `-u`: Show UDP ports
187+
- `-l`: Display only listening ports (services accepting connections)
188+
- `-n`: Show numeric addresses/ports
189+
190+
**Example Output:**
191+
```console
192+
Active Internet connections (only servers)
193+
Proto Recv-Q Send-Q Local Address Foreign Address State
194+
tcp 0 0 :::3000 :::* LISTEN
195+
```
196+
197+
- Shows Node.js listening on port 3000
198+
- No output means the service isn't running properly
199+
200+
## 4. Interactive Debugging
201+
### Executing into Containers
202+
203+
For Node.js application debugging:
204+
```bash
205+
docker exec -it nodejs-app sh
206+
```
207+
208+
This gives you full shell access inside the container where you can run any Linux command (as long as the tool exists in the container). Following are few things that you can do.
209+
210+
1. Check running processes: `ps aux`
211+
```bash
212+
ps aux
213+
```
214+
215+
**Output:**
216+
217+
```console
218+
PID USER TIME COMMAND
219+
1 root 0:00 {MainThread} node app.js
220+
28 root 0:00 sh
221+
45 root 0:00 ps aux
222+
```
223+
- Shows all running processes in the container
224+
- `PID1`: Your Node.js application (node app.js)
225+
- `PID28`: The shell session you just started
226+
- `PID 45`: The `ps aux` command itself
227+
- Confirms your application is running as the main process
228+
229+
2. Examine environment variables: `printenv`
230+
```bash
231+
printenv
232+
```
233+
234+
**Output:**
235+
```console
236+
NODE_VERSION=23.11.0
237+
HOSTNAME=ada88201c429
238+
YARN_VERSION=1.22.22
239+
SHLVL=1
240+
HOME=/root
241+
TERM=xterm
242+
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
243+
PWD=/app
244+
```
245+
- Displays all environment variables to your Node.js app
246+
247+
3. Test Redis connectivity: `redis-cli -h redis ping`
248+
```bash
249+
redis-cli -h redis ping
250+
```
251+
252+
**Output:**
253+
```console
254+
PONG
255+
```
256+
257+
- Tests connectivity to your Redis container
258+
- `PONG` response confirms network connectivity
259+
260+
### Debugging Nginx Configuration
261+
```bash
262+
docker exec -it debug-docker_nginx_1 nginx -t
263+
```
264+
265+
**Example Output:**
266+
```console
267+
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
268+
nginx: configuration file /etc/nginx/nginx.conf test is successful
269+
```
270+
271+
## 5. Health Checks and Readiness Probes
272+
Docker health checks are automated tests that periodically verify if a container is functioning properly. Health checks transform your containers from static processes into self aware services that can catch issues like application crashes, frozen processes, dependency failures, etc.
273+
Let's enhance our `docker-compose.yml` with health checks:
274+
275+
```yaml
276+
services:
277+
nodejs-app:
278+
# ... existing config ...
279+
healthcheck:
280+
test: ["CMD", "curl", "-f", "http://localhost:3000"]
281+
interval: 30s
282+
timeout: 10s
283+
retries: 3
284+
285+
redis:
286+
# ... existing config ...
287+
healthcheck:
288+
test: ["CMD", "redis-cli", "ping"]
289+
interval: 30s
290+
timeout: 5s
291+
retries: 3
292+
```
293+
294+
- Docker runs the `test` command at your specified `interval` (e.g., every 30s)
295+
- The service is marked `healthy` only if the command succeeds (exit code 0)
296+
- After `retries` consecutive failures, it's marked `unhealthy`
297+
298+
Now check the container health:
299+
```bash
300+
docker ps --format "table {{.Names}}\t{{.Status}}"
301+
```
302+
303+
**Example Output:**
304+
```console
305+
NAMES STATUS
306+
debug-docker_nginx_1 Up 5 minutes
307+
debug-docker_nodejs-app_1 Up 5 minutes (healthy)
308+
debug-docker_redis_1 Up 5 minutes (healthy)
309+
```
310+
311+
## 6. Temporary Debug Containers
312+
Sometimes you need additional tools that are not part of the containers you are using, for that create a temporary debug container with the needed tools in the same network:
313+
```bash
314+
docker run -it --rm --network debug-docker_default alpine sh
315+
```
316+
317+
Now from this container you can:
318+
1. Test DNS resolution:
319+
```bash
320+
nslookup nodejs-app
321+
```
322+
323+
**Example Output:**
324+
```console
325+
Server: 127.0.0.11
326+
Address: 127.0.0.11:53
327+
328+
Non-authoritative answer:
329+
Name: nodejs-app
330+
Address: 172.19.0.3
331+
```
332+
333+
- Confirms Docker's internal DNS is working
334+
- Shows the service name resolves to the correct container IP (172.19.03)
335+
336+
2. Check connectivity:
337+
```bash
338+
wget -qO- http://nodejs-app:3000
339+
```
340+
341+
**Example Output:**
342+
```console
343+
Hello from Node.js!
344+
```
345+
346+
- Shows successful TCP connection to port 3000
347+
- Returns the actual HTTP response from your Node.js app
348+
349+
## 7. Docker System Diagnostics
350+
When facing resource issues:
351+
```bash
352+
docker system df
353+
```
354+
355+
**Example Output:**
356+
```console
357+
TYPE TOTAL ACTIVE SIZE RECLAIMABLE
358+
Images 17 3 2.374GB 2.295GB (96%)
359+
Containers 3 3 2B 0B (0%)
360+
Local Volumes 2 1 88B 88B (100%)
361+
Build Cache 108 0 37.82MB 37.82MB
362+
```
363+
364+
Check detailed resource usage:
365+
```bash
366+
docker stats
367+
```
368+
369+
**Example Output:**
370+
```console
371+
CONTAINER ID NAME CPU % MEM USAGE/LIMIT MEM % NET I/O BLOCK I/O PIDS
372+
5111f46d640b debug-docker_nginx_1 0.00% 9.633MiB/31.06GiB 0.03% 41.9kB/126B 0B/4.1kB 13
373+
e701e4d02bb0 debug-docker_nodejs-app_1 0.00% 13.31MiB/31.06GiB 0.04% 45.3kB/3.49kB 0B/0B 7
374+
3e0399cc7510 debug-docker_redis_1 0.93% 4.691MiB/31.06GiB 0.01% 42.3kB/126B 1.43MB/0B 6
375+
```
376+
377+
## Conclusion
378+
Effective Docker debugging requires a systematic approach. By mastering these techniques, you'll be able to diagnose and resolve even the most complex Docker issues in production environments.
327 KB
Loading

0 commit comments

Comments
 (0)