Commit c2ce760
enhance: add timeout resilience to Docker acquisition (socket proxy errors) (#3873)
* feat: add timeout resilience to Docker acquisition
- Add configurable retry mechanism for Docker log readers
- Implement per-container log options to prevent race conditions
- Add intelligent consecutive failure tracking with reset logic
- Use typed errors for better failure classification
- Preserve EOF-resilient behavior with proper Since timestamp management
Fixes socket proxy timeout issues that previously caused containers
to be permanently removed from monitoring after temporary failures.
* fix: address linting errors in docker acquisition
- Fix indent-error-flow: remove unnecessary else clause
- Fix early-return: invert conditions to reduce nesting
- Improve code readability and follow Go best practices
* feat: add configurable retry mechanism for Docker acquisition
- Add RetryBackoffDuration config option (default: 5s)
- Fix race condition with shared Since timestamp by creating per-container log options
- Implement container/service health checks via Docker API instead of log activity
- Add EOF-resilient retry logic that distinguishes between container death and network issues
- Simplify retry mechanism: indefinite retries for healthy containers with backoff
- Fix test mocks to properly initialize ContainerState and add ServiceInspectWithRaw
Fixes timeout issues when using Docker socket proxies by retrying failed connections
while ensuring containers that are actually stopped are removed from monitoring.
* enhance: remove retry count as we rely on container status
* enhance: add log once reconnected and remove retry code since we know its running
* enhance: clear up log confusion
* fix: docker tests, what a mess
* fix: docker tests, love you too CI
* fix: docker tests, love you too CI
* dry / extract constructor from closures
* wip
* backoff * 3
* simpler tests
* deps: update github.com/cenkalti/backoff
* lint
---------
Co-authored-by: marco <[email protected]>1 parent 757c323 commit c2ce760
File tree
4 files changed
+428
-196
lines changed- pkg/acquisition/modules/docker
4 files changed
+428
-196
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
20 | 20 | | |
21 | 21 | | |
22 | 22 | | |
23 | | - | |
| 23 | + | |
24 | 24 | | |
25 | 25 | | |
26 | 26 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
109 | 109 | | |
110 | 110 | | |
111 | 111 | | |
112 | | - | |
113 | | - | |
| 112 | + | |
| 113 | + | |
114 | 114 | | |
115 | 115 | | |
116 | 116 | | |
| |||
0 commit comments