Steps to reproduce
- Create a *.txt with atleast 1 URL
- Define a proxy (via yml or CLI)
- Run
zeno get list urls.txt
2026-03-20T19:56:55+01:00 DEBUG worker.go:174 | worker stopped component=archiver.worker worker_id=1
2026-03-20T19:56:55+01:00 DEBUG worker.go:174 | worker stopped component=archiver.worker worker_id=0
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x8 pc=0x1012cb3e0]
goroutine 1 [running]:
github.com/internetarchive/Zeno/internal/pkg/archiver.Stop()
/Users/altay/Desktop/Development/zeno/internal/pkg/archiver/worker.go:125 +0xa0
github.com/internetarchive/Zeno/internal/pkg/controler.stopPipeline()
/Users/altay/Desktop/Development/zeno/internal/pkg/controler/pipeline.go:180 +0xd4
github.com/internetarchive/Zeno/internal/pkg/controler.Stop(...)
/Users/altay/Desktop/Development/zeno/internal/pkg/controler/controler.go:13
Fails before properly closing the WARC writer, had to find this bug the hard way 🫣
Root cause
|
// Instantiate WARC client |
|
var err error |
|
if config.Get().Proxy != "" { |
|
proxiedWARCSettings := WARCSettings |
|
proxiedWARCSettings.Proxy = config.Get().Proxy |
|
globalArchiver.ClientWithProxy, err = warc.NewWARCWritingHTTPClient(proxiedWARCSettings) |
...
|
// Even if a proxied client has been set, we want to create an non-proxied one |
|
if config.Get().Proxy == "" { |
|
globalArchiver.Client, err = warc.NewWARCWritingHTTPClient(WARCSettings) |
If
proxy != "" a
globalArchiver.ClientWithProxy will be created, but despite the comment
globalArchiver.Client will be nil
But the archiver/worker.go expects the globalArchiver.Client to not be nil, causing the crash.
|
globalArchiver.Client.WaitGroup.Wait() |
|
stopLocalWatcher <- struct{}{} |
|
logger.Debug("WARC writing finished") |
|
globalArchiver.Client.Close() |
|
if globalArchiver.ClientWithProxy != nil { |
|
globalArchiver.ClientWithProxy.WaitGroup.Wait() |
|
globalArchiver.ClientWithProxy.Close() |
|
} |
Steps to reproduce
zeno get list urls.txtFails before properly closing the WARC writer, had to find this bug the hard way 🫣
Root cause
Zeno/internal/pkg/archiver/warc.go
Lines 63 to 68 in dfedbc7
...
Zeno/internal/pkg/archiver/warc.go
Lines 81 to 83 in dfedbc7
If
proxy != ""aglobalArchiver.ClientWithProxywill be created, but despite the commentglobalArchiver.Clientwill be nilBut the
archiver/worker.goexpects theglobalArchiver.Clientto not be nil, causing the crash.Zeno/internal/pkg/archiver/worker.go
Lines 125 to 132 in dfedbc7