Skip to content

SIGSEGV when using get list with proxy #580

@AltayAkkus

Description

@AltayAkkus

Steps to reproduce

  1. Create a *.txt with atleast 1 URL
  2. Define a proxy (via yml or CLI)
  3. Run zeno get list urls.txt
2026-03-20T19:56:55+01:00 DEBUG worker.go:174       | worker stopped component=archiver.worker worker_id=1
2026-03-20T19:56:55+01:00 DEBUG worker.go:174       | worker stopped component=archiver.worker worker_id=0
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x8 pc=0x1012cb3e0]

goroutine 1 [running]:
github.com/internetarchive/Zeno/internal/pkg/archiver.Stop()
        /Users/altay/Desktop/Development/zeno/internal/pkg/archiver/worker.go:125 +0xa0
github.com/internetarchive/Zeno/internal/pkg/controler.stopPipeline()
        /Users/altay/Desktop/Development/zeno/internal/pkg/controler/pipeline.go:180 +0xd4
github.com/internetarchive/Zeno/internal/pkg/controler.Stop(...)
        /Users/altay/Desktop/Development/zeno/internal/pkg/controler/controler.go:13

Fails before properly closing the WARC writer, had to find this bug the hard way 🫣

Root cause

// Instantiate WARC client
var err error
if config.Get().Proxy != "" {
proxiedWARCSettings := WARCSettings
proxiedWARCSettings.Proxy = config.Get().Proxy
globalArchiver.ClientWithProxy, err = warc.NewWARCWritingHTTPClient(proxiedWARCSettings)

...
// Even if a proxied client has been set, we want to create an non-proxied one
if config.Get().Proxy == "" {
globalArchiver.Client, err = warc.NewWARCWritingHTTPClient(WARCSettings)

If proxy != "" a globalArchiver.ClientWithProxy will be created, but despite the comment globalArchiver.Client will be nil

But the archiver/worker.go expects the globalArchiver.Client to not be nil, causing the crash.

globalArchiver.Client.WaitGroup.Wait()
stopLocalWatcher <- struct{}{}
logger.Debug("WARC writing finished")
globalArchiver.Client.Close()
if globalArchiver.ClientWithProxy != nil {
globalArchiver.ClientWithProxy.WaitGroup.Wait()
globalArchiver.ClientWithProxy.Close()
}

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions