Got "unexpected fault address" when torrent downloading

[penglongli]

I created a p2p network on my local area network, seeded the content on machine A, and downloaded it on other machines.

The other machines panicked while downloading the torrent:

unexpected fault address 0x7f210e6d9000
fatal error: fault
unexpected fault address 0x7f20f3033020
fatal error: fault
unexpected fault address 0x7f20dc00b020
fatal error: fault
unexpected fault address 0x7f20d0377020
fatal error: fault
unexpected fault address 0x7f21125bf020
fatal error: fault
unexpected fault address 0x7f20dfd1b020
fatal error: fault
unexpected fault address 0x7f20fd4c3020
fatal error: fault
[signal SIGBUS: bus error code=0x2 addr=0x7f210e6d9000 pc=0x15a3320]

goroutine 928696 gp=0xc002eeb880 m=55 mp=0xc003506008 [running]:
runtime.throw({0x4002865?, 0x300000002?})
        /root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.3.linux-amd64/src/runtime/panic.go:1101 +0x48 fp=0xc002890ed0 sp=0xc002890ea0 pc=0x1599c48
runtime.sigpanic()
        /root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.3.linux-amd64/src/runtime/signal_unix.go:922 +0x10a fp=0xc002890f30 sp=0xc002890ed0 pc=0x159bf0a
runtime.memmove()
        /root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.3.linux-amd64/src/runtime/memmove_amd64.s:390 +0x460 fp=0xc002890f38 sp=0xc002890f30 pc=0x15a3320
github.com/anacrolix/torrent/mmap_span.copyBytes(...)
        /image-proxy/vendor/github.com/anacrolix/torrent/mmap_span/mmap_span.go:79
github.com/anacrolix/torrent/mmap_span.(*MMapSpan).locateCopy.func1(0x0?, {0x0?, 0xc002891068?})
        /image-proxy/vendor/github.com/anacrolix/torrent/mmap_span/mmap_span.go:86 +0xc6 fp=0xc002890fc8 sp=0xc002890f38 pc=0x2f3e066
github.com/anacrolix/torrent/segments.Index.Locate.func2(0x1536614?, {0x1536614?, 0x4b6?})
        /image-proxy/vendor/github.com/anacrolix/torrent/segments/index.go:60 +0x1e fp=0xc002890ff0 sp=0xc002890fc8 pc=0x2f3be3e
github.com/anacrolix/torrent/segments.ScanConsecutive(0xc002891090, {0x1597d9f?, 0x58?}, 0xc0028910a8)
        /image-proxy/vendor/github.com/anacrolix/torrent/segments/segments.go:54 +0xf1 fp=0xc002891048 sp=0xc002890ff0 pc=0x2f3c091
github.com/anacrolix/torrent/segments.Index.Locate({{0xc0027e1950?, 0x100000?, 0x100000?}}, {0x100000?, 0x4b6?}, 0xc006a890e0?)
        /image-proxy/vendor/github.com/anacrolix/torrent/segments/index.go:59 +0x10f fp=0xc0028910d0 sp=0xc002891048 pc=0x2f3bdaf
github.com/anacrolix/torrent/mmap_span.(*MMapSpan).locateCopy(0xab2e038e?, 0x4415e80?, {0xc00883c000, 0x4000, 0x4000}, 0xc002891218?)
        /image-proxy/vendor/github.com/anacrolix/torrent/mmap_span/mmap_span.go:83 +0x78 fp=0xc002891140 sp=0xc0028910d0 pc=0x2f3df38
github.com/anacrolix/torrent/mmap_span.(*MMapSpan).WriteAt(0xc0028911e0?, {0xc00883c000?, 0x4000, 0x4b6?}, 0xc000793420?)
        /image-proxy/vendor/github.com/anacrolix/torrent/mmap_span/mmap_span.go:102 +0xd4 fp=0xc0028911b8 sp=0xc002891140 pc=0x2f3e274
github.com/anacrolix/missinggo/v2.(*SectionWriter).WriteAt(0x6?, {0xc00883c000?, 0xc002891268?, 0x396ffa0?}, 0xc002891260?)
        /image-proxy/vendor/github.com/anacrolix/missinggo/v2/section_writer.go:22 +0x4c fp=0xc0028911f0 sp=0xc0028911b8 pc=0x2ef5f0c
github.com/anacrolix/torrent/storage.(*mmapStoragePiece).WriteAt(0x396ffa0?, {0xc00883c000?, 0xc002891280?, 0x15b0603?}, 0xc005c9c288?)
        <autogenerated>:1 +0x2a fp=0xc002891228 sp=0xc0028911f0 pc=0x2f6408a
github.com/anacrolix/torrent/storage.Piece.WriteAt({{0x4445580?, 0xc00201afc0?}, {0xc000793420?, 0x31de36d?}}, {0xc00883c000, 0x4000, 0x4000}, 0x7c000)
        /image-proxy/vendor/github.com/anacrolix/torrent/storage/wrappers.go:85 +0x123 fp=0xc0028912a0 sp=0xc002891228 pc=0x2f619e3
github.com/anacrolix/torrent.(*Torrent).writeChunk(0xc000052a70?, 0x2f40525?, 0x7c000, {0xc00883c000, 0x4000, 0x4000})
        /image-proxy/vendor/github.com/anacrolix/torrent/torrent.go:1077 +0x69 fp=0xc0028912f0 sp=0xc0028912a0 pc=0x3203a09
github.com/anacrolix/torrent.(*Peer).receiveChunk.func6(0xc000052a08, 0xc002891638, 0xc001bf0a88, 0xc00309f360)
        /image-proxy/vendor/github.com/anacrolix/torrent/peer.go:718 +0xec fp=0xc002891378 sp=0xc0028912f0 pc=0x31de20c
github.com/anacrolix/torrent.(*Peer).receiveChunk(0xc0025e0a88, 0xc00309f360)
        /image-proxy/vendor/github.com/anacrolix/torrent/peer.go:719 +0x93a fp=0xc0028916d0 sp=0xc002891378 pc=0x31dd87a
github.com/anacrolix/torrent.(*PeerConn).mainReadLoop(0xc0025e0a88)
        /image-proxy/vendor/github.com/anacrolix/torrent/peerconn.go:837 +0xa05 fp=0xc0028919e0 sp=0xc0028916d0 pc=0x31e4bc5
github.com/anacrolix/torrent.(*Torrent).runHandshookConn(0xc001bf0a88, 0xc0025e0a88)
        /image-proxy/vendor/github.com/anacrolix/torrent/client.go:1131 +0x51a fp=0xc002891ba8 sp=0xc0028919e0 pc=0x31ce55a
github.com/anacrolix/torrent.(*Torrent).logRunHandshookConn(0xc001bf0a88, 0x0?, 0x0, {0x0?})
        /image-proxy/vendor/github.com/anacrolix/torrent/torrent.go:1899 +0x39 fp=0xc002891df0 sp=0xc002891ba8 pc=0x3209539
github.com/anacrolix/torrent.(*Torrent).runHandshookConnLoggingErr(...)
        /image-proxy/vendor/github.com/anacrolix/torrent/torrent.go:1906
github.com/anacrolix/torrent.(*Client).outgoingConnection(0xc000052a08, {{{0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...}, ...}, ...}, ...)
        /image-proxy/vendor/github.com/anacrolix/torrent/client.go:907 +0x327 fp=0xc002891f70 sp=0xc002891df0 pc=0x31cc467
github.com/anacrolix/torrent.initiateConn.gowrap1()
        /image-proxy/vendor/github.com/anacrolix/torrent/torrent.go:2719 +0x3e fp=0xc002891fe0 sp=0xc002891f70 pc=0x321211e
runtime.goexit({})
        /root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.3.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc002891fe8 sp=0xc002891fe0 pc=0x15a20e1
created by github.com/anacrolix/torrent.initiateConn in goroutine 928667
        /image-proxy/vendor/github.com/anacrolix/torrent/torrent.go:2719 +0x2a5

Version

1.58.1

Torrent Client Init

clientConfig := torrent.NewDefaultClientConfig()
clientConfig.DataDir = th.op.TorrentPath
clientConfig.Seed = true
clientConfig.ListenPort = int(th.op.TorrentPort)
clientConfig.DefaultStorage = storage.NewMMap(th.op.TorrentPath)
clientConfig.DisableUTP = true
clientConfig.MaxUnverifiedBytes = 4096 << 20
clientConfig.PieceHashersPerTorrent = 4
clientConfig.NoDHT = true
clientConfig.DisablePEX = false
clientConfig.EstablishedConnsPerTorrent = 200
clientConfig.HalfOpenConnsPerTorrent = 100
clientConfig.TotalHalfOpenConns = 500
clientConfig.TorrentPeersHighWater = 2000
clientConfig.TorrentPeersLowWater = 200
clientConfig.MaxAllocPeerRequestDataPerConn = 4 << 20
clientConfig.DialRateLimiter = rate.NewLimiter(100, 200)
clientConfig.DisableAcceptRateLimiting = true
clientConfig.AcceptPeerConnections = true
tc, err := torrent.NewClient(clientConfig)
if err != nil {
        return errors.Wrapf(err, "create torrent client failed")
}
th.client = tc
th.pc, err = storage.NewDefaultPieceCompletionForDir(".")
if err != nil {
        return errors.Wrapf(err, "new piece completion for dir '%s' failed", th.op.TorrentPath)
}

How-to-generate-torrent

pieceLength := metainfo.ChoosePieceLength(fi.Size())
info := metainfo.Info{
        PieceLength: pieceLength,
}
if err = info.BuildFromFilePath(layerFile); err != nil {
        return nil, errors.Wrapf(err, "build torrent metainfo from file '%s' failed", layerFile)
}
mi := metainfo.MetaInfo{
        InfoBytes: bencode.MustMarshal(info),
}
ih := mi.HashInfoBytes()
to, _ := th.client.AddTorrentOpt(torrent.AddTorrentOpts{
        InfoHash: ih,
        Storage:  storage.NewMMapWithCompletion(th.op.TorrentPath, th.pc),
        ChunkSize: 131072,
})
if err = to.MergeSpec(&torrent.TorrentSpec{
        DisplayName: digest,
        InfoBytes:   mi.InfoBytes,
        Trackers:    [][]string{{th.op.TorrentAnnounce}}, // Custom Torrent Announce URL
}); err != nil {
        return nil, errors.Wrapf(err, "setting trackers failed")
}
logctx.Infof(ctx, "generate torrent success")

serveMi := serveTo.Metainfo()
var buffer bytes.Buffer
if err = serveMi.Write(&buffer); err != nil {
        return "", errors.Wrapf(err, "get torrent bytes failed")
}
torrentBase64 = base64.StdEncoding.EncodeToString(buffer.Bytes())
logctx.Infof(ctx, "generate serve torrent success")

Download torrent

torrentBytes, err := base64.StdEncoding.DecodeString(torrentBase64)
if err != nil {
        return true, 0, errors.Wrapf(err, "base64 decode '%s' failed", torrentBase64)
}
mi, err := metainfo.Load(bytes.NewBuffer(torrentBytes))
if err != nil {
        return true, 0, errors.Wrapf(err, "load metainfo '%s' failed", torrentBase64)
}
t, err := th.client.AddTorrent(mi)
if err != nil {
        return true, 0, errors.Wrapf(err, "add torrent '%s' failed", torrentBase64)
}
if err = th.gotTorrentInfo(t); err != nil {
        return true, 0, err
}
// ignore chunk error
t.SetOnWriteChunkError(func(err error) {})
t.DownloadAll()
logctx.Infof(ctx, "torrent start downloading")
start := time.Now()
done := make(chan struct{})

interval := 5 * time.Second
ticker := time.NewTicker(interval)
defer ticker.Stop()
var currentBytes, prevBytes int64
completedSlice := make([]int64, 0)
allSize := formatutils.FormatSize(t.Length())
for {
        currentBytes = t.BytesCompleted()
        byteRate := (currentBytes - prevBytes) * int64(time.Second) / int64(interval)
        var progress float64 = 0
        if t.Length() != 0 {
                progress = float64(currentBytes) / float64(t.Length()) * 100
        }
        select {
        case <-ticker.C:
                logctx.Infof(ctx, "torrent downloading(%v): %s/%s: %v/s, completed(bytes): %.2f%%",
                        time.Since(start),
                        formatutils.FormatSize(currentBytes),
                        allSize,
                        formatutils.FormatSize(byteRate),
                        float64(t.BytesCompleted())/float64(t.Length())*100,
                )
                if currentBytes == t.Length() {
                        close(done)
                        break
                }
                completedSlice = append(completedSlice, currentBytes)
                prevBytes = currentBytes

        case <-ctx.Done():
                return true, 0, errors.Errorf("download torrent context exceeded")
        case <-done:
                logctx.Infof(ctx, "torrent download completed")
                return true, 0, nil
        }
}

[anacrolix]

Could you provide your OS, and architecture? go env if possible.

Have you tried with v1.59.1?

In more recent versions, a mix of mmap and file storage is the default, you may find that works better. Any reason you are using the mmap storage?

[penglongli]

Could you provide your OS, and architecture? go env if possible.

Have you tried with v1.59.1?

In more recent versions, a mix of mmap and file storage is the default, you may find that works better. Any reason you are using the mmap storage?

Runtime OS:

Linux VM-120-218-centos 5.4.119-19-0013_plus #1 SMP Mon Jan 16 10:26:09 CST 2023 x86_64 x86_64 x86_64 GNU/Linux

Golang Build Runtime:

GO111MODULE='on'
GOARCH='amd64'
GOBIN=''
GOCACHE='/root/.cache/go-build'
GOENV='/root/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/root/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/root/go'
GOPRIVATE=''
GOROOT='/usr/local/go'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/usr/local/go/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.22.5'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='1'
GOMOD='/dev/null'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build213288463=/tmp/go-build -gno-record-gcc-switches'

I've read that using mmap is a more performant approach, but I don't necessarily have a preference for storage.

Are there any best practices for generate torrents and downloading torrents source code? If there are best practices, I can learn and improve my code.

[anacrolix]

I would just switch to the regular storage, and/or the latest version of anacrolix/torrent.

[penglongli]

I would just switch to the regular storage, and/or the latest version of anacrolix/torrent.

I'll try the version v1.59.1.

Then I have another question. After my machine A generates a torrent, when another machine B downloads it, it sometimes takes 60 seconds to reach the download speed. How can I solve this problem?

I1024 16:13:21.309893   1 proxy.go:363] download layer with torrent is starting, t-request=a0455b95-32cd-4e56-8bd9-551e6b8fbc0a
I1024 16:13:21.324664   1 torrent.go:487] torrent start downloading, t-request=a0455b95-32cd-4e56-8bd9-551e6b8fbc0a
I1024 16:13:26.325432   1 torrent.go:512] torrent downloading(5.00064666s): 0.00B /204.79MB: 0.00B /s, completed(bytes): 0.00%, t-request=a0455b95-32cd-4e56-8bd9-551e6b8fbc0a
I1024 16:13:31.325526   1 torrent.go:512] torrent downloading(10.000744576s): 0.00B /204.79MB: 0.00B /s, completed(bytes): 0.00%, t-request=a0455b95-32cd-4e56-8bd9-551e6b8fbc0a
I1024 16:13:36.325887   1 torrent.go:512] torrent downloading(15.00113416s): 0.00B /204.79MB: 0.00B /s, completed(bytes): 0.00%, t-request=a0455b95-32cd-4e56-8bd9-551e6b8fbc0a
I1024 16:13:41.325514   1 torrent.go:512] torrent downloading(20.00073793s): 0.00B /204.79MB: 0.00B /s, completed(bytes): 0.00%, t-request=a0455b95-32cd-4e56-8bd9-551e6b8fbc0a
I1024 16:13:46.325751   1 torrent.go:512] torrent downloading(25.000975892s): 0.00B /204.79MB: 0.00B /s, completed(bytes): 0.00%, t-request=a0455b95-32cd-4e56-8bd9-551e6b8fbc0a
I1024 16:13:51.325541   1 torrent.go:512] torrent downloading(30.000764861s): 0.00B /204.79MB: 0.00B /s, completed(bytes): 0.00%, t-request=a0455b95-32cd-4e56-8bd9-551e6b8fbc0a
I1024 16:13:56.325557   1 torrent.go:512] torrent downloading(35.000777667s): 0.00B /204.79MB: 0.00B /s, completed(bytes): 0.00%, t-request=a0455b95-32cd-4e56-8bd9-551e6b8fbc0a
I1024 16:14:01.325874   1 torrent.go:512] torrent downloading(40.001097225s): 0.00B /204.79MB: 0.00B /s, completed(bytes): 0.00%, t-request=a0455b95-32cd-4e56-8bd9-551e6b8fbc0a
I1024 16:14:06.325493   1 torrent.go:512] torrent downloading(45.000711351s): 0.00B /204.79MB: 0.00B /s, completed(bytes): 0.00%, t-request=a0455b95-32cd-4e56-8bd9-551e6b8fbc0a
I1024 16:14:11.325564   1 torrent.go:512] torrent downloading(50.000780725s): 0.00B /204.79MB: 0.00B /s, completed(bytes): 0.00%, t-request=a0455b95-32cd-4e56-8bd9-551e6b8fbc0a
I1024 16:14:16.325476   1 torrent.go:512] torrent downloading(55.000693093s): 0.00B /204.79MB: 0.00B /s, completed(bytes): 0.00%, t-request=a0455b95-32cd-4e56-8bd9-551e6b8fbc0a
I1024 16:14:21.324940   1 torrent.go:512] torrent downloading(1m0.00011245s): 0.00B /204.79MB: 0.00B /s, completed(bytes): 4.11%, t-request=a0455b95-32cd-4e56-8bd9-551e6b8fbc0a
I1024 16:14:26.324898   1 torrent.go:512] torrent downloading(1m5.00047461s): 8.42MB/204.79MB: 1.68MB/s, completed(bytes): 56.55%, t-request=a0455b95-32cd-4e56-8bd9-551e6b8fbc0a
I1024 16:14:31.324838   1 torrent.go:512] torrent downloading(1m10.00069447s): 115.81MB/204.79MB: 21.47MB/s, completed(bytes): 81.29%, t-request=a0455b95-32cd-4e56-8bd9-551e6b8fbc0a
I1024 16:14:36.325601   1 torrent.go:512] torrent downloading(1m15.00034687s): 166.48MB/204.79MB: 10.13MB/s, completed(bytes): 100.00%, t-request=a0455b95-32cd-4e56-8bd9-551e6b8fbc0a
I1024 16:14:41.324971   1 torrent.go:512] torrent download completed, t-request=a0455b95-32cd-4e56-8bd9-551e6b8fbc0a

[buryni]

I'm hitting the same SIGBUS with the default file storage (not NewMMap), via the mmapFileIo code path in file-io-mmap.go.

Stack trace (v1.61.0, Linux amd64, Go 1.26):

unexpected fault address 0x738964198000
fatal error: fault
[signal SIGBUS: bus error code=0x2 addr=0x738964198000 pc=0x71b590]

crypto/sha1.blockSHANI(...)
    .../go/src/crypto/sha1/sha1block_amd64.s:1825
crypto/sha1.(*digest).Write(...)
    .../go/src/crypto/sha1/sha1.go:146
io.(*multiWriter).Write(...)
storage.(*mmapFileHandle).writeToN(...)
    .../torrent@v1.61.0/storage/file-io-mmap.go:289
storage.(*filePieceImpl).writeFileTo(...)
    .../torrent@v1.61.0/storage/file-piece.go:390
...
(*Torrent).hashPiece → (*Torrent).pieceHasher
    .../torrent@v1.61.0/torrent.go:1280

Root cause: mmapFileIo.openForWrite calls f.Truncate(size) while a concurrent pieceHasher goroutine is reading from the same file through a separate mmap handle (writeToN at line 289). The truncation invalidates the mmap region, causing SIGBUS when the hasher reads beyond the new file end.

Workaround: Setting TORRENT_STORAGE_DEFAULT_FILE_IO=classic before the process starts switches to classicFileIo (regular os.File ops), which handles truncation gracefully (returns io.EOF instead of crashing). This env var is read by file-io-mmap.go:init().

This happens under moderate concurrency (~5 parallel torrents being partially downloaded), with piece hashing running on multiple goroutines simultaneously.

[anacrolix]

Could you create unit tests that reproduce this?