fix: non-range HTTP downloaded bytes exceeding total file size (#1317)

Author: monkeyWie

README.md

@@ -16,7 +16,7 @@
 
 ## 🚀 Introduction
 
-Gopeed (full name Go Speed), a high-speed downloader developed by `Golang` + `Flutter`, supports (HTTP, BitTorrent, Magnet) protocol, and supports all platforms. In addition to basic download functions, Gopeed is also a highly customizable downloader that supports implementing more features through integration with [APIs](https://gopeed.com/docs/dev-api) or installation and development of [extensions](https://gopeed.com/docs/dev-extension).
+Gopeed (full name Go Speed), a high-speed downloader developed by `Golang` + `Flutter`, supports (HTTP, BitTorrent, Magnet, ED2K) protocol, and supports all platforms. In addition to basic download functions, Gopeed is also a highly customizable downloader that supports implementing more features through integration with [APIs](https://gopeed.com/docs/dev-api) or installation and development of [extensions](https://gopeed.com/docs/dev-extension).
 
 Visit ✈ [Official Website](https://gopeed.com) | 📖 [Official Docs](https://gopeed.com/docs)
 

README_ja-JP.md

@@ -16,7 +16,7 @@
 
 ## 🚀 はじめに
 
-Gopeed (正式名 Go Speed) は `Golang` + `Flutter` によって開発された高速ダウンローダーで、(HTTP、BitTorrent、Magnet) プロトコルをサポートし、すべてのプラットフォームをサポートします。基本的なダウンロード機能に加え、[APIs](https://gopeed.com/docs/dev-api)との連動や[拡張機能](https://gopeed.com/docs/dev-extension)のインストール・開発による追加機能にも対応した、カスタマイズ性の高いダウンローダーです。
+Gopeed (正式名 Go Speed) は `Golang` + `Flutter` によって開発された高速ダウンローダーで、(HTTP、BitTorrent、Magnet、ED2K) プロトコルをサポートし、すべてのプラットフォームをサポートします。基本的なダウンロード機能に加え、[APIs](https://gopeed.com/docs/dev-api)との連動や[拡張機能](https://gopeed.com/docs/dev-extension)のインストール・開発による追加機能にも対応した、カスタマイズ性の高いダウンローダーです。
 
 見て下さい ✈ [公式ウェブサイト](https://gopeed.com) | 📖 [開発ドキュメント](https://gopeed.com/docs)
 

README_vi-VN.md

@@ -16,7 +16,7 @@
 
 ## 🚀 Giới thiệu
 
-Gopeed (tên đầy đủ Go Speed), một công cụ tải xuống tốc độ cao được phát triển bởi `Golang` + `Flutter`, hỗ trợ giao thức (HTTP, BitTorrent, Magnet) và hỗ trợ tất cả các nền tảng. Ngoài các chức năng tải xuống cơ bản, Gopeed còn là một công cụ tải xuống có thể tùy chỉnh cao cho phép triển khai thêm tính năng thông qua việc tích hợp với [APIs](https://gopeed.com/docs/dev-api) hoặc cài đặt và phát triển các [tiện ích mở rộng](https://gopeed.com/docs/dev-extension).
+Gopeed (tên đầy đủ Go Speed), một công cụ tải xuống tốc độ cao được phát triển bởi `Golang` + `Flutter`, hỗ trợ giao thức (HTTP, BitTorrent, Magnet, ED2K) và hỗ trợ tất cả các nền tảng. Ngoài các chức năng tải xuống cơ bản, Gopeed còn là một công cụ tải xuống có thể tùy chỉnh cao cho phép triển khai thêm tính năng thông qua việc tích hợp với [APIs](https://gopeed.com/docs/dev-api) hoặc cài đặt và phát triển các [tiện ích mở rộng](https://gopeed.com/docs/dev-extension).
 
 Truy cập ✈ [Trang web chính thức](https://gopeed.com) | 📖 [Tài liệu chính thức](https://gopeed.com/docs)
 

README_zh-CN.md

@@ -16,7 +16,7 @@
 
 ## 🚀 介绍
 
-Gopeed（全称 Go Speed），直译过来中文名叫做`够快下载器`（不是狗屁下载器！），是一款由`Golang`+`Flutter`开发的高速下载器，支持（HTTP、BitTorrent、Magnet）协议下载，并且支持全平台使用。除了基本的下载功能外，Gopeed 还是一款高度可定制化的下载器，支持通过对接[APIs](https://gopeed.com/docs/dev-api)或者安装和开发[扩展](https://gopeed.com/docs/dev-extension)来实现更多的功能。
+Gopeed（全称 Go Speed），直译过来中文名叫做`够快下载器`（不是狗屁下载器！），是一款由`Golang`+`Flutter`开发的高速下载器，支持（HTTP、BitTorrent、Magnet、ED2K）协议下载，并且支持全平台使用。除了基本的下载功能外，Gopeed 还是一款高度可定制化的下载器，支持通过对接[APIs](https://gopeed.com/docs/dev-api)或者安装和开发[扩展](https://gopeed.com/docs/dev-extension)来实现更多的功能。
 
 访问 ✈ [官方网站](https://gopeed.com/zh-CN) | 📖 [官方文档](https://gopeed.com/docs)
 

README_zh-TW.md

@@ -16,7 +16,7 @@
 
 ## 🚀 簡介
 
-Gopeed（全稱 Go Speed），是一款使用`Golang`+`Flutter`編寫的高速下載軟體，支援（HTTP、BitTorrent、Magnet）協定，同時支援所有的平台。
+Gopeed（全稱 Go Speed），是一款使用`Golang`+`Flutter`編寫的高速下載軟體，支援（HTTP、BitTorrent、Magnet、ED2K）協定，同時支援所有的平台。
 
 前往 ✈ [主頁](https://gopeed.com/zh-CN) | 📖 [文檔](https://gopeed.com/docs)
 

internal/protocol/http/fetcher.go

@@ -618,6 +618,7 @@ func (f *Fetcher) doStart() error {
 		for _, conn := range f.connections {
 			// Reset connections that can be retried
 			if !conn.Completed && conn.State != connCompleted {
+				f.resetConnectionForRestart(conn)
 				conn.State = connNotStarted
 				conn.failed = false
 				conn.retryTimes = 0
@@ -1413,6 +1414,27 @@ func (f *Fetcher) helpOtherConnection(helper *connection) bool {
 	return true
 }
 
+func (f *Fetcher) resetConnectionForRestart(conn *connection) {
+	if f.meta.Res.Range {
+		return
+	}
+
+	// Without range support a new request always starts from byte 0,
+	// so pause/retry must restart instead of continuing from the old offset.
+	if conn.Chunk == nil {
+		conn.Chunk = newChunk(0, 0)
+	} else {
+		conn.Chunk.Begin = 0
+		conn.Chunk.End = 0
+		conn.Chunk.Downloaded = 0
+	}
+	conn.Downloaded = 0
+	conn.Completed = false
+	conn.speed = 0
+	conn.lastSpeedCheck = 0
+	conn.lastSpeedDownload = 0
+}
+
 func (f *Fetcher) resumeConnections() {
 	// Collect connections to resume while holding the lock
 	var toResume []*connection
@@ -1436,6 +1458,7 @@ func (f *Fetcher) resumeConnections() {
 				continue
 			}
 		}
+		f.resetConnectionForRestart(conn)
 		// Reset the connection state for resume
 		conn.ctx, conn.cancel = context.WithCancel(f.ctx)
 		conn.State = connNotStarted
@@ -1475,7 +1498,16 @@ func (f *Fetcher) onDownloadComplete() {
 	// Check if all chunks are complete (no remaining bytes)
 	allChunksComplete := true
 	for _, conn := range f.connections {
-		if conn.Chunk != nil && conn.Chunk.remain() > 0 && !conn.Completed && conn.State != connCompleted {
+		needsMoreData := false
+		if f.meta.Res.Range {
+			needsMoreData = conn.Chunk != nil && conn.Chunk.remain() > 0
+		} else if f.meta.Res.Size > 0 {
+			needsMoreData = conn.Downloaded < f.meta.Res.Size
+		} else {
+			needsMoreData = !conn.Completed && conn.State != connCompleted
+		}
+
+		if needsMoreData && !conn.Completed && conn.State != connCompleted {
 			// This connection has remaining work and isn't done
 			// Check if it failed with 403 (server limit) - these can be ignored if other connections completed the work
 			if conn.State == connFailed && conn.failed {

internal/protocol/http/fetcher_test.go

@@ -299,6 +299,53 @@ func TestFetcher_DownloadContinue(t *testing.T) {
 	downloadContinue(listener, 16, t)
 }
 
+func TestFetcher_DownloadContinue_NoRangeRestart(t *testing.T) {
+	listener := test.StartTestNoRangeSlowServer(time.Millisecond)
+	defer listener.Close()
+
+	fetcher := downloadReady(listener, 4, t)
+	if err := fetcher.Start(); err != nil {
+		t.Fatal(err)
+	}
+
+	time.Sleep(20 * time.Millisecond)
+
+	stats := fetcher.Stats().(*http.Stats)
+	if len(stats.Connections) != 1 {
+		t.Fatalf("expected a single non-range connection, got %d", len(stats.Connections))
+	}
+	if stats.Connections[0].Downloaded <= 0 || stats.Connections[0].Downloaded >= test.BuildSize {
+		t.Fatalf("expected partial download before pause, got %d", stats.Connections[0].Downloaded)
+	}
+
+	if err := fetcher.Pause(); err != nil {
+		t.Fatal(err)
+	}
+	if err := fetcher.Start(); err != nil {
+		t.Fatal(err)
+	}
+	if err := fetcher.Wait(); err != nil {
+		t.Fatal(err)
+	}
+
+	finalStats := fetcher.Stats().(*http.Stats)
+	if len(finalStats.Connections) != 1 {
+		t.Fatalf("expected a single non-range connection after resume, got %d", len(finalStats.Connections))
+	}
+	if finalStats.Connections[0].Downloaded != test.BuildSize {
+		t.Fatalf("downloaded bytes should restart cleanly: got %d, want %d", finalStats.Connections[0].Downloaded, test.BuildSize)
+	}
+	if total := fetcher.Progress().TotalDownloaded(); total != test.BuildSize {
+		t.Fatalf("progress total = %d, want %d", total, test.BuildSize)
+	}
+
+	want := test.FileMd5(test.BuildFile)
+	got := test.FileMd5(test.DownloadFile)
+	if want != got {
+		t.Errorf("Download() got = %v, want %v", got, want)
+	}
+}
+
 func TestFetcher_DownloadChunked(t *testing.T) {
 	listener := test.StartTestCustomServer()
 	defer listener.Close()
@@ -1384,8 +1431,8 @@ func TestFetcher_Patch_Extra(t *testing.T) {
 			Method: "POST", // Update method
 			// Body is empty, should NOT update
 			Header: map[string]string{
-				"X-Custom":   "modified", // Overwrite existing
-				"X-New":      "added",    // Add new
+				"X-Custom": "modified", // Overwrite existing
+				"X-New":    "added",    // Add new
 				// Authorization is not in patch, should remain
 			},
 		},

internal/test/httptest.go

@@ -335,6 +335,43 @@ func StartTestOneTimeServer() net.Listener {
 	})
 }
 
+// StartTestNoRangeSlowServer creates a server that always returns the full file
+// with Content-Length but does not support Range requests.
+func StartTestNoRangeSlowServer(delayPerChunk time.Duration) net.Listener {
+	return startTestServer(func(sl *shutdownListener) http.Handler {
+		mux := http.NewServeMux()
+		mux.HandleFunc("/"+BuildName, func(writer http.ResponseWriter, request *http.Request) {
+			writer.Header().Set("Content-Length", fmt.Sprintf("%d", BuildSize))
+
+			file, err := os.Open(BuildFile)
+			if err != nil {
+				return
+			}
+			defer file.Close()
+
+			buf := make([]byte, 256*1024)
+			for !sl.isShutdown {
+				n, readErr := file.Read(buf)
+				if n > 0 {
+					if _, writeErr := writer.Write(buf[:n]); writeErr != nil {
+						return
+					}
+					if flusher, ok := writer.(http.Flusher); ok {
+						flusher.Flush()
+					}
+					if delayPerChunk > 0 {
+						time.Sleep(delayPerChunk)
+					}
+				}
+				if readErr != nil {
+					return
+				}
+			}
+		})
+		return mux
+	})
+}
+
 // StartTestExpiringRedirectServer creates a server that simulates expiring redirect URLs.
 // The original URL redirects to a temporary URL that expires after a specified number of requests.
 // When the temporary URL expires (returns 403), the client should retry with the original URL