Refactor(Network):用 HttpClient 代替已经过时的 WebRequest、WebClient、ServicePointManager by LinQingYuu · Pull Request #6407 · Meloong-Git/PCL

LinQingYuu · 2025-05-26T05:47:38Z

PR 改动

技术性更改

将 WebRequest 和 WebClient 替换为 HttpClient 以实现更高的网络性能

支持在发送网络请求时读取 GZip 等压缩格式的响应以节约双方的网络传输成本

可能修复了以下 Bug

Resolve #6418

特别感谢（重构期间帮了大忙）

@wuliaodexiaoluo （帮助解决了标头误用的问题）
@sulingjiang（提出了可以实现继承了 IWebProxy 类来动态修改代理的想法）
@qiuyue712（协助测试）
@jiangyanyue（协助测试期间偶然发现了 CTS 超时会作用于整个请求流程，以及一个下载的 Bug）

pref(ModNet):使用 HttpClient 替代过时的 WebRequest 和 WebClient

LinQingYuu · 2025-05-27T08:25:56Z

先上了一点测试，发现了比较有意思的现象，使用 HttpClient 的版本下载速度要快于使用 WebRequest 的版本，包括我还拿了自己写的核心库和官版比较，结果还是核心库更快一不止点（而且核心库还是一个文件一个线程，一分钟就装完了，PCL 卡半天还没好）

~~暂不知道为什么会这样子~~

经过验证，似乎是 WebRequest 性能太差了....

sync

CO-authored-by: qiuyue712 <qiuyue712@outlook.com>

MoYuan-CN · 2025-06-02T03:13:26Z

是否可以同时 Resolve 掉 #6467? @shimoranla

LinQingYuu · 2025-06-02T03:27:29Z

是否可以同时 Resolve 掉 #6467? @shimoranla

尝试在 WebRequest 下设置 CachePolicy 为 Revalidate，完全没有 304 响应，看起来 WebRequest 不会加 If-None-Match 或 If-Modified-Since 标头

如果需要自己处理的话那这玩意折腾起来比较坐牢，需要在加载器的缓存保留 ETag 标头，这样整个地方都要改成使用 HttpRequestMessage 才能正确处理 304 响应

看龙猫短期内有没有意向做掉，如果没有的话我再考虑折腾

cc @LTCatt

LinQingYuu · 2025-06-02T05:35:44Z

为啥 Build Failed 了？

~~有无大佬救下？~~

算了 Rebase 得了

CO-Authored-by: jiangyanyue <jiangyanyueawa@outlook.com>

ghost · 2025-06-02T09:12:18Z

在能稳定复现的计算机上使用构建产物进行全量下载，速度恢复正常（基本吃满带宽）

LTCatt · 2025-06-03T07:34:41Z

看龙猫短期内有没有意向做掉，如果没有的话我再考虑折腾

这个和 #6467 都是优先的（

baiyuexiao496

这下面一堆 Return 改 Exit Sub 真的对吗.....

CO-Authored-By: baiyuexiao496 <mibaiyuexiao@outlook.com>

Co-authored-by: tangge233 <50769997+tangge233@users.noreply.github.com>

LTCatt · 2025-07-13T17:35:49Z

因为没注释，问点问题……

HttpClientFactory 为什么要 6 个小时更换一个？
HttpWebProxyFactory 是什么作用？
GetRequestMessage 应当同样可以使用 HttpClient，就像 SendRequest 一样？

多谢！

LinQingYuu · 2025-07-14T01:42:21Z

因为没注释，问点问题……

(寄)

HttpClientFactory 为什么要 6 个小时更换一个？

因为微软文档上写的 HttpClient 实际上会一直存储域名解析信息，如果用户开长一点就会出现缓存解析导致连接出问题，所以当时写的时候做了轮换

不过合并前一段时间看了 ce 日志，这东西底层还是 HttpWebRequest 貌似不会这个有问题，所以理论上也可以删掉换成单例

(.net fw 只是加了异步，网络栈是在 .NET Core 重写的，这些我后来才了解到）

btw: HttpClientFactory 没有被更换，只是更换了内部的 Client

HttpWebProxyFactory 是什么作用？

不清楚能不能在 UseProxy=True 的情况下不传 Proxy，所以保险起见直接打了个来确保代理正常工作

做这个类目的之一是适配 #5770 可以手动更换代理地址，正如我在 https://github.com/Meloong-Git/PCL/issues/6418#issuecomment-2916523442提到的那样，因为没办法在实例化 HttpClient 后动态修改实例内部的 Handler，所以代理地址会被硬编码在里面直到 6 小时后被销毁然后重新创建

GetRequestMessage 应当同样可以使用 HttpClient，就像 SendRequest 一样？

这个只是觉得每个 Client 都得来个 .GetAwaiter.GetResult() 重复太多了所以糊了个类弄这些东西

~~重新看了一遍代码，感觉当初完全就是在写更大的石山~~

LTCatt

血压有点高……

LTCatt · 2025-07-14T09:00:31Z

+        Catch ex As ThreadInterruptedException
+                Throw
+            Catch ex As TaskCanceledException
+                Return New WebException($"连接服务器超时，请检查你的网络环境是否良好（{ex.Message}，{Url}）", ex)


Return New WebException???????
WTF?????????

Return New WebException??????? WTF?????????

当时应该是忘记关掉 Copilot 的补全了 Orz

LTCatt · 2025-07-14T09:00:49Z

    ''' <param name="ContentType">请求的套接字类型。</param>
    ''' <param name="DontRetryOnRefused">当返回 40x 时不重试。</param>
-    Public Function NetRequestRetry(Url As String, Method As String, Data As Object, ContentType As String, Optional DontRetryOnRefused As Boolean = True, Optional Headers As Dictionary(Of String, String) = Nothing) As String
+    Public Function NetRequestRetry(Url As String, Method As String, Data As Object, ContentType As String, Optional DontRetryOnRefused As Boolean = True, Optional Headers As Dictionary(Of String, String) = Nothing, Optional RequireReturnRespObj As Boolean = False) As String


RequireReturnRespObj 永远是 false，why?

看了一圈只有多线程引擎那里用了 RequireReturnRespObj 参数，这应该把多线程引擎那里单独写一份，搞不懂为啥要在 NetRequestRetry 加这个参数

LTCatt · 2025-07-14T09:01:34Z

+        ' 获取HttpMethod
+        Private Function GetHttpMethod(Method As String) As HttpMethod
+            Select Case Method?.Trim().ToUpperInvariant()
+                Case "HEAD"
+                    Return HttpMethod.Head
+                Case "POST"
+                    Return HttpMethod.Post
+                Case "PUT"
+                    Return HttpMethod.Put
+                Case "DELETE"
+                    Return HttpMethod.Delete
+                Case Else
+                    Return HttpMethod.Get
+            End Select
+        End Function


它有自带方法……

New HttpMethod(Method.Trim.ToUpper)

LTCatt · 2025-07-14T09:05:13Z

+                Dim Client = ClientFactory.GetHttpClient()
+                Dim Resp As HttpResponseMessage = Client.SendAsync(Req, HttpCompletionOption.ResponseHeadersRead, CTS.Token).GetAwaiter().GetResult()
+
+                If RequireReturnRespObj Then Return Resp


我没有找到任何一个地方需要用到 RequireReturnRespObj 的，这一堆处理是什么原因？

LTCatt · 2025-07-14T09:06:28Z

+    Private Request As New HttpRequest()
+    Public Class HttpRequest
+        Public Sub New()
+        End Sub


它只有静态的 Function，为什么要做成单例？

它只有静态的 Function，为什么要做成单例？

CSharp 写习惯了所以干啥都 Public Class Orz

LTCatt · 2025-07-14T09:08:43Z

-            SecretHeadersSign(Url, Request, UseBrowserUserAgent)
-            Return Request.DownloadString(Url)
+            Using Req As HttpRequestMessage = Request.GetRequestMessage(Url, Headers:=Headers, RequireHeaderSign:=True, RequireCdnSign:=True, UseBrowserUA:=UseBrowserUserAgent)
+                Req.Headers.AcceptLanguage.Add(New StringWithQualityHeaderValue("en-US,en;q=0.5"))


完全运行不了，在跑 OptiFine 版本列表获取时报错。

Suggested change

Req.Headers.AcceptLanguage.Add(New StringWithQualityHeaderValue("en-US,en;q=0.5"))

Req.Headers.AcceptLanguage.Add(New StringWithQualityHeaderValue("en-US", 0.5))

完全运行不了，在跑 OptiFine 版本列表获取时报错。

草为什么我本地测没问题啊（

加载线程 DlOptiFineList Official (36) 出错，已完成 2%：获取结果失败，值“en-US,en”的格式无效。（https://optifine.net/downloads）在 PCL.ModNet.NetGetCodeByClient(String Url, Encoding Encoding, String Accept, Boolean UseBrowserUserAgent) 位置 F:\Projects\Plain Craft Launcher 2\本体代码\Plain Craft Launcher 2\Modules\Base\ModNet.vb:行号 131 在 PCL.ModDownload.DlOptiFineListOfficialMain(LoaderTask`2 Loader) 位置 F:\Projects\Plain Craft Launcher 2\本体代码\Plain Craft Launcher 2\Modules\Minecraft\ModDownload.vb:行号 380 在 PCL.ModLoader.LoaderTask`2._Closure$__13-0._Lambda$__0() 位置 F:\Projects\Plain Craft Launcher 2\本体代码\Plain Craft Launcher 2\Modules\Base\ModLoader.vb:行号 319 错误类型：System.Net.WebException → 值“en-US,en”的格式无效。在 System.Net.Http.Headers.HeaderUtilities.CheckValidToken(String value, String parameterName) 在 System.Net.Http.Headers.StringWithQualityHeaderValue..ctor(String value, Double quality) 在 PCL.ModNet.NetGetCodeByClient(String Url, Encoding Encoding, Int32 Timeout, String Accept, Boolean UseBrowserUserAgent) 位置 F:\Projects\Plain Craft Launcher 2\本体代码\Plain Craft Launcher 2\Modules\Base\ModNet.vb:行号 140 错误类型：System.FormatException

LTCatt · 2025-07-14T09:09:12Z

        Using Client As New WebClient
            Try
-                SecretHeadersSign(Url, Client, UseBrowserUserAgent)
+                'SecretHeadersSign(Url, Client, UseBrowserUserAgent)


Why?

GetRequestMessage 会自己做这个任务（

但是这里根本没有调 GetRequestMessage 啊？

但是这里根本没有调 GetRequestMessage 啊？

不建议代码挤一起的原因 +1：容易漏掉某些东西 Orz

LTCatt · 2025-07-14T09:10:23Z

                        Throw New WebException($"获取到的分段大小不一致：Range 起始于 {Info.DownloadStart}，预期 ContentLength 为 {FileSize - Info.DownloadStart}，返回 ContentLength 为 {ContentLength}，总文件大小 {FileSize}")
                    End If
-                    'Log($"[Download] {LocalName} {Info.Uuid}#：通过大小检查，文件大小 {FileSize}，起始点 {Info.DownloadStart}，ContentLength {ContentLength}")
+                    Log($"[Download] {LocalName} {Info.Uuid}#：通过大小检查，文件大小 {FileSize}，起始点 {Info.DownloadStart}，ContentLength {ContentLength}")


这里是刻意注释的，这玩意儿会打一大堆 Log

LTCatt · 2025-07-14T09:10:31Z

                    State = NetState.Merge
                Else
-                    Return
+                    Exit Sub


???

这后面一堆应该是当时让 Copilot 重新整理代码代码的时候 Copilot 批量替换了，没仔细审代码 Orz

不是，这应该是把我某个版本之后的更改全部回滚了……

LTCatt · 2025-07-14T09:10:40Z

+                    Dim Info As New FileInfo(LocalPath)
+                    Info.Directory.Create()


LinQingYuu · 2025-07-14T09:15:21Z

好像写混了，当时没认真看 Orz

LTCatt

我在扫雷高级取得了一整天的好成绩，你也来试试吧.jpg

LTCatt · 2025-07-14T13:01:35Z

+                        If RequestMessage.Content.Headers.Contains(Header.Key) Then Continue For
+                        If Not RequestMessage.Content.Headers.Contains(Header.Key) Then


那我问你那我问你，它还能不是 True 的吗

LTCatt · 2025-07-14T13:01:48Z

+                        If Data Is Nothing OrElse RequestMessage.Content Is Nothing Then Continue For
+                        If RequestMessage.Content.Headers.Contains(Header.Key) Then Continue For
+                        If Not RequestMessage.Content.Headers.Contains(Header.Key) Then
+                            If Header.Key.ToLower() = "Content-Type" Then RequestMessage.Content.Headers.ContentType = New MediaTypeHeaderValue(Header.Value)


.ToLower() = "Content-Type"

LTCatt · 2025-07-14T13:32:24Z

                Else
-                    ex = New ResponsedWebException($"服务器返回错误（{ex.Status}，{ex.Message}，{Url}）{vbCrLf}{Res}", Res, ex)
+                    If Not ReqHeaders.ContainsKey("Content-Type") Then ReqHeaders.Add("Content-Type", ContentType)
                End If


这一堆全没啦？
ResponsedWebException 在判密码错误的时候还要用的……

LinQingYuu added 2 commits April 13, 2025 15:21

pref(ModNet):使用 HttpClient 替代过时的 WebRequest 和 WebClient

74657d5

Merge pull request #49 from LuoYun-Team/modnet

9a0ad08

pref(ModNet):使用 HttpClient 替代过时的 WebRequest 和 WebClient

LinQingYuu marked this pull request as draft May 26, 2025 05:47

LinQingYuu added · 优化社区处理中社区正在调查或处理该项 labels May 26, 2025

zkitefly requested review from LTCatt and removed request for LTCatt May 26, 2025 06:18

LinQingYuu changed the title ~~[WIP] Refactor(Network):用 HttpClient 代替已经过时的 WebRequest、HttpWebRequest、WebClient、ServicePointManager~~ [WIP] Refactor(Network):用 HttpClient 代替已经过时的 WebRequest、WebClient、ServicePointManager May 28, 2025

LTCatt mentioned this pull request May 28, 2025

下载游戏时速度很慢 #6418

Closed

3 tasks

LTCatt linked an issue May 28, 2025 that may be closed by this pull request

下载游戏时速度很慢 #6418

Closed

3 tasks

Merge pull request #51 from Hex-Dragon/main

d2e2103

sync

LinQingYuu force-pushed the HttpClient branch from 8c62c11 to d2e2103 Compare June 1, 2025 08:54

Refactor(ModNet): 使用 HttpClient 代替过失函数 Feat(ModNet): 支持 GZip 等压缩格式响应的读取

5a94854

CO-authored-by: qiuyue712 <qiuyue712@outlook.com>

LinQingYuu added 2 commits June 2, 2025 13:20

revert: 错误的文件更改

9c9216c

revert: 错误的文件更改

70a2f28

LinQingYuu and others added 2 commits June 2, 2025 13:40

fix: 编译错误

3adf47a

CO-Authored-by: jiangyanyue <jiangyanyueawa@outlook.com>

chore: 弄错了

7ecf306

LinQingYuu marked this pull request as ready for review June 2, 2025 09:02

LinQingYuu changed the title ~~[WIP] Refactor(Network):用 HttpClient 代替已经过时的 WebRequest、WebClient、ServicePointManager~~ Refactor(Network):用 HttpClient 代替已经过时的 WebRequest、WebClient、ServicePointManager Jun 2, 2025

LinQingYuu requested a review from LTCatt June 2, 2025 09:13

LinQingYuu added 等待确认已经过社区确认，等待开发者确认 and removed 社区处理中社区正在调查或处理该项 labels Jun 2, 2025

LTCatt added 处理中开发者正在调查或处理该项 🟪 极高优先度：极高 and removed 等待确认已经过社区确认，等待开发者确认 labels Jun 3, 2025

baiyuexiao496 suggested changes Jun 4, 2025

View reviewed changes

Comment thread Plain Craft Launcher 2/Modules/Base/ModNet.vb Outdated

Comment thread Plain Craft Launcher 2/Modules/Base/ModNet.vb Outdated

Comment thread Plain Craft Launcher 2/Modules/Base/ModNet.vb Outdated

fix: 少了特定标头和异常抛出

6dbb5c9

CO-Authored-By: baiyuexiao496 <mibaiyuexiao@outlook.com>

LinQingYuu force-pushed the HttpClient branch from 7478d4d to 6dbb5c9 Compare June 7, 2025 06:19

LinQingYuu mentioned this pull request Jun 20, 2025

feat(Network): 下载之前进行 Prefetch #6573

Closed

chore: 删除无用代码

65812b8

tangge233 suggested changes Jun 23, 2025

View reviewed changes

Comment thread Plain Craft Launcher 2/Modules/Base/ModNet.vb Outdated

Update Plain Craft Launcher 2/Modules/Base/ModNet.vb

83e34b8

Co-authored-by: tangge233 <50769997+tangge233@users.noreply.github.com>

LTCatt merged commit f98dcd8 into Meloong-Git:main Jul 13, 2025
2 checks passed

HexDragon-Bot added 完成已被处理，将在下次更新之后生效 and removed 处理中开发者正在调查或处理该项 labels Jul 13, 2025

LinQingYuu deleted the HttpClient branch July 13, 2025 14:02

LTCatt reviewed Jul 14, 2025

View reviewed changes

LinQingYuu mentioned this pull request Jul 15, 2025

更新Curseforge源的模组，会返回403禁止报错 #6578

Closed

3 tasks

LTCatt mentioned this pull request Jul 20, 2025

feat(mcim): 重新启用 MCIM 镜像源 #6657

Closed

	Req.Headers.AcceptLanguage.Add(New StringWithQualityHeaderValue("en-US,en;q=0.5"))
	Req.Headers.AcceptLanguage.Add(New StringWithQualityHeaderValue("en-US", 0.5))

		If RequestMessage.Content.Headers.Contains(Header.Key) Then Continue For
		If Not RequestMessage.Content.Headers.Contains(Header.Key) Then

Uh oh!

Conversation

LinQingYuu commented May 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR 改动

技术性更改

可能修复了以下 Bug

特别感谢（重构期间帮了大忙）

Uh oh!

LinQingYuu commented May 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MoYuan-CN commented Jun 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

LinQingYuu commented Jun 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

LinQingYuu commented Jun 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ghost commented Jun 2, 2025

Uh oh!

LTCatt commented Jun 3, 2025

Uh oh!

baiyuexiao496 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

LTCatt commented Jul 13, 2025

Uh oh!

LinQingYuu commented Jul 14, 2025

Uh oh!

LTCatt left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

LinQingYuu Jul 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

LinQingYuu commented May 26, 2025 •

edited

Loading

LinQingYuu commented May 27, 2025 •

edited

Loading

MoYuan-CN commented Jun 2, 2025 •

edited

Loading

LinQingYuu commented Jun 2, 2025 •

edited

Loading

LinQingYuu commented Jun 2, 2025 •

edited

Loading

LinQingYuu Jul 14, 2025 •

edited

Loading