fix: tighten tutorial validation and site metadata#49
Open
WtxwNs wants to merge 1 commit into
Open
Conversation
Billy1900
approved these changes
Jul 12, 2026
Billy1900
left a comment
Billy1900
left a comment
Contributor
There was a problem hiding this comment.
我对于其中的内容改进没有任何意见；基本都是 语句改写+章数纠正
| # 第 6 章 · 策略梯度方法 | ||
|
|
||
| 第 4 章走了路线一——先学 $Q(s,a)$ 给每个动作打分，再选分数最高的（回顾：[Q(s,a) 与贪心策略](../chapter03_mdp/value-q)）。DQN 在 CartPole 和 Atari 上效果很好，但它有一个根本性的局限：**只能处理有限个离散动作。** | ||
| 第 4 章走了路线一——先学 $Q(s,a)$ 给每个动作打分，再选分数最高的（回顾：[Q(s,a) 与贪心策略](../chapter03_mdp/value-q)）。经典 DQN 在 CartPole 和 Atari 上效果很好，但它需要枚举网络输出的所有动作再取最大值，因此直接适用于有限离散动作空间。 |
| 第 4 章走了路线一——先学 $Q(s,a)$ 给每个动作打分，再选分数最高的（回顾：[Q(s,a) 与贪心策略](../chapter03_mdp/value-q)）。经典 DQN 在 CartPole 和 Atari 上效果很好，但它需要枚举网络输出的所有动作再取最大值，因此直接适用于有限离散动作空间。 | ||
|
|
||
| CartPole 只有"左推"和"右推"两个选项，DQN 给每个动作算一个 $Q$ 值，取最大的就行。但如果要控制一个机械臂呢？肩关节、肘关节、腕关节各有多个自由度，每个自由度能施加连续的力矩——可能的动作组合有无穷多种，不可能给每种组合都算一个 $Q$ 值。更不用说大模型生成文本——每一步要从几万个 token 中采样，策略本身是一个连续的概率分布，$\arg\max$ 的思路完全行不通。 | ||
| CartPole 只有"左推"和"右推"两个选项，DQN 给每个动作算一个 $Q$ 值，取最大的就行。但如果要控制一个机械臂呢？肩关节、肘关节、腕关节各有多个自由度，每个自由度都能施加连续力矩，无法为无限多个动作逐一输出 $Q$ 值。大模型的词表虽然有限，但生成需要在庞大的序列状态空间中逐步决策，还要保留随机采样能力，因此也更适合直接参数化策略分布。 |
Author
|
改的都是我对着学的时候一眼没太看懂的地方，和一些太绝对的表述 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
scripts/verify.shno longer losesset -euon Windows and falsely reports successbaseValidation
npx prettier --checkon all changed Markdown/JavaScript filesnpm run lintnpm run buildnpm run sitemap(355 published pages; excluded targets absent)Existing baseline issue
npm run verifynow fails correctly instead of falsely passing. It currently stops at the repository-wideprettier --check .because 406 pre-existing files do not match the configured Prettier format. This PR intentionally does not reformat the entire repository.