fix: preprocessing_data crash for favorite/collection API mode - #764
Open
WenhuaXia wants to merge 2 commits into
Open
fix: preprocessing_data crash for favorite/collection API mode#764WenhuaXia wants to merge 2 commits into
WenhuaXia wants to merge 2 commits into
Conversation
审阅者指南调整抖音账号预处理逻辑，以在 API 模式下正确处理“喜欢/收藏”标签页：将其逻辑与普通作品处理分离，并确保会拉取用户信息；当基于 API 模式下
|
| 变更 | 详情 | 文件 |
|---|---|---|
| 为“喜欢/收藏”标签页单独拆分预处理路径，并在基于 sec_uid 的选择失败时提供健壮的回退方案。 |
|
src/extract/extractor.py |
| 确保在 API 模式下，“喜欢/收藏”标签页仍会获取用户信息，以便为预处理提供元数据。 |
|
src/application/main_terminal.py |
提示与命令
与 Sourcery 交互
- 触发新的审查： 在 Pull Request 中评论
@sourcery-ai review。 - 继续讨论： 直接回复 Sourcery 的审查评论。
- 从审查评论生成 GitHub Issue： 回复 Sourcery 的审查评论，请求其基于该评论创建 Issue。你也可以直接回复该评论
@sourcery-ai issue来从中创建 Issue。 - 生成 Pull Request 标题： 在 Pull Request 标题的任意位置写上
@sourcery-ai，即可随时生成标题。你也可以在 Pull Request 中评论@sourcery-ai title来（重新）生成标题。 - 生成 Pull Request 概要： 在 Pull Request 正文任意位置写上
@sourcery-ai summary，即可在对应位置生成 PR 概要。你也可以在 Pull Request 中评论@sourcery-ai summary来在任意时间（重新）生成概要。 - 生成审阅者指南： 在 Pull Request 中评论
@sourcery-ai guide，即可在任意时间（重新）生成审阅者指南。 - 一次性解决所有 Sourcery 评论： 在 Pull Request 中评论
@sourcery-ai resolve，即可标记解决所有 Sourcery 评论。如果你已经处理了所有评论且不想再看到它们，这会很有用。 - 一次性忽略所有 Sourcery 审查： 在 Pull Request 中评论
@sourcery-ai dismiss，即可忽略所有现有的 Sourcery 审查。特别适合当你希望从一次全新的审查开始——别忘了再评论@sourcery-ai review触发新的审查！
自定义你的使用体验
打开你的 控制面板 以：
- 启用或停用审查功能，比如 Sourcery 自动生成的 Pull Request 概要、审阅者指南等。
- 更改审查语言。
- 添加、删除或编辑自定义审查说明。
- 调整其他审查相关设置。
获取帮助
Original review guide in English
Reviewer's Guide
Adjusts Douyin account preprocessing to correctly handle favorite/collection tabs in API mode by separating their logic from normal post handling and ensuring user info is fetched, with a graceful fallback when sec_uid-based matching fails.
Sequence diagram for deal_account_detail user info fetching in API mode
sequenceDiagram
actor Client
participant MainTerminal
participant UserInfoService
Client->>MainTerminal: deal_account_detail(api, tab)
alt [api and tab not in {favorite, collection}]
MainTerminal->>MainTerminal: info = None
else [not api or tab in {favorite, collection}]
MainTerminal->>UserInfoService: get_user_info_data(user_id)
UserInfoService-->>MainTerminal: info
end
MainTerminal-->>Client: processed account detail
Flow diagram for preprocessing_data handling favorite/collection tabs
flowchart TD
A[preprocessing_data called] --> B{mode}
B -->|favorite or collection| C[try __select_item with sec_uid]
C --> D[__extract_pretreatment_data with uid and nickname]
D --> E[return id_, name, mark]
C -->|DownloaderError| F[cleaner.filter_name mark user_id]
F --> G[return user_id, filtered_name, filtered_name]
B -->|other modes| H[existing processing paths]
File-Level Changes
| Change | Details | Files |
|---|---|---|
| Separate preprocessing path for favorite/collection tabs with a robust fallback when sec_uid-based selection fails. |
|
src/extract/extractor.py |
| Ensure user info is still fetched in API mode for favorite/collection tabs so metadata is available for preprocessing. |
|
src/application/main_terminal.py |
Tips and commands
Interacting with Sourcery
- Trigger a new review: Comment
@sourcery-ai reviewon the pull request. - Continue discussions: Reply directly to Sourcery's review comments.
- Generate a GitHub issue from a review comment: Ask Sourcery to create an
issue from a review comment by replying to it. You can also reply to a
review comment with@sourcery-ai issueto create an issue from it. - Generate a pull request title: Write
@sourcery-aianywhere in the pull
request title to generate a title at any time. You can also comment
@sourcery-ai titleon the pull request to (re-)generate the title at any time. - Generate a pull request summary: Write
@sourcery-ai summaryanywhere in
the pull request body to generate a PR summary at any time exactly where you
want it. You can also comment@sourcery-ai summaryon the pull request to
(re-)generate the summary at any time. - Generate reviewer's guide: Comment
@sourcery-ai guideon the pull
request to (re-)generate the reviewer's guide at any time. - Resolve all Sourcery comments: Comment
@sourcery-ai resolveon the
pull request to resolve all Sourcery comments. Useful if you've already
addressed all the comments and don't want to see them anymore. - Dismiss all Sourcery reviews: Comment
@sourcery-ai dismisson the pull
request to dismiss all existing Sourcery reviews. Especially useful if you
want to start fresh with a new review - don't forget to comment
@sourcery-ai reviewto trigger a new review!
Customizing Your Experience
Access your dashboard to:
- Enable or disable review features such as the Sourcery-generated pull request
summary, the reviewer's guide, and others. - Change the review language.
- Add, remove or edit custom review instructions.
- Adjust other review settings.
Getting Help
- Contact our support team for questions or feedback.
- Visit our documentation for detailed guides and information.
- Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey - 我发现了 1 个问题，并且留下了一些整体性的反馈：
- 在新的
case "favorite" | "collection"分支中，多次重复使用(self.extract_params_tiktok if tiktok else self.extract_params)[...]查找会让代码更难阅读和维护；可以考虑先把选择好的参数字典赋值给一个局部变量，然后重复使用它。 except DownloaderError:代码块会悄悄吞掉所有下载器错误，并通过返回一个兜底元组改变原有行为；如果可能的话，尽量收窄异常捕获范围，或者至少记录/追踪原始错误以便调试。- 在
favorite/collection的兜底返回中，name和mark字段都来源于self.cleaner.filter_name(mark, user_id)；请再次确认这两个输出是否确实应该完全相同，还是说mark应该保留不同的语义信息。
面向 AI Agent 的提示
Please address the comments from this code review:
## Overall Comments
- In the new `case "favorite" | "collection"` branch, the repeated `(self.extract_params_tiktok if tiktok else self.extract_params)[...]` lookups make the code harder to read and maintain; consider assigning the chosen params dict to a local variable and reusing it.
- The `except DownloaderError:` block silently swallows all downloader errors and changes behavior by returning a fallback tuple; if possible, narrow the exception scope or at least log/trace the original error to aid debugging.
- In the fallback return for `favorite`/`collection`, both the `name` and `mark` fields are derived from `self.cleaner.filter_name(mark, user_id)`; double-check whether these two outputs are intended to be identical or if `mark` should preserve a different semantic value.
## Individual Comments
### Comment 1
<location path="src/extract/extractor.py" line_range="898" />
<code_context>
mark,
)
return id_, name, mark
+ case "favorite" | "collection":
+ try:
+ item = self.__select_item(
</code_context>
<issue_to_address>
**issue (complexity):** 考虑引入一个局部的 params 变量，并在需要时再抽取一个辅助方法，以避免在新的 favorite/collection 分支中重复相同的三元表达式和参数列表。
你可以通过引入本地 `params` 变量并“拉平”调用结构，来减少这个新分支中的重复代码和嵌套。这能在保持行为不变的前提下，让代码更容易阅读和维护。
例如，将下面的代码：
```python
case "favorite" | "collection":
try:
item = self.__select_item(
data,
user_id,
(self.extract_params_tiktok if tiktok else self.extract_params)[
"sec_uid"
],
)
id_, name, mark = self.__extract_pretreatment_data(
item,
(self.extract_params_tiktok if tiktok else self.extract_params)[
"uid"
],
(self.extract_params_tiktok if tiktok else self.extract_params)[
"nickname"
],
mark,
)
return id_, name, mark
except DownloaderError:
return (
user_id,
self.cleaner.filter_name(mark, user_id),
self.cleaner.filter_name(mark, user_id),
)
```
重构为：
```python
case "favorite" | "collection":
params = self.extract_params_tiktok if tiktok else self.extract_params
try:
item = self.__select_item(
data,
user_id,
params["sec_uid"],
)
id_, name, mark = self.__extract_pretreatment_data(
item,
params["uid"],
params["nickname"],
mark,
)
return id_, name, mark
except DownloaderError:
cleaned = self.cleaner.filter_name(mark, user_id)
return user_id, cleaned, cleaned
```
如果在其他分支中也存在类似模式，你还可以抽取一个小的辅助方法，进一步统一流程：
```python
def _pretreat_user_item(self, data, user_id, params, mark):
item = self.__select_item(data, user_id, params["sec_uid"])
return self.__extract_pretreatment_data(
item,
params["uid"],
params["nickname"],
mark,
)
```
然后在分支中这么写：
```python
case "favorite" | "collection":
params = self.extract_params_tiktok if tiktok else self.extract_params
try:
id_, name, mark = self._pretreat_user_item(data, user_id, params, mark)
return id_, name, mark
except DownloaderError:
cleaned = self.cleaner.filter_name(mark, user_id)
return user_id, cleaned, cleaned
```
</issue_to_address>帮我变得更有用！请在每条评论上点 👍 或 👎，我会根据你的反馈来改进后续的评审。
Original comment in English
Hey - I've found 1 issue, and left some high level feedback:
- In the new
case "favorite" | "collection"branch, the repeated(self.extract_params_tiktok if tiktok else self.extract_params)[...]lookups make the code harder to read and maintain; consider assigning the chosen params dict to a local variable and reusing it. - The
except DownloaderError:block silently swallows all downloader errors and changes behavior by returning a fallback tuple; if possible, narrow the exception scope or at least log/trace the original error to aid debugging. - In the fallback return for
favorite/collection, both thenameandmarkfields are derived fromself.cleaner.filter_name(mark, user_id); double-check whether these two outputs are intended to be identical or ifmarkshould preserve a different semantic value.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- In the new `case "favorite" | "collection"` branch, the repeated `(self.extract_params_tiktok if tiktok else self.extract_params)[...]` lookups make the code harder to read and maintain; consider assigning the chosen params dict to a local variable and reusing it.
- The `except DownloaderError:` block silently swallows all downloader errors and changes behavior by returning a fallback tuple; if possible, narrow the exception scope or at least log/trace the original error to aid debugging.
- In the fallback return for `favorite`/`collection`, both the `name` and `mark` fields are derived from `self.cleaner.filter_name(mark, user_id)`; double-check whether these two outputs are intended to be identical or if `mark` should preserve a different semantic value.
## Individual Comments
### Comment 1
<location path="src/extract/extractor.py" line_range="898" />
<code_context>
mark,
)
return id_, name, mark
+ case "favorite" | "collection":
+ try:
+ item = self.__select_item(
</code_context>
<issue_to_address>
**issue (complexity):** Consider introducing a local params variable and possibly a helper method to avoid repeating the same ternary and argument lists in the new favorite/collection case branch.
You can reduce duplication and nesting in the new branch by introducing a local `params` variable and flattening the calls. This keeps behavior identical while making the code easier to scan and maintain.
For example, refactor this:
```python
case "favorite" | "collection":
try:
item = self.__select_item(
data,
user_id,
(self.extract_params_tiktok if tiktok else self.extract_params)[
"sec_uid"
],
)
id_, name, mark = self.__extract_pretreatment_data(
item,
(self.extract_params_tiktok if tiktok else self.extract_params)[
"uid"
],
(self.extract_params_tiktok if tiktok else self.extract_params)[
"nickname"
],
mark,
)
return id_, name, mark
except DownloaderError:
return (
user_id,
self.cleaner.filter_name(mark, user_id),
self.cleaner.filter_name(mark, user_id),
)
```
into:
```python
case "favorite" | "collection":
params = self.extract_params_tiktok if tiktok else self.extract_params
try:
item = self.__select_item(
data,
user_id,
params["sec_uid"],
)
id_, name, mark = self.__extract_pretreatment_data(
item,
params["uid"],
params["nickname"],
mark,
)
return id_, name, mark
except DownloaderError:
cleaned = self.cleaner.filter_name(mark, user_id)
return user_id, cleaned, cleaned
```
If similar patterns exist in other cases, you could also extract a small helper to further standardize the flow:
```python
def _pretreat_user_item(self, data, user_id, params, mark):
item = self.__select_item(data, user_id, params["sec_uid"])
return self.__extract_pretreatment_data(
item,
params["uid"],
params["nickname"],
mark,
)
```
and then:
```python
case "favorite" | "collection":
params = self.extract_params_tiktok if tiktok else self.extract_params
try:
id_, name, mark = self._pretreat_user_item(data, user_id, params, mark)
return id_, name, mark
except DownloaderError:
cleaned = self.cleaner.filter_name(mark, user_id)
return user_id, cleaned, cleaned
```
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
- Extract repeated conditional expression into `params` variable - Add warning log when falling back to cleaned user info - Fix mark fallback to use filtered name as default instead of user_id Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bug: When calling `/douyin/account` API with `tab=favorite` or `tab=collection`, the endpoint crashes with `DownloaderError: 提取账号信息或合集信息失败`.
Root Cause:
collectionmodes, video authors are original creators, not the target user — `sec_uid` never matches.Fix:
collectionin `preprocessing_data`. Add try/except fallback so when video data can't be matched, gracefully return `user_id` + `mark`.collectionmodes always query user info even in API mode, ensuring accurate metadata.Summary by Sourcery
在预处理和 API 流程中正确处理“收藏”和“合集”账号模式，以避免崩溃并确保用户元数据可用。
Bug 修复：
Original summary in English
Summary by Sourcery
Handle favorite and collection account modes correctly in preprocessing and API flows to avoid crashes and ensure user metadata is available.
Bug Fixes: