fix(workers): honor RSS/Atom entry date as bookmark created date - #2943
Open
TowyTowy wants to merge 1 commit into
Open
fix(workers): honor RSS/Atom entry date as bookmark created date#2943TowyTowy wants to merge 1 commit into
TowyTowy wants to merge 1 commit into
Conversation
Feed entries carry a publication date (RSS `<pubDate>`, Atom `<updated>`/`<published>`), which rss-parser normalizes into `isoDate`. The feed worker ignored it and always created bookmarks with the fetch-time default, so imported entries were dated when the feed was polled rather than when they were published. Extract the entry's published date in the feed parser and pass it as `createdAt` when creating the bookmark, falling back to the server default when the feed omits a date. This mirrors how file imports already honor their source date (`sourceAddedAt`). Fixes karakeep-app#2334 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
Greptile SummaryThis PR makes feed imports preserve entry publication dates. The main changes are:
Confidence Score: 5/5This looks safe to merge after a small date-fallback cleanup.
apps/workers/workers/utils/feedParser.ts Important Files Changed
Prompt To Fix All With AIFix the following 1 code review issue. Work through them one at a time, proposing concise fixes.
---
### Issue 1 of 1
apps/workers/workers/utils/feedParser.ts:43
**Invalid Normalized Date Blocks Fallback**
When `rss-parser` provides an `isoDate` string that is present but not parseable, this chooses it before checking `pubDate`. A feed item with a usable raw `pubDate` can then get `publishedAt` set to `undefined`, so the worker stores the bookmark with the fetch-time default instead of the entry date.
```suggestion
publishedAt: parseFeedDate(item.isoDate) ?? parseFeedDate(item.pubDate),
```
Reviews (1): Last reviewed commit: "fix(workers): honor RSS/Atom entry date ..." | Re-trigger Greptile |
| .transform((item) => ({ | ||
| ...item, | ||
| guid: item.guid ?? item.id ?? item.link, | ||
| publishedAt: parseFeedDate(item.isoDate ?? item.pubDate), |
Contributor
There was a problem hiding this comment.
Invalid Normalized Date Blocks Fallback
When rss-parser provides an isoDate string that is present but not parseable, this chooses it before checking pubDate. A feed item with a usable raw pubDate can then get publishedAt set to undefined, so the worker stores the bookmark with the fetch-time default instead of the entry date.
Suggested change
| publishedAt: parseFeedDate(item.isoDate ?? item.pubDate), | |
| publishedAt: parseFeedDate(item.isoDate) ?? parseFeedDate(item.pubDate), |
Prompt To Fix With AI
This is a comment left during a code review.
Path: apps/workers/workers/utils/feedParser.ts
Line: 43
Comment:
**Invalid Normalized Date Blocks Fallback**
When `rss-parser` provides an `isoDate` string that is present but not parseable, this chooses it before checking `pubDate`. A feed item with a usable raw `pubDate` can then get `publishedAt` set to `undefined`, so the worker stores the bookmark with the fetch-time default instead of the entry date.
```suggestion
publishedAt: parseFeedDate(item.isoDate) ?? parseFeedDate(item.pubDate),
```
How can I resolve this? If you propose a fix, please make it concise.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
RSS/Atom feed entries carry a publication date (
<pubDate>for RSS,<updated>/<published>for Atom), which rss-parser normalizes intoisoDate. The feed worker ignored it and always created bookmarks with the fetch-time default, so entries imported from a feed were dated when the feed was polled instead of when they were published (e.g. a Reddit saved-posts feed shows "today" for every item).This makes RSS ingestion consistent with file imports, which already honor their source date via
sourceAddedAt->createdAtin the import worker.Changes:
feedParser: parse the entry's published date (isoDate, falling back to rawpubDate) into apublishedAt: Date | undefined.feedWorker: pass it ascreatedAtwhen creating the bookmark; when the feed omits a date we sendundefinedand the server default is used.Scope: this fixes the date half of the issue. The author part is a separate, crawler-side concern (link author is populated by the crawl and
createBookmarkhas no author field for links), so it's intentionally left out of this PR.Fixes #2334
How Has This Been Tested?
feedParserunit tests (vitest): RSS<pubDate>, Atom<updated>(Reddit-style), and the no-date fallback (publishedAtundefined). Fail before the fix, pass after.pnpm --filter @karakeep/workers typecheck, oxfmt--check, oxlint all clean.Checklist:
Please describe to which degree, if any, an LLM was used in creating this pull request.
This PR was written with substantial help from an AI coding assistant (Anthropic Claude), which located the bug, wrote the code and tests, and ran the checks. All changes were reviewed by me before submission, and I take responsibility for them.