New Components - social_fetch#21091
Open
Lukem121 wants to merge 5 commits into
Open
Conversation
Replace stub social_fetch.app.mjs with full runtime, route helpers, and workflow components. Relates to PipedreamHQ#21083
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
dannyroosevelt
moved this to Ready for PR Review
in Component (Source and Action) Backlog
Collaborator
|
Thank you so much for submitting this! We've added it to our backlog to review, and our team has been notified. |
Collaborator
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds a Social Fetch integration: constants/utilities, routing, app HTTP plumbing with error formatting and auth, five Pipedream actions, a scheduled profile-change source that diffs and emits events, package metadata, and a README. ChangesSocial Fetch Integration
Sequence DiagramssequenceDiagram
participant Action as Action Module
participant App as social_fetch.app
participant Routing as routing.mjs
participant API as Social Fetch API
Action->>App: call getProfile/getPost/listProfilePosts/getTranscript/getCreditBalance
App->>Routing: resolve route (platform, inputs)
Routing-->>App: return {path, params}
App->>App: _makeRequest(path, params) with x-api-key
App->>API: HTTP GET to API_BASE_URL + path
API-->>App: JSON response or HTTP error
App-->>Action: return data or throw formatted Error
sequenceDiagram
participant Timer as Scheduled Timer
participant Source as new-profile-update
participant App as social_fetch.app
participant DB as persistent DB
participant Stream as Event Stream
Timer->>Source: deploy/run triggered
Source->>DB: _getLastProfile()
DB-->>Source: lastProfile or undefined
Source->>App: getProfile(platform, handle, profileUrl)
App-->>Source: current profile
Source->>Source: getObjectDiff(lastProfile, profile)
alt diff is non-empty
Source->>DB: _setLastProfile(profile)
Source->>Source: profileSummaryLabel()
Source->>Stream: $emit({profile, diff, metadata})
end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Suggested reviewers
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 inconclusive)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@components/social_fetch/sources/new-profile-update/new-profile-update.mjs`:
- Around line 73-75: The emitted metadata currently uses runtime clock values
for id and ts (id: Date.now(), ts: Date.parse(new Date())), which breaks dedupe
and timestamp guarantees; replace those with a stable, event-derived id (use the
source event's unique id field or a deterministic hash of event properties such
as profile identifier + event timestamp) and set ts from the API event timestamp
(e.g., parse event.published/event.updated/event.timestamp) instead of
Date.now()/new Date(); keep summary using label unchanged. Target the id and ts
assignments in the object that builds the emission (the id, ts, summary entries)
and ensure the new id is deterministic across restarts/polls.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: f671ee01-72e3-4174-a84a-ad50c8ace693
⛔ Files ignored due to path filters (1)
components/social_fetch/pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (13)
components/social_fetch/README.mdcomponents/social_fetch/actions/get-credit-balance/get-credit-balance.mjscomponents/social_fetch/actions/get-post/get-post.mjscomponents/social_fetch/actions/get-profile/get-profile.mjscomponents/social_fetch/actions/get-transcript/get-transcript.mjscomponents/social_fetch/actions/list-profile-posts/list-profile-posts.mjscomponents/social_fetch/common/constants.mjscomponents/social_fetch/common/routing.mjscomponents/social_fetch/common/utils.mjscomponents/social_fetch/package.jsoncomponents/social_fetch/social_fetch.app.mjscomponents/social_fetch/sources/new-profile-update/new-profile-update.mjscomponents/social_fetch/sources/new-profile-update/test-event.mjs
…entation clarity - Cleaned up error message formatting in formatApiError function. - Enhanced descriptions for various properties in social_fetch.app.mjs to improve clarity for users. - Updated pnpm-lock.yaml to reflect changes in dependencies and versions.
Co-authored-by: Cursor <cursoragent@cursor.com>
Author
|
Thanks for the review! Addressed the CodeRabbit feedback on emit metadata in New Profile Update (commit 5a4481e):
All 5 actions and auth were tested locally via |
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
components/social_fetch/social_fetch.app.mjs (1)
108-118:⚠️ Potential issue | 🟡 MinorMerge caller-supplied
headerswith auth headers in_makeRequest()(avoid clobberingx-api-key)
_makeRequest()setsheaders: this._headers(), but then spreads...opts, so anyopts.headersfrom the caller will replace the auth headers. Destructureheadersand merge it with_headers(), and switch tobaseURL+url.🔧 Suggested refactor
async _makeRequest({ - $ = this, path, params, ...opts + $ = this, + path, + params, + headers = {}, + ...opts } = {}) { let response; try { response = await axios($, { - url: `${API_BASE_URL}${path}`, - headers: this._headers(), + baseURL: API_BASE_URL, + url: path, + headers: { + ...this._headers(), + ...headers, + }, params, ...opts, }); } catch (error) {🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@components/social_fetch/social_fetch.app.mjs` around lines 108 - 118, In _makeRequest(), callers' opts.headers currently overwrite the auth headers from this._headers(); fix by destructuring headers from opts (e.g., { headers: callerHeaders, ...restOpts } = opts), merge them with this._headers() (callerHeaders taking precedence or vice versa per desired behavior) and pass the merged headers into axios; also change the axios call to use baseURL: API_BASE_URL and url: path (instead of concatenating) and spread restOpts into the request so all other options remain intact (refer to _makeRequest and this._headers()).
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@components/social_fetch/social_fetch.app.mjs`:
- Around line 41-42: The prop description strings in
components/social_fetch/social_fetch.app.mjs (the description fields for the
social platform and the related input props in the same block) are written as UI
instructions; replace them with input-contract style descriptions that specify
the expected value, type, and conditional requirements — e.g., for the platform
prop state the allowed values (e.g., "twitter", "instagram", "facebook",
"linkedin"), for the handle input state "a username without @ (string) required
when platform is a handle-based service", and for the profileUrl input state "a
full profile URL (string, starting with http/https) required when platform is
facebook or linkedin"; ensure each description is concise, agent-focused, and
mentions validation/format expectations rather than telling a human how to
select fields.
- Around line 66-67: Update the prop description strings in
components/social_fetch/social_fetch.app.mjs for the properties handle,
profileUrl, postUrl, and contentType to include concrete inline examples and
remove UI-centric phrasing; e.g., change the handle description to "Username or
handle (without @), e.g. 'nasa'", profileUrl to "Full profile URL, e.g.
'https://www.linkedin.com/in/janedoe/'", postUrl to "Full post/video URL, e.g.
'https://www.tiktok.com/@nasa/video/1234567890'", and contentType to "Type of
content, e.g. 'videos'"; apply the same example-style updates where similar
descriptions appear on the other indicated lines (73-74, 80-81, 91-92).
---
Outside diff comments:
In `@components/social_fetch/social_fetch.app.mjs`:
- Around line 108-118: In _makeRequest(), callers' opts.headers currently
overwrite the auth headers from this._headers(); fix by destructuring headers
from opts (e.g., { headers: callerHeaders, ...restOpts } = opts), merge them
with this._headers() (callerHeaders taking precedence or vice versa per desired
behavior) and pass the merged headers into axios; also change the axios call to
use baseURL: API_BASE_URL and url: path (instead of concatenating) and spread
restOpts into the request so all other options remain intact (refer to
_makeRequest and this._headers()).
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 993695f9-4dd2-404f-a006-2ce050ca55b3
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (1)
components/social_fetch/social_fetch.app.mjs
Co-authored-by: Cursor <cursoragent@cursor.com>
Author
|
Follow-up on the latest CodeRabbit review — addressed in commit 9cf0505:
The earlier dedupe fix (5a4481e) remains in place for the polling source. |
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
components/social_fetch/social_fetch.app.mjs (1)
164-175: 🧹 Nitpick | 🔵 Trivial | 💤 Low valueAdd JSDoc to public methods for consistency.
getPost,listProfilePosts, andgetTranscriptlack JSDoc annotations, while similar methods (getCreditBalance,getProfile,profileSummaryLabel) include them. Consistent documentation improves maintainability and helps agents understand method contracts.📝 Suggested JSDoc additions
+ /** + * `@param` {{ platform?: string; postUrl?: string; [key: string]: unknown }} [args] + */ getPost({ platform, postUrl, ...opts } = {}) {+ /** + * `@param` {{ platform?: string; contentType?: string; handle?: string; + * profileUrl?: string; cursor?: string; [key: string]: unknown }} [args] + */ listProfilePosts({ platform, contentType,+ /** + * `@param` {{ platform?: string; mediaUrl?: string; [key: string]: unknown }} [args] + */ getTranscript({ platform, mediaUrl, ...opts } = {}) {Also applies to: 176-194, 195-206
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@components/social_fetch/social_fetch.app.mjs` around lines 164 - 175, Add JSDoc comments to the public methods getPost, listProfilePosts, and getTranscript to match the style used for getCreditBalance/getProfile/profileSummaryLabel: include a short description, `@param` annotations for each parameter (platform, postUrl or other options object, and any spread/opts), types and whether optional, and an `@returns` annotation describing the returned Promise/response shape; place the JSDoc immediately above each function definition (getPost, listProfilePosts, getTranscript) so documentation and editors can surface the method contract consistently.
♻️ Duplicate comments (1)
components/social_fetch/social_fetch.app.mjs (1)
47-47:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winRewrite prop descriptions to be agent-focused and include concrete examples.
These descriptions still contain UI-centric phrasing ("Select the platform", "Paste the URL from the browser address bar") and lack inline examples for non-obvious formats.
As per coding guidelines, prop
descriptionfields must be written for AI agent consumption, avoid UI-centric language, and include concrete inline examples for non-obvious inputs.✏️ Example rewrites
platform: { type: "string", label: "Platform", description: - "Select the social platform. Handle-based platforms need a username; Facebook and LinkedIn need a full profile URL.", + "Social media platform. Use a username (handle) for TikTok, Instagram, Twitter/X, or Threads; use a full profile URL for Facebook or LinkedIn. Example: `tiktok`, `instagram`, `facebook`, `linkedin`.", options: PROFILE_PLATFORMS, },handle: { type: "string", label: "Handle", description: - "Username or handle (without @) for TikTok, Instagram, Twitter/X, or Threads.", + "Username without the leading @ symbol. Example: `nasa`.", optional: true, },profileUrl: { type: "string", label: "Profile URL", description: - "Full profile page URL for Facebook or LinkedIn. Paste the URL from the browser address bar.", + "Full Facebook or LinkedIn profile URL. Example: `https://www.linkedin.com/in/janedoe/`.", optional: true, },postUrl: { type: "string", label: "Post URL", description: - "Full URL of the post, video, or tweet. Paste the link from your browser address bar.", + "Full URL of the post, video, or tweet. Example: `https://www.tiktok.com/@nasa/video/1234567890`.", },contentType: { type: "string", label: "Content Type", description: - "Videos for TikTok; Posts or Reels for Instagram; Tweets for Twitter / X; Posts for Threads and Facebook.", + "Type of content to list. Use `videos` for TikTok, `posts` or `reels` for Instagram, `tweets` for Twitter/X, `posts` for Threads or Facebook.", options: CONTENT_TYPE_OPTIONS, },Also applies to: 72-72, 79-79, 86-86, 97-97
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@components/social_fetch/social_fetch.app.mjs` at line 47, Update the prop "description" strings in components/social_fetch/social_fetch.app.mjs to be agent-focused (address the AI directly), remove UI-centric phrasing, and include concrete inline examples for non-obvious formats: for the platform selector description explain what each platform expects (e.g., "use handle for Twitter/Instagram: 'elonmusk'; use full profile URL for Facebook/LinkedIn: 'https://www.facebook.com/zuck'"), for the username/handle prop give example handles and clarifying format (e.g., "no leading @, e.g. 'jack' or 'spotify'"), and for URL props show exact example URLs and any required components (e.g., "include https:// and full path, e.g. 'https://www.linkedin.com/in/janedoe/'"). Apply this style to all description fields in the file (the other occurrences noted in the review) so every description is written to the agent and includes at least one concrete example.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@components/social_fetch/sources/new-profile-update/new-profile-update.mjs`:
- Around line 50-66: The generated id in _buildEmitMetadata can exceed 64 chars
when identity (e.g., profileUrl) is long; replace the raw identity in the id
with a short cryptographic hash of the identity (e.g., sha256 and take a 16-char
hex slice) and then build the id as platform + separator + identityHash +
separator + diffHash, and finally enforce a max length of 64 characters (slice
or truncate the composed string) so emitted ids always remain <=64 chars.
- Around line 57-60: The emitted metadata `id` built from
`${this.platform}:${identity}:${diffHash}` exceeds the 64-char dedupe-id limit
and will break deduplication; change the `id` construction in
new-profile-update.mjs (where `createHash("sha256")` produces `diffHash`) to
produce a <=64-char deterministic id by folding/hashing the full identifying
payload into a short fingerprint (e.g., hash the concatenation of platform,
identity and diffHash and slice to 64 chars as in
components/linkly/sources/new-link-clicked/new-link-clicked.mjs) and, if you
must allow repeated identical diffs to emit, incorporate a persisted monotonic
counter or the API-provided update/event timestamp into the input used to derive
the id (do not use Date.now() if an API timestamp is available).
---
Outside diff comments:
In `@components/social_fetch/social_fetch.app.mjs`:
- Around line 164-175: Add JSDoc comments to the public methods getPost,
listProfilePosts, and getTranscript to match the style used for
getCreditBalance/getProfile/profileSummaryLabel: include a short description,
`@param` annotations for each parameter (platform, postUrl or other options
object, and any spread/opts), types and whether optional, and an `@returns`
annotation describing the returned Promise/response shape; place the JSDoc
immediately above each function definition (getPost, listProfilePosts,
getTranscript) so documentation and editors can surface the method contract
consistently.
---
Duplicate comments:
In `@components/social_fetch/social_fetch.app.mjs`:
- Line 47: Update the prop "description" strings in
components/social_fetch/social_fetch.app.mjs to be agent-focused (address the AI
directly), remove UI-centric phrasing, and include concrete inline examples for
non-obvious formats: for the platform selector description explain what each
platform expects (e.g., "use handle for Twitter/Instagram: 'elonmusk'; use full
profile URL for Facebook/LinkedIn: 'https://www.facebook.com/zuck'"), for the
username/handle prop give example handles and clarifying format (e.g., "no
leading @, e.g. 'jack' or 'spotify'"), and for URL props show exact example URLs
and any required components (e.g., "include https:// and full path, e.g.
'https://www.linkedin.com/in/janedoe/'"). Apply this style to all description
fields in the file (the other occurrences noted in the review) so every
description is written to the agent and includes at least one concrete example.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 9fb39dcc-e6fb-44e1-8769-aa46d58e7771
📒 Files selected for processing (3)
components/social_fetch/common/utils.mjscomponents/social_fetch/social_fetch.app.mjscomponents/social_fetch/sources/new-profile-update/new-profile-update.mjs
Co-authored-by: Cursor <cursoragent@cursor.com>
Author
|
Addressed the latest CodeRabbit emit-id feedback in fddec70:
Re: repeated identical diffs (A->B->A->B) being suppressed — intentional with dedupe unique for now; our API has no update timestamp to disambiguate. Happy to add a persisted emit counter if reviewers want every re-occurrence to fire. |
Contributor
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
components/social_fetch/sources/new-profile-update/new-profile-update.mjs (1)
82-103:⚠️ Potential issue | 🟠 Major | ⚡ Quick winMove state update after emit to prevent skipped events on partial failure.
The call to
_setLastProfile(profile)on line 83 happens before$emit()on line 92. If the emit fails, the state is already updated, causing the event to be permanently skipped on retry.Proposed fix
if (Object.keys(diff).length > 0) { - this._setLastProfile(profile); const label = this.app.profileSummaryLabel( this.platform, this.handle, this.profileUrl, ); const { id, ts, } = this._buildEmitMetadata(profile, diff); this.$emit( { profile, diff, }, { id, summary: `Profile update for ${label}`, ts, }, ); + this._setLastProfile(profile); }As per coding guidelines: "Update stored state AFTER emitting (so partial failures don't cause skipped events on the next run)."
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@components/social_fetch/sources/new-profile-update/new-profile-update.mjs` around lines 82 - 103, The state update is happening before the emit which can cause events to be skipped if the emit fails; move the call to _setLastProfile(profile) so it runs only after $emit(...) completes successfully. Keep building the label and metadata as-is using app.profileSummaryLabel(...) and _buildEmitMetadata(profile, diff), call this.$emit(...) first with the constructed payload and metadata (id, summary, ts), and then call this._setLastProfile(profile) immediately after the emit returns to ensure state is only advanced on successful emit.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@components/social_fetch/sources/new-profile-update/new-profile-update.mjs`:
- Around line 82-103: The state update is happening before the emit which can
cause events to be skipped if the emit fails; move the call to
_setLastProfile(profile) so it runs only after $emit(...) completes
successfully. Keep building the label and metadata as-is using
app.profileSummaryLabel(...) and _buildEmitMetadata(profile, diff), call
this.$emit(...) first with the constructed payload and metadata (id, summary,
ts), and then call this._setLastProfile(profile) immediately after the emit
returns to ensure state is only advanced on successful emit.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 9308c21b-1f37-4d75-8c10-d34b3f40b225
📒 Files selected for processing (1)
components/social_fetch/sources/new-profile-update/new-profile-update.mjs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds Social Fetch workflow components for the registered
social_fetchapp:Replaces the stub
social_fetch.app.mjswith the full app runtime (API key auth viax-api-key, route helpers, error formatting).Relates to #21083
Test plan
npx eslint components/social_fetchsfk_…API key; verify connection test (GET /v1/whoami)Auth
x-api-keyapi_keyGET https://api.socialfetch.dev/v1/whoamiSummary by CodeRabbit
New Features
Documentation