Skip to content

Move log backfill off the public API Gateway endpoint - #8595

Open
huydhn wants to merge 11 commits into
gh/huydhn/6/basefrom
gh/huydhn/6/head
Open

Move log backfill off the public API Gateway endpoint#8595
huydhn wants to merge 11 commits into
gh/huydhn/6/basefrom
gh/huydhn/6/head

Conversation

@huydhn

@huydhn huydhn commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Stack from ghstack (oldest at bottom):

Impact: backfillMissingLog, which Dr.CI calls when a failed job has no log
Risk: medium -- changes a live Dr.CI path

What

backfillMissingLog now invokes the gha-log-uploader lambda directly instead
of POSTing a synthetic action: "backfill" event at
jqogootqqe.execute-api.us-east-1.amazonaws.com. Adds
POST /api/log-uploader/backfill for callers outside HUD, authenticated with a
shared secret in LOG_UPLOADER_BOT_KEY, matching the DRCI_BOT_KEY and
FLAKY_TEST_BOT_KEY routes.

[ghstack-poisoned]
@vercel

vercel Bot commented Aug 20, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
torchci Ignored Ignored Aug 21, 2026 10:51pm

Request Review

huydhn added 2 commits August 20, 2026 13:01
[ghstack-poisoned]
[ghstack-poisoned]
huydhn added 3 commits August 20, 2026 15:17
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
@huydhn
huydhn marked this pull request as ready for review August 20, 2026 23:55
@huydhn
huydhn requested a review from izaitsevfb August 20, 2026 23:55
huydhn added 4 commits August 20, 2026 23:41
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]

@izaitsevfb izaitsevfb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, but let's make sure that the manual setup is done

Comment thread torchci/lib/jobUtils.ts
error
);
return false;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's not forget about the manual setup, as otherwise the failures would be silent


🟡 Confirm the uploader's invoke grant is in place before this deploys — without it every Dr.CI backfill fails and nothing downstream sees it. (ai-generated section)

The uploader is reachable only through an IAM-authenticated invoke, and the grant for the principal behind the HUD's OUR_AWS_ACCESS_KEY_ID is documented as one-time manual AWS setup rather than something the deploy performs. If it is not in place when this ships, the invoke throws on every call; the new catch logs the error and returns false, and the sole caller of backfillMissingLog ignores that value — so Dr.CI finishes normally and log backfill just stops.

Reviewed by codex gpt-5.6-sol at xhigh effort, against 5099e87.

[ghstack-poisoned]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants