Skip to content

Commit dbc830f

Browse files
feat: support for creating upload v2 notifier jobs (#5729)
# Description This PR introduces support for creation of upload_v2 notifier jobs required for batching of staging files **Key Changes:** - Files are grouped into v1 and v2 based on the presence of BytesPerTable and a feature flag. - Extracted common logic for v1 and v2 paths into separate methods - For now, v2 notifier jobs include only one staging file per job, similar to v1. This is temporary — batching logic will be implemented in the next PR. - We will no longer track the processing status (success/failure) of staging files in v2. - v2 jobs will always force creation of load files (see state_generate_load_files.go). ## Linear Ticket https://linear.app/rudderstack/issue/WAR-461/enable-creation-of-upload-v2-notifier-jobs ## Security - [x] The code changed/added as part of this pull request won't create any security issues with how the software is being used.
1 parent fd46379 commit dbc830f

File tree

6 files changed

+626
-193
lines changed

6 files changed

+626
-193
lines changed

services/notifier/notifier.go

+3-2
Original file line numberDiff line numberDiff line change
@@ -35,8 +35,9 @@ const (
3535
type JobType string
3636

3737
const (
38-
JobTypeUpload JobType = "upload"
39-
JobTypeAsync JobType = "async_job"
38+
JobTypeUpload JobType = "upload"
39+
JobTypeUploadV2 JobType = "upload_v2"
40+
JobTypeAsync JobType = "async_job"
4041
)
4142

4243
type JobStatus string

0 commit comments

Comments
 (0)