Skip to content

Support an optional created_at on pipeline creation - #1119

Closed
QnJ1c2kNCg wants to merge 1 commit into
ArroyoSystems:masterfrom
QnJ1c2kNCg:broy/specify-create_at
Closed

Support an optional created_at on pipeline creation#1119
QnJ1c2kNCg wants to merge 1 commit into
ArroyoSystems:masterfrom
QnJ1c2kNCg:broy/specify-create_at

Conversation

@QnJ1c2kNCg

@QnJ1c2kNCg QnJ1c2kNCg commented Jul 29, 2026

Copy link
Copy Markdown
Collaborator

PipelinePost takes an optional created_at in microseconds since the Unix epoch, applied to both the pipeline and its initial job. When omitted the inserts fall back to CURRENT_TIMESTAMP via COALESCE, preserving the existing database-clock behavior.

Testing
Tested a few different pipelines and validated via the UI:

gitlab/arroyo-all/arroyo  master [$!?]
❯ curl -sS -X POST http://localhost:5115/api/v1/pipelines \
  -H 'Content-Type: application/json' \
  -d '{
    "name": "backdated_test",
    "parallelism": 1,
    "created_at": 1583393889123456,
    "query": "create table impulse with (connector = '"'"'impulse'"'"', event_rate = '"'"'10'"'"');\nselect counter from impulse;"
  }' | jq '{id, name, created_at}'
{
  "id": "pl_RaUQ01xlBW",
  "name": "backdated_test",
  "created_at": 1583393889123456
}

gitlab/arroyo-all/arroyo  master [$!?]
❯ CREATED_AT=$(( $(date -v-7d +%s) * 1000000 ))
curl -sS -X POST http://localhost:5115/api/v1/pipelines \
  -H 'Content-Type: application/json' \
  -d "{
    \"name\": \"backdated_7d\",
    \"parallelism\": 1,
    \"created_at\": $CREATED_AT,
    \"query\": \"create table impulse with (connector = 'impulse', event_rate = '10');\\nselect counter from impulse;\"
  }" | jq '{id, created_at}'
{
  "id": "pl_OjeqgbnYlb",
  "created_at": 1784746618000000
}

gitlab/arroyo-all/arroyo  master [$!?]
❯ curl -sS -X POST http://localhost:5115/api/v1/pipelines \
  -H 'Content-Type: application/json' \
  -d '{
    "name": "wallclock_test",
    "parallelism": 1,
    "query": "create table impulse with (connector = '"'"'impulse'"'"', event_rate = '"'"'10'"'"');\nselect counter from impulse;"
  }' | jq '{id, created_at}'
{
  "id": "pl_pH9pETsKhs",
  "created_at": 1785351432000000
}

Open in Devin Review

PipelinePost takes an optional created_at in microseconds since the Unix
epoch, applied to both the pipeline and its initial job. When omitted the
inserts fall back to CURRENT_TIMESTAMP via COALESCE, preserving the
existing database-clock behavior.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 1 additional finding.

Open in Devin Review

@QnJ1c2kNCg QnJ1c2kNCg closed this Jul 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant