Tracking flow run provenance from external applications

[sghegoiu-rokos]

Is there a recommended way to track which external application or user triggered a flow run? I see the server tracks created_by for automations and deployments, but I'm not sure how to identify flow runs initiated by a web app/api.

Are there existing patterns for passing custom provenance metadata through the API? Im not sure whether it's possible to do server auth on the OSS & maybe source it from the token

[zzstoatzz]

Great question! There are a few mechanisms for tracking provenance on flow runs.

1. created_by field (automatic with Cloud auth)

If you're using Prefect Cloud, the created_by field is automatically populated based on the API key used:

flow_run = await client.create_flow_run_from_deployment(deployment_id=...)
print(flow_run.created_by)
# CreatedBy(id=UUID('...'), type='USER', display_value='username')

This works for any authenticated API call - the server tracks which user/service account made the request.

2. Labels (recommended for custom provenance)

Labels are key-value pairs that support typed values (strings, numbers, booleans). They're the cleanest way to attach provenance metadata:

from prefect.client.orchestration import get_client

async with get_client() as client:
    flow_run = await client.create_flow_run_from_deployment(
        deployment_id=deployment_id,
        labels={
            "triggered_by": "external-scheduler",
            "source_app": "jenkins",
            "build_number": 456,
            "is_production": True,
        },
    )

Labels are queryable and visible in the UI. Prefect also auto-populates system labels like prefect.deployment.id, prefect.flow.name, etc.

3. REST API

Same approach works via REST:

curl -X POST "${API_URL}/deployments/${DEPLOYMENT_ID}/create_flow_run" \
  -H "Authorization: Bearer ${API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "labels": {
      "triggered_by": "external-orchestrator",
      "request_id": "req-123",
      "priority": 1
    }
  }'

4. Incoming Webhooks (Cloud only)

If you're on Prefect Cloud, you can also use incoming webhooks to trigger runs from external systems. These create events that can trigger automations.

---

Recommendation: Use labels for custom provenance metadata - they're structured, queryable, and purpose-built for this. The created_by field handles auth-based tracking automatically if you're on Cloud.

[sghegoiu-rokos]

Labels are queryable and visible in the UI

@zzstoatzz Im not sure this is actually true. I can see a FlowRunFilterTags exists but there is no FlowRunFilterLabels.

[zzstoatzz]

hi @sghegoiu-rokos - you're right, i didn't realize that this filter did not yet exist. opened this PR. you can use tags in the meantime if you want

[sghegoiu-rokos]

@zzstoatzz would it be possible to also add a FlowRunFilterCreatedBy so that we can query based on schedule provenance?