Skip to content

feat(sigil): Add sigil.receiver and sigil.write components#6029

Open
alexander-akhmetov wants to merge 1 commit intomainfrom
alexander-akhmetov/sigil-http
Open

feat(sigil): Add sigil.receiver and sigil.write components#6029
alexander-akhmetov wants to merge 1 commit intomainfrom
alexander-akhmetov/sigil-http

Conversation

@alexander-akhmetov
Copy link
Copy Markdown

@alexander-akhmetov alexander-akhmetov commented Apr 9, 2026

Brief description of Pull Request

Add sigil.receiver and sigil.write components for proxying Sigil AI generation ingest traffic through Alloy.

sigil.receiver listens on POST /api/v1/generations:export and forwards raw request bytes to downstream GenerationsReceiver targets.

sigil.write implements GenerationsReceiver by forwarding to remote Sigil ingest endpoints with configurable backoff/retry, tenant header rewriting, and fan-out to multiple backends.

Both components use the raw-proxy pattern (like pyroscope.receive_http). Request and response bodies are forwarded as opaque bytes without deserialization. Both are marked StabilityExperimental.

Pull Request Details

Sigil SDKs currently send generation records directly to Sigil ingest, bypassing Alloy. This means operators who route all observability traffic through Alloy have a gap in their telemetry gateway for AI signals.

These components close that gap: SDKs point at Alloy's sigil.receiver, Alloy proxies the traffic through sigil.write to the upstream, applying auth header rewriting, retry/backoff, and multi-backend fan-out transparently.

Notes to the Reviewer

PR Checklist

  • Documentation added
  • Tests updated
  • Config converters updated (N/A — new signal type, no prior config format)

@cla-assistant
Copy link
Copy Markdown

cla-assistant bot commented Apr 9, 2026

CLA assistant check
All committers have signed the CLA.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 9, 2026

💻 Deploy preview available (feat(sigil): Add sigil.receiver and sigil.write components):

@alexander-akhmetov alexander-akhmetov force-pushed the alexander-akhmetov/sigil-http branch 2 times, most recently from db40e88 to 9a73970 Compare April 9, 2026 23:50
@grafana grafana deleted a comment from cursor bot Apr 9, 2026
@alexander-akhmetov alexander-akhmetov force-pushed the alexander-akhmetov/sigil-http branch 2 times, most recently from c2d91a7 to 1f1475c Compare April 10, 2026 19:58
@alexander-akhmetov alexander-akhmetov marked this pull request as ready for review April 10, 2026 21:36
@alexander-akhmetov alexander-akhmetov force-pushed the alexander-akhmetov/sigil-http branch from 1f1475c to ab41f6a Compare April 10, 2026 22:17
sigil.receiver listens on POST /api/v1/generations:export and forwards
raw request bytes to one or more downstream GenerationsReceiver targets.

sigil.write implements GenerationsReceiver by forwarding to remote Sigil
ingest endpoints with configurable backoff/retry and tenant header
rewriting.

Both components are marked StabilityExperimental.
@alexander-akhmetov alexander-akhmetov force-pushed the alexander-akhmetov/sigil-http branch from ab41f6a to 7606264 Compare April 10, 2026 22:39
@kalleep
Copy link
Copy Markdown
Contributor

kalleep commented Apr 13, 2026

Hey thanks for the pr. Mind explaining why we would want something like this in alloy? AFAICT can see this is just adding a generic proxy to alloy.

From sigil docs:

OpenTelemetry-native: Sigil follows the OTel GenAI semantic conventions, emits standard traces and metrics over OTLP, and works with existing OTel pipelines

So you should be able to ingest this with otel pipelines right? But I am probably missing something

@alexander-akhmetov
Copy link
Copy Markdown
Author

alexander-akhmetov commented Apr 13, 2026

Hi! Yes, this is basically a proxy. I based it mostly on pyroscope.receive_http.

The reason I thought it could make sense in Alloy is that Sigil is not just OTLP. Traces and metrics go through the normal OTel pipeline, but the full generation data (messages, tools, thinking information, etc.) goes through a separate Sigil ingest API. So even if someone already uses Alloy for OTel, they still need to configure the app separately or add another place for auth, tenant headers, retries, and routing for that generation ingest path. My idea was to let Alloy handle that too.

But I also agree the obvious question is whether the right fix is to improve Sigil and upstream OTel standards to make them work together instead of adding this path to Alloy. That is a fair concern. I mainly opened this PR because the separate Sigil ingest path exists today, and this seemed like a practical integration point for users who already run Alloy. So I totally understand if this isn't something you want in Alloy, feel free to close the PR if so :)

@kalleep
Copy link
Copy Markdown
Contributor

kalleep commented Apr 13, 2026

I based it mostly on pyroscope.receive_http

Yeah so while the profiles in pyroscope is just a byte blob it still have labels that you can e.g. perform relabeling on them.

So even if someone already uses Alloy for OTel, they still need to configure the app separately or add another place for auth, tenant headers, retries, and routing for that generation ingest path

I can see this being annoying / not ideal but you would still have to configure the app to send it to the endpoint you expose with correct port etc but there is a value in centralize e.g. authentication.

But I also agree the obvious question is whether the right fix is to improve Sigil and upstream OTel standards to make them work together instead of adding this path to Alloy

Do you think this "generation data" will ever become something like a native otel signal or something we would want to do processing on before storing at the final destination?

@alexander-akhmetov
Copy link
Copy Markdown
Author

Do you think this "generation data" will ever become something like a native otel signal

I think it could, but I'm not sure if / when that could happen.

something we would want to do processing on before storing at the final destination?

I think processing could be useful, even if just relabeling, filtering or routing based on labels / metadata, etc. but I made it just a raw proxy to start with.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants