[All] Support `CallerOrg` in Go SDK to identify callers in `x-zerobus-sdk` header

[zuckerberg-db]

SDK

All

Description

The ZeroBus Go SDK provides no way for callers to identify themselves in the x-zerobus-sdk gRPC metadata header without abandoning the built-in OAuth authentication flow.

Today, the x-zerobus-sdk header is hardcoded to zerobus-sdk-rs/<version> inside OAuthHeadersProvider. The only way to inject a custom identifier is to implement HeadersProvider manually via CreateStreamWithHeadersProvider, which requires the caller to re-implement OAuth token acquisition — a significant burden for a simple identification need.

This makes it difficult for platform teams to track which applications or organizations are sending data through ZeroBus.

Proposed Solution

Add a CallerOrg field to StreamConfigurationOptions (and ArrowStreamConfigurationOptions) that gets appended to the x-zerobus-sdk header value as a callerOrg-<value> segment. When CallerOrg is empty (default), behavior is unchanged.

This follows the same pattern used in the Delta Sharing SDK, which added a callerOrg option that threads through config layers and is appended to the User-Agent HTTP header.

Resulting header format:

x-zerobus-sdk: zerobus-sdk-rs/0.5.0 callerOrg-<organization>

Go SDK usage:

options := zerobus.DefaultStreamConfigurationOptions()
options.CallerOrg = "<organization>"

stream, err := sdk.CreateStream(tableProps, clientID, clientSecret, options)
// x-zerobus-sdk header now includes: "zerobus-sdk-rs/0.5.0 callerOrg-<organization>"

Implementation approach:

• Add CallerOrg string to Go's StreamConfigurationOptions and ArrowStreamConfigurationOptions in go/types.go
• Thread it through the C FFI struct (go/ffi.go, go/arrow_ffi.go) and the Rust FFI #[repr(C)] struct (rust/ffi/src/lib.rs)
• In the Rust FFI layer, wrap the headers provider with a CallerOrgHeadersProvider decorator that appends callerOrg-<value> to the x-zerobus-sdk header
• No Rust SDK core changes needed — the wrapper lives entirely in the FFI crate

struct CallerOrgHeadersProvider {
    inner: Arc<dyn HeadersProvider>,
    caller_org: String,
}

#[async_trait]
impl HeadersProvider for CallerOrgHeadersProvider {
    async fn get_headers(&self) -> ZerobusResult<HashMap<&'static str, String>> {
        let mut headers = self.inner.get_headers().await?;
        if let Some(existing) = headers.get("x-zerobus-sdk") {
            headers.insert("x-zerobus-sdk", format!("{} callerOrg-{}", existing, self.caller_org));
        } else {
            headers.insert("x-zerobus-sdk", format!("callerOrg-{}", self.caller_org));
        }
        Ok(headers)
    }
}

Additional Context

• Backward compatible — CallerOrg defaults to empty string, preserving existing behavior
• All stream types — covers both standard streams and Arrow Flight streams
• Both auth paths — works with OAuth (CreateStream) and custom auth (CreateStreamWithHeadersProvider)
• Precedent — Delta Sharing SDK implemented the identical pattern: delta-io/delta-sharing@d205144
• The x-zerobus-sdk header is defined in rust/sdk/src/headers_provider.rs and was renamed from user-agent for clearer SDK identification in gRPC metadata

[teodordelibasic-db]

Hey, thanks for filing this and for the detailed explanation! It makes sense to do this and we already had plans to do something similar, so I have a couple of questions regarding acceptance criteria for this:

1. Is it necessary for this to be called CallerOrg? If yes, what is the reason for that, e.g. how will it be used on server side?
2. Do you think this information needs to be passed in the x-zerobus-sdk header? It makes sense to do it that way, just trying to understand the full context behind CallerOrg.

[zuckerberg-db]

Your questions raise some good points.

1/ It's not necessary for it be called CallerOrg. In fact, CallerOrg deviates from what's used in Databricks core SDKs. Databricks core SDKs prepend the User-Agent HTTP header. This would probably be the ideal way to implement it.

• Python SDK - configures it through useragent.with_partner()
• Java SDK - configures itthrough UserAgent.withPartner()
• Go SDK - configures it through useragent.WithPartner()

2/ And passing it in the x-zerobus-sdk is not the right approach. We need to pass the Zerobus API calls to the HTTP UserAgent tag so we can identify the calls in logs such as the system tables.

[teodordelibasic-db]

I agree that using the UserAgent header is ideal. In fact, that's what we implemented at first instead of x-zerobus-sdk, but then I found out that the version of tonic (Rust gRPC crate) we use overrides whatever is set for UserAgent and just sets tonic/x.y.z. Upgrading to this new version is something we have on our roadmap, but bumping pretty much any of tonic/prost/arrow pulls in a lot of transitive dependency bumps. This can be a pain for customers and is even a pain for us for version bumps in Universe.

Does it make sense to you to wait for us to bump to new tonic version, then expose API for setting CallerOrg/Partner which will be added to the value of the UserAgent header?

[zuckerberg-db]

@teodordelibasic-db what's the timeline for the tonic upgrade? I think that's the only viable path here.

[teodordelibasic-db]

Ideally we will release a new major version next week with the dependency versions bumped and then we can integrate this as well.