Skip to content

[bot] Google GenAI batch generation (:batchGenerateContent) requests are not traced #211

Description

@braintrust-bot

<!-- provider-gap-audit: genai-batchgeneratecontent-not-traced -->

What instrumentation is missing

The Gemini API's Batch Mode endpoint (POST .../models/{model}:batchGenerateContent) submits an asynchronous batch generation job — a real model-execution surface (50% cheaper, higher-throughput content generation), not a CRUD/admin endpoint. This repo's GenAI tracing middleware does not recognize or trace this endpoint at all: requests to it pass through the wrapped http.Client completely untraced.

Where it breaks

trace/contrib/genai/tracegenai.go, genaiRouter (lines 143-157) dispatches based on containsGenerateContent/containsEmbedContent:

func containsGenerateContent(path string) bool {
	return strings.Contains(path, ":generateContent") ||
		strings.Contains(path, "/generateContent") ||
		strings.Contains(path, ":streamGenerateContent") ||
		strings.Contains(path, "/streamGenerateContent")
}

A batch request path is .../models/{model}:batchGenerateContent. The substring ":generateContent" requires a colon immediately before generateContent, but in :batchGenerateContent the colon precedes batch, not generateContent — so this check (and the /generateContent check) both return false. containsEmbedContent doesn't match it either. genaiRouter therefore returns nil and no span is created for batch generation requests.

Braintrust docs status: not_found

https://www.braintrust.dev/docs/integrations/ai-providers/gemini explicitly enumerates the traced Go methods as GenerateContent/GenerateContentStream plus embedding calls (EmbedContent/batchEmbedContents) — batch content generation is not listed as traced for any language on this page.

Upstream sources

Local repo files inspected

  • trace/contrib/genai/tracegenai.gogenaiRouter, containsGenerateContent, containsEmbedContent, isStreamingPath, extractModelFromPath (lines 142-212)
  • trace/contrib/genai/generatecontent.go — generation tracer, only invoked for paths already matched by the router
  • trace/contrib/genai/embedcontent.go — embedding tracer, confirms batchEmbedContents is handled while no equivalent batch-generation tracer exists

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions