Skip to content

README.md

Latest commit

 

History

History
367 lines (291 loc) · 11.5 KB

README.md

File metadata and controls

367 lines (291 loc) · 11.5 KB

Grafana Sigil TypeScript/JavaScript SDK

Sigil records normalized LLM generation and tool-execution telemetry using your OpenTelemetry tracer/meter setup.

Installation

pnpm add @grafana/sigil-sdk-js

Validation

Run the shared core conformance suite for the JavaScript SDK from the repo root:

mise run test:ts:sdk-conformance

Run the cross-language aggregate core conformance suite from the repo root:

mise run sdk:conformance

Quick Start

import { SigilClient } from "@grafana/sigil-sdk-js";

const client = new SigilClient({
  generationExport: {
    protocol: "http",
    endpoint: "http://localhost:8080/api/v1/generations:export",
    auth: { mode: "tenant", tenantId: "dev-tenant" },
  },
  api: {
    endpoint: "http://localhost:8080",
  },
});

await client.startGeneration(
  {
    conversationId: "conv-1",
    model: { provider: "openai", name: "gpt-5" },
  },
  async (recorder) => {
    const outputText = "Hello from model";
    recorder.setResult({
      output: [{ role: "assistant", content: outputText }],
    });
  }
);

await client.shutdown();

Configure OTEL exporters (traces/metrics) in your application OTEL SDK setup. You can optionally pass tracer and meter directly to SigilClient.

Quick OTEL setup pattern before creating the Sigil client:

import { NodeSDK } from "@opentelemetry/sdk-node";

const otel = new NodeSDK();
await otel.start();

Core API

  • startGeneration(...) and startStreamingGeneration(...)
  • startToolExecution(...)
  • Recorder methods: setResult(...), setCallError(...), end(), getError()
  • Lifecycle: flush(), shutdown()

Manual try/finally style

const recorder = client.startGeneration({
  model: { provider: "anthropic", name: "claude-sonnet-4-5" },
});

try {
  recorder.setResult({
    output: [{ role: "assistant", content: "Done" }],
  });
} catch (error) {
  recorder.setCallError(error);
  throw error;
} finally {
  recorder.end();
}

Embedding Observability

Use startEmbedding(...) for embedding API calls. Embedding recording creates OTel spans and SDK metrics only, and does not enqueue generation exports.

await client.startEmbedding(
  {
    agentName: "retrieval-worker",
    agentVersion: "1.0.0",
    model: { provider: "openai", name: "text-embedding-3-small" },
  },
  async (recorder) => {
    const response = await openai.embeddings.create(request);
    recorder.setResult({
      inputCount: request.input.length,
      inputTokens: response.usage?.prompt_tokens ?? 0,
      inputTexts: request.input,
      responseModel: response.model,
    });
  }
);

Input text capture is opt-in:

const client = new SigilClient({
  embeddingCapture: {
    captureInput: true,
    maxInputItems: 20,
    maxTextLength: 1024,
  },
});

embeddingCapture.captureInput may expose PII/document content in spans. Keep it disabled by default and enable it only for scoped debugging.

TraceQL examples:

  • traces{gen_ai.operation.name="embeddings"}
  • traces{gen_ai.operation.name="embeddings" && gen_ai.request.model="text-embedding-3-small"}
  • traces{gen_ai.operation.name="embeddings" && error.type!=""}

Tool Execution Example

await client.startToolExecution(
  {
    toolName: "weather",
    includeContent: true,
  },
  async (recorder) => {
    recorder.setResult({
      arguments: { city: "Paris" },
      result: { temp_c: 18 },
    });
  }
);

Provider Helpers

  • OpenAI: docs/providers/openai.md
  • Anthropic: docs/providers/anthropic.md
  • Gemini: docs/providers/gemini.md

Framework Handlers

Use module subpath exports for framework callback integrations:

  • LangChain: @grafana/sigil-sdk-js/langchain
  • LangGraph: @grafana/sigil-sdk-js/langgraph
  • OpenAI Agents: @grafana/sigil-sdk-js/openai-agents
  • LlamaIndex: @grafana/sigil-sdk-js/llamaindex
  • Google ADK: @grafana/sigil-sdk-js/google-adk
  • Vercel AI SDK: @grafana/sigil-sdk-js/vercel-ai-sdk
  • LangChain guide: docs/frameworks/langchain.md
  • LangGraph guide: docs/frameworks/langgraph.md
  • OpenAI Agents guide: docs/frameworks/openai-agents.md
  • LlamaIndex guide: docs/frameworks/llamaindex.md
  • Google ADK guide: docs/frameworks/google-adk.md
  • Vercel AI SDK guide: docs/frameworks/vercel-ai-sdk.md
import { SigilClient } from "@grafana/sigil-sdk-js";
import { withSigilLangChainCallbacks } from "@grafana/sigil-sdk-js/langchain";
import { withSigilLangGraphCallbacks } from "@grafana/sigil-sdk-js/langgraph";
import { withSigilOpenAIAgentsHooks } from "@grafana/sigil-sdk-js/openai-agents";
import { withSigilLlamaIndexCallbacks } from "@grafana/sigil-sdk-js/llamaindex";
import { withSigilGoogleAdkPlugins } from "@grafana/sigil-sdk-js/google-adk";
import { createSigilVercelAiSdk } from "@grafana/sigil-sdk-js/vercel-ai-sdk";
import { Runner } from "@openai/agents";
import { CallbackManager } from "llamaindex";

const client = new SigilClient();
const langChainConfig = withSigilLangChainCallbacks(undefined, client, { providerResolver: "auto" });
const langGraphConfig = withSigilLangGraphCallbacks(undefined, client, { providerResolver: "auto" });
const runner = new Runner();
const openAIAgentsHooks = withSigilOpenAIAgentsHooks(runner, client, { providerResolver: "auto" });
const callbackManager = new CallbackManager();
const llamaIndexConfig = withSigilLlamaIndexCallbacks({ callbackManager }, client, { providerResolver: "auto" });
const googleAdkRunnerConfig = withSigilGoogleAdkPlugins(undefined, client, { providerResolver: "auto" });
const vercelAiSdk = createSigilVercelAiSdk(client, { agentName: "vercel-agent" });

Each framework handler injects:

  • sigil.framework.name (langchain, langgraph, openai-agents, llamaindex, google-adk, or vercel-ai-sdk)
  • sigil.framework.source (handler for existing callback handlers, framework for Vercel AI SDK hooks)
  • sigil.framework.language (javascript for existing callback handlers, typescript for Vercel AI SDK hooks)
  • metadata["sigil.framework.run_id"]
  • metadata["sigil.framework.thread_id"] (when present)
  • metadata["sigil.framework.parent_run_id"] (when available)
  • metadata["sigil.framework.component_name"]
  • metadata["sigil.framework.run_type"]
  • metadata["sigil.framework.tags"]
  • metadata["sigil.framework.retry_attempt"] (when available)
  • metadata["sigil.framework.event_id"] (when available)
  • metadata["sigil.framework.langgraph.node"] (LangGraph when available)

Conversation mapping is conversation-first:

  • conversation_id / session_id / group_id from framework context first
  • then thread_id
  • deterministic fallback sigil:framework:<framework_name>:<run_id>

When present in generation metadata, low-cardinality framework keys are copied onto generation span attributes.

For LangGraph persistence, pass configurable.thread_id and reuse it across invocations:

const threadConfig = {
  ...withSigilLangGraphCallbacks(undefined, client, { providerResolver: "auto" }),
  configurable: { thread_id: 'customer-42' },
};
await graph.invoke({ prompt: 'Remember my timezone is UTC+1.', answer: '' }, threadConfig);
await graph.invoke({ prompt: 'What timezone did I give you?', answer: '' }, threadConfig);

Behavior

  • Generation modes are explicit: SYNC and STREAM.
  • Generation export supports HTTP, gRPC, and none (instrumentation-only).
  • Traces/metrics use config.tracer/config.meter when provided, otherwise OTEL globals.
  • Exports are asynchronous with bounded queueing and retry/backoff.
  • flush() drains queued generations; shutdown() flushes and closes generation exporters.
  • Empty tool names produce a no-op tool recorder.
  • Generation/tool spans always include SDK identity attributes:
    • sigil.sdk.name=sdk-js
  • Normalized generation metadata always includes the same SDK identity key; conflicting caller values are overwritten.
  • Raw provider artifacts are opt-in (rawArtifacts: true).

Instrumentation-only mode (no generation send)

Set generationExport.protocol to "none" to keep generation/tool instrumentation and spans while disabling generation transport.

const client = new SigilClient({
  generationExport: {
    protocol: "none",
  },
});

SDK metrics

The SDK emits these OTel histograms through your configured OTEL meter provider:

  • gen_ai.client.operation.duration
  • gen_ai.client.token.usage
  • gen_ai.client.time_to_first_token
  • gen_ai.client.tool_calls_per_operation

Generation export auth modes

Auth is configured for generationExport.

  • mode: "none"
  • mode: "tenant" (requires tenantId, injects X-Scope-OrgID)
  • mode: "bearer" (requires bearerToken, injects Authorization: Bearer <token>)
  • mode: "basic" (requires basicPassword + basicUser or tenantId, injects Authorization: Basic <base64(user:password)>; also injects X-Scope-OrgID when tenantId is set — for self-hosted multi-tenancy only, not needed for Grafana Cloud)

Invalid mode/field combinations throw during client config resolution.

If explicit headers already contain Authorization or X-Scope-OrgID, explicit headers take precedence.

const client = new SigilClient({
  generationExport: {
    protocol: "http",
    endpoint: "http://localhost:8080/api/v1/generations:export",
    auth: { mode: "tenant", tenantId: "prod-tenant" },
  },
  api: {
    endpoint: "http://localhost:8080",
  },
});

Grafana Cloud auth (basic)

For Grafana Cloud, use basic auth mode. The username is your Grafana Cloud instance/tenant ID and the password is your Grafana Cloud API key:

const client = new SigilClient({
  generationExport: {
    protocol: "http",
    endpoint: "https://<your-stack>.grafana.net/api/v1/generations:export",
    auth: {
      mode: "basic",
      tenantId: process.env.GRAFANA_CLOUD_INSTANCE_ID,
      basicPassword: process.env.GRAFANA_CLOUD_API_KEY,
    },
  },
});

If your deployment requires a distinct username, set basicUser explicitly:

auth: {
  mode: "basic",
  tenantId: process.env.GRAFANA_CLOUD_INSTANCE_ID,
  basicUser: process.env.GRAFANA_CLOUD_INSTANCE_ID,
  basicPassword: process.env.GRAFANA_CLOUD_API_KEY,
},

Env-secret wiring example

The SDK does not auto-load env vars. Resolve env secrets in your app and map them into config.

const generationBearerToken = (process.env.SIGIL_GEN_BEARER_TOKEN ?? "").trim();

const client = new SigilClient({
  generationExport: {
    protocol: "http",
    endpoint: "http://localhost:8080/api/v1/generations:export",
    auth:
      generationBearerToken.length > 0
        ? { mode: "bearer", bearerToken: generationBearerToken }
        : { mode: "tenant", tenantId: "dev-tenant" },
  },
  api: {
    endpoint: "http://localhost:8080",
  },
});

Common topology:

  • Grafana Cloud: generation basic mode with instance ID and API key.
  • Self-hosted direct to Sigil: generation tenant mode.
  • Traces/metrics via OTEL Collector/Alloy: configure exporters in your app OTEL SDK setup.
  • Enterprise proxy: generation bearer mode to proxy; proxy authenticates and forwards tenant header upstream.

Conversation Ratings

Use the SDK helper to submit user-facing ratings:

const result = await client.submitConversationRating("conv-123", {
  ratingId: "rat-123",
  rating: "CONVERSATION_RATING_VALUE_BAD",
  comment: "Answer ignored user context",
  metadata: { channel: "assistant-ui" },
  source: "sdk-js",
});

console.log(result.rating.rating, result.summary.hasBadRating);

submitConversationRating sends requests to api.endpoint (default http://localhost:8080) and uses the same generation-export auth headers (tenant or bearer) already configured on the SDK client.