LGTM Exporter

Status
Stability	alpha: logs, metrics, traces
Distributions	contrib
Issues

Overview

The LGTM exporter is an OpenTelemetry Collector component that sends logs, metrics, and traces to Grafana's LGTM stack (Loki, Grafana, Tempo, Mimir). This exporter is designed to work with the native OTLP endpoints provided by each component of the LGTM stack while handling multi-tenancy through flexible tenant configuration.

Key features:

OpenTelemetry Collector exporter for logs, metrics, and traces
Native integration with Loki, Mimir, and Tempo OTLP endpoints
Multi-tenancy support with flexible tenant ID extraction from resource attributes
Separate HTTP client configuration for each signal type
Support for authentication extensions (Basic Auth, Bearer Token, OAuth2, etc.)
Built-in retry and queue configurations
Telemetry metrics for monitoring exporter performance

Configuration

The following settings are required:

At least one of logs, metrics, or traces must be configured with an endpoint

The following settings are optional:

Signal Endpoint Configuration

Configure endpoints for the signals you want to export. Each signal (logs, metrics, traces) can be configured independently with its own HTTP client settings.

Logs

logs.endpoint (no default): The Loki OTLP endpoint URL (e.g., http://loki:3100/otlp/v1/logs)
Supports all HTTP Client Configuration options below (prefixed with logs.)

Metrics

metrics.endpoint (no default): The Mimir OTLP endpoint URL (e.g., http://mimir:8080/otlp/v1/metrics)
Supports all HTTP Client Configuration options below (prefixed with metrics.)

Traces

traces.endpoint (no default): The Tempo endpoint URL (e.g., http://tempo:3201/v1/traces)
Supports all HTTP Client Configuration options below (prefixed with traces.)

HTTP Client Configuration

Each signal type (logs, metrics, traces) supports the following HTTP client configuration options. Simply prefix the option with the signal name (e.g., logs.timeout, metrics.compression, traces.tls):

endpoint (no default): The target URL to send data to
timeout (default = 0s/unlimited): HTTP request timeout
headers (no default): Additional HTTP headers to send with requests
tls (no default): TLS configuration settings (see configtls)
proxy_url (no default): Proxy URL to use for HTTP requests
auth (no default): Authentication extension configuration. Supports all standard OpenTelemetry authentication extensions including Basic Auth, Bearer Token, OAuth2, and more. Each signal can use a different authenticator. (see configauth)
compression (default = gzip): Compression type (gzip, snappy, zstd, zlib, deflate, or none)
compression_params (no default): Advanced compression configuration
read_buffer_size (default = 0): HTTP client read buffer size
write_buffer_size (default = 0): HTTP client write buffer size
max_idle_conns (default = 100): Maximum idle HTTP connections (0 = no limit)
max_idle_conns_per_host (default = system default): Maximum idle HTTP connections per host
max_conns_per_host (default = 0/unlimited): Maximum total connections per host
idle_conn_timeout (default = 90s): Maximum time a connection remains open before closing
disable_keep_alives (default = false): Disable HTTP keep-alives
http2_read_idle_timeout (default = 0s): HTTP/2 health check ping interval
http2_ping_timeout (default = 15s): HTTP/2 ping response timeout
force_attempt_http2 (default = true): Force HTTP/2 protocol usage

For the complete list of HTTP client configuration options, see confighttp.

Authentication Configuration

The exporter supports OpenTelemetry Collector authentication extensions through the auth configuration option. Each signal (logs, metrics, traces) can be configured with its own authenticator, allowing you to use different authentication methods for different backends.

To use authentication:

Configure one or more authentication extensions in the extensions section
Reference the extension ID in the signal's auth.authenticator field
Add the extension to the service's extensions list

Supported authentication extensions include:

Basic Auth (basicauthextension): Username/password authentication
Bearer Token (bearertokenauthextension): Token-based authentication
OAuth2 (oauth2clientauthextension): OAuth 2.0 client credentials flow
OIDC (oidcauthextension): OpenID Connect authentication
Sigv4 (sigv4authextension): AWS Signature Version 4 authentication

Example configuration with different authenticators per signal:

extensions:
  basicauth/loki:
    client_auth:
      username: loki-user
      password: loki-pass
  bearertokenauth/mimir:
    token: mimir-token-here

exporters:
  lgtm:
    logs:
      endpoint: https://loki.example.com/otlp/v1/logs
      auth:
        authenticator: basicauth/loki
    metrics:
      endpoint: https://mimir.example.com/otlp/v1/metrics
      auth:
        authenticator: bearertokenauth/mimir
    traces:
      endpoint: https://tempo.example.com/v1/traces
      # No auth configured for traces

service:
  extensions: [basicauth/loki, bearertokenauth/mimir]
  pipelines:
    logs:
      exporters: [lgtm]

For more details on authentication, see configauth.

Content Type Configuration

The exporter supports configuring the content type for the request body sent to LGTM backends:

content_type (default = protobuf): The format used to encode telemetry data in the request body
- protobuf: Sends data in Protocol Buffers binary format with Content-Type: application/x-protobuf header (recommended)
- json: Sends data in JSON format with Content-Type: application/json header

Protocol Buffers (protobuf) is the recommended format as it is more efficient in terms of size and serialization performance. JSON format may be useful for debugging or compatibility with specific backend configurations.

Tenant Configuration

The exporter supports multi-tenancy by extracting tenant IDs from resource attributes and setting them in HTTP headers:

tenant.label (default = tenant.id): Primary resource attribute name to look for tenant ID
tenant.labels (default = []): Additional resource attribute names to search for tenant ID (fallback order)
tenant.format (default = %s): Format string for the tenant ID value (uses Go fmt syntax)
tenant.header (default = X-Scope-OrgID): HTTP header name to set with the tenant ID
tenant.default (default = default): Default tenant ID when no tenant attribute is found

The exporter searches for tenant IDs in the following order:

tenant.label attribute
Each attribute in tenant.labels (in order)
Falls back to tenant.default if none found

Queue and Retry Configuration

The exporter supports standard OpenTelemetry Collector queue and retry settings:

sending_queue:
- enabled (default = true): Enable the sending queue
- num_consumers (default = 10): Number of consumers that dequeue batches
- queue_size (default = 1000): Maximum number of batches kept in memory before dropping data
retry_on_failure:
- enabled (default = true): Enable retry on failure
- initial_interval (default = 5s): Time to wait after the first failure before retrying
- max_interval (default = 30s): Upper bound on backoff interval
- max_elapsed_time (default = 300s): Maximum amount of time spent trying to send a batch

For more details, see exporterhelper configuration.

Examples

Basic Configuration

This example shows a basic HTTP configuration:

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317

processors:
  batch:

exporters:
  lgtm:
    logs:
      endpoint: http://loki:3100/otlp/v1/logs
      timeout: 30s
      tls:
        insecure: true
    metrics:
      endpoint: http://mimir:8080/otlp/v1/metrics
      timeout: 30s
      tls:
        insecure: true
    traces:
      endpoint: http://tempo:3200/v1/traces
      timeout: 30s
      tls:
        insecure: true
    tenant:
      label: tenant.id
      labels:
        - tenant_id
        - tenantID
        - tenantId
      header: X-Scope-OrgID
      format: "%s-dev"
      default: default
    content_type: protobuf
    retry_on_failure:
      enabled: true
      initial_interval: 5s
      max_interval: 30s
    sending_queue:
      enabled: true
      num_consumers: 10
      queue_size: 1000

service:
  pipelines:
    logs:
      receivers: [otlp]
      processors: [batch]
      exporters: [lgtm]
    metrics:
      receivers: [otlp]
      processors: [batch]
      exporters: [lgtm]
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [lgtm]

Configuration with TLS, Custom Headers and Auth Extensions

This example shows HTTPS configuration with TLS certificates, custom headers, and different authentication methods for each signal:

extensions:
  # Basic auth for Loki
  basicauth/loki:
    client_auth:
      username: ${LOKI_USERNAME}
      password: ${LOKI_PASSWORD}
  
  # Bearer token for Mimir
  bearertokenauth/mimir:
    token: ${MIMIR_TOKEN}

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317

processors:
  batch:

exporters:
  lgtm:
    logs:
      endpoint: https://loki.example.com/otlp/v1/logs
      timeout: 30s
      auth:
        authenticator: basicauth/loki
      headers:
        - name: X-Custom-Header
          value: custom-value
      tls:
        insecure: false
        cert_file: /etc/certs/client-cert.pem
        key_file: /etc/certs/client-key.pem
        ca_file: /etc/certs/ca.pem
      compression: gzip
    metrics:
      endpoint: https://mimir.example.com/otlp/v1/metrics
      timeout: 30s
      auth:
        authenticator: bearertokenauth/mimir
      headers:
        - name: X-Custom-Header
          value: custom-value
        - name: X-Environment
          value: production
      tls:
        insecure: false
        cert_file: /etc/certs/client-cert.pem
        key_file: /etc/certs/client-key.pem
        ca_file: /etc/certs/ca.pem
      compression: gzip
    traces:
      endpoint: https://tempo.example.com/v1/traces
      timeout: 30s
      headers:
        - name: X-Custom-Header
          value: custom-value
        - name: X-Trace-Source
          value: collector
      tls:
        insecure: false
        cert_file: /etc/certs/client-cert.pem
        key_file: /etc/certs/client-key.pem
        ca_file: /etc/certs/ca.pem
      compression: gzip
    tenant:
      label: tenant.id
      labels:
        - tenant_id
        - k8s.namespace.name
      format: "%s"
      header: X-Scope-OrgID
      default: anonymous
    content_type: protobuf
    retry_on_failure:
      enabled: true
      initial_interval: 5s
      max_interval: 30s
      max_elapsed_time: 300s
    sending_queue:
      enabled: true
      num_consumers: 10
      queue_size: 1000

service:
  extensions: [basicauth/loki, bearertokenauth/mimir]
  pipelines:
    logs:
      receivers: [otlp]
      processors: [batch]
      exporters: [lgtm]
    metrics:
      receivers: [otlp]
      processors: [batch]
      exporters: [lgtm]
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [lgtm]

Configuration with JSON Content Type

This example shows how to configure the exporter to send data in JSON format instead of the default Protocol Buffers format:

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317

processors:
  batch:

exporters:
  lgtm:
    content_type: json  # Send data as JSON instead of protobuf
    logs:
      endpoint: http://loki:3100/otlp/v1/logs
    metrics:
      endpoint: http://mimir:8080/otlp/v1/metrics
    traces:
      endpoint: http://tempo:3200/v1/traces
    tenant:
      label: tenant.id
      default: default

service:
  pipelines:
    logs:
      receivers: [otlp]
      processors: [batch]
      exporters: [lgtm]
    metrics:
      receivers: [otlp]
      processors: [batch]
      exporters: [lgtm]
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [lgtm]

Note: Protocol Buffers (protobuf) is recommended for production use as it provides better performance and smaller payload sizes. JSON format is primarily useful for debugging or specific backend requirements.

Telemetry

The exporter emits the following metrics to monitor its performance:

otelcol_exporter_lgtm_sent_samples_total: Number of samples sent to LGTM (split by signal type, tenant, and HTTP status code)
otelcol_exporter_lgtm_sent_requests_total: Number of requests sent to LGTM (split by signal type, tenant, and HTTP status code)
otelcol_exporter_lgtm_write_latency_milliseconds: Latency of write requests to LGTM in milliseconds (split by HTTP status code and URL)

Testing and Development

The exporter includes a comprehensive test suite with equal coverage across all signal types (logs, metrics, traces):

Unit Tests: Complete test coverage for processor operations including partitioning, sending, and dispatching telemetry data
Integration Tests: Docker Compose-based integration test environment in the test/ directory

Integration Test Environment

The test/ directory contains a complete Docker Compose stack for testing the exporter:

docker-compose.yml: Orchestrates the full LGTM stack with the OpenTelemetry Collector
grafana/: Grafana configuration with pre-configured datasources for Loki, Mimir, and Tempo
loki/: Loki configuration for receiving OTLP logs
mimir/: Mimir configuration for receiving OTLP metrics
tempo/: Tempo configuration for receiving OTLP traces
otel-collector/: OpenTelemetry Collector configuration using the LGTM exporter
test-client/: Test scripts for sending sample telemetry data
- send-logs.sh: Send sample logs via OTLP
- send-metrics.sh: Send sample metrics via OTLP
- send-traces.sh: Send sample traces via OTLP
- send-telemetry.sh: Send all signal types (runs automatically in docker-compose)

Testing

The test/ directory contains a complete integration test environment using Docker Compose with the full LGTM stack:

Test Infrastructure

The docker-compose setup includes:

Loki: Log aggregation system
Mimir: Metrics storage
Tempo: Distributed tracing backend
Grafana: Visualization and dashboards
OpenTelemetry Collector: Configured with the LGTM exporter
test-client: Automated telemetry generator

Running Tests

cd test
docker compose up

The test-client service automatically generates and sends sample logs, metrics, and traces to the OTEL Collector, which then exports them to the appropriate LGTM stack components. No manual data sending is required - the test-client runs the send-telemetry.sh script automatically on startup.

Access Grafana at http://localhost:3000 (admin/admin) to visualize the exported data:

Logs: Query Loki datasource for tenant labels
Metrics: Browse Mimir metrics
Traces: Search Tempo traces

Test Data

The test-client generates telemetry for multiple tenants (tenant-a, tenant-b) and services (web-app, api-service, database, cache, auth-service), demonstrating multi-tenancy support. Plus randomly removes the tenant label to generate data for the default tenant.

Compatibility

This exporter is compatible with:

Grafana Loki v2.9+ (OTLP endpoint)
Grafana Mimir v2.10+ (OTLP endpoint)
Grafana Tempo v2.2+
Any other system compatible with the LGTM stack's OTLP endpoints

Name		Name	Last commit message	Last commit date
Latest commit History 16 Commits
.github		.github
internal/metadata		internal/metadata
test		test
testdata		testdata
.gitignore		.gitignore
Dockerfile		Dockerfile
README.md		README.md
builder-config.yaml		builder-config.yaml
config.go		config.go
config_test.go		config_test.go
doc.go		doc.go
exporter.go		exporter.go
exporter_test.go		exporter_test.go
factory.go		factory.go
factory_test.go		factory_test.go
go.mod		go.mod
go.sum		go.sum
metadata.yaml		metadata.yaml
processor.go		processor.go
processor_test.go		processor_test.go

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

LGTM Exporter

Overview

Configuration

Signal Endpoint Configuration

Logs

Metrics

Traces

HTTP Client Configuration

Authentication Configuration

Content Type Configuration

Tenant Configuration

Queue and Retry Configuration

Examples

Basic Configuration

Configuration with TLS, Custom Headers and Auth Extensions

Configuration with JSON Content Type

Telemetry

Testing and Development

Integration Test Environment

Testing

Test Infrastructure

Running Tests

Test Data

Compatibility

About

Uh oh!

Releases 6

Packages

Uh oh!

Contributors

Uh oh!

Languages

Folders and files

Latest commit

History

Repository files navigation

LGTM Exporter

Overview

Configuration

Signal Endpoint Configuration

Logs

Metrics

Traces

HTTP Client Configuration

Authentication Configuration

Content Type Configuration

Tenant Configuration

Queue and Retry Configuration

Examples

Basic Configuration

Configuration with TLS, Custom Headers and Auth Extensions

Configuration with JSON Content Type

Telemetry

Testing and Development

Integration Test Environment

Testing

Test Infrastructure

Running Tests

Test Data

Compatibility

About

Resources

Uh oh!

Stars

Watchers

Forks

Releases 6

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Packages