sacloud-otel-collector

OpenTelemetry collector for sacloud.

Usage

Release Binary

Pre-built binaries are available on GitHub Releases:

1. Download the archive for your platform from the latest release (tar.gz for Linux/macOS, zip for Windows)
2. Extract the archive: tar xzf sacloud-otel-collector_*.tar.gz (Linux/macOS) or unzip sacloud-otel-collector_*.zip (Windows)
3. Run with your configuration:

./sacloud-otel-collector --config config.yml

On Windows:

.\sacloud-otel-collector.exe --config config.yml

Container Image

Container images are available on GitHub Container Registry:

docker pull ghcr.io/sacloud/sacloud-otel-collector:latest

Run the collector with your configuration:

docker run --rm -v $(pwd)/config.yml:/config.yml ghcr.io/sacloud/sacloud-otel-collector:latest --config /config.yml

RPM/DEB Package

RPM and DEB packages are available on GitHub Releases.

Installation

For Debian/Ubuntu:

wget https://github.com/sacloud/sacloud-otel-collector/releases/download/v<version>/sacloud-otel-collector_<version>_linux_amd64.deb
sudo dpkg -i sacloud-otel-collector_<version>_linux_amd64.deb

For RHEL/CentOS/Fedora:

wget https://github.com/sacloud/sacloud-otel-collector/releases/download/v<version>/sacloud-otel-collector_<version>_linux_amd64.rpm
sudo rpm -i sacloud-otel-collector_<version>_linux_amd64.rpm

Configuration

Edit the configuration file at /etc/sacloud-otel-collector/config.yaml:

sudo vi /etc/sacloud-otel-collector/config.yaml

The default configuration includes receivers for OTLP, host metrics, and file logs, with exporters configured for SAKURA Cloud Monitoring Suite. You need to replace the **** placeholders with your actual monitoring suite credentials.

Service Management

Enable and start the service:

sudo systemctl enable sacloud-otel-collector
sudo systemctl start sacloud-otel-collector

Check service status:

sudo systemctl status sacloud-otel-collector

View logs:

sudo journalctl -u sacloud-otel-collector -f

Reload configuration after changes:

sudo systemctl reload sacloud-otel-collector

Components

The sacloud-otel-collector includes the following OpenTelemetry components:

For more details, see builder-config.yaml.

Receivers

Component	Description	Documentation
otlp	OpenTelemetry Protocol receiver	Documentation
prometheus	Prometheus metrics receiver	Documentation
hostmetrics	Host metrics receiver	Documentation
jaeger	Jaeger traces receiver	Documentation
kafka	Kafka receiver	Documentation
filelog	File log receiver	Documentation
fluentforward	Fluent Forward receiver	Documentation
journald	Journald receiver	Documentation

Processors

Component	Description	Documentation
batch	Batch processor	Documentation
memorylimiter	Memory limiter processor	Documentation
resource	Resource processor	Documentation
attributes	Attributes processor	Documentation
transform	Transform processor	Documentation
filter	Filter processor	Documentation
deltatocumulative	Delta to cumulative processor	Documentation
resourcedetection	Resource detection processor	Documentation
tailsampling	Tail sampling processor	Documentation
groupbyattrs	Group by attributes processor	Documentation

Exporters

Component	Description	Documentation
debug	Debug exporter	Documentation
otlp	OpenTelemetry Protocol exporter	Documentation
prometheusremotewrite	Prometheus Remote Write exporter	Documentation
otlphttp	OTLP HTTP exporter	Documentation
file	File exporter	Documentation
elasticsearch	Elasticsearch exporter	Documentation
awss3	AWS S3 exporter	Documentation
kafka	Kafka exporter	Documentation
sacloud	SAKURA Cloud Monitoring Suite exporter	See Config examples
mackerelotlp	Mackerel OTLP exporter	Documentation

Extensions

Component	Description	Documentation
health_check	Health check extension	Documentation
file_storage	File storage extension	Documentation

Build

See Building a custom collector.

1. Install Go.
2. make See Makefile for details.

sacloud-otel-collector will be created.

Config examples

Sakuracloud Monitoring Suite

See manual.

Using the sacloud exporter (recommended)

The sacloud exporter simplifies configuration for SAKURA Cloud Monitoring Suite with sensible defaults.

Features:

Feature	Description
Endpoint	Endpoint ID (e.g., 123456789012) or full URL
Compression	snappy for metrics, gzip for logs/traces
Timeout	30s (configurable)
Retry	Exponential backoff (configurable)
Queue	Optimized for 5MB per request limit

See also SAKURA Cloud Monitoring Suite limits.

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318
  hostmetrics:
    collection_interval: 10s
    scrapers:
      cpu:
        metrics:
          system.cpu.utilization:
            enabled: true
      memory:
        metrics:
          system.memory.utilization:
            enabled: true
      disk:
      filesystem:
        metrics:
          system.filesystem.utilization:
            enabled: true
      network:
      paging:
        metrics:
          system.paging.utilization:
            enabled: true
  filelog:
    start_at: end
    exclude: []
    include:
      - /var/log/example.log

processors:
  resourcedetection:
    detectors: [system]
    system:
      hostname_sources: [os]

# Replace endpoint identifiers and tokens with your monitoring suite's configurations.
exporters:
  sacloud:
    metrics:
      endpoint: "123456789012" # or "https://123456789012.metrics.monitoring.global.api.sacloud.jp/prometheus/api/v1/write"
      token: "${SACLOUD_METRICS_TOKEN}" # met-***************
    logs:
      endpoint: "123456789012" # or "https://123456789012.logs.monitoring.global.api.sacloud.jp"
      token: "${SACLOUD_LOGS_TOKEN}" # log-***************
    traces:
      endpoint: "123456789012" # or "https://123456789012.traces.monitoring.global.api.sacloud.jp"
      token: "${SACLOUD_TRACES_TOKEN}" # trc-***************

service:
  pipelines:
    metrics:
      receivers: [hostmetrics]
      processors: [resourcedetection]
      exporters: [sacloud]
    logs:
      receivers: [filelog]
      processors: [resourcedetection]
      exporters: [sacloud]
    traces:
      receivers: [otlp]
      processors: [resourcedetection]
      exporters: [sacloud]

Advanced: Timeout and Retry Configuration

The exporter includes sensible defaults for timeout (30 seconds) and retry (enabled with exponential backoff). You can customize these settings if needed:

exporters:
  sacloud:
    # Timeout for HTTP requests (default: 30s)
    timeout: 30s
    # Retry configuration (default: enabled with exponential backoff)
    retry_on_failure:
      enabled: true
      initial_interval: 5s    # Time to wait after first failure
      max_interval: 30s       # Maximum backoff interval
      max_elapsed_time: 5m    # Maximum total retry time before giving up
    metrics:
      endpoint: "123456789012"
      token: "${SACLOUD_METRICS_TOKEN}"
    logs:
      endpoint: "123456789012"
      token: "${SACLOUD_LOGS_TOKEN}"
    traces:
      endpoint: "123456789012"
      token: "${SACLOUD_TRACES_TOKEN}"

Using standard exporters

Alternatively, you can use standard exporters directly:

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318
  hostmetrics:
    collection_interval: 10s
    scrapers:
      cpu:
        metrics:
          system.cpu.utilization:
            enabled: true
      memory:
        metrics:
          system.memory.utilization:
            enabled: true
      disk:
      filesystem:
        metrics:
          system.filesystem.utilization:
            enabled: true
      network:
      paging:
        metrics:
          system.paging.utilization:
            enabled: true
  filelog:
    start_at: end
    exclude: []
    include:
      - /var/log/example.log

processors:
  resourcedetection:
    detectors: [system]
    system:
      hostname_sources: [os]
  batch:
    timeout: 1s
    # Adjust the send_batch_size/send_batch_max_size so that
    # the payload size of a single request does not exceed 5 MiB.
    send_batch_size: 4096
    send_batch_max_size: 4096

# You should replace `****` to your monitoring suite's configurations.
exporters:
  otlphttp/sakura-monitoring-suite-log:
    endpoint: https://****.logs.monitoring.global.api.sacloud.jp
    headers:
      Authorization: "Bearer ****"
  prometheusremotewrite/sakura-monitoring-suite-metrics:
    endpoint: https://****.metrics.monitoring.global.api.sacloud.jp/prometheus/api/v1/write
    headers:
      Authorization: "Bearer ****"
    resource_to_telemetry_conversion:
      enabled: true
  otlphttp/sakura-monitoring-suite-trace:
    endpoint: https://****.traces.monitoring.global.api.sacloud.jp
    headers:
      Authorization: "Bearer ****"

service:
  pipelines:
    metrics:
      receivers:
        - hostmetrics
      processors:
        - resourcedetection
        - batch
      exporters:
        - prometheusremotewrite/sakura-monitoring-suite-metrics
    logs:
      receivers:
        - filelog
      processors:
        - resourcedetection
        - batch
      exporters:
        - otlphttp/sakura-monitoring-suite-log
    traces:
      receivers:
        - otlp
      processors:
        - resourcedetection
        - batch
      exporters:
        - otlphttp/sakura-monitoring-suite-trace

Contributing

If you want to add a new component of otel collector, modify builder-config.yaml. Then run make build-src && make to build the collector.

E2E test

CI runs an end-to-end test on Linux, macOS and Windows for each push to the main branch: a filelog receiver reads a file and the sacloud exporter sends the logs to the sakumock monitoring-suite data plane, asserting the dumped payload. To run it locally:

make
go install github.com/sacloud/sakumock/monitoringsuite/cmd/sakumock-monitoringsuite@v0.6.0
cd e2e && go test -v ./...

Automation

GitHub Actions are used to automate the build and release process. The following steps are performed:

1. When new branches are merged into the main branch, tagpr creates/updates a new pull request for the release.
2. When the release pull request is merged, the following steps are performed:
   1. The main branch is tagged with a new version number.
   2. The main branch is built and released to GitHub Releases.

License

sacloud-otel-collector Copyright (C) 2025 The sacloud/sacloud-otel-collector authors. This project is published under Apache 2.0 License.