Skip to content

feat: Oracle exporter can scrape more than one DB#6008

Open
ptodev wants to merge 5 commits intomainfrom
ptodev/test-oracle
Open

feat: Oracle exporter can scrape more than one DB#6008
ptodev wants to merge 5 commits intomainfrom
ptodev/test-oracle

Conversation

@ptodev
Copy link
Copy Markdown
Contributor

@ptodev ptodev commented Apr 8, 2026

Pull Request Details

The prometheus.exporter.oracledb component is currently not able to scrape more than one Oracle DB from one prometheus.exporter.oracledb. In order to do this, you'd need to setup more than one component. However, the upstream exporter contains global variables which cause bugs when more than one instance is used.

This PR updates Alloy to a more recent version of the upstream exporter. That version is able to scrape more than one DB. This PR will also change the configuration parameters of the component so that it's possible to set up different connection strings, usernames, passwords for each DB being scraped.

Notes to the Reviewer

I'd like to add integration tests later on, in a separate PR. I drafted them locally but they need more polish before I open them for review.

For now I tested this change locally, using this configuration:

Alloy config
prometheus.exporter.oracledb "oracles" {
  custom_metrics = ["/Users/paulintodev/Documents/GitHub/alloy-10/integration-tests/docker/tests/oracledb/custom-metrics.toml"]

  database {
    name = "db1"
    connection_string = "oracle://system:testpassword@localhost:1521/FREEPDB1"
  }
  database {
    name = "db2"
    connection_string = "oracle://system:testpassword@localhost:1522/FREEPDB1"
  }
  database {
    name = "db3"
    connection_string = "oracle://system:testpassword@localhost:1523/FREEPDB1"
  }
  database {
    name = "db4"
    connection_string = "oracle://system:testpassword@localhost:1524/FREEPDB1"
  }
  database {
    name = "db5"
    connection_string = "oracle://system:testpassword@localhost:1525/FREEPDB1"
  }

  query_timeout = 30
}

prometheus.scrape "oracles" {
  targets         = prometheus.exporter.oracledb.oracles.targets
  forward_to      = [prometheus.remote_write.oracledb_metrics.receiver]
  scrape_interval = "60s"
  scrape_timeout  = "55s"
}

prometheus.remote_write "oracledb_metrics" {
  endpoint {
    url = "https://prometheus-prod-05-gb-south-0.grafana.net/api/prom/push"

    basic_auth {
      username = ""
      password = ""
    }
    metadata_config {
      send_interval = "1s"
    }
    queue_config {
      max_samples_per_send = 100
    }
  }
  external_labels = {
    test_name = "oracledb_metrics",
  }
}

livedebugging {
  enabled = true
}
custom-metrics.toml
[[metric]]
context = "startup_time"
metricsdesc = {hours_up = "DBSTtartuptime"}
labels = ["db_name", "startup_time"]
request = '''
SELECT
        instance_name as db_name,
       startup_Time as startup_time,
        (sysdate - startup_time)*12 as hours_up
FROM
        V$instance
'''
Docker compose
name: alloy-integration-tests
services:
  oracledb:
    image: gvenzl/oracle-free:23-slim
    container_name: oracledb
    ports:
      - "1521:1521"
    environment:
      ORACLE_PASSWORD: testpassword
    healthcheck:
      test: ["CMD", "healthcheck.sh"]
      interval: 10s
      timeout: 10s
      retries: 30
      start_period: 90s
    networks:
      - integration-tests

  oracledb2:
    image: gvenzl/oracle-free:23-slim
    container_name: oracledb2
    ports:
      - "1522:1521"
    environment:
      ORACLE_PASSWORD: testpassword
    healthcheck:
      test: ["CMD", "healthcheck.sh"]
      interval: 10s
      timeout: 10s
      retries: 30
      start_period: 90s
    networks:
      - integration-tests

  oracledb3:
    image: gvenzl/oracle-free:23-slim
    container_name: oracledb3
    ports:
      - "1523:1521"
    environment:
      ORACLE_PASSWORD: testpassword
    healthcheck:
      test: ["CMD", "healthcheck.sh"]
      interval: 10s
      timeout: 10s
      retries: 30
      start_period: 90s
    networks:
      - integration-tests

  oracledb4:
    image: gvenzl/oracle-free:23-slim
    container_name: oracledb4
    ports:
      - "1524:1521"
    environment:
      ORACLE_PASSWORD: testpassword
    healthcheck:
      test: ["CMD", "healthcheck.sh"]
      interval: 10s
      timeout: 10s
      retries: 30
      start_period: 90s
    networks:
      - integration-tests

  oracledb5:
    image: gvenzl/oracle-free:23-slim
    container_name: oracledb5
    ports:
      - "1525:1521"
    environment:
      ORACLE_PASSWORD: testpassword
    healthcheck:
      test: ["CMD", "healthcheck.sh"]
      interval: 10s
      timeout: 10s
      retries: 30
      start_period: 90s
    networks:
      - integration-tests

networks:
  integration-tests:
    driver: bridge

PR Checklist

  • Documentation added
  • Tests updated
  • Config converters updated

@ptodev ptodev requested review from a team and clayton-cornell as code owners April 8, 2026 13:51
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 8, 2026

💻 Deploy preview available (feat: Oracle exporter can scrape more than one DB):

# TODO: Try enabling caching later. It might use up too much disk space on runners so needs extra testing.
cache: false
- run: GO_TAGS="embedalloyui promtail_journal_enabled" GOOS=${{ matrix.os }} GOARCH=${{ matrix.arch }} GOARM= make alloy
- run: GO_TAGS="embedalloyui promtail_journal_enabled godror" GOOS=${{ matrix.os }} GOARCH=${{ matrix.arch }} GOARM= make alloy
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I opened an upstream PR for this tag to not be necessary.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would we always build and run test with this tag?

Maybe we could just append it to GO_TAGS in our make file and make sure everything pass those tags, like integration tests etc. Then we don't have to do it for every workflow, WDYT?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the future we can switch to a different driver (go ora) which would mean Alloy no longer needs the Oracle Client to be installed. I opened an issue for it - #6011

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would we always build and run test with this tag?

Yes, until the next version of the exporter is released. My upstream PR to not require tag by default was merged. I hope there will be a new release soon - when we upgrade to it, we can delete this tag.

I opened an issue upstream to make the driver configurable in a more flexible way, without build tags. If this happens we can also make a new driver config attribute for people who want to choose the more basic driver that doesn't need an Instant Client installation.

we could just append it to GO_TAGS in our make file

True! I updated the PR to only do this amendment in the makefile. The changes to Actions workflows have been reverted.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 8, 2026

🔍 Dependency Review

Below is an assessment of each changed dependency in the PR, with required code updates (if any), linked evidence, and concise diffs where appropriate.

Note: I only assessed dependencies that changed in go.mod files (including indirects where they affect existing behavior), per the instructions.


github.com/oracle/oracle-db-appdev-monitoring v0.0.0-20250516154730-1d8025fde3b0 -> v0.0.0-20260323143131-1e9bbcecf9eb — ❌ Changes Needed

Impact

  • This is a significant update of the embedded oracledb exporter with:
    • Multi-database support (configure multiple DB targets with labels).
    • API changes in exporter construction and configuration (pointer fields).
    • Removal of implicit driver assumption and new need to pick driver (godror enabled via build tag).
    • A non-nil scrape interval in the metrics files configuration is required by the collector path.

Evidence

  • PR code changes in this repo adapting to new API:

    • Multi-DB support in component and integration layers:
      • internal/component/prometheus/exporter/oracledb/oracledb.go
      • internal/static/integrations/oracledb_exporter/oracledb_exporter.go
    • Documentation and config changes enabling multi-database and warning about single-process usage:
      • docs/sources/reference/components/prometheus/prometheus.exporter.oracledb.md
    • Build changes adding godror tag:
      • .golangci.yml and Makefile (GO_TAGS_COMMA)
  • References in code comments to upstream changes:

Key API/Behavior Changes to Apply (already applied in this PR)

  1. Exporter constructor behavior/contract
  • The construction path now expects a configuration that includes:
    • A Databases map keyed by name
    • DatabaseConfig.ConnectConfig fields as pointers
    • MetricsConfiguration.Metrics.ScrapeInterval must be set (non-nil)

Code update:

- oeExporter, err := oe.NewExporter(slogLogger, &oe.MetricsConfiguration{
-   Databases: map[string]oe.DatabaseConfig{ ... },
-   Metrics: oe.MetricsFilesConfig{
-     Custom:  c.CustomMetrics,
-     Default: c.DefaultMetrics,
-   },
- })
- if err != nil { return nil, err }
+ scrapeInterval := time.Duration(0)
+ oeExporter := oe.NewExporter(slogLogger, &oe.MetricsConfiguration{
+   Databases: databases, // built with ConnectConfig pointer fields
+   Metrics: oe.MetricsFilesConfig{
+     Custom:         c.CustomMetrics,
+     Default:        c.DefaultMetrics,
+     ScrapeInterval: &scrapeInterval,
+   },
+ })
+ if oeExporter == nil {
+   return nil, errors.New("failed to create oracledb exporter")
+ }
  1. ConnectConfig fields are now pointers
  • MaxIdleConns, MaxOpenConns, QueryTimeout must be set via pointers.

Code update:

- ConnectConfig: oe.ConnectConfig{
-   MaxIdleConns: c.MaxIdleConns,
-   MaxOpenConns: c.MaxOpenConns,
-   QueryTimeout: c.QueryTimeout,
- },
+ ConnectConfig: oe.ConnectConfig{
+   MaxIdleConns: ptr.To(c.MaxIdleConns),
+   MaxOpenConns: ptr.To(c.MaxOpenConns),
+   QueryTimeout: ptr.To(c.QueryTimeout),
+ },
  1. Multi-database support
  • The project introduced a first-class multi-target config:
    • A new database block in the component
    • Internal normalization of connection strings for both oracle://-style and host/service style
    • Passing a map[string]DatabaseConfig to the exporter with optional labels.

Code updates (component):

 type DatabaseTarget struct {
   Name             string            `alloy:"name,attr"`
   ConnectionString alloytypes.Secret `alloy:"connection_string,attr"`
   Username         string            `alloy:"username,attr,optional"`
   Password         alloytypes.Secret `alloy:"password,attr,optional"`
   Labels           map[string]string `alloy:"labels,attr,optional"`
 }
 
 type Arguments struct {
- ConnectionString alloytypes.Secret `alloy:"connection_string,attr"`
+ ConnectionString alloytypes.Secret `alloy:"connection_string,attr,optional"` // Deprecated
  ...
- Username string `alloy:"username,attr,optional"`
- Password alloytypes.Secret `alloy:"password,attr,optional"`
+ Username string `alloy:"username,attr,optional"` // Deprecated
+ Password alloytypes.Secret `alloy:"password,attr,optional"` // Deprecated
+ Databases DatabaseTargets `alloy:"database,block,optional"`
 }
 
+var (
+  errBothConfigModes = errors.New("cannot set connection_string together with database blocks; use one or the other")
+  errDuplicateDBName = errors.New("duplicate database name")
+)
 
 func (a *Arguments) Validate() error {
- if a.ConnectionString == "" {
-   return errNoConnectionString
- }
+ if len(a.Databases) > 0 && a.ConnectionString != "" { return errBothConfigModes }
+ if len(a.Databases) == 0 && a.ConnectionString == "" { return errNoConnectionString }
+ // validate each database block (non-empty name, unique, valid URL, etc.)
 }

Code updates (integration):

- // required driver for integration
- _ "github.com/sijms/go-ora/v2"
+// driver now selected via build tags; use godror tag and instant client

+func NormalizeConnectionString(connectionString, username, password string) (string, string, string) {
+  // Handle oracle://user:pass@host:port/svc => split out user/pass and strip scheme
+}
 
- func (c *Config) InstanceKey(agentKey string) (string, error) {
-   // previously derived from top-level connection_string
- }
+ func (c *Config) InstanceKey(defaultKey string) (string, error) {
+   // If multiple DBs => return defaultKey to avoid unstable high-cardinality instance
+ }
 
+ type DatabaseInstance struct {
+   Name             string             `yaml:"name"`
+   ConnectionString config_util.Secret `yaml:"connection_string"`
+   Username         string             `yaml:"username,omitempty"`
+   Password         config_util.Secret `yaml:"password,omitempty"`
+   Labels           map[string]string  `yaml:"labels,omitempty"`
+ }
 
 func New(logger log.Logger, c *Config) (integrations.Integration, error) {
-  oeExporter, err := oe.NewExporter(slogLogger, &oe.MetricsConfiguration{ ... })
-  if err != nil { return nil, err }
+  // Build Databases map[string]DatabaseConfig from single or multi-target config
+  // Set ScrapeInterval pointer
+  oeExporter := oe.NewExporter(slogLogger, &oe.MetricsConfiguration{ ... })
+  if oeExporter == nil { return nil, errors.New("failed to create oracledb exporter") }
 }
  1. Driver and environment requirements
  • The PR enables the “godror” build tag and documents Instant Client requirements.
    • Makefile ensures the “godror” tag is on by default.
    • docs updated to instruct setting DYLD_LIBRARY_PATH on Mac ARM and note Instant Client requirement.
  • These are operational/runtime requirements when using godror.

Recommended follow-up (already in PR)


github.com/godror/godror v0.48.1 -> v0.50.0 — ⚠️ Needs Review

Impact

  • Oracle DB driver for database/sql used by the embedded exporter. The project now opts into this path with the “godror” build tag.
  • Upgrading to 0.50.0 typically doesn’t require code changes unless you use driver-specific APIs; this project doesn’t import godror directly.
  • Operational impact: Using godror requires Oracle Instant Client and CGO. The docs and Makefile changes in this PR already cover this.

Evidence

  • Build/tag changes and docs updates present in this PR.
  • knownpb bump to v0.3.0 aligns with newer godror.

Action

  • No code changes in this repository beyond what’s included in this PR.
  • Ensure runtime env includes Oracle Instant Client and required env vars as documented.

github.com/godror/knownpb v0.1.2 -> v0.3.0 — ✅ Safe
  • Indirect support package for godror. Not used directly in this repository. No code changes required.

github.com/hashicorp/vault/api v1.20.0 -> v1.22.0 — ✅ Safe

Impact

  • Vault API minor updates. Existing code generally continues to compile with no changes.
  • No Vault API usage changes were made in this PR, suggesting no breaking changes affected current usage paths.

Action

  • No code changes needed here.

github.com/hashicorp/go-secure-stdlib/parseutil v0.1.6 -> v0.2.0 — ✅ Safe
  • Minor update in a utility library used indirectly. No usage in this repo that requires code changes.

github.com/oracle/oci-go-sdk/v65 v65.89.3 -> v65.108.3 — ✅ Safe
  • Indirect dependency. No code in this repository imports it directly. No action required.

github.com/Azure/azure-sdk-for-go/sdk/keyvault/azsecrets v0.12.0 -> github.com/Azure/azure-sdk-for-go/sdk/security/keyvault/azsecrets v1.4.0 — ✅ Safe

Impact

  • The Azure Key Vault Secrets SDK moved under sdk/security/keyvault and advanced to a GA v1 line.
  • In this repository it’s an indirect dependency; there is no direct usage to update here.

Note

  • If you ever adopt it directly, the package path changes and some surface details changed across the GA cut, but that is not applicable here.

github.com/Azure/azure-sdk-for-go/sdk/security/keyvault/internal v1.1.1 -> v1.2.0 — ✅ Safe
  • Internal helper package for the Azure SDK. Indirect dependency. No code updates needed.

github.com/BurntSushi/toml v1.5.0 -> v1.6.0 — ✅ Safe
  • Minor release with bug fixes and small improvements. No breaking API changes for typical use. No updates required.

github.com/sony/gobreaker v0.5.0 -> v1.0.0 — ✅ Safe
  • gobreaker v1.0.0 did not introduce disruptive API changes for standard use. In this repository it appears only indirectly. No code updates needed.

Notes

  • Net-new indirect: github.com/VictoriaMetrics/easyproto v1.1.3 — not used directly here; no action required.
  • Changes to build tags and environment for the oracledb exporter:
    • A “godror” build tag is enforced in the Makefile and linter configuration to ensure the driver is included, pending upstream change (see Makefile TODO and link to upstream PR).
    • Operational docs were improved to note Oracle Instant Client requirements and DYLD_LIBRARY_PATH on Mac ARM.

If you need sample configuration/code updates for the oracledb exporter (multi-database), the PR already contains working patterns. Here are minimal diffs to mirror them if integrating elsewhere:

  • Component arguments validation and multi-db wiring:
+ Databases        DatabaseTargets   `alloy:"database,block,optional"`
+ // Validate: forbid mixing top-level connection_string with database blocks.
+ if len(a.Databases) > 0 && a.ConnectionString != "" { return errBothConfigModes }
  • Normalization of oracle:// connection strings:
+ u, user, pass := oracledb_exporter.NormalizeConnectionString(string(d.ConnectionString), d.Username, string(d.Password))
+ cfg.Databases = append(cfg.Databases, oracledb_exporter.DatabaseInstance{ ... })
  • Exporter construction with non-nil pointer fields:
+ scrapeInterval := time.Duration(0)
+ oeExporter := oe.NewExporter(logger, &oe.MetricsConfiguration{
+   Databases: dbs,
+   Metrics: oe.MetricsFilesConfig{
+     Custom:         c.CustomMetrics,
+     Default:        c.DefaultMetrics,
+     ScrapeInterval: &scrapeInterval,
+   },
+ })

These changes maintain existing behavior (single DB config works as before) while enabling multi-database support.

Copy link
Copy Markdown
Contributor

@kalleep kalleep left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, some nits

# TODO: Try enabling caching later. It might use up too much disk space on runners so needs extra testing.
cache: false
- run: GO_TAGS="embedalloyui promtail_journal_enabled" GOOS=${{ matrix.os }} GOARCH=${{ matrix.arch }} GOARM= make alloy
- run: GO_TAGS="embedalloyui promtail_journal_enabled godror" GOOS=${{ matrix.os }} GOARCH=${{ matrix.arch }} GOARM= make alloy
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would we always build and run test with this tag?

Maybe we could just append it to GO_TAGS in our make file and make sure everything pass those tags, like integration tests etc. Then we don't have to do it for every workflow, WDYT?

@ptodev ptodev force-pushed the ptodev/test-oracle branch from 43cf816 to 2842de6 Compare April 10, 2026 11:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants