Skip to content

discovery.relabel's targets doesn't parse local.file.target.content correctly #6163

@suspiciousRaccoon

Description

@suspiciousRaccoon

Component(s)

local.file, discovery.relabel

What's wrong?

When a local.file.target.content is passed as __target__ in a discovery.relabel.targets block the content is read as json {true "value"} instead of a string value. Changing is_secret to false will only change the boolean of the json to false: {false "value"}

Wrapping it in convert.nonsensitive() is a workaround that seems to work.

Steps to reproduce

Run the following commands:

touch service_ip 
touch docker-compose.yaml
touch config.alloy
cat "value" >> service_ip
# copy config from Configuration
docker compose up

System information

Kernel Version: 6.18.13-200.fc43.x86_64 (64-bit)

Software version

Grafana Alloy 1.15.1

Configuration

// config.alloy

local.file "service_ip" {
	filename  = "/etc/alloy/service_ip"
	is_secret = true
}

discovery.relabel "backend_relabel" {
	targets = [{
		__address__ = local.file.service_ip.content, 
	}]	
}
logging {
  level = "debug"
}
prometheus.scrape "backend_scrape" {
	targets    = discovery.relabel.backend_relabel.output
	forward_to = [prometheus.remote_write.prometheus.receiver]
}

prometheus.remote_write "prometheus" {
	endpoint {
		url = "https://example.com/v1/write"
	}
}



// docker-compose.yaml
services:
  alloy:
    container_name: alloy
    image: grafana/alloy:${GRAFANA_ALLOY_VERSION:-v1.15.1}
    privileged: true
    network_mode: host
    environment:
        ALLOY_DEPLOY_MODE: docker
    volumes:
        - ./config.alloy:/etc/alloy/config.alloy
        - ./service_ip:/etc/alloy/service_ip
    command: run --server.http.listen-addr=127.0.0.1:12345 --storage.path=/var/lib/alloy/data /etc/alloy/config.alloy
    devices:
          - /dev/kmsg

Logs

alloy  | ts=2026-04-30T19:44:56.84712229Z level=info boringcrypto_enabled=false
alloy  | ts=2026-04-30T19:44:56.834163268Z level=info source=/go/pkg/mod/github.com/!kim!machine!gun/automemlimit@v0.7.5/memlimit/memlimit.go:175 msg="memory is not limited, skipping" package=github.com/KimMachineGun/automemlimit/memlimit
alloy  | ts=2026-04-30T19:44:56.847203043Z level=info msg="no peer discovery configured: both join and discover peers are empty" service=cluster
alloy  | ts=2026-04-30T19:44:56.847212702Z level=debug msg="Running alloy controller" controller_id=""
alloy  | ts=2026-04-30T19:44:56.847219324Z level=info msg="running usage stats reporter"
alloy  | ts=2026-04-30T19:44:56.847224224Z level=info msg="starting complete graph evaluation" controller_path=/ controller_id="" trace_id=eb42702d2d9a611daebcf2bcf7cc514f
alloy  | ts=2026-04-30T19:44:56.84723835Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=eb42702d2d9a611daebcf2bcf7cc514f node_id=livedebugging duration=7.855µs
alloy  | ts=2026-04-30T19:44:56.847247818Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=eb42702d2d9a611daebcf2bcf7cc514f node_id=labelstore duration=2.334µs
alloy  | ts=2026-04-30T19:44:56.847256284Z level=info msg="replaying WAL, this may take a while" component_path=/ component_id=prometheus.remote_write.prometheus subcomponent=wal dir=/var/lib/alloy/data/prometheus.remote_write.prometheus/wal
alloy  | ts=2026-04-30T19:44:56.84726442Z level=info msg="WAL segment loaded" component_path=/ component_id=prometheus.remote_write.prometheus subcomponent=wal segment=0 maxSegment=1
alloy  | ts=2026-04-30T19:44:56.847271703Z level=info msg="WAL segment loaded" component_path=/ component_id=prometheus.remote_write.prometheus subcomponent=wal segment=1 maxSegment=1
alloy  | ts=2026-04-30T19:44:56.847278116Z level=info msg="Starting WAL watcher" component_path=/ component_id=prometheus.remote_write.prometheus subcomponent=rw remote_name=48f759 url=https://example.com/v1/write queue=48f759
alloy  | ts=2026-04-30T19:44:56.847287073Z level=info msg="Starting scraped metadata watcher" component_path=/ component_id=prometheus.remote_write.prometheus subcomponent=rw remote_name=48f759 url=https://example.com/v1/write
alloy  | ts=2026-04-30T19:44:56.847295689Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=eb42702d2d9a611daebcf2bcf7cc514f node_id=prometheus.remote_write.prometheus duration=2.975896ms
alloy  | ts=2026-04-30T19:44:56.847306069Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=eb42702d2d9a611daebcf2bcf7cc514f node_id=local.file.service_ip duration=143.101µs
alloy  | ts=2026-04-30T19:44:56.847314555Z level=info msg="Replaying WAL" component_path=/ component_id=prometheus.remote_write.prometheus subcomponent=rw remote_name=48f759 url=https://example.com/v1/write queue=48f759
alloy  | ts=2026-04-30T19:44:56.847322189Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=eb42702d2d9a611daebcf2bcf7cc514f node_id=discovery.relabel.backend_relabel duration=64.071µs
alloy  | ts=2026-04-30T19:44:56.847330395Z level=debug msg="Tailing WAL" component_path=/ component_id=prometheus.remote_write.prometheus subcomponent=rw remote_name=48f759 url=https://example.com/v1/write lastCheckpoint="" checkpointIndex=0 currentSegment=0 lastSegment=1
alloy  | ts=2026-04-30T19:44:56.847340103Z level=debug msg="Processing segment" component_path=/ component_id=prometheus.remote_write.prometheus subcomponent=rw remote_name=48f759 url=https://example.com/v1/write currentSegment=0
alloy  | ts=2026-04-30T19:44:56.847348679Z level=debug msg="scrape config was updated" component_path=/ component_id=prometheus.scrape.backend_scrape
alloy  | ts=2026-04-30T19:44:56.847355222Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=eb42702d2d9a611daebcf2bcf7cc514f node_id=prometheus.scrape.backend_scrape duration=8.80253ms
alloy  | ts=2026-04-30T19:44:56.847366012Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=eb42702d2d9a611daebcf2bcf7cc514f node_id=tracing duration=13.496µs
alloy  | ts=2026-04-30T19:44:56.847378115Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=eb42702d2d9a611daebcf2bcf7cc514f node_id=logging duration=314.736µs
alloy  | ts=2026-04-30T19:44:56.847488494Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=eb42702d2d9a611daebcf2bcf7cc514f node_id=remotecfg duration=92.766µs
alloy  | ts=2026-04-30T19:44:56.847534982Z level=info msg="applying non-TLS config to HTTP server" service=http
alloy  | ts=2026-04-30T19:44:56.847545562Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=eb42702d2d9a611daebcf2bcf7cc514f node_id=http duration=31.009µs
alloy  | ts=2026-04-30T19:44:56.84756569Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=eb42702d2d9a611daebcf2bcf7cc514f node_id=ui duration=4.569µs
alloy  | ts=2026-04-30T19:44:56.847585798Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=eb42702d2d9a611daebcf2bcf7cc514f node_id=cluster duration=4.248µs
alloy  | ts=2026-04-30T19:44:56.847615184Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=eb42702d2d9a611daebcf2bcf7cc514f node_id=otel duration=15.168µs
alloy  | ts=2026-04-30T19:44:56.847632176Z level=info msg="finished complete graph evaluation" controller_path=/ controller_id="" trace_id=eb42702d2d9a611daebcf2bcf7cc514f duration=12.737172ms
alloy  | ts=2026-04-30T19:44:56.847712148Z level=debug msg="changing node state" service=cluster from=viewer to=participant
alloy  | ts=2026-04-30T19:44:56.847832996Z level=info msg="scheduling loaded components and services" controller_id=""
alloy  | ts=2026-04-30T19:44:56.848087358Z level=debug msg="finished node evaluation" controller_path=/ controller_id="" node_id=prometheus.scrape.backend_scrape duration=203.185µs
alloy  | ts=2026-04-30T19:44:56.848221512Z level=debug msg="finished node evaluation" controller_path=/ controller_id="" node_id=prometheus.scrape.backend_scrape duration=82.757µs
alloy  | ts=2026-04-30T19:44:56.848240478Z level=debug msg="Starting task" controller_id="" taskID=otel id=otel
alloy  | ts=2026-04-30T19:44:56.848292166Z level=debug msg="Starting task" controller_id="" taskID=prometheus.remote_write.prometheus id=prometheus.remote_write.prometheus
alloy  | ts=2026-04-30T19:44:56.848384972Z level=debug msg="Starting task" controller_id="" taskID=local.file.service_ip id=local.file.service_ip
alloy  | ts=2026-04-30T19:44:56.848413916Z level=debug msg="Starting task" controller_id="" taskID=discovery.relabel.backend_relabel id=discovery.relabel.backend_relabel
alloy  | ts=2026-04-30T19:44:56.848432802Z level=debug msg="Starting task" controller_id="" taskID=prometheus.scrape.backend_scrape id=prometheus.scrape.backend_scrape
alloy  | ts=2026-04-30T19:44:56.848189702Z level=debug msg="Starting task" controller_id="" taskID=labelstore id=labelstore
alloy  | ts=2026-04-30T19:44:56.848240608Z level=debug msg="finished node evaluation" controller_path=/ controller_id="" node_id=discovery.relabel.backend_relabel duration=326.428µs
alloy  | ts=2026-04-30T19:44:56.848266167Z level=debug msg="Starting task" controller_id="" taskID=remotecfg id=remotecfg
alloy  | ts=2026-04-30T19:44:56.848496643Z level=debug msg="Starting task" controller_id="" taskID=http id=http
alloy  | ts=2026-04-30T19:44:56.848509467Z level=debug msg="Starting task" controller_id="" taskID=cluster id=cluster
alloy  | ts=2026-04-30T19:44:56.848517703Z level=debug msg="Starting task" controller_id="" taskID=livedebugging id=livedebugging
alloy  | ts=2026-04-30T19:44:56.848525548Z level=debug msg="Starting task" controller_id="" taskID=ui id=ui
alloy  | ts=2026-04-30T19:44:56.848570122Z level=info msg="starting cluster node" service=cluster peers_count=0 peers="" advertise_addr=127.0.0.1:12345 minimum_cluster_size=0 minimum_size_wait_timeout=0s
alloy  | ts=2026-04-30T19:44:56.84860601Z level=debug msg="passed new targets to scrape manager" component_path=/ component_id=prometheus.scrape.backend_scrape
alloy  | ts=2026-04-30T19:44:56.848662106Z level=info msg="peers changed" service=cluster peers_count=1 min_cluster_size=0 peers=fedora
alloy  | ts=2026-04-30T19:44:56.848768007Z level=info msg="failed to register collector with remote server" service=remotecfg id=46d8c210-2248-4e10-9849-63eeb48ab0cb name="" err="noop client"
alloy  | ts=2026-04-30T19:44:56.848852958Z level=debug msg="Running alloy controller" controller_id=remotecfg
alloy  | ts=2026-04-30T19:44:56.849762761Z level=info msg="now listening for http traffic" service=http addr=127.0.0.1:12345
alloy  | ts=2026-04-30T19:45:11.838951155Z level=debug msg="Watcher is reading the WAL due to timeout, haven't received any write notifications recently" component_path=/ component_id=prometheus.remote_write.prometheus subcomponent=rw remote_name=48f759 url=https://example.com/v1/write timeout=15s
alloy  | ts=2026-04-30T19:45:26.679231198Z level=debug msg="Scrape failed" component_path=/ component_id=prometheus.scrape.backend_scrape scrape_pool=prometheus.scrape.backend_scrape target=http://%7Btrue%20value%7D/metrics err="parse \"http://%7Btrue%20value%7D/metrics\": invalid URL escape \"%7B\"

Tip

React with 👍 if this issue is important to you.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions