loadgen not running and / or no telemetry?

[agardnerIT]

Bug Report

Which version of the demo you are using? 2.2.0

Symptom

I only see traces for flagd and cart but nothing "end to end".

The loadgen readme tells me that http://localhost:8080/loadgen should be available but I get upstream request timeout.

Loadgen Logs

# kubectl logs -f load-generator-6999c9b97c-5n72m

Helm

# helm list -A
NAME                    NAMESPACE       REVISION        UPDATED                                 STATUS          CHART                           APP VERSION
dynatrace-collector     default         3               2026-04-02 03:33:42.268327743 +0000 UTC deployed        opentelemetry-collector-0.147.1 0.147.0    
my-otel-demo            default         1               2026-04-01 01:46:18.663460328 +0000 UTC deployed        opentelemetry-demo-0.40.6       2.2.0  

Load Gen Pod Details

# kubectl describe pod/load-generator-6999c9b97c-5n72m
Name:             load-generator-6999c9b97c-5n72m
Namespace:        default
Priority:         0
Service Account:  my-otel-demo
Node:             obslab-otel-collector-data-ingest-cluster-control-plane/172.18.0.2
Start Time:       Wed, 01 Apr 2026 01:46:20 +0000
Labels:           app.kubernetes.io/component=load-generator
                  app.kubernetes.io/name=load-generator
                  app.kubernetes.io/version=2.2.0
                  opentelemetry.io/name=load-generator
                  pod-template-hash=6999c9b97c
Annotations:      resource.opentelemetry.io/service.namespace: otel-demo
Status:           Running
IP:               10.244.0.9
IPs:
  IP:           10.244.0.9
Controlled By:  ReplicaSet/load-generator-6999c9b97c
Containers:
  load-generator:
    Container ID:   containerd://f3e47fc2097b5d9407069fb98a917eb9e5e626fa14f1bd131a73630efb7aada9
    Image:          ghcr.io/open-telemetry/demo:2.2.0-load-generator
    Image ID:       ghcr.io/open-telemetry/demo@sha256:40eddf7ddd63616f93c6dfbb6d2d695b2983ab166e2bf50642a4189388b3bb2c
    Port:           8089/TCP (service)
    Host Port:      0/TCP (service)
    State:          Running
      Started:      Thu, 02 Apr 2026 01:44:09 +0000
    Last State:     Terminated
      Reason:       Error
      Exit Code:    1
      Started:      Wed, 01 Apr 2026 23:26:54 +0000
      Finished:     Thu, 02 Apr 2026 01:44:09 +0000
    Ready:          True
    Restart Count:  4
    Limits:
      memory:  500Mi
    Requests:
      memory:  500Mi
    Environment:
      OTEL_SERVICE_NAME:                        (v1:metadata.labels['app.kubernetes.io/component'])
      OTEL_COLLECTOR_NAME:                     dynatrace-collector-opentelemetry-collector
      LOCUST_WEB_HOST:                         0.0.0.0
      LOCUST_WEB_PORT:                         8089
      LOCUST_USERS:                            10
      LOCUST_SPAWN_RATE:                       1
      LOCUST_HOST:                             http://frontend-proxy:8080
      LOCUST_HEADLESS:                         false
      LOCUST_AUTOSTART:                        true
      LOCUST_BROWSER_TRAFFIC_ENABLED:          true
      PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION:  python
      FLAGD_HOST:                              flagd
      FLAGD_PORT:                              8013
      FLAGD_OFREP_PORT:                        8016
      OTEL_EXPORTER_OTLP_ENDPOINT:             http://$(OTEL_COLLECTOR_NAME):4317
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-5wzfg (ro)
Conditions:
  Type                        Status
  PodReadyToStartContainers   True 
  Initialized                 True 
  Ready                       True 
  ContainersReady             True 
  PodScheduled                True 
Volumes:
  kube-api-access-5wzfg:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    Optional:                false
    DownwardAPI:             true
QoS Class:                   Burstable
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:                      <none>

_{Tip: React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.}

[julianocosta89]

Hello @agardnerIT 👋🏽

Everything on the Demo is exposed via the frontend-proxy (envoy).
I believe this is the culprit, could you check if it is running?

[agardnerIT]

Here's what I've found so far using Python 3.14:

Locust UI warns that: One or more User class in your locustfile has no host attribute set. Please provide one in the field above.

So a host line should be included there too.

class WebsiteUser(HttpUser):
    host = os.environ.get("LOCUST_HOST", "http://localhost:8080")

--skip-log-setup

export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
export LOCUST_BROWSER_TRAFFIC_ENABLED=true
locust --skip-log-setup WebsiteBrowserUser

This just hangs. Claude Sonnet 4.6 gave me a very convincing sounding explanation (but who knows!?):

--skip-log-setup is interfering with the gevent/asyncio bridge that PlaywrightUser depends on. Specifically, Locust's normal log setup does more than just configure logging — it also ensures gevent's monkey-patching is applied in the right order. Skipping it causes a deadlock before any user is instantiated.

What I do know is that locust WebsiteBrowserUser does get to the message about running the UI on 8089 and indeed I can get to the UI. I need to manually trigger a test though.

But, when I do trigger the test, I see messages like this:

[2026-04-07 07:22:15,023] 3ac820a1ea4b/ERROR/root: Error in add to cart task: Page.goto: Protocol error (Page.navigate): Cannot navigate to invalid URL
Call log:
  - navigating to "/", waiting until "domcontentloaded"

I will keep investigating as I feel closer now that I was before

[julianocosta89]

@agardnerIT may I ask why are you exposing the locust itself?

Everything on the Demo is exposed via the frontend-proxy (envoy).

[agardnerIT]

@agardnerIT may I ask why are you exposing the locust itself?

Everything on the Demo is exposed via the frontend-proxy (envoy).

I'm not as standard, this was all experimentation to see why it wasn't firing traffic.