Intermittent Traces from App

[atesavci-th]

Describe your question

Hello, our self-hosted app was working fine until couple days ago, then traces stopped showing up on langfuse. We were able to retreive prompts, make requests such as /api/public/health, /api/public/projects, and ingestion endpoint returned success, but traces weren't visible.

Saw #8206 and updated to the latest Langfuse and python SDK versions. Then we were able to send traces with local script using /ingest endpoint, but wasn't able to see traces from our app. After restarting langfuse, we were able to get traces from our app, but it stopped working after a while without any code changes on our side. We're still able to send traces via below
RESPONSE=$(curl`` -s -w "HTTP %{http_code}" -X POST "${HOST}/api/public/ingestion" \ -H "Content-Type: application/json" \ -H "Authorization: Basic ${AUTH}" \ -d ``"${PAYLOAD}")
But traces from our app are intermittent. It starts to work after restarting langfuse instance, then traces stop coming after a while. On the other hand, requests return success and we're able to retreive prompts etc without problems.

Langfuse Cloud or Self-Hosted?

Self-Hosted

If self-hosted, what version are you running?

v3.133.0

SDK and integration versions

python sdk version 3.10.1

Pre-Submission Checklist

• I have checked for existing issues/discussions and consulted Langfuse AI.

[inkeep-ai-assistant[bot]]

Based on the available sources, your intermittent trace visibility issue after upgrading to v3.133.0 with Python SDK 3.10.1 suggests a potential worker container problem(1).

The Langfuse v3 architecture includes a worker container that asynchronously processes events(2). When traces appear after restarting but stop working later, this often indicates the worker isn't running properly or has stopped processing the ingestion queue(1).

Key troubleshooting steps:

1. Verify worker container status - Ensure the langfuse-worker container is running and hasn't crashed. Check worker logs for errors(1).

2. Check all required components - Langfuse v3 requires:

   • Langfuse Web container
   • Langfuse Worker container
   • PostgreSQL
   • ClickHouse
   • Redis/Valkey cache
   • S3/Blob storage(2)

3. Review environment variables - Confirm OTEL_EXPORTER_OTLP_ENDPOINT is correctly set if using OpenTelemetry. Connection errors to localhost:4318 can occur if this isn't configured properly(3).

4. Monitor background migrations - Check if background migrations are stuck at "Queued" status, which can prevent trace processing(4).

Since your /api/public/ingestion endpoint returns success but traces don't appear, the issue is likely in the asynchronous processing pipeline rather than ingestion itself. The worker container processes events from the queue after they're accepted by the ingestion endpoint(2).

For additional troubleshooting resources, check the Troubleshooting & FAQ(4) and review related discussions on worker container issues(4)(1).

📚 Sources:

1. bug: OTEL Endpoint /api/public/otel/v1/traces Hangs Indefinitely on Self-Hosted v3.120.0
2. Self-host Langfuse
3. bug: Noticing errors in Langfuse sdk logs
4. Troubleshooting & FAQ

---

Have another question? Just tag @inkeep.

[Lotte-Verheyden]

Hi @atesavci-th, was Inkeep bot's answer helpful for you?

[atesavci-th]

Hi @Lotte-Verheyden
We've checked all listed components and worker logs, haven't seen any errors. Also no background migrations are stuck at "Queued" status. And we don't use OTEL_EXPORTER_OTLP_ENDPOINT.

Currently, it continues to work intermittently from the app without any changes.
We've checked clickhouse and only the traces that appear in langfuse exists, no record of the traces that don't make it to langfuse UI.