tracing/langfuse.py: sync_trace() returns mock data; LangfuseTrace.filter() is a no-op

[Harsh23Kashyap]

Problem

src/ragas/integrations/tracing/langfuse.py:97-143 defines async def sync_trace(trace_id, max_retries=10, delay=2) with a retry loop. The loop's body unconditionally constructs a TraceWithFullDetails(id=trace_id, latency=0, totalCost=0.0, observations=[], ...) mock and returns it on iteration 1. The max_retries and delay parameters are dead. The function's docstring promises to "wait for a Langfuse trace to be synced to the server" but it never talks to Langfuse.

Companion: LangfuseTrace.filter(span_name) (lines 91–94) is a no-op — always returns []. The inline comment labels it a "placeholder implementation for backward compatibility."

Why it matters

The package docstring in src/ragas/integrations/tracing/__init__.py (lines 21–32) literally shows users calling sync_trace() and trace.get_url(). Anyone following the docs gets a TraceWithFullDetails with totalCost=0.0 and observations=[] — silently misleading them about whether their trace was actually synced. The function is part of the public ragas.integrations.tracing.langfuse API surface (re-exported via __getattr__ in __init__.py).

Reproduction

import asyncio
from ragas.integrations.tracing.langfuse import sync_trace, LangfuseTrace

# Set LANGFUSE_SECRET_KEY / LANGFUSE_PUBLIC_KEY / LANGFUSE_HOST first.
# Then run any @observe() block; the trace_id is returned by Langfuse.

async def main():
    trace = await sync_trace(trace_id="abc", max_retries=10, delay=0.1)
    # Despite max_retries=10 and delay=0.1, this returns on iteration 1
    # with a mock object: latency=0, totalCost=0.0, observations=[].
    print(trace.trace.latency)          # 0
    print(trace.trace.totalCost)        # 0.0
    print(trace.trace.observations)     # []
    print(trace.filter("any_span_name"))  # [] (always empty)

asyncio.run(main())

Expected behavior

Either (a) sync_trace actually fetches the trace from Langfuse using the v3 API (e.g. Langfuse().get(trace_id) or the equivalent api.trace.get(trace_id)), polling until the trace is available, and returns a real TraceWithFullDetails; or (b) the function is removed/replaced with a working alternative.

LangfuseTrace.filter(span_name) should actually filter the trace's observations by name.

Reference

src/ragas/integrations/tracing/mlflow.py (MLflowTrace.sync_trace + get_filter) is the correct shape. MLflowTrace.sync_trace calls get_last_active_trace_id() + get_trace() and get_filter(span_name) calls self.trace.search_spans(name=span_name). The Langfuse file in the same package should match.

Out of scope

The broader Langfuse Python SDK v3 → v4 migration tracked in #2853. This issue is about the existing v3 stub being a no-op, not about a v4 upgrade. A v4-migration PR can build on this fix.

Acceptance criteria

• sync_trace(trace_id=...) either returns a real TraceWithFullDetails fetched from Langfuse or raises a clear error with a message pointing at Upgrade Langfuse tracing integration to Python SDK v4 #2853.
• LangfuseTrace.filter(span_name) returns the matching observations (empty list when none match is fine; a hard-coded [] is not).
• New unit tests cover:
  • (a) the function no longer returns mock data on the happy path (when Langfuse is reachable and the trace exists)
  • (b) the retry loop is exercised on transient failures
  • (c) filter returns matching spans
  • (d) behaviour is unchanged when LANGFUSE_AVAILABLE=False (stub classes still importable; sync_trace raises the existing ValueError("No trace id found ..."))
• Backward compatibility: public signatures (sync_trace(trace_id=None, max_retries=10, delay=2) and LangfuseTrace.filter(span_name)) are unchanged.
• ruff check and pyright on the touched file are clean.

Environment

• ragas main (post 0.4.3, commit 298b682)
• Python 3.12
• Reproducible without Langfuse credentials (the bug is on the happy path; the mock is returned regardless of input)