Unable to link traces to prompts in Spring AI

[marcjaner]

For a AI agents project that I'm developing in Spring AI (+ Kotlin) we're using Langfuse as our observability tool. One of the goals behind this decision is to be able to measure the performance of each prompt. However, right now we're not able to do so because our generation observations are not being linked to a prompt.

Some context about the setup and solutions we've tried: The project involves multiple agents so for convenience we've defined a TracingAspect that applies to the function in charge of generating the completion of each agent. This aspect ¹ handles the creation of a parent trace.

Creating this parent trace is necessary in order to define the userId, sessionId as well as tags. This also allows all the observations from a user <-> agent interaction to be grouped under a parent trace. For context, this would be a simplified example of our aspect:

import io.micrometer.tracing.Tracer
import org.apache.logging.log4j.LogManager
import org.aspectj.lang.ProceedingJoinPoint
import org.aspectj.lang.annotation.Around
import org.aspectj.lang.annotation.Aspect
import java.util.*

@Aspect
@Component
class TraceCompletionAspect(
    private val tracer: Tracer
) {
    companion object {
        private val log = LogManager.getLogger()
    }
     
     // `traceCompletion` is the defined annotation we use in generateCompletion() functions
    @Around("@annotation(traceCompletion)") 
    fun around(pjp: ProceedingJoinPoint, traceCompletion: TraceCompletion): Any? {

        val span = tracer.nextSpan().name(traceCompletion.getOperation()).start()
        
        span.tag("langfuse.user.id", getUserIdDummy())
        span.tag("langfuse.session.id", getSessionIdDummy())

        span.tagOfStrings("ai.telemetry.metadata.tags", traceCompletion.getTagsDummy())

        return tracer.withSpan(span).use {
            try {
               // Add the interaction input to parent trace
                span.tag("langfuse.trace.input", traceCompletion.getInputDummy())

                val result = pjp.proceed() // i.e.: Agent.generateCompletion()

                span.tag("langfuse.trace.output", traceCompletion.getOutputDummy())

                result
            } catch (t: Throwable) {
                span.error(t)
                throw t
            } finally {
                span.end()
            }
        }
    }

The first approach we've tried is to link the trace to the prompt in this aspect by doing:

span.tag("langfuse.observation.prompt.name", traceCompletion.getPromptNameDummy())
span.tag(""langfuse.observation.prompt.version", traceCompletion.getPromptVersionDummy())

However this will not work because prompt metadata can only be added to generation observations ².

The next approach we tried was extending the ObservationFilter that we added when configuring Langfuse in our Spring Boot project according to the documentation³ (adapted to kotlin):

@Component
class ChatModelCompletionContentObservationFilter : ObservationFilter  {
    override fun map(context: Observation.Context): Observation.Context{
        val chatCtx = context as? ChatModelObservationContext ?: return context

        val prompts = processPrompts(chatCtx)
        val completions = processCompletion(chatCtx)

        chatCtx.addHighCardinalityKeyValue(
            KeyValue.of("gen_ai.prompt", ObservabilityHelper.concatenateStrings(prompts))
        )
        chatCtx.addHighCardinalityKeyValue(
            KeyValue.of("gen_ai.completion", ObservabilityHelper.concatenateStrings(completions))
        )

        return chatCtx
    }

    private fun processPrompts(ctx: ChatModelObservationContext): List<String>{
        val instructions = ctx.request.instructions
        return instructions.mapNotNull { it.text }.filter { it.isNotBlank() }
    }

    private fun processCompletion(ctx: ChatModelObservationContext): List<String>{
        val results = ctx.response?.results ?: return emptyList()
        if(results.isEmpty()) return emptyList()
        return results.mapNotNull { it -> it.output.text?.takeIf {t -> t.isNotBlank() } }
    }
}

We updated the above ObservationFilter to have something like this:

@Component()
class ChatModelCompletionContentObservationFilter : ObservationFilter {
    // added this
    companion object {
        private val log = LogManager.getLogger()
        private val promptContext = ThreadLocal<PromptInfo>()

        data class PromptInfo(val name: String, val version: Int)

        fun setPrompt(name: String, version: Int) {
            promptContext.set(PromptInfo(name, version))
        }

        fun clear() {
            promptContext.remove()
        }
    }

    override fun map(context: Observation.Context): Observation.Context {
        if (context !is ChatModelObservationContext) return context

       val chatCtx = context as? ChatModelObservationContext ?: return context

        val prompts = processPrompts(chatCtx)
        val completions = processCompletion(chatCtx)

        chatCtx.addHighCardinalityKeyValue(
            KeyValue.of("gen_ai.prompt", ObservabilityHelper.concatenateStrings(prompts))
        )
        chatCtx.addHighCardinalityKeyValue(
            KeyValue.of("gen_ai.completion", ObservabilityHelper.concatenateStrings(completions))
        )
        
        // link trace to prompt
        promptContext.get()?.let { info ->
            log.info("[completion-filter] Linking prompt: ${info.name} v${info.version}") // for debugging

            context.addHighCardinalityKeyValue(KeyValue.of("langfuse.observation.prompt.name", info.name))
            context.addHighCardinalityKeyValue(KeyValue.of("langfuse.observation.prompt.version", info.version.toString()))

            promptContext.remove() // clear context
        }

        return context
    }

   // ... rest of code
}

However, despite in our logs being able to see the [completion-filter] Linking ... log, the result in Langfuse is not the expected one: not only the observation is not linked to the prompt, but the generation observation is no longer recorded by Langfuse.

Our understanding is that the prompt details should be added to the trace/observation resulting from the ChatClient.call() method, which is the one that is of type generation, but we're unable to get it to work.

Has anyone faced a similar issue before? Would be great to get an example of this working. Thanks

Footnotes

1. Aspect Oriented Programming in SpringBoot ↩

2. Get started with prompt management ↩

3. Integrating Langfuse with Spring AI ↩

[inkeep-ai-assistant[bot]]

Based on the available documentation, the Spring AI integration with Langfuse does not currently support directly linking prompts to generation observations through the OpenTelemetry instrumentation(1).

The Spring AI documentation shows how to set userId and sessionId on traces using the langfuse.user.id and langfuse.session.id attributes, but it does not provide a mechanism for linking prompts to generation observations(1).

For other SDKs, prompt linking is typically done by passing a prompt object when creating the generation observation. For example, in the Python SDK, you would use langfuse.update_current_generation(prompt=prompt)(2) or pass prompt=prompt when starting a generation(3). However, these approaches are not available in the Spring AI integration since it relies on OpenTelemetry auto-instrumentation rather than direct SDK calls.

The issue you're experiencing where the generation observation disappears when adding the prompt attributes suggests that the attributes you're adding may be interfering with the observation creation process. The Spring AI integration uses specific attribute patterns that the LangfuseSpanProcessor expects, and adding custom attributes in the ObservationFilter may be disrupting this.

Potential workarounds:

1. Use the Langfuse Java SDK directly: You could use the Langfuse Java SDK alongside Spring AI to update traces after they've been created, though this would require additional implementation work.

2. Track prompts via metadata: Instead of linking prompts directly, you could add prompt information to the trace metadata using tags, which would allow you to filter and analyze by prompt version in Langfuse, even if the formal prompt link isn't established.

3. Feature request: This appears to be a gap in the current Spring AI integration. You might want to open a discussion on the Langfuse GitHub repository to request this feature.

The Spring AI integration documentation focuses primarily on basic tracing setup and user/session tracking, but doesn't cover prompt management integration(1).

📚 Sources:

1. Integrating Langfuse with Spring AI
2. Link Prompts to Traces
3. Get Started with Prompt Management

---

Have another question? Just tag @inkeep.

[viacheslav-dobrynin]

@marcjaner, Hello!

Can you please tell if you have managed to find a solution?

[marcjaner]

No, we haven't found any solution to this yet.

[viacheslav-dobrynin]

Thanks!

[viacheslav-dobrynin]

@marcjaner

Looks like the problem in the type of langfuse.observation.prompt.version tag and current micrometer behaviour.
The type of langfuse.observation.prompt.version is integer.
But micrometer now supports only string type, see KeyValue and the related issue.

My reproducible example

payload.json (original JSON was taken during the debug of the real spring ai+kotlin app):

{
  "resourceSpans": [
    {
      "resource": {
        "attributes": [
          {
            "key": "service.name",
            "value": {
              "stringValue": "test-service"
            }
          },
          {
            "key": "telemetry.sdk.language",
            "value": {
              "stringValue": "java"
            }
          },
          {
            "key": "telemetry.sdk.name",
            "value": {
              "stringValue": "opentelemetry"
            }
          },
          {
            "key": "telemetry.sdk.version",
            "value": {
              "stringValue": "1.38.0"
            }
          }
        ]
      },
      "scopeSpans": [
        {
          "scope": {
            "name": "org.springframework.boot",
            "version": "3.5.7",
            "attributes": []
          },
          "spans": [
            {
              "traceId": "efc40c98d40d3ba167b6ead920f3e020",
              "spanId": "73f0082eae157c08",
              "name": "test-parent-span",
              "kind": 1,
              "startTimeUnixNano": "1763734711496069000",
              "endTimeUnixNano": "1763734719607932739",
              "attributes": [
                {
                  "key": "langfuse.environment",
                  "value": {
                    "stringValue": "test_environment"
                  }
                }
              ],
              "events": [],
              "links": [],
              "status": {
                "code": 1
              },
              "flags": 257
            }
          ]
        }
      ]
    },
    {
      "resource": {
        "attributes": [
          {
            "key": "service.name",
            "value": {
              "stringValue": "test-service"
            }
          },
          {
            "key": "telemetry.sdk.language",
            "value": {
              "stringValue": "java"
            }
          },
          {
            "key": "telemetry.sdk.name",
            "value": {
              "stringValue": "opentelemetry"
            }
          },
          {
            "key": "telemetry.sdk.version",
            "value": {
              "stringValue": "1.38.0"
            }
          }
        ]
      },
      "scopeSpans": [
        {
          "scope": {
            "name": "org.springframework.boot",
            "version": "3.5.7",
            "attributes": []
          },
          "spans": [
            {
              "traceId": "efc40c98d40d3ba167b6ead920f3e020",
              "spanId": "54124ffa1eb6f6e9",
              "parentSpanId": "73f0082eae157c08",
              "name": "chat gemini-2.5-flash",
              "kind": 1,
              "startTimeUnixNano": "1763734713931838741",
              "endTimeUnixNano": "1763734719497912672",
              "attributes": [
                {
                  "key": "gen_ai.completion",
                  "value": {
                    "stringValue": "Hello"
                  }
                },
                {
                  "key": "gen_ai.operation.name",
                  "value": {
                    "stringValue": "chat"
                  }
                },
                {
                  "key": "gen_ai.prompt",
                  "value": {
                    "stringValue": "Hello"
                  }
                },
                {
                  "key": "gen_ai.request.model",
                  "value": {
                    "stringValue": "gemini-2.5-flash"
                  }
                },
                {
                  "key": "gen_ai.request.temperature",
                  "value": {
                    "stringValue": "0.7"
                  }
                },
                {
                  "key": "gen_ai.request.top_p",
                  "value": {
                    "stringValue": "1.0"
                  }
                },
                {
                  "key": "gen_ai.response.finish_reasons",
                  "value": {
                    "stringValue": "[\"STOP\"]"
                  }
                },
                {
                  "key": "gen_ai.response.model",
                  "value": {
                    "stringValue": "gemini-2.5-flash"
                  }
                },
                {
                  "key": "gen_ai.system",
                  "value": {
                    "stringValue": "google_genai"
                  }
                },
                {
                  "key": "gen_ai.usage.input_tokens",
                  "value": {
                    "stringValue": "1"
                  }
                },
                {
                  "key": "gen_ai.usage.output_tokens",
                  "value": {
                    "stringValue": "1"
                  }
                },
                {
                  "key": "gen_ai.usage.total_tokens",
                  "value": {
                    "stringValue": "2"
                  }
                },
                {
                  "key": "langfuse.environment",
                  "value": {
                    "stringValue": "test_environment"
                  }
                },
                {
                  "key": "langfuse.observation.prompt.name",
                  "value": {
                    "stringValue": "ReplaceToYourPromptName"
                  }
                },
                {
                  "key": "langfuse.observation.prompt.version",
                  "value": {
                    "stringValue": "1"
                  }
                },
                {
                  "key": "spring.ai.model.request.tool.names",
                  "value": {
                    "stringValue": "[\"testTool1\", \"testTool2\", \"testTool3\"]"
                  }
                }
              ],
              "events": [],
              "links": [],
              "status": {
                "code": 1
              },
              "flags": 257
            }
          ]
        }
      ]
    }
  ]
}

post-payload.py:

import base64
import json
import os
from time import time_ns
from typing import Tuple

import dotenv
import requests
from google.protobuf.json_format import ParseDict
from opentelemetry.proto.collector.trace.v1.trace_service_pb2 import (
    ExportTraceServiceRequest,
)


def load_auth() -> Tuple[str, str]:
    dotenv.load_dotenv()
    public_key = os.getenv("LANGFUSE_PUBLIC_KEY")
    secret_key = os.getenv("LANGFUSE_SECRET_KEY")
    if not public_key or not secret_key:
        raise SystemExit("Missing LANGFUSE_PUBLIC_KEY or LANGFUSE_SECRET_KEY")
    return public_key, secret_key


def main():
    base_url = os.getenv("LANGFUSE_BASE_URL", "https://cloud.langfuse.com").rstrip("/")
    endpoint = f"{base_url}/api/public/otel/v1/traces"
    public_key, secret_key = load_auth()

    with open("payload.json") as f:
        data = json.load(f)

    # Shift all span timestamps to now so they appear in the current UI window.
    now = time_ns()
    first_span = data["resourceSpans"][0]["scopeSpans"][0]["spans"][0]
    delta = now - int(first_span["startTimeUnixNano"])
    for resource_span in data.get("resourceSpans", []):
        for scope_span in resource_span.get("scopeSpans", []):
            for span in scope_span.get("spans", []):
                span["startTimeUnixNano"] = str(int(span["startTimeUnixNano"]) + delta)
                span["endTimeUnixNano"] = str(int(span["endTimeUnixNano"]) + delta)

    req = ExportTraceServiceRequest()
    ParseDict(data, req)  # fill the protobuf message
    payload_bytes = req.SerializeToString()

    auth = base64.b64encode(f"{public_key}:{secret_key}".encode()).decode()
    resp = requests.post(
        endpoint,
        headers={
            "Content-Type": "application/x-protobuf",
            "Accept": "application/json",
            "Authorization": f"Basic {auth}",
        },
        data=payload_bytes,
        timeout=10,
    )
    print("Status:", resp.status_code)
    try:
        print("Response:", resp.json())
    except Exception:
        print("Response text:", resp.text)
    resp.raise_for_status()
    print("Sent protobuf payload to", endpoint)


if __name__ == "__main__":
    main()

Steps:

• Run post-payload.py and go to UI: https://cloud.langfuse.com/project/{your_project_id}/observations (for some reason these traces don't appear in traces tab, only on observations)
• You will see only one trace (parent one)
• Replace stringValue to intValue in JSON for langfuse.observation.prompt.version tag and repeat
• After this you will see both observations and linked prompt too if it exists

I think this problem could be fixed both sides (micrometer or langfuse).
Maybe you can change type on string for langfuse.observation.prompt.version tag if applicable.
Or add extra langfuse.observation.prompt.label tag as workaround.
And I would also appreciate it if the errors about the invalid event could be seen somewhere.

@langfuse, @hassiebp, @marliessophie what do you think?

[hassiebp]

@viacheslav-dobrynin Thanks a lot for digging deep here and uncovering the underlying issue! Our OpenTelemetry mapping guide states that langfuse.prompt.version must be a integer: https://langfuse.com/integrations/native/opentelemetry#prompt

What is the framework you were using that created those spans?

[viacheslav-dobrynin]

I use Spring AI with Spring Boot, which uses micrometer under the hood for tracing and everything else related to observability.

[viacheslav-dobrynin]

Is it possible to add langfuse.observation.prompt.label: string tag as workaround?

[viacheslav-dobrynin]

@marcjaner, added new issue to micrometer: micrometer-metrics/micrometer#6916

kindly ask you to upvote if it is relevant for you

[marcjaner]

seen @viacheslav-dobrynin! thanks for the deep dive on the topic. already upvoted!