Is it possible to add data retention (TTL) for traces, scores etc. in the UI?

[erantr1]

Describe the feature or potential improvement

Issue Description:
Over time, old traces, scores, and other observability data accumulate and consume significant disk space in our self-hosted Langfuse instance. While this historical data is valuable initially, much of it becomes unnecessary after a certain period.

Current Limitations:
The existing options for managing and deleting old data in self-hosted deployments are not convenient for regular maintenance.

Proposed Solution:
Add a data retention/TTL (Time-To-Live) configuration option directly in the Langfuse UI that would allow administrators to:

1. Set automatic deletion policies for traces, scores, and related data after a specified time period (e.g., 30, 60, 90 days)
2. Configure retention periods per project or globally
3. Optionally preview the storage that would be freed before applying retention policies

Additional information

No response

[jannikmaierhoefer]

Hi @erantr1, thanks for adding this here.

[erantr1]

@jannikmaierhoefer
Thank you for your comment :-)

I thought about it a bit further, and if this feature will indeed be added, maybe it will be nice to have an extra "layer" - an option to mark specific items (e.g., a specific trace) as "live forever". I have some traces I use to demonstrate some things, and for them, I don't necessarily want them to be deleted after a certain period of time.

[SAGONVS]

I see in te UI there is Data Retention Period to set, would that be useful?, also we are thinking on having a process which can send data to S3 in archival mode for more than 3O days traces.

[erantr1]

@SAGONVS
I see that Data Retention was indeed added last January (https://langfuse.com/changelog/2025-01-30-data-retention).

But I don't see "Data Retention Period" in the UI.
I guess it's related to the fact I'm self-hosting langfuse (https://langfuse.com/docs/administration/data-retention)?

The archival mode sounds like a good idea!
Would you also consider the ability to mark specific traces as excluded from Data Retention (sort of live forever traces)?