Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion src/langsmith/online-evaluations-llm-as-judge.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -46,10 +46,12 @@ If you also have a webhook automation rule on this project and want the webhook

Configure a sampling rate to control the percentage of filtered runs that trigger the automation action. For example, to control costs, you may want to set a filter to only apply the evaluator to 10% of traces. In order to do this, you would set the sampling rate to 0.1.

## Apply a rule to past runs
## Apply a rule to past runs or threads

Apply a rule to past runs by toggling the **Apply to past runs** and entering a "Backfill from" date. This is only possible upon rule creation.

For [thread-level evaluation rules](/langsmith/online-evaluations-multi-turn), you can also apply the rule to past threads. LangSmith finds eligible threads with matching trace activity between the selected start date and the rule's creation time, then applies the rule's filters. After each selected thread is idle, LangSmith evaluates its root-run history and attaches the resulting feedback to a representative trace in the thread. When the backfill completes, the rule remains active and evaluates new matching thread activity through the normal live flow.

<Note>
The backfill is processed as a background job, so you will not see the results immediately.
</Note>
Expand Down
Loading