(help wanted) Fetch scores from external datastore for a field in a featureset.

[vimarshchaturvedi]

Hello everyone,

I am currently working on a project using the Elasticsearch Learning to Rank (LTR) plugin, and I've encountered a challenge that I hope to get some insights on.

In my use case, I need to incorporate external scores (fetched from an external service) into the ranking process of Elasticsearch search results. These external scores cannot not indexed in Elasticsearch and need to be dynamically retrieved and applied during the search query.

In other words, I want to create a featureset where some fields need to be hydrated from an external api call. So, the flow might go like this -

1. Perform the retrieval query which might fetch 300-500 documents.
2. Before performing the sltr query I want fetch certain scores for each document.
3. When I perform the sltr query I want to pass the external scores. Here, the LTR model has been trained on these external scores (obviously).

After going over the docs and searching the internet here are the two solutions I found and wanted your thoughts -

1. Extra Logging Can I use the this feature in Scala to add extra features from an external source? It states that it will add values to the log_entry which can be later used in training. So, if it ok to expect that I can inject external values this way?
2. Can I pass external values in "params" in the sltr query?

[preston-andritsch]

Also looking to do something similar, add contextual params at time of search into the LearnToRank stlr query. My understanding is only fields on the document can be used as features in LTR