Submission checklist
Package (Required)
Feature Description
I would like LangChain to support a ProfileAwareRouter — a new router type that uses classical ML (KNN or Random Forest) to select among prompt variants based on a structured feature vector extracted from the input payload.
This is distinct from existing semantic routers which use embedding similarity. The use case is structured computation pipelines where prompt performance varies predictably by input profile characteristics (e.g. which form fields or schema elements are present).
Use Case
I'm building a structured computation pipeline where LLM outputs vary significantly based on input profile characteristics. No single prompt performs best across all input profiles — different prompt versions win on different profile types, and the winning prompt is predictable from a small set of boolean features extracted from the input.
Currently I have to implement custom routing logic outside LangChain. Existing routers (semantic router, LLM router) are not suited for this because the routing signal is not semantic similarity — it is structural features of the input payload.
Example: in tax computation, whether Schedule 1 is present or absent is the dominant predictor of which prompt formula produces the correct output. A KNN trained on a small labeled evaluation matrix routes with far higher accuracy than semantic similarity or LLM self-routing.
Proposed Solution
ProfileAwareRouter as a new router in langchain.chains.router
from langchain.chains.router import ProfileAwareRouter
from langchain.chains.router.ml_routing import KNNRoutingModel
router = ProfileAwareRouter(
destination_chains={
"v1": llm_chain_v1,
"v2": llm_chain_v2,
"v3": llm_chain_v3,
},
feature_extractor=extract_profile_features,
routing_model=KNNRoutingModel(n_neighbors=5),
confidence_threshold=0.7,
fallback_chain=ensemble_chain,
)
result = router.invoke({"payload": student_tax_payload})
### Alternatives Considered
I tried semantic routing but the routing signal here is not semantic similarity — it's structural features of the payload, so embedding distance is not meaningful. I tried LLM routing but it adds latency and cost, and is less reliable than a classifier trained on ground-truth labels. I tried hard-coded if/else logic but it doesn't generalize, provide confidence scores, or handle ambiguous profiles.
### Additional Context
The routing model would expose standard sklearn validation: silhouette score for cluster quality, out-of-bag error for Random Forest, calibration curve for confidence score reliability, per-segment precision/recall breakdown, and routing lift vs best single prompt baseline. This pattern generalizes to any domain where structured payloads feed LLM computation and prompt performance varies by input characteristics — tax computation, medical coding, insurance underwriting, financial aid calculation.
Submission checklist
Package (Required)
Feature Description
I would like LangChain to support a
ProfileAwareRouter— a new router type that uses classical ML (KNN or Random Forest) to select among prompt variants based on a structured feature vector extracted from the input payload.This is distinct from existing semantic routers which use embedding similarity. The use case is structured computation pipelines where prompt performance varies predictably by input profile characteristics (e.g. which form fields or schema elements are present).
Use Case
I'm building a structured computation pipeline where LLM outputs vary significantly based on input profile characteristics. No single prompt performs best across all input profiles — different prompt versions win on different profile types, and the winning prompt is predictable from a small set of boolean features extracted from the input.
Currently I have to implement custom routing logic outside LangChain. Existing routers (semantic router, LLM router) are not suited for this because the routing signal is not semantic similarity — it is structural features of the input payload.
Example: in tax computation, whether Schedule 1 is present or absent is the dominant predictor of which prompt formula produces the correct output. A KNN trained on a small labeled evaluation matrix routes with far higher accuracy than semantic similarity or LLM self-routing.
Proposed Solution
ProfileAwareRouteras a new router inlangchain.chains.router