feat: consensus strategies and examples #57
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Implemented advanced aggregation functions and strategies, and included examples to test the functions.
Example files:
For less redundancy, each example file uses sample LLM responses based on an example prompt. This is an extensible approach, since this framework is meant to be used for consensus approaches where a single agent is an individual trained format and consensus strategies are applied on the outputs.
Several advanced aggregation functions have been implemented, providing logging and mathematical approaches for recording Shapley values, clustering similar responses (for text based situations), identifying hallucinations, and more. Certain consensus strategies are more applicable in certain contexts, and the example files identify some appropriate cases where this can be done.
All logic code and strategies are structured under aggregator/, since this is where all of the aggregation logic should exist.