update docs

Author: magurh

README.md

@@ -7,16 +7,16 @@ Flare AI SDK for Consensus Learning.
 
 ### 🚀 Key Features
 
-- **Consensus Learning Implementation**  
+- **Consensus Learning Implementation**
   A Python implementation of single-node, multi-model Consensus Learning (CL). CL is a decentralized ensemble learning paradigm introduced in [arXiv:2402.16157](https://arxiv.org/abs/2402.16157).
 
-- **300+ LLM Support**  
+- **300+ LLM Support**
   Leverages OpenRouter to access over 300 models via a unified interface.
 
-- **Iterative Feedback Loop**  
+- **Iterative Feedback Loop**
   Employs an aggregation process where multiple LLM outputs are refined over configurable iterations.
 
-- **Modular & Configurable**  
+- **Modular & Configurable**
   Easily customize models, conversation prompts, and aggregation parameters through a simple JSON configuration.
 
 ## 📌 Prerequisites
@@ -45,26 +45,27 @@ Set your OpenRouter API key in your environment:
 export OPENROUTER_API_KEY=<your-openrouter-api-key>
 ```
 
-Verify your available credits with:
+Verify your available credits and get all supported models with:
 
 ```bash
 uv run python -m tests.credits
+uv run python -m tests.models
 ```
 
 ## 🚀 Running Consensus Learning
 
 Configure your consensus learning instance in `src/input.json`, including:
 
-- **Models:**  
+- **Models:**
   Specify each LLM's OpenRouter `id`, along with parameters like `max_tokens` and `temperature`.
 
-- **Initial Conversation:**  
+- **Initial Conversation:**
   Set up the conversation context (e.g., a `system` message and an initial `user` query).
 
-- **Aggregator Settings:**  
+- **Aggregator Settings:**
   Define the aggregator model, additional context, aggregation prompt, and specify how aggregated responses are handled.
 
-- **Iterations:**  
+- **Iterations:**
   Determine the number of iterations for the feedback loop.
 
 Once configured, start the process with:

src/README.md

@@ -1,33 +1,38 @@
-[![python](https://img.shields.io/badge/Python-3.11-3776AB.svg?style=flat&logo=python&logoColor=white)](https://www.python.org)
+[![Python](https://img.shields.io/badge/Python-3.12-3776AB.svg?style=flat&logo=python&logoColor=white)](https://www.python.org)
 [![Flare](https://img.shields.io/badge/flare-network-e62058.svg?logo=data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIzNCIgaGVpZ2h0PSIzNCI+PHBhdGggZD0iTTkuNC0uMWEzMjAuMzUgMzIwLjM1IDAgMCAwIDIuOTkuMDJoMi4yOGExMTA2LjAxIDExMDYuMDEgMCAwIDEgOS4yMy4wNGMzLjM3IDAgNi43My4wMiAxMC4xLjA0di44N2wuMDEuNDljLS4wNSAyLTEuNDMgMy45LTIuOCA1LjI1YTkuNDMgOS40MyAwIDAgMS02IDIuMDdIMjAuOTJsLTIuMjItLjAxYTQxNjEuNTcgNDE2MS41NyAwIDAgMS04LjkyIDBMMCA4LjY0YTIzNy4zIDIzNy4zIDAgMCAxLS4wMS0xLjUxQy4wMyA1LjI2IDEuMTkgMy41NiAyLjQgMi4yIDQuNDcuMzcgNi43LS4xMiA5LjQxLS4wOXoiIGZpbGw9IiNFNTIwNTgiLz48cGF0aCBkPSJNNy42NSAxMi42NUg5LjJhNzU5LjQ4IDc1OS40OCAwIDAgMSA2LjM3LjAxaDMuMzdsNi42MS4wMWE4LjU0IDguNTQgMCAwIDEtMi40MSA2LjI0Yy0yLjY5IDIuNDktNS42NCAyLjUzLTkuMSAyLjVhNzA3LjQyIDcwNy40MiAwIDAgMC00LjQtLjAzbC0zLjI2LS4wMmMtMi4xMyAwLTQuMjUtLjAyLTYuMzgtLjAzdi0uOTdsLS4wMS0uNTVjLjA1LTIuMSAxLjQyLTMuNzcgMi44Ni01LjE2YTcuNTYgNy41NiAwIDAgMSA0LjgtMnoiIGZpbGw9IiNFNjIwNTciLz48cGF0aCBkPSJNNi4zMSAyNS42OGE0Ljk1IDQuOTUgMCAwIDEgMi4yNSAyLjgzYy4yNiAxLjMuMDcgMi41MS0uNiAzLjY1YTQuODQgNC44NCAwIDAgMS0zLjIgMS45MiA0Ljk4IDQuOTggMCAwIDEtMi45NS0uNjhjLS45NC0uODgtMS43Ni0xLjY3LTEuODUtMy0uMDItMS41OS4wNS0yLjUzIDEuMDgtMy43NyAxLjU1LTEuMyAzLjM0LTEuODIgNS4yNy0uOTV6IiBmaWxsPSIjRTUyMDU3Ii8+PC9zdmc+&colorA=FFFFFF)](https://dev.flare.network/)
 
-# Consensus Learning with LLMs
+# Flare AI Consensus
 
 ## OpenRouter Clients
 
 We implement two OpenRouter clients for interacting with the OpenRouter API: a standard sync client and an asynchronous client.
-Both of these are constructed on top of some base client (`BaseClient` and `AsyncBaseClient`, respectively) which includes simple logic for API interaction.
-Namely, the base clients introduce the `get` and `post` requests.
+Both of these are constructed on top of base classes (`BaseRouter` and `AsyncBaseRouter`, respectively) which include simple logic for API interaction.
+Namely, the base classes introduce the `get` and `post` requests.
 
-The `OpenRouterClient` (and `AsyncOpenRouterClient`) build on these base clients by providing specific API endpoints for interacting with the OpenRouter API.
+The `OpenRouterProvider` (and `AsyncOpenRouterProvider`) build on these base classes by providing specific API endpoints for interacting with the OpenRouter API.
 
 ## Messaging
 
-Messages sent to the LLMs are built and within `src/consensus`.
-Standard (*i.e.* non-aggregated) messages are sent using the `send_round()` method within `consensus/consensus.py`:
+Every message sent to the **Chat Completion** endpoint must specify one of the following roles:
 
-* Initial messages (first round messages) consist of the `initial_conversation` defined within `input.json`.
-* Follow-up conversations are built using the `build_improvement_conversation()` method and contain the following:
+* `system`: Provides context and guidelines for the conversation.
+* `user`: Contains the queries or instructions directed to the model.
+* `assistant`: Represents the model’s responses. Any message with this role is treated as a previous response from the model.
+
+For the consensus learning algorithm, messages are sent to the LLMs via OpenRouter and are constructed.
+Standard (*i.e.* non-aggregated) messages are sent using the `send_round()` method in `consensus/consensus.py`:
+
+* **Initial messages** (first round messages): These consist of the `initial_conversation` defined within `input.json`.
+* **Follow-up conversations**: Built using the `build_improvement_conversation()` method and contain include:
   * The initial conversation defined within `input.json`.
   * The aggregated prompt from the aggregator, with the role assigned in the `aggregated_prompt_type` field of `input.json`.
   * The `improvement_prompt` defined in `input.json`, which is assigned a `user` role.
 
 ## Aggregator
 
-The simplest aggregation method does not rely on an LLM, and instead does a simple concatenation of multiple responses.
-For this one can use the `concatenate_aggregator()` method of `consensus/aggregator.py`.
+The simplest aggregation method is a simple concatenation of multiple responses, and can be achieved using the `_concatenate_aggregator()` method of `consensus/aggregator.py`.
 
-The `centralized_llm_aggregator()` method relies on a specified model to which the individual responses are sent to.
+In contrast, the `centralized_llm_aggregator()` method relies on a specified LLM to which the individual responses are sent to.
 The prompt is built as follows:
 
 * A first message defined in the `aggregator_context` field of the `input.json`.
@@ -36,7 +41,7 @@ The prompt is built as follows:
 
 ## Config
 
-The data from the `input.json` is loaded into a set of classes, defined within `consensus/config.py`:
+The data from the `input.json` is loaded into a set of classes, defined within `settings.py`:
 
 * `ModelConfig`: a class for specifying an LLM.
 * `AggregatorConfig`: a class dedicated to the aggregator model.

src/data/chat_response.json

@@ -18,15 +18,15 @@
         },
         {
             "role": "assistant",
-            "content": "Certainly! In the original \"Pokemon the Series,\" Brock is widely recognized as the second best trainer due to his exceptional skills and dedication to the art of Pokemon training. Here are some precise arguments that support this claim:\n\n1. **Expertise in Rock-type Pokemon**: Brock is known for his mastery of rock-type Pokemon, which is a significant advantage in the world of Pokemon. His knowledge and experience in this area have made him a formidable opponent in battles.\n\n2. **Leadership in Pewter City**: As the leader of the Pewter City Gym, Brock has demonstrated his ability to train and lead Pokemon effectively. His gym is one of the toughest in the region, and he has successfully defended it against numerous challenges.\n\n3. **Competitive Spirit**: Brock's competitive spirit and determination to win battles are evident throughout the series. He has consistently shown a willingness to push himself and his Pokemon to their limits, which is a hallmark of a great trainer.\n\n4. **Team Composition**: Brock's team is well-rounded and includes a variety of Pokemon, which showcases his ability to train and utilize different types of Pokemon effectively. This versatility is crucial in the dynamic world of Pokemon battles.\n\n5. **Influence on the Series**: Brock's character has had a significant impact on the series, influencing the development of other characters and the overall narrative. His presence in the series has added depth and realism to the world of Pokemon.\n\nIn summary, Brock's expertise, leadership, competitive spirit, team composition, and influence on the series collectively make him the second best trainer in the original \"Pokemon the Series.\""
+            "content": "Certainly! In the original \"Pokemon the Series,\" Brock is widely recognized as the second best trainer due to his exceptional skills and dedication to the art of Pokemon training. Here are some precise arguments to support this claim:\n\n1. **Expertise in Rock-type Pokemon**: Brock is a master of rock-type Pokemon, which is evident from his ability to train and battle with these types of Pokemon effectively. His knowledge and mastery of rock-type Pokemon strategies are unparalleled among the main characters in the series.\n\n2. **Leadership in Pewter City**: As the leader of the Pewter City Gym, Brock is responsible for training and preparing Pokemon for battles. His dedication to his role and his ability to guide and train other trainers make him a significant figure in the series.\n\n3. **Competitive Spirit**: Brock's competitive spirit is evident in his participation in the Johto League Championships. His determination to win and his ability to strategize and adapt during battles are commendable.\n\n4. **Team Composition**: Brock's team is composed of rock-type Pokemon, which showcases his specialization and expertise in this type. His team includes Pokemon like Onix, Geodude, and Rhyhorn, which are all rock-type Pokemon.\n\n5. **Character Development**: Throughout the series, Brock's character development is notable. He starts as a simple gym leader but grows into a more strategic and thoughtful trainer, showing his growth and improvement as a Pokemon trainer.\n\nThese points collectively demonstrate Brock's exceptional skills and dedication, making him the second best trainer in the original \"Pokemon the Series.\""
         },
         {
             "role": "user",
             "content": "Can you improve on your previous answers with more precise arguments?"
         },
         {
             "role": "assistant",
-            "content": "Certainly! Here are more precise arguments to support the claim that Brock is the second best trainer in the original \"Pokemon the Series\":\n\n1. **Rock-type Mastery**: Brock's specialization in rock-type Pokemon is unparalleled. His knowledge of rock moves and strategies is extensive, making him a formidable opponent in battles. His ability to counter and exploit the weaknesses of other Pokemon types is a testament to his expertise.\n\n2. **Gym Leader**: As the leader of the Pewter City Gym, Brock has demonstrated his ability to train and lead Pokemon effectively. His gym is one of the toughest in the region, and he has successfully defended it against numerous challenges, showcasing his strategic prowess and the strength of his team.\n\n3. **Competitive Spirit**: Brock's competitive spirit and determination to win battles are evident throughout the series. He has consistently shown a willingness to push himself and his Pokemon to their limits, which is a hallmark of a great trainer. His unwavering commitment to victory has inspired many other trainers in the series.\n\n4. **Team Composition**: Brock's team is well-rounded and includes a variety of Pokemon, which showcases his ability to train and utilize different types of Pokemon effectively. This versatility is crucial in the dynamic world of Pokemon battles, where a well-balanced team can adapt to various situations and strategies.\n\n5. **Influence on the Series**: Brock's character has had a significant impact on the series, influencing the development of other characters and the overall narrative. His presence in the series has added depth and realism to the world of Pokemon, making him a beloved and influential figure among fans.\n\nIn conclusion, Brock's expertise in rock-type Pokemon, his leadership as a Gym leader, his competitive spirit, his well-rounded team, and his influence on the series collectively make him the second best trainer in the original \"Pokemon the Series.\""
+            "content": "Certainly! Here are more precise arguments to support the claim that Brock is the second best trainer in the original \"Pokemon the Series\":\n\n1. **Rock-type Expertise**: Brock's specialization in rock-type Pokemon is unparalleled. His knowledge of rock-type Pokemon's strengths, weaknesses, and strategies is extensive, making him a formidable opponent in battles.\n\n2. **Leadership Skills**: As the leader of the Pewter City Gym, Brock demonstrates exceptional leadership skills. He not only trains his own Pokemon but also helps other trainers improve their skills, showcasing his dedication to the art of Pokemon training.\n\n3. **Competitive Success**: Brock's participation in the Johto League Championships highlights his competitive spirit and ability to perform under pressure. His strategic thinking and ability to adapt during battles are crucial factors in his success.\n\n4. **Team Composition**: Brock's team is meticulously composed of rock-type Pokemon, which showcases his deep understanding of the type's capabilities and weaknesses. This strategic team composition is a testament to his expertise and planning.\n\n5. **Character Development**: Throughout the series, Brock's character development is evident. He starts as a straightforward gym leader but evolves into a more complex and strategic trainer. His growth and the lessons he imparts to other trainers demonstrate his commitment to the art of Pokemon training.\n\nThese points collectively highlight Brock's exceptional skills, dedication, and strategic thinking, making him the second best trainer in the original \"Pokemon the Series.\""
         }
     ]
 }