Model Configuration

ROSA supports both the OpenAI API and Azure OpenAI for its language model. Users can configure and pass either a ChatOpenAI, a AzureChatOpenAI instance, or a ChatOllama instance to the ROSA class. Here's an overview of how to set up and use these LLMs:

Using ChatOpenAI

To use the standard OpenAI API with the ChatOpenAI model:

1. Ensure you have your OpenAI API key.

2. Set up your environment variable:

   Add the following to your .env file or set it in your system environment:

   OPENAI_API_KEY=your_openai_api_key
   

3. Create a ChatOpenAI instance:

   import os
   from dotenv import load_dotenv
   from langchain_openai import ChatOpenAI
   
   load_dotenv()  # This loads the variables from .env file
   
   openai_llm = ChatOpenAI(
       model_name="gpt-4",  # or your preferred model
       temperature=0,
       max_tokens=None,
       timeout=None,
       max_retries=2,
       openai_api_key=os.getenv("OPENAI_API_KEY"),  # Using environment variable
   )
   
   # Pass the LLM to ROSA
   rosa_instance = ROSA(ros_version=2, llm=openai_llm, ...)

Using AzureChatOpenAI

To use Azure OpenAI, you'll need to create an AzureChatOpenAI instance with the appropriate configuration. There are two ways to set this up:

1. Using Azure API Management (APIM) with Tenant ID, Client ID, and Client Secret:

   Required Environment Variables:

   • APIM_SUBSCRIPTION_KEY (if required by your APIM setup)
   • AZURE_TENANT_ID
   • AZURE_CLIENT_ID
   • AZURE_CLIENT_SECRET
   • DEPLOYMENT_ID
   • API_VERSION
   • API_ENDPOINT

   Add these to your .env file or set them in your system environment.

   import os
   from dotenv import load_dotenv
   from langchain_openai import AzureChatOpenAI
   from azure.identity import ClientSecretCredential, get_bearer_token_provider
   
   load_dotenv()
   
   # Set up Azure authentication
   credential = ClientSecretCredential(
       tenant_id=os.getenv("AZURE_TENANT_ID"),
       client_id=os.getenv("AZURE_CLIENT_ID"),
       client_secret=os.getenv("AZURE_CLIENT_SECRET"),
       authority="https://login.microsoftonline.com",
   )
   
   token_provider = get_bearer_token_provider(
       credential, "https://cognitiveservices.azure.com/.default"
   )
   
   # Create AzureChatOpenAI instance
   azure_llm = AzureChatOpenAI(
       azure_deployment=os.getenv("DEPLOYMENT_ID"),
       azure_ad_token_provider=token_provider,
       openai_api_type="azure_ad",
       api_version=os.getenv("API_VERSION"),
       azure_endpoint=os.getenv("API_ENDPOINT"),
       default_headers={"Ocp-Apim-Subscription-Key": os.getenv("APIM_SUBSCRIPTION_KEY")} if os.getenv("APIM_SUBSCRIPTION_KEY") else {},
   )
   
   # Pass the LLM to ROSA
   rosa = ROSA(ros_version=2, llm=azure_llm, ...)

2. Using API Key:

   Required Environment Variables:

   • AZURE_OPENAI_API_KEY
   • DEPLOYMENT_ID
   • API_ENDPOINT

   Add these to your .env file or set them in your system environment.

   import os
   from dotenv import load_dotenv
   from langchain_openai import AzureChatOpenAI
   
   load_dotenv()
   
   # Create AzureChatOpenAI instance
   azure_llm = AzureChatOpenAI(
       azure_deployment=os.getenv("DEPLOYMENT_ID"),
       openai_api_key=os.getenv("AZURE_OPENAI_API_KEY"),
       azure_endpoint=os.getenv("API_ENDPOINT"),
   )
   
   # Pass the LLM to ROSA
   rosa = ROSA(ros_version=2, llm=azure_llm, ...)

Using ChatOllama

To use Ollama with the ChatOllama model:

1. Ensure you have Ollama installed and running on your system.

2. Create a ChatOllama instance:

   from langchain_ollama import ChatOllama
   
   ollama_llm = ChatOllama(
       model="llama3.1",  # or your preferred model
       temperature=0,
       num_ctx=4096,  # adjust based on your model's context window
   )
   
   # Pass the LLM to ROSA
   rosa_instance = ROSA(ros_version=2, llm=ollama_llm, ...)

   You can customize the ChatOllama instance with additional parameters:

   • base_url: The base URL of your Ollama instance (default is "http://localhost:11434")
   • callback_manager: A callback manager for handling events
   • verbose: Whether to print verbose output

3. If you're running Ollama on a different machine or port, specify the base_url:

   ollama_llm = ChatOllama(
       model="llama3.1",
       base_url="http://your-ollama-server:11434",
       # ... other parameters ...
   )

Important

Ollama runs locally on your machine, so you don't need to set up API keys or environment variables. However, ensure that you have sufficient system resources to run your chosen model.

Note: Ensure that you have the necessary environment variables set in your .env file or system environment. Always handle your API keys and secrets securely.

For more detailed information on using these models, including advanced features like tool calling, streaming, and fine-tuning, refer to the official documentation:

• Azure OpenAI Documentation
• OpenAI Documentation
• Ollama Documentation

Remember to handle your API keys and secrets securely, preferably using environment variables or a secure secret management system.