Model Configuration

ROSA supports both the OpenAI API and Azure OpenAI for its language model. Users can configure and pass either a ChatOpenAI or AzureChatOpenAI instance to the ROSA class. Here's an overview of how to set up and use these LLMs:

Using ChatOpenAI

To use the standard OpenAI API with the ChatOpenAI model:

1. Ensure you have your OpenAI API key.

2. Create a ChatOpenAI instance:

   from langchain_openai import ChatOpenAI
   
   openai_llm = ChatOpenAI(
       model_name="gpt-4o",  # or your preferred model
       openai_api_key="your_openai_api_key",
   )
   
   # Pass the LLM to ROSA
   rosa_instance = ROSA(ros_version=2, llm=openai_llm, ...)

3. Additional Configuration Options:

   • temperature: Controls randomness (0.0 to 2.0, default 0.7).
   • max_tokens: Limits the response length.
   • request_timeout: Sets timeout for API requests.

   Example with additional options:

   openai_llm = ChatOpenAI(
       model_name="gpt-4",
       openai_api_key="your_openai_api_key",
       temperature=0.2,
       max_tokens=512,
       request_timeout=60,
   )

⚠️ Ensure that you have the necessary environment variables set in your .env file or system environment. Always handle your API keys and secrets securely. ⚠️

Using AzureChatOpenAI

To use Azure OpenAI, you'll need to create an AzureChatOpenAI instance with the appropriate configuration. There are two ways to set this up:

1. Using Azure API Management (APIM) with Tenant ID, Client ID, and Client Secret:

   Required Environment Variables:

   • APIM_SUBSCRIPTION_KEY (if required by your APIM setup)
   • AZURE_TENANT_ID
   • AZURE_CLIENT_ID
   • AZURE_CLIENT_SECRET
   • DEPLOYMENT_ID
   • API_VERSION
   • API_ENDPOINT

   import os
   from dotenv import load_dotenv
   from langchain_openai import AzureChatOpenAI
   from azure.identity import ClientSecretCredential, get_bearer_token_provider
   
   load_dotenv()
   
   # Set up Azure authentication
   credential = ClientSecretCredential(
       tenant_id=os.getenv("AZURE_TENANT_ID"),
       client_id=os.getenv("AZURE_CLIENT_ID"),
       client_secret=os.getenv("AZURE_CLIENT_SECRET"),
       authority="https://login.microsoftonline.com",
   )
   
   token_provider = get_bearer_token_provider(
       credential, "https://cognitiveservices.azure.com/.default"
   )
   
   # Create AzureChatOpenAI instance
   azure_llm = AzureChatOpenAI(
       azure_deployment=os.getenv("DEPLOYMENT_ID"),
       azure_ad_token_provider=token_provider,
       openai_api_type="azure_ad",
       api_version=os.getenv("API_VERSION"),
       azure_endpoint=os.getenv("API_ENDPOINT"),
       default_headers={"Ocp-Apim-Subscription-Key": os.getenv("APIM_SUBSCRIPTION_KEY")} if os.getenv("APIM_SUBSCRIPTION_KEY") else {},
   )
   
   # Pass the LLM to ROSA
   rosa = ROSA(ros_version=2, llm=azure_llm, ...)

2. Using API Key:

   Required Environment Variables:

   • AZURE_OPENAI_API_KEY
   • DEPLOYMENT_ID
   • API_ENDPOINT

   import os
   from dotenv import load_dotenv
   from langchain_openai import AzureChatOpenAI
   
   load_dotenv()
   
   # Create AzureChatOpenAI instance
   azure_llm = AzureChatOpenAI(
       azure_deployment=os.getenv("DEPLOYMENT_ID"),
       openai_api_key=os.getenv("AZURE_OPENAI_API_KEY"),
       azure_endpoint=os.getenv("API_ENDPOINT"),
   )
   
   # Pass the LLM to ROSA
   rosa = ROSA(ros_version=2, llm=azure_llm, ...)

Note: If you're using the turtle_agent demo, you can refer to the rosa/src/turtle_agent/scripts/llm.py file for an example of how to set up the LLM. However, when using ROSA in your own project, you have the flexibility to configure and pass the LLM instance directly to the ROSA class as shown in the examples above.

Remember to handle your API keys and secrets securely, preferably using environment variables or a secure secret management system.