Python, Phi-4 returns 500 error

[geosword]

Describe the bug
A clear and concise description of what the bug is.
Phi-4 does not work - returns a 500 error

To Reproduce
Steps to reproduce the behavior:

1. Create a new app
2. Custom engine app
3. Basic AI Chatbot
4. Python
5. Azure OpenAI
6. Enter AOAI key (from Deployment in AI Foundry)
7. Enter the endpoint (from Deployment in AI Foundry)
8. run local env (after checking .env.local.user has the appropriate values)
9. The bot returns "an error occurred getting the response" (or something to that effect) in the teams chat. and checking the terminal output shows:
   """
   File "/Users/dylanh/development/devclub/devclub67/aoaibot/.venv/lib/python3.12/site-packages/teams/ai/planners/action_planner.py", line 100, in begin_task
   return await self.continue_task(context, state)
   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/Users/dylanh/development/devclub/devclub67/aoaibot/.venv/lib/python3.12/site-packages/teams/ai/planners/action_planner.py", line 111, in continue_task
   raise ApplicationError(res.error or "[ActionPlanner]: failed task")
   teams.app_error.ApplicationError:
   The chat completion API returned an error
   status of InternalServerError: Error code: 500 - {'error': {'code': 'InternalServerError', 'message': 'Backend returned unexpected response. Please contact Microsoft for help.'}}
   """
   Expected behavior
   I expected the model to return a response

VS Code Extension Information (please complete the following information):

• OS: Mac OS
• Version 14.6.1 (23G93)

Version: 1.97.2
Commit: e54c774e0add60467559eb0d1e229c6452cf8447
Date: 2025-02-12T23:20:35.343Z (3 wks ago)
Electron: 32.2.7
ElectronBuildId: 10982180
Chromium: 128.0.6613.186
Node.js: 20.18.1
V8: 12.8.374.38-electron.0
OS: Darwin arm64 23.6.0

Additional context
Add any other context about the problem here.

[teams-toolkit-assistant]

👋 Hi @geosword,

Thank you for reporting this issue! We want to let you know that:

⏱️ Our team will personally review your issue
🎯 You can expect a response within 2 business days

✨ This is an AI-generated initial response
🤖 AI-Generated Preliminary Analysis:

It appears that this issue may be related to Azure OpenAI and its interaction with the Teams Toolkit. Here are a few potential solutions and steps you can take to troubleshoot and resolve the issue:

Potential Issues and Solutions:

1. Azure Subscription Validation:

   • Ensure that the Azure subscription being used is active and has the required services enabled. Log in to the Azure portal and confirm the subscription status.

2. Configuration and Deployment:

   • Double-check the configuration details in your .env.local.user file to ensure the Azure OpenAI key and endpoint are correct.
   • Verify that your deployment steps in AI Foundry are followed accurately.

3. Examine the Internal Server Error:

   • A 500 error generally indicates an issue on the server side. Investigate the logs on Azure to identify any specific errors.

   • Ensure you have valid and accurate keys and endpoints:

     AZURE_OPENAI_ENDPOINT=<your-azure-openai-endpoint>
     AZURE_OPENAI_API_KEY=<your-azure-openai-api-key>

4. Error Handling and Response Size:

   • Microsoft Teams has specific error codes and descriptions which might help you pinpoint the issue. Here are some error codes:

     Error code	Error name	Description
     100	NOT_SUPPORTED_ON_PLATFORM	API isn't supported on the current platform.
     500	INTERNAL_ERROR	Internal error encountered while performing operation.
     4000	INVALID_ARGUMENTS	One or more arguments are invalid.
     10000	SIZE_EXCEEDED	Return value is too large and has exceeded the platform size boundaries.

   • Ensure your responses do not exceed the allowable size for Teams, which is 40 KB. Adjust your prompt or reduce response characters to avoid exceeding this limit.

Steps to Troubleshoot:

1. Validate Azure Credentials:

   • Make sure your Azure OpenAI credentials are correct and valid.
   • Renew or regenerate keys if necessary.

2. Review Azure Logs and Detailed Errors:

   • Access Azure's diagnostic logs for more specific error information.
   • Focus on the specific responses caught in the log files that mention the 500 error.

3. Debug Locally:

   • Try debugging your chatbot locally before deployment to ensure it is functioning correctly without API errors. Use the Teams Toolkit debug feature for this purpose.

4. Check Related GitHub Issues:

   • Similar issues such as this one may provide additional clues or workarounds.

5. Update Teams Toolkit and Dependencies:

   • Ensure your Teams Toolkit and all dependencies are up-to-date to avoid potential bugs fixed in newer versions.

6. Sample Code:

   • Here’s a simplified example of endpoint handling in Python:

     import openai
     
     openai.api_base = "<your-azure-openai-endpoint>"
     openai.api_key = "<your-azure-openai-api-key>"
     
     try:
         response = openai.Completion.create(
             engine="davinci",
             prompt="Tell me a joke.",
             max_tokens=50
         )
         print(response.choices[0].text.strip())
     except openai.error.OpenAIError as e:
         print(f"An error occurred: {e}")

Additional Resources:

1. Error Handling in Teams:

   • Microsoft Docs: Integrate media capabilities - Error handling
   • Integrate People Picker - Error handling
   • Integrate QR or barcode scanner capability - Error handling

2. Official Documentation:

   • Review the Microsoft Documentation on error codes for related errors in Teams development.

By following these steps and reviewing the provided links, you should be able to identify and address the cause of the 500 error when using Azure OpenAI in your Teams app. If you encounter persistent issues, consider seeking further assistance from the Microsoft support community or opening an issue on the Teams Toolkit's GitHub repository.

---

Note: This is an automated first response generated by AI. A human team member will review your issue and provide a more detailed response soon. We appreciate your patience!

[geosword]

BTW, I was initially of the opinion that I'd just configured something incorrectly (I suppose that could still be the case!) However, As a validator, I used the "Deploy as web app" feature in Chat Playground

And used the resulting app, it too seems to suffer from the same problem:
"""
Error
Error code: 500 - {'error': {'code': 'InternalServerError', 'message': 'Backend returned unexpected response. Please contact Microsoft for help.'}}
"""

This is presented in the Chat UI itself.

I haven't dissected the code of this application, but the only notable difference in terms of environment variables is the endpoint this app uses, vs the one communicated in Azure AI Foundry:

app uses: https://hello-m6m57c28-francecentral.openai.azure.com/
Foundry Endpoint: https://hello-m6m57c28-francecentral.services.ai.azure.com/

All that said, they do CNAME to the same place, so perhaps this is not relevant.

[sayali-MSFT]

Thank you for your inquiry about your Teams app development issue!

We are checking the issue. We will get back to you shortly.

[frankqianms]

Hi @geosword, the teams-ai SDK used in Basic AI Chatbot Custom Engine Agent support only AzureOpenAI or OpenAI models currently. Only these two models' api-key and endpoint can work directly. Some suffixes of url are fixed in the SDK, so that other enpoints do not work.
Any other models deployed on Azure AI hub need to be customized something when creating the model and planner in your code. Here is an example repo you can refer to: https://github.com/[frankqianms/customModel-py](https://github.com/frankqianms/customModel-py)
Good luck~