Python: Error when running step8_assistant_vision.py: Invalid value for purpose

[anu43]

Description

When running step8_assistant_vision.py from the main repository, an error occurs indicating an invalid value for the purpose parameter.

Steps to Reproduce

1. Clone the semantic-kernel repository
2. Navigate to python/samples/getting_started_with_agents/
3. Run python step8_assistant_vision.py

Expected Behavior

The script should run without errors and process the cat image.

Actual Behavior

The script fails with an error: openai.BadRequestError: Error code: 400 - {'error': {'code': 'invalidPayload', 'message': 'Invalid value for purpose.'}}

Environment

• Operating System: macOS 15.2
• Python version: Python 3.10.13
• Semantic Kernel version: semantic-kernel==1.18.2
• OpenAI library version: openai==1.59.7

Error Message

Traceback (most recent call last):
  File "/Users/nihat/Documents/epam/semantic-kernel/python/samples/getting_started_with_agents/step8_assistant_vision.py", line 137, in <module>
    asyncio.run(main())
  File "/opt/homebrew/anaconda3/envs/sem-kernel/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/opt/homebrew/anaconda3/envs/sem-kernel/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
    return future.result()
  File "/Users/nihat/Documents/epam/semantic-kernel/python/samples/getting_started_with_agents/step8_assistant_vision.py", line 101, in main
    file_id = await agent.add_file(cat_image_file_path, purpose="vision")
  File "/opt/homebrew/anaconda3/envs/sem-kernel/lib/python3.10/site-packages/semantic_kernel/agents/open_ai/open_ai_assistant_base.py", line 553, in add_file
    file = await self.client.files.create(file=file, purpose=purpose)  # type: ignore
  File "/opt/homebrew/anaconda3/envs/sem-kernel/lib/python3.10/site-packages/openai/resources/files.py", line 443, in create
    return await self._post(
  File "/opt/homebrew/anaconda3/envs/sem-kernel/lib/python3.10/site-packages/openai/_base_client.py", line 1849, in post
    return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)
  File "/opt/homebrew/anaconda3/envs/sem-kernel/lib/python3.10/site-packages/openai/_base_client.py", line 1543, in request
    return await self._request(
  File "/opt/homebrew/anaconda3/envs/sem-kernel/lib/python3.10/site-packages/openai/_base_client.py", line 1644, in _request
    raise self._make_status_error_from_response(err.response) from None
openai.BadRequestError: Error code: 400 - {'error': {'code': 'invalidPayload', 'message': 'Invalid value for purpose.'}}

[moonbox3]

Hi @anu43, are you running the sample out of the box, without modifications? Please let me know what OpenAI model you're using. I am able to run it with gpt-4o, as an example.

# AuthorRole.USER: 'Describe this image.'
# AuthorRole.ASSISTANT: The image depicts the bustling atmosphere of Times Square in New York City. It's a vibrant scene filled with bright lights and large digital billboards advertising various brands and shows. The scene is crowded with people walking and taking photos, and there are numerous taxis and vehicles on the streets. Visible logos and advertisements include well-known brands such as McDonald's, Samsung, and various Broadway shows. The architecture is tall and modern, with a towering skyline. Signs and billboards contribute to the lively, energetic feel characteristic of Times Square, a major commercial and tourist hub.
# AuthorRole.USER: 'What is the main color in this image?'
# AuthorRole.ASSISTANT: The main color in this image is blue. This is evident from the vast expanse of water surrounding the great white shark and the school of fish. The shades of blue range from light to dark, dominating the image and contributing to the underwater environment's overall appearance.
# AuthorRole.USER: 'Is there an animal in this image?'
# AuthorRole.ASSISTANT: Yes, there is an animal in this image. It features a cat sitting gracefully in a lush, colorful garden. The garden is filled with a variety of flowers and greenery, creating a serene and picturesque setting.

[anu43]

I ran this by Azure OpenAI's gpt-4o 2024-02-15-preview. What Azure OpenAI model would you suggest?

[moonbox3]

You’ll want to use at least a May API version. I was testing with 2024-09-01-preview. Admittedly we can do better at documenting which AOAI api versions work. I will create a work item to do this.

[moonbox3]

Tracking the docs improvements around which AOAI api versions to use for the sample in #10231.