Skip to content

MakeMeSay get_content fails on its documented dict response type #1805

Description

@sylvesterkaczmarek

Describe the bug

evals.elsuite.make_me_say.utils.get_content() is annotated to accept either a dict or a CompletionResult:

def get_content(response: Union[dict, CompletionResult]) -> str:

It handles CompletionResult through get_completions(), but the fallback assumes attribute-style OpenAI response objects:

return response.choices[0].message.content

A normal dictionary response such as:

{"choices": [{"message": {"content": "hello"}}]}

therefore raises AttributeError even though dict is explicitly part of the public input type.

Expected behavior

Dictionary responses should return response["choices"][0]["message"]["content"]; existing CompletionResult and attribute-style response behavior should remain unchanged.

Proposed fix

Add an explicit dictionary branch and focused regression coverage for dict and CompletionResult inputs.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions