Skip to content

openai api compatibility: ChatCompletionContentPartText array #10552

Description

@edk4971

LocalAI version:
LocalAI v4.5.0 (deb430f)

Environment, CPU architecture, OS, and Version:
docker on ubuntu-server, running inside proxmox vm

Describe the bug
Repeat of #10524
I'm using mealie's openai tool to parse recipes. it passes a content array in the user prompt, which results in:
rpc error: code = InvalidArgument desc = unsupported content[].type

To Reproduce

{
  "messages": [
    {
      "role": "system",
      "content": "Parse ingredient strings into components. You will receive a list of one or more ingredients. It is critical that you return the same number of ingredients as given and in the same order.\n\nWhen parsing:\n- If uncertain about quantity, unit, or food, put the entire string in the note field\n- Respect grammar rules for multiple languages\n- Interpret singular/plural/grammatical variations\n- Text in parentheses = notes (e.g., \"3 potatoes (roughly chopped)\" → note: \"roughly chopped\")\n- Correct common typos (e.g., \"potatos\" → \"potatoes\")\n- Recognize units: tablespoon, teaspoon, gram, tsp, tbsp, oz, sprig, can, bundle, bunch, unit, cube, package, pinch\n- For ranges (e.g., \"3-5\", \"1 to 2\"), use the lower number\n- All text must appear somewhere, or otherwise be accounted for; if converting \"2 dozen\" → \"24\", don't put \"dozen\" elsewhere. If you're unsure, put extra text in the notes\n\n\n###\nBelow is a list of units found in the units database. While parsing, you should reference this list when determining which part of the input is the unit. You may find a unit in the input that does not exist in this list. This should not prevent you from parsing that text as a unit.\n---\n\n[\"sprigs\",\"dash\",\"gallons\",\"mg\",\"kilogram\",\"tsp\",\"pinches\",\"dashes\",\"ml\",\"gram\",\"servings\",\"tbsp\",\"bunch\",\"pint\",\"gallon\",\"fluid ounces\",\"teaspoon\",\"liters\",\"pound\",\"ounce\",\"lbs\",\"pt\",\"g\",\"quarts\",\"oz\",\"stalk\",\"sprig\",\"gal\",\"l\",\"heads\",\"cans\",\"milliliters\",\"bunches\",\"milligrams\",\"fluid ounce\",\"can\",\"milligram\",\"qt\",\"pints\",\"cup\",\"tablespoon\",\"c\",\"milliliter\",\"pinch\",\"teaspoons\",\"ounces\",\"splash\",\"serving\",\"splashes\",\"kilograms\",\"packs\",\"cloves\",\"clove\",\"kg\",\"grams\",\"cups\",\"pounds\",\"head\",\"fl oz\",\"pack\",\"lb\",\"tablespoons\",\"liter\",\"quart\"]\n"
    },
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "[\"1/4 cup brown sugar, packed\",\"1/4 cup reduced sodium soy sauce\",\"2 teaspoons sesame oil\",\"1/2 teaspoon crushed red-pepper flakes, or more to taste\",\"1/4 teaspoon ground ginger\",\"1 tablespoon vegetable oil\",\"3 cloves garlic, minced\",\"1 pound ground beef\",\"2 green onions, thinly sliced\",\"1/4 teaspoon sesame seeds\",\"1 cup brown rice\"]"
        }
      ]
    }
  ],
  "model": "gemma-4-12b-it-GGUF",
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "schema": {
        "$defs": {
          "OpenAIIngredient": {
            "properties": {
              "quantity": {
                "anyOf": [
                  {
                    "type": "number"
                  },
                  {
                    "type": "null"
                  }
                ],
                "default": 0,
                "description": "The numerical quantity as a whole number or float. Convert fractions to decimals.",
                "title": "Quantity"
              },
              "unit": {
                "anyOf": [
                  {
                    "type": "string"
                  },
                  {
                    "type": "null"
                  }
                ],
                "description": "The unit of measurement, e.g., 'cups', 'lbs', 'teaspoons'.",
                "title": "Unit"
              },
              "food": {
                "anyOf": [
                  {
                    "type": "string"
                  },
                  {
                    "type": "null"
                  }
                ],
                "description": "The ingredient itself, e.g., 'onions' or 'chicken breast'.",
                "title": "Food"
              },
              "note": {
                "anyOf": [
                  {
                    "type": "string"
                  },
                  {
                    "type": "null"
                  }
                ],
                "description": "Preparation details, e.g., 'chopped', 'drained', 'peeled and minced'. If there are any elements you're not sure about, put them here.",
                "title": "Note"
              }
            },
            "title": "OpenAIIngredient",
            "type": "object",
            "additionalProperties": false,
            "required": [
              "quantity",
              "unit",
              "food",
              "note"
            ]
          }
        },
        "properties": {
          "ingredients": {
            "description": "List of parsed ingredients.",
            "items": {
              "$ref": "#/$defs/OpenAIIngredient"
            },
            "title": "Ingredients",
            "type": "array"
          }
        },
        "title": "OpenAIIngredients",
        "type": "object",
        "additionalProperties": false,
        "required": [
          "ingredients"
        ]
      },
      "name": "OpenAIIngredients",
      "strict": true
    }
  },
  "stream": false
}

Expected behavior
Returns a parsed list

Logs
attached.

backend-logs-gemma-4-12b-it-GGUF-2026-06-27.json

Additional context

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions