Skip to content

Deepseek R1 with JSON returns partial message text (a bunch of new lines) #5816

Open
@pdevito3

Description

@pdevito3

Description

When making a curl call with deepseek r1 like this

curl --request POST \
  --url http://localhost:11434/api/generate \
  --header 'Content-Type: application/json; charset=utf-8' \
  --data '{
  "model": "deepseek-r1",
  "prompt": "[INST] Generate 5 product category names for an online retailer\n of high-tech outdoor adventure goods and related clothing/electronics/etc.\n Each category name is a single descriptive term, so it does not use the word '\''and'\''.\n Category names should be interesting and novel, e.g., \"Mountain Unicycles\", \"AI Boots\",\n or \"High-volume Water Filtration Plants\", not simply \"Tents\".\n This retailer sells relatively technical products.\n\n Each category has a list of up to 8 brand names that make products in that category. All brand names are\n purely fictional. Brand names are usually multiple words with spaces and/or special characters, e.g.\n \"Orange Gear\", \"Aqua Tech US\", \"Livewell\", \"E & K\", \"JAXⓇ\".\n Many brand names are used in multiple categories. Some categories have only 2 brands.\n \n The response should be in a JSON format like below with the exact batch count of objects. It is very important you make sure it is in this format and that it is valid JSON. \n { \"categories\": [{\"name\":\"Tents\", \"brands\":[\"Rosewood\", \"Summit Kings\"]}] } [/INST]",
  "format": "json",
  "response_format": {
    "type": "json_object"
  },
  "options": {
    "temperature": 0.9,
    "numPredict": 1750,
    "topP": 1
  },
  "raw": true,
  "stream": false,
  "stop": [
    "[/TOOL_CALLS]"
  ]
}'

you get a response like

{
  "model": "deepseek-r1",
  "created_at": "2025-01-26T17:26:53.933014Z",
  "response": "{\n \"categories\": [\n   {\n     \"name\": \"Mountain Unicycles\",\n     \"brands\": [\"Quark Waves\", \"Throttle Pro\"]\n   },\n   {\n     \"name\": \"AI Boots\",\n     \"brands\": [\"Nanite Steps\", \"RoboGuard\"]\n   },\n   {\n     \"name\": \"High-End Hiking Boots\",\n     \"brands\": [\"Pinnacle Trails\", \"Cobra X\"]\n   },\n   {\n     \"name\": \"Electric Kites\",\n     \"brands\": [\"EcoWings\", \"SkyDance\"]\n   },\n   {\n     \"name\": \"Smartwater Rides\",\n     \"brands\": [\"Voyager Pro\", \"Oasis Explore\"]\n   }\n ]\n}\n \n \n\n\n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n",
  "done": false
}

This is fine other than the weird new lines at the end -- they're causing a problem when interacting with MS AI though as they are the only part of the message text response.

Note

fwiw, i get that maybe this is a bug that needs to be resolved by them but wanted to at least call it out for y'all and other's awareness

Reproduction Steps

When doing a comparable call (at least i believe so) with MS AI:

var chatOptions = new ChatOptions
        {
            ResponseFormat = ChatResponseFormat.Json,
            // response_format neded for their json setup -- https://api-docs.deepseek.com/guides/json_mode
            AdditionalProperties = new AdditionalPropertiesDictionary
            {
                ["response_format"] = new
                {
                    type = "json_object"
                }
            }
        };

        var prompt = "Generate 5 product category names for an online retailer\n of high-tech outdoor adventure goods and related clothing/electronics/etc.\n Each category name is a single descriptive term, so it does not use the word 'and'.\n Category names should be interesting and novel, e.g., \"Mountain Unicycles\", \"AI Boots\",\n or \"High-volume Water Filtration Plants\", not simply \"Tents\".\n This retailer sells relatively technical products.\n\n Each category has a list of up to 8 brand names that make products in that category. All brand names are\n purely fictional. Brand names are usually multiple words with spaces and/or special characters, e.g.\n \"Orange Gear\", \"Aqua Tech US\", \"Livewell\", \"E & K\", \"JAXⓇ\".\n Many brand names are used in multiple categories. Some categories have only 2 brands.\n \n The response should be in a JSON format like below with the exact batch count of objects. It is very important you make sure it is in this format and that it is valid JSON. \n { \"categories\": [{\"name\":\"Tents\", \"brands\":[\"Rosewood\", \"Summit Kings\"]}] }";

        var chatCompletion = await chatClient.CompleteAsync(prompt, chatOptions);
        var parsedJson = JsonSerializer.Deserialize<CategoriesResponse>(chatCompletion.Message.Text, 
new(JsonSerializerDefaults.Web)
    {
        WriteIndented = true
    });

You just get the new lines as the message text response:
Image

Expected behavior

get the full text response

Actual behavior

getting a partial response with just new lines

Regression?

Kind of if you're talking overall capabilities of parsing a message -- message text comes back fine for other models, but not deepseek, but could be classified as an enhancement to handle more scenarios like the deepseek one

Known Workarounds

None with MS AI that I know of, I'd need to manually make the http call like the curl call and parse the response from there.

Configuration

.NET 9
Mac Sequoia 15.2
AFAIK, not specific to this config

Other information

No response

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area-aiMicrosoft.Extensions.AI librariesenhancementThis issue represents an ask for new feature or an enhancement to an existing one

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions