Description
Description
When making a curl call with deepseek r1 like this
curl --request POST \
--url http://localhost:11434/api/generate \
--header 'Content-Type: application/json; charset=utf-8' \
--data '{
"model": "deepseek-r1",
"prompt": "[INST] Generate 5 product category names for an online retailer\n of high-tech outdoor adventure goods and related clothing/electronics/etc.\n Each category name is a single descriptive term, so it does not use the word '\''and'\''.\n Category names should be interesting and novel, e.g., \"Mountain Unicycles\", \"AI Boots\",\n or \"High-volume Water Filtration Plants\", not simply \"Tents\".\n This retailer sells relatively technical products.\n\n Each category has a list of up to 8 brand names that make products in that category. All brand names are\n purely fictional. Brand names are usually multiple words with spaces and/or special characters, e.g.\n \"Orange Gear\", \"Aqua Tech US\", \"Livewell\", \"E & K\", \"JAXⓇ\".\n Many brand names are used in multiple categories. Some categories have only 2 brands.\n \n The response should be in a JSON format like below with the exact batch count of objects. It is very important you make sure it is in this format and that it is valid JSON. \n { \"categories\": [{\"name\":\"Tents\", \"brands\":[\"Rosewood\", \"Summit Kings\"]}] } [/INST]",
"format": "json",
"response_format": {
"type": "json_object"
},
"options": {
"temperature": 0.9,
"numPredict": 1750,
"topP": 1
},
"raw": true,
"stream": false,
"stop": [
"[/TOOL_CALLS]"
]
}'
you get a response like
{
"model": "deepseek-r1",
"created_at": "2025-01-26T17:26:53.933014Z",
"response": "{\n \"categories\": [\n {\n \"name\": \"Mountain Unicycles\",\n \"brands\": [\"Quark Waves\", \"Throttle Pro\"]\n },\n {\n \"name\": \"AI Boots\",\n \"brands\": [\"Nanite Steps\", \"RoboGuard\"]\n },\n {\n \"name\": \"High-End Hiking Boots\",\n \"brands\": [\"Pinnacle Trails\", \"Cobra X\"]\n },\n {\n \"name\": \"Electric Kites\",\n \"brands\": [\"EcoWings\", \"SkyDance\"]\n },\n {\n \"name\": \"Smartwater Rides\",\n \"brands\": [\"Voyager Pro\", \"Oasis Explore\"]\n }\n ]\n}\n \n \n\n\n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n",
"done": false
}
This is fine other than the weird new lines at the end -- they're causing a problem when interacting with MS AI though as they are the only part of the message text response.
Note
fwiw, i get that maybe this is a bug that needs to be resolved by them but wanted to at least call it out for y'all and other's awareness
Reproduction Steps
When doing a comparable call (at least i believe so) with MS AI:
var chatOptions = new ChatOptions
{
ResponseFormat = ChatResponseFormat.Json,
// response_format neded for their json setup -- https://api-docs.deepseek.com/guides/json_mode
AdditionalProperties = new AdditionalPropertiesDictionary
{
["response_format"] = new
{
type = "json_object"
}
}
};
var prompt = "Generate 5 product category names for an online retailer\n of high-tech outdoor adventure goods and related clothing/electronics/etc.\n Each category name is a single descriptive term, so it does not use the word 'and'.\n Category names should be interesting and novel, e.g., \"Mountain Unicycles\", \"AI Boots\",\n or \"High-volume Water Filtration Plants\", not simply \"Tents\".\n This retailer sells relatively technical products.\n\n Each category has a list of up to 8 brand names that make products in that category. All brand names are\n purely fictional. Brand names are usually multiple words with spaces and/or special characters, e.g.\n \"Orange Gear\", \"Aqua Tech US\", \"Livewell\", \"E & K\", \"JAXⓇ\".\n Many brand names are used in multiple categories. Some categories have only 2 brands.\n \n The response should be in a JSON format like below with the exact batch count of objects. It is very important you make sure it is in this format and that it is valid JSON. \n { \"categories\": [{\"name\":\"Tents\", \"brands\":[\"Rosewood\", \"Summit Kings\"]}] }";
var chatCompletion = await chatClient.CompleteAsync(prompt, chatOptions);
var parsedJson = JsonSerializer.Deserialize<CategoriesResponse>(chatCompletion.Message.Text,
new(JsonSerializerDefaults.Web)
{
WriteIndented = true
});
You just get the new lines as the message text response:
Expected behavior
get the full text response
Actual behavior
getting a partial response with just new lines
Regression?
Kind of if you're talking overall capabilities of parsing a message -- message text comes back fine for other models, but not deepseek, but could be classified as an enhancement to handle more scenarios like the deepseek one
Known Workarounds
None with MS AI that I know of, I'd need to manually make the http call like the curl call and parse the response from there.
Configuration
.NET 9
Mac Sequoia 15.2
AFAIK, not specific to this config
Other information
No response
Activity