Skip to content

Compatibility Issues with DeepSeek-R1 Model Response Format #200

@darielnoel

Description

@darielnoel

Describe the Bug

The DeepSeek-R1 model outputs a response format incompatible with KaibanJS's default prompt engineering setup, which expects JSON-formatted responses. This mismatch causes workflows using the DeepSeek-R1 model to fail during execution. The reasoning-based response structure from the model needs specific adjustments to the parsing logic or prompt tuning in KaibanJS.


Steps to Reproduce

  1. Configure an agent in KaibanJS to use the DeepSeek-R1 model:

    const reasoningAgent = new Agent({
      name: 'Reasoner',
      role: 'Analyze complex data patterns',
      llmConfig: {
        provider: 'deepseek',
        model: 'deepseek-r1',
      },
    });
  2. Execute a workflow with the agent.

Image
  1. Observe that the workflow fails due to the model's reasoning-based response format.

Current vs. Expected Behavior

  • Current Behavior: The agent fails to process the output from DeepSeek-R1 due to its non-JSON response format, leading to workflow interruptions.
  • Expected Behavior: The agent should successfully process the model's responses and complete the workflow without issues.

Additional Notes

  • DeepSeek-R1 provides a reasoning-based response format, as detailed in their API documentation: DeepSeek Reasoning Model Guide.
  • This format requires adjustments to KaibanJS's response handling or prompt engineering to accommodate the reasoning model's output style.
  • Reference issue in LangChain regarding similar compatibility concerns: LangChain Issue #29317.
  • Potential solutions include extending the prompt engineering to support reasoning-style models or adapting KaibanJS's JSON parsing logic.

Metadata

Metadata

Assignees

No one assigned

    Labels

    coreFundamental operations and essential features of AgenticJSintegrationsAdding support for third-party tools, agents, or serviceslibraryIssues and enhancements directly related to the codebase of the library component of the project.llms

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions