Skip to content

feat(dashscope): Support stream tool call#133

Open
mengnankkkk wants to merge 6 commits intospring-ai-alibaba:mainfrom
mengnankkkk:hotfix/tool_call_stream
Open

feat(dashscope): Support stream tool call#133
mengnankkkk wants to merge 6 commits intospring-ai-alibaba:mainfrom
mengnankkkk:hotfix/tool_call_stream

Conversation

@mengnankkkk
Copy link
Contributor

@mengnankkkk mengnankkkk commented Jan 21, 2026

Describe what this PR does / why we need it

在 DashScopeChatOptions 中新增了 enableStreamToolCalls 配置项(默认 false)
当设置为 true 时,工具调用的片段会在工具执行之前就开始流式返回

Does this pull request fix one issue?

Close #111

Describe how you did it

测试代码:

 void testStreamToolCallsWithRealApiIntegration() {
        // Get API key from environment
        String apiKey = System.getenv("AI_DASHSCOPE_API_KEY");
        if (apiKey == null || apiKey.isEmpty()) {
            throw new IllegalStateException("AI_DASHSCOPE_API_KEY environment variable is not set");
        }

        // Create real DashScope API instance
        DashScopeApi realApi = DashScopeApi.builder().apiKey(apiKey).build();

        // Create a weather request class
        record WeatherRequest(String location) {}

        // Create a real tool callback for getting weather
        ToolCallback weatherTool = FunctionToolCallback.builder("get_weather", 
                (WeatherRequest request) -> {
                    // Simulate weather API call
                    return "The weather in " + request.location() + " is sunny with 25°C";
                })
                .description("Get the current weather for a given location")
                .inputType(WeatherRequest.class)
                .build();

        // Configure options with enableStreamToolCalls=true
        DashScopeChatOptions options = DashScopeChatOptions.builder()
                .model("qwen-plus") // Use a model that supports tool calls
                .stream(true)
                .enableStreamToolCalls(true)
                .toolCallbacks(List.of(weatherTool))
                .build();

        // Create chat model with real API
        DashScopeChatModel realChatModel = DashScopeChatModel.builder()
                .dashScopeApi(realApi)
                .defaultOptions(options)
                .build();

        // Create a prompt that should trigger tool execution
        UserMessage userMessage = new UserMessage("What's the weather like in Beijing today?");
        Prompt prompt = new Prompt(List.of(userMessage), options);

        // Track states during streaming
        AtomicBoolean receivedToolCallChunks = new AtomicBoolean(false);
        AtomicBoolean receivedToolExecutionResult = new AtomicBoolean(false);
        AtomicBoolean hasToolCalls = new AtomicBoolean(false);

        // Execute streaming request
        List<ChatResponse> responses = realChatModel.stream(prompt)
                .doOnNext(response -> {
                    System.out.println("=== Received Chunk ===");
                    System.out.println("Text: " + response.getResult().getOutput().getText());
                    
                    // Check if this chunk has tool calls
                    if (!response.getResult().getOutput().getToolCalls().isEmpty()) {
                        hasToolCalls.set(true);
                        receivedToolCallChunks.set(true);
                        System.out.println("Tool Calls: " + response.getResult().getOutput().getToolCalls());
                    }

                    // Check finish reason
                    String finishReason = response.getResult().getMetadata().getFinishReason();
                    System.out.println("Finish Reason: " + finishReason);

                    // Check if tool execution result is present
                    if (response.getResult().getOutput().getText().contains("sunny") 
                            || response.getResult().getOutput().getText().contains("25°C")) {
                        receivedToolExecutionResult.set(true);
                        System.out.println("Tool execution result detected in response");
                    }
                })
                .collectList()
                .block();

        // Verify results
        assertThat(responses).isNotNull();
        assertThat(responses).isNotEmpty();

        // Verify that we received tool call chunks during streaming
        assertThat(receivedToolCallChunks.get())
                .as("Should have received tool call chunks during streaming")
                .isTrue();

        // Verify that tool execution happened and result was received
        assertThat(receivedToolExecutionResult.get())
                .as("Should have received tool execution result in final response")
                .isTrue();

        // Print summary
        System.out.println("\n=== Test Summary ===");
        System.out.println("Total chunks received: " + responses.size());
        System.out.println("Tool calls detected: " + hasToolCalls.get());
        System.out.println("Tool execution result received: " + receivedToolExecutionResult.get());

        // Verify the final response contains expected content
        String finalText = responses.stream()
                .map(r -> r.getResult().getOutput().getText())
                .reduce("", (a, b) -> a + b);
        
        System.out.println("Final combined text: " + finalText);
        
        // The final text should contain either the tool call information or the weather result
        assertThat(finalText)
                .as("Final response should contain weather-related information")
                .isNotEmpty();
    }

得到结果

Running com.alibaba.cloud.ai.dashscope.chat.DashScopeChatModelTests
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
21:57:13.649 [main] WARN com.alibaba.cloud.ai.tool.validator.DefaultToolCallValidator -- Filtering out toolCall with null function name: ToolCall[id=tool-call-id, type=function, function=ChatCompletionFunction[name=null, arguments={"location": "Beijing"}], index=null]
=== Received Chunk ===
Text:
Tool Calls: [ToolCall[id=call_1f37553f6d0540208c9493, type=function, name=get_weather, arguments={"location": "]]
Finish Reason:
21:57:15.047 [HttpClient-2-Worker-0] WARN com.alibaba.cloud.ai.tool.validator.DefaultToolCallValidator -- Filtering out toolCall with null function name: ToolCall[id=, type=function, function=ChatCompletionFunction[name=null, arguments=Beijing"}], index=0]
=== Received Chunk ===
Text:
Finish Reason: TOOL_CALLS
=== Received Chunk ===
Text: The
Finish Reason:
=== Received Chunk ===
Text:  weather in
Finish Reason:
=== Received Chunk ===
Text:  Beijing today is
Finish Reason:
=== Received Chunk ===
Text:  sunny with a
Finish Reason:
Tool execution result detected in response
=== Received Chunk ===
Text:  temperature of 25°C
Finish Reason:
Tool execution result detected in response
=== Received Chunk ===
Text: .
Finish Reason:
=== Received Chunk ===
Text:
Finish Reason: STOP

=== Test Summary ===
Total chunks received: 9
Tool calls detected: true
Tool execution result received: true
Final combined text: The weather in Beijing today is sunny with a temperature of 25°C.

Describe how to verify it

Special notes for reviews

@mengnankkkk mengnankkkk marked this pull request as draft January 21, 2026 14:09
@mengnankkkk mengnankkkk marked this pull request as ready for review January 22, 2026 11:13
@guanxuc guanxuc self-requested a review January 22, 2026 13:54
@guanxuc guanxuc force-pushed the hotfix/tool_call_stream branch from f5d7465 to d447702 Compare January 26, 2026 01:50
@guanxuc
Copy link
Collaborator

guanxuc commented Jan 26, 2026

DashScopeChatModelTests里麻烦加一下流式输出toolcall的单测

@mengnankkkk mengnankkkk requested a review from guanxuc January 31, 2026 07:26
@guanxuc guanxuc changed the title feat(dashscope): add tool stream feat(dashscope): Support stream tool call Jan 31, 2026
guanxuc and others added 2 commits January 31, 2026 22:31
Co-authored-by: guanxu <86234262+guanxuc@users.noreply.github.com>
Co-authored-by: guanxu <86234262+guanxuc@users.noreply.github.com>
@guanxuc
Copy link
Collaborator

guanxuc commented Jan 31, 2026

@mengnankkkk stream tool call好像丢失了arguments参数,这个有空帮看下

non-stream tool call:

toolCalls=[ToolCall[id=call_d5c655db312244389a49b7, type=function, name=getCurrentWeather, arguments={"location": "San Francisco, CA", "unit": "C"}]]

stream tool call:

toolCalls=[ToolCall[id=call_f469e0c8e41945aa912b02, type=function, name=getCurrentWeather, arguments=]]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Tool call support streamable output

2 participants