ai: Support tool calls in RAG methods

[scotttrinh]

Our previous methods only supported returning text that was generated by an assistant, but if you provide tools, we were never returning a rich enough object that would allow you to call the tools with the provided arguments.

This is a breaking change because instead of returning string | null, we now return a whole AssistantMessage from the main methods. See the updated README for additional details.

[Copilot]

Pull Request Overview

This PR adds comprehensive support for tool calling functionality to the AI package, expanding its capabilities to handle OpenAI-style function calls in both streaming and non-streaming modes. The changes include extensive test infrastructure improvements, new API types, and detailed documentation.

Key changes:

• Implements OpenAI-style tool calling support with proper streaming and non-streaming handling
• Adds comprehensive test coverage for tool calling scenarios
• Refactors test infrastructure with better debugging and shared setup utilities

Reviewed Changes

Copilot reviewed 10 out of 11 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
packages/ai/test/test-setup.ts	New shared test environment setup utility
packages/ai/test/mockHttpServer.ts	Enhanced mock server with tool calling support and debug logging
packages/ai/test/globalSetup.ts	Added performance timing for test setup
packages/ai/test/core.test.ts	Refactored tests and added tool calling test cases
packages/ai/src/utils.ts	Improved error message formatting
packages/ai/src/types.ts	Added tool definition types and streaming message types
packages/ai/src/core.ts	Updated to return AssistantMessage objects with tool call support
packages/ai/package.json	Added debug dependency
packages/ai/jest.config.js	Added test concurrency limits and timeout configuration
packages/ai/README.md	Added comprehensive tool calling documentation with examples

Comments suppressed due to low confidence (1)

packages/ai/src/types.ts:110

• The property name 'args' is ambiguous and inconsistent with the function calling convention. Consider renaming to 'arguments' to match the OpenAI API standard used elsewhere in the codebase.

      args?: Record<string, unknown> | null;