Core Package Tests #25265
wardpeetsecrets.test-core.yml
on: workflow_run
Annotations
10 errors
|
src/loop/loop.test.ts > Loop Tests > AISDK v5 > loopFn > output = "object" > options.onFinish > should be called when object doesn't match the schema without destructuring:
packages/core/src/loop/test-utils/streamObject.ts#L1031
Error: Snapshot `Loop Tests > AISDK v5 > loopFn > output = "object" > options.onFinish > should be called when object doesn't match the schema without destructuring 1` mismatched
- Expected
+ Received
- [Error: Structured output validation failed
- ✖ Required
- → at content
- ]
+ [Error: Structured output validation failed: - content: Invalid input: expected string, received undefined]
❯ src/loop/test-utils/streamObject.ts:1031:25
❯ src/loop/test-utils/streamObject.ts:1030:11
|
|
src/agent/agent.test.ts > Agent Tests > test schema compat structured output > should convert optional fields to nullable for openai and succeed without error:
packages/core/src/llm/model/aisdk/v5/model.ts#L45
AI_APICallError: Invalid schema for response_format 'response': In context=(), 'required' is required to be supplied and to be an array including every key in properties. Missing 'windSpeed'.
❯ ../../node_modules/.pnpm/@AI-SDK+provider-utils@3.0.17_zod@4.3.5/node_modules/@ai-sdk/provider-utils/dist/index.mjs:889:14
❯ postToApi ../../node_modules/.pnpm/@AI-SDK+provider-utils@3.0.17_zod@4.3.5/node_modules/@ai-sdk/provider-utils/dist/index.mjs:734:28
❯ OpenAIResponsesLanguageModel.doGenerate ../../node_modules/.pnpm/@AI-SDK+openai@2.0.69_zod@4.3.5/node_modules/@ai-sdk/openai/dist/index.mjs:3426:9
❯ AISDKV5LanguageModel.doGenerate src/llm/model/aisdk/v5/model.ts:45:20
❯ pRetry.default.retries src/stream/aisdk/v5/execute.ts:144:34
❯ Proxy.pRetry ../../node_modules/.pnpm/p-retry@7.1.0/node_modules/p-retry/index.js:195:19
❯ createStream src/stream/aisdk/v5/execute.ts:138:16
❯ Object.start src/stream/base/input.ts:22:26
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Serialized Error: { url: 'https://api.openai.com/v1/responses', requestBodyValues: { model: 'gpt-4o', input: [ { role: 'system', content: 'You are a weather agent. When asked about weather in any city, use the weather info tool with the city name as the input.' }, { role: 'user', content: [ { type: 'input_text', text: 'What is the weather in London? You can omit wind speed, precipitation, and barometric pressure.' } ] } ], temperature: +0, top_p: undefined, max_output_tokens: undefined, text: { format: { type: 'json_schema', strict: true, name: 'response', description: undefined, schema: { '$schema': 'http://json-schema.org/draft-07/schema#', type: 'object', properties: { weather: { type: 'string' }, temperature: { type: 'number' }, humidity: { type: 'number' }, windSpeed: { type: 'string' }, barometricPressure: { anyOf: [ { type: 'number' }, { type: 'null' } ] }, precipitation: { anyOf: [ { type: 'number' }, { type: 'null' } ] } }, required: [ 'weather', 'temperature', 'humidity', 'precipitation' ], additionalProperties: false, '~standard': { validate: 'Function<validate>', vendor: 'zod', version: 1, jsonSchema: { input: 'Function<anonymous>', output: 'Function<anonymous>' } } } } }, conversation: undefined, max_tool_calls: undefined, metadata: undefined, parallel_tool_calls: undefined, previous_response_id: undefined, store: undefined, user: undefined, instructions: undefined, service_tier: undefined, include: undefined, prompt_cache_key: undefined, prompt_cache_retention: undefined, safety_identifier: undefined, top_logprobs: undefined, truncation: undefined, tools: [ { type: 'function', name: 'weatherInfo', description: 'Fetches the current weather information for a given city', parameters: { '$schema': 'https://json-schema.org/draft/2020-12/schema', type: 'object', properties: { city: { type: 'string' } }, required: [ 'city' ], additionalProperties: false }, strict: true } ], tool_choice: 'auto' }, statusCode: 400, responseHeaders: { 'alt-svc': 'h3=":443"; ma=86400', 'cf-cache-status': 'DYNAMIC', 'cf-ray': '9c4bd7228eab28d3-DFW', connection: 'keep-alive', 'content-length': '319', 'content-type': 'application/json', date: 'Tue, 27 Jan 2026 22:55:55 GMT', 'openai-organization': 'mastra-qb2kpb', 'openai-processing-ms': '13', 'openai-project': 'proj_Cvw1JiXoxHTeEWgnnK3t9AYG', 'openai-version': '2020-10-01', server: 'cloudflare', 'set-cookie': '_cfuvid=omRj.wQ2uyKVdjRnaiUHwovofZCr6z9dUOtUvw5nrAw-1769554555331-0.0.1.1-604800000; path=/; domain=.api.openai.com; HttpOnly; Secure; SameSite=None', 'strict-transport-security': 'max-age=31536000; includeSubDomains; preload', 'x-content-type-options': 'nosniff', 'x-request-id': 'req_143d9610265d4ea2a5c5ffce889cabc0' }, responseBody: '{\n "error": {\n "message": "Invalid schema for response_format \'response\': In context=(), \'required\' is required to be supplied and to be an array including every key in properties. Missing \'windSpeed\'.",\n "type": "invalid_request_error",\n "param": "text.format.schema",\n "code": "invalid_json_schema"\n }
|
|
src/agent/agent.test.ts > Agent Tests > v1 - agent > should call tool without input or output schemas:
packages/core/src/llm/model/model.ts#L282
Error: Invalid schema for function 'noSchemaTool': schema must be a JSON Schema of 'type: "object"', got 'type: "None"'.
❯ MastraLLMV1.__text src/llm/model/model.ts:282:27
❯ AgentLegacyHandler.generateLegacy src/agent/agent-legacy.ts:923:22
❯ src/agent/agent.test.ts:582:20
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Serialized Error: { domain: 'LLM', category: 'THIRD_PARTY', code: 'LLM_GENERATE_TEXT_AI_SDK_EXECUTION_FAILED', details: { modelId: 'gpt-4o', modelProvider: 'openai.chat', runId: '825abcba-45f8-4f4f-b854-fe31b75d9900', threadId: 'unknown', resourceId: 'unknown' } }
Caused by: Caused by: AI_APICallError: Invalid schema for function 'noSchemaTool': schema must be a JSON Schema of 'type: "object"', got 'type: "None"'.
❯ ../../node_modules/.pnpm/@AI-SDK+provider-utils@2.2.8_zod@4.3.5/node_modules/@ai-sdk/provider-utils/dist/index.mjs:667:14
❯ postToApi ../../node_modules/.pnpm/@AI-SDK+provider-utils@2.2.8_zod@4.3.5/node_modules/@ai-sdk/provider-utils/dist/index.mjs:567:28
❯ OpenAIChatLanguageModel.doGenerate ../../node_modules/.pnpm/@AI-SDK+openai@1.3.24_zod@4.3.5/node_modules/@ai-sdk/openai/dist/index.mjs:622:9
❯ fn ../../node_modules/.pnpm/ai@4.3.19_react@19.2.3_zod@3.25.76/node_modules/ai/core/generate-text/generate-text.ts:407:30
❯ ../../node_modules/.pnpm/ai@4.3.19_react@19.2.3_zod@3.25.76/node_modules/ai/core/telemetry/record-span.ts:18:22
❯ _retryWithExponentialBackoff ../../node_modules/.pnpm/ai@4.3.19_react@19.2.3_zod@3.25.76/node_modules/ai/util/retry-with-exponential-backoff.ts:36:12
❯ fn ../../node_modules/.pnpm/ai@4.3.19_react@19.2.3_zod@3.25.76/node_modules/ai/core/generate-text/generate-text.ts:363:32
❯ ../../node_modules/.pnpm/ai@4.3.19_react@19.2.3_zod@3.25.76/node_modules/ai/core/telemetry/record-span.ts:18:22
❯ MastraLLMV1.__text src/llm/model/model.ts:256:52
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Serialized Error: { url: 'https://api.openai.com/v1/chat/completions', requestBodyValues: { model: 'gpt-4o', logit_bias: undefined, logprobs: undefined, top_logprobs: undefined, user: undefined, parallel_tool_calls: undefined, max_tokens: undefined, temperature: +0, top_p: undefined, frequency_penalty: undefined, presence_penalty: undefined, response_format: undefined, stop: undefined, seed: undefined, max_completion_tokens: undefined, store: undefined, metadata: undefined, prediction: undefined, reasoning_effort: undefined, messages: [ { role: 'system', content: 'You are an agent that can use the noSchemaTool to get test data.' }, { role: 'user', content: 'Use the noSchemaTool to get test data' } ], tools: [ { type: 'function', function: { name: 'noSchemaTool', description: 'Returns test data with arbitrary structure', parameters: { '$schema': 'http://json-schema.org/draft-07/schema#' }, strict: undefined } } ], tool_choice: 'required', functions: undefined, function_call: undefined }, statusCode: 400, responseHeaders: { 'access-control-expose-headers': 'X-Request-ID', 'alt-svc': 'h3=":443"; ma=86400', 'cf-cache-status': 'DYNAMIC', 'cf-ray': '9c4bd6af6f6628d3-DFW', connection: 'keep-alive', 'content-length': '280', 'content-type': 'application/json', date: 'Tue, 27 Jan 2026 22:55:37 GMT', 'openai-organization': 'mastra-qb2kpb', 'openai-processing-ms': '17', 'openai-project': 'proj_Cvw1JiXoxHTeEWgnnK3t9AYG', 'openai-version': '2020-10-01', server: 'cloudflare', 'set-cookie': '_cfuvid=rmDrFU7zIBumyE6uAPcsdSrMBnBDu.h.vYf4o1mof4A-1769554537177-0.0.1.1-604800000; path=/; domain=.api.openai.com; HttpOnly; Secure; SameSite=None', 'strict-transport-security': 'max-age=31536000; includeSubDomains; preload', 'x-content-type-options': 'nosniff', 'x-openai-proxy-wasm': 'v0.1', 'x-ratelimit-limit-requests': '10000', 'x-ratelimit-limit-tokens': '2000000', 'x-ratelimit-remaining-requests': '9999', 'x-ratelimit-remaining-tokens': '1999971', 'x-ratelimit-reset-requests': '6ms', 'x-ratelimit-reset-tokens': '0s', 'x-request-id': 'req_5fabdcc59910454595a683fb4c3af7db' }, responseBody: '
|
|
src/agent/agent.test.ts > Agent Tests > v1 - agent > should call tool without input or output schemas:
packages/core/src/llm/model/model.ts#L282
Error: Invalid schema for function 'noSchemaTool': schema must be a JSON Schema of 'type: "object"', got 'type: "None"'.
❯ MastraLLMV1.__text src/llm/model/model.ts:282:27
❯ AgentLegacyHandler.generateLegacy src/agent/agent-legacy.ts:923:22
❯ src/agent/agent.test.ts:491:20
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Serialized Error: { domain: 'LLM', category: 'THIRD_PARTY', code: 'LLM_GENERATE_TEXT_AI_SDK_EXECUTION_FAILED', details: { modelId: 'gpt-4o', modelProvider: 'openai.chat', runId: 'f2f4989a-0dae-467e-9e8e-6c0c19de3d73', threadId: 'unknown', resourceId: 'unknown' } }
Caused by: Caused by: AI_APICallError: Invalid schema for function 'noSchemaTool': schema must be a JSON Schema of 'type: "object"', got 'type: "None"'.
❯ ../../node_modules/.pnpm/@AI-SDK+provider-utils@2.2.8_zod@4.3.5/node_modules/@ai-sdk/provider-utils/dist/index.mjs:667:14
❯ postToApi ../../node_modules/.pnpm/@AI-SDK+provider-utils@2.2.8_zod@4.3.5/node_modules/@ai-sdk/provider-utils/dist/index.mjs:567:28
❯ OpenAIChatLanguageModel.doGenerate ../../node_modules/.pnpm/@AI-SDK+openai@1.3.24_zod@4.3.5/node_modules/@ai-sdk/openai/dist/index.mjs:622:9
❯ fn ../../node_modules/.pnpm/ai@4.3.19_react@19.2.3_zod@3.25.76/node_modules/ai/core/generate-text/generate-text.ts:407:30
❯ ../../node_modules/.pnpm/ai@4.3.19_react@19.2.3_zod@3.25.76/node_modules/ai/core/telemetry/record-span.ts:18:22
❯ _retryWithExponentialBackoff ../../node_modules/.pnpm/ai@4.3.19_react@19.2.3_zod@3.25.76/node_modules/ai/util/retry-with-exponential-backoff.ts:36:12
❯ fn ../../node_modules/.pnpm/ai@4.3.19_react@19.2.3_zod@3.25.76/node_modules/ai/core/generate-text/generate-text.ts:363:32
❯ ../../node_modules/.pnpm/ai@4.3.19_react@19.2.3_zod@3.25.76/node_modules/ai/core/telemetry/record-span.ts:18:22
❯ MastraLLMV1.__text src/llm/model/model.ts:256:52
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Serialized Error: { url: 'https://api.openai.com/v1/chat/completions', requestBodyValues: { model: 'gpt-4o', logit_bias: undefined, logprobs: undefined, top_logprobs: undefined, user: undefined, parallel_tool_calls: undefined, max_tokens: undefined, temperature: +0, top_p: undefined, frequency_penalty: undefined, presence_penalty: undefined, response_format: undefined, stop: undefined, seed: undefined, max_completion_tokens: undefined, store: undefined, metadata: undefined, prediction: undefined, reasoning_effort: undefined, messages: [ { role: 'system', content: 'You are an agent that can use the noSchemaTool to get test data.' }, { role: 'user', content: 'Use the noSchemaTool to get test data' } ], tools: [ { type: 'function', function: { name: 'noSchemaTool', description: 'Returns test data with arbitrary structure', parameters: { '$schema': 'http://json-schema.org/draft-07/schema#' }, strict: undefined } } ], tool_choice: 'required', functions: undefined, function_call: undefined }, statusCode: 400, responseHeaders: { 'access-control-expose-headers': 'X-Request-ID', 'alt-svc': 'h3=":443"; ma=86400', 'cf-cache-status': 'DYNAMIC', 'cf-ray': '9c4bd6a619b628d3-DFW', connection: 'keep-alive', 'content-length': '280', 'content-type': 'application/json', date: 'Tue, 27 Jan 2026 22:55:35 GMT', 'openai-organization': 'mastra-qb2kpb', 'openai-processing-ms': '15', 'openai-project': 'proj_Cvw1JiXoxHTeEWgnnK3t9AYG', 'openai-version': '2020-10-01', server: 'cloudflare', 'set-cookie': '_cfuvid=GAFmAXS90gwON8e6A0R0a08NSQfE6teJjkVv8ILRBh8-1769554535454-0.0.1.1-604800000; path=/; domain=.api.openai.com; HttpOnly; Secure; SameSite=None', 'strict-transport-security': 'max-age=31536000; includeSubDomains; preload', 'x-content-type-options': 'nosniff', 'x-openai-proxy-wasm': 'v0.1', 'x-ratelimit-limit-requests': '10000', 'x-ratelimit-limit-tokens': '2000000', 'x-ratelimit-remaining-requests': '9999', 'x-ratelimit-remaining-tokens': '1999971', 'x-ratelimit-reset-requests': '6ms', 'x-ratelimit-reset-tokens': '0s', 'x-request-id': 'req_cdd5e7a893ec499cae4d9438ade14da5' }, responseBody: '
|
|
src/agent/agent.test.ts > Agent Tests > test schema compat structured output > should convert optional fields to nullable for openai and succeed without error:
packages/core/src/llm/model/aisdk/v5/model.ts#L45
AI_APICallError: Invalid schema for response_format 'response': In context=(), 'required' is required to be supplied and to be an array including every key in properties. Missing 'windSpeed'.
❯ ../../node_modules/.pnpm/@AI-SDK+provider-utils@3.0.17_zod@4.3.5/node_modules/@ai-sdk/provider-utils/dist/index.mjs:889:14
❯ postToApi ../../node_modules/.pnpm/@AI-SDK+provider-utils@3.0.17_zod@4.3.5/node_modules/@ai-sdk/provider-utils/dist/index.mjs:734:28
❯ OpenAIResponsesLanguageModel.doGenerate ../../node_modules/.pnpm/@AI-SDK+openai@2.0.69_zod@4.3.5/node_modules/@ai-sdk/openai/dist/index.mjs:3426:9
❯ AISDKV5LanguageModel.doGenerate src/llm/model/aisdk/v5/model.ts:45:20
❯ pRetry.default.retries src/stream/aisdk/v5/execute.ts:144:34
❯ Proxy.pRetry ../../node_modules/.pnpm/p-retry@7.1.0/node_modules/p-retry/index.js:195:19
❯ createStream src/stream/aisdk/v5/execute.ts:138:16
❯ Object.start src/stream/base/input.ts:22:26
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Serialized Error: { url: 'https://api.openai.com/v1/responses', requestBodyValues: { model: 'gpt-4o', input: [ { role: 'system', content: 'You are a weather agent. When asked about weather in any city, use the weather info tool with the city name as the input.' }, { role: 'user', content: [ { type: 'input_text', text: 'What is the weather in London? You can omit wind speed, precipitation, and barometric pressure.' } ] } ], temperature: +0, top_p: undefined, max_output_tokens: undefined, text: { format: { type: 'json_schema', strict: true, name: 'response', description: undefined, schema: { '$schema': 'http://json-schema.org/draft-07/schema#', type: 'object', properties: { weather: { type: 'string' }, temperature: { type: 'number' }, humidity: { type: 'number' }, windSpeed: { type: 'string' }, barometricPressure: { anyOf: [ { type: 'number' }, { type: 'null' } ] }, precipitation: { anyOf: [ { type: 'number' }, { type: 'null' } ] } }, required: [ 'weather', 'temperature', 'humidity', 'precipitation' ], additionalProperties: false, '~standard': { validate: 'Function<validate>', vendor: 'zod', version: 1, jsonSchema: { input: 'Function<anonymous>', output: 'Function<anonymous>' } } } } }, conversation: undefined, max_tool_calls: undefined, metadata: undefined, parallel_tool_calls: undefined, previous_response_id: undefined, store: undefined, user: undefined, instructions: undefined, service_tier: undefined, include: undefined, prompt_cache_key: undefined, prompt_cache_retention: undefined, safety_identifier: undefined, top_logprobs: undefined, truncation: undefined, tools: [ { type: 'function', name: 'weatherInfo', description: 'Fetches the current weather information for a given city', parameters: { '$schema': 'https://json-schema.org/draft/2020-12/schema', type: 'object', properties: { city: { type: 'string' } }, required: [ 'city' ], additionalProperties: false }, strict: true } ], tool_choice: 'auto' }, statusCode: 400, responseHeaders: { 'alt-svc': 'h3=":443"; ma=86400', 'cf-cache-status': 'DYNAMIC', 'cf-ray': '9c4bd6a56f5c28d3-DFW', connection: 'keep-alive', 'content-length': '319', 'content-type': 'application/json', date: 'Tue, 27 Jan 2026 22:55:35 GMT', 'openai-organization': 'mastra-qb2kpb', 'openai-processing-ms': '15', 'openai-project': 'proj_Cvw1JiXoxHTeEWgnnK3t9AYG', 'openai-version': '2020-10-01', server: 'cloudflare', 'set-cookie': '_cfuvid=qxo1SBSmvVLKIFhsueMk.DLFVA4ejkcHX73Zd4OiGZU-1769554535309-0.0.1.1-604800000; path=/; domain=.api.openai.com; HttpOnly; Secure; SameSite=None', 'strict-transport-security': 'max-age=31536000; includeSubDomains; preload', 'x-content-type-options': 'nosniff', 'x-request-id': 'req_82570678dc4e48b488967985fcf53208' }, responseBody: '{\n "error": {\n "message": "Invalid schema for response_format \'response\': In context=(), \'required\' is required to be supplied and to be an array including every key in properties. Missing \'windSpeed\'.",\n "type": "invalid_request_error",\n "param": "text.format.schema",\n "code": "invalid_json_schema"\n }
|
|
src/agent/agent.test.ts > Agent Tests > prepareStep > should execute a new tool added in prepareStep with toolChoice required:
packages/core/src/agent/agent.test.ts#L6555
AssertionError: expected [ { type: 'function', …(4) } ] to match object [ { type: 'function', …(3) } ]
(1 matching property omitted from actual)
- Expected
+ Received
@@ -1,11 +1,11 @@
[
{
"description": "tool1",
"name": "tool1",
"parameters": {
- "$schema": "http://json-schema.org/draft-07/schema#",
+ "$schema": "https://json-schema.org/draft/2020-12/schema",
"additionalProperties": false,
"properties": {
"value": {
"type": "string",
},
❯ src/agent/agent.test.ts:6555:51
|
|
src/agent/agent-gemini.test.ts > Gemini Model Compatibility Tests > Tool suspension and resumption > should call findUserWorkflow with suspend and resume via generate when autoResumeSuspendedTools is true:
packages/core/src/agent/agent-gemini.test.ts#L1158
AssertionError: expected undefined to be 'Dero Israel' // Object.is equality
- Expected:
"Dero Israel"
+ Received:
undefined
❯ src/agent/agent-gemini.test.ts:1158:20
|
|
src/agent/agent-gemini.test.ts > Gemini Model Compatibility Tests > Tool suspension and resumption > should call findUserWorkflow with suspend and resume via stream when autoResumeSuspendedTools is true:
packages/core/src/agent/agent-gemini.test.ts#L1051
AssertionError: expected undefined to be 'Dero Israel' // Object.is equality
- Expected:
"Dero Israel"
+ Received:
undefined
❯ src/agent/agent-gemini.test.ts:1051:22
|
|
src/utils.test.ts > makeCoreTool > should convert a Vercel tool correctly:
packages/core/src/utils.test.ts#L183
AssertionError: expected Error: Schema validation failed due to an… { …(4) } to deeply equal { result: 'success' }
- Expected
+ Received
- {
- "result": "success",
+ MastraError {
+ "message": "Schema validation failed due to an invalid schema definition. This often happens when a union schema (z.union or z.or) has undefined options. Please check that all schema options are properly defined. Original error: Cannot read properties of undefined (reading 'validate')",
+ "cause": Error {
+ "message": "Schema validation failed due to an invalid schema definition. This often happens when a union schema (z.union or z.or) has undefined options. Please check that all schema options are properly defined. Original error: Cannot read properties of undefined (reading 'validate')",
+ },
+ "id": "TOOL_EXECUTION_FAILED",
+ "domain": "TOOL",
+ "category": "USER",
+ "details": {
+ "argsJson": "{\"name\":\"test\"}",
+ "errorMessage": "Failed tool execution",
+ "model": "",
+ },
}
❯ src/utils.test.ts:183:20
|
|
Unhandled error:
packages/core/src/loop/test-utils/streamObject.ts#L1719
TypeCheckError: Type 'Schema<unknown>' is not assignable to type 'StandardSchemaWithJSON<undefined>'.
Property '"~standard"' is missing in type 'Schema<unknown>' but required in type 'StandardSchemaV1<undefined, undefined>'.
❯ src/loop/test-utils/streamObject.ts:1719:15
|