Skip to content

Heavy Tests

Heavy Tests #900

Manually triggered December 17, 2025 15:41
Status Failure
Total duration 12m 7s
Artifacts 9

heavy-tests.yml

on: workflow_dispatch
Matrix: integration-tests
Fit to window
Zoom out
Zoom in

Annotations

21 errors
ModelCapabilitiesIntegrationTest.[67] LLModel(provider=Google, id=gemini-2.5-flash, capabilities=[Temperature, Completion, MultipleChoices, Image, Video, Audio, Tools, ToolChoice, Basic, Standard], contextLength=1048576, maxOutputTokens=65536), Image[jvm]: integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/capabilities/ModelCapabilitiesIntegrationTest.kt#L180
ai.koog.prompt.executor.clients.LLMClientException: Error from client: GoogleLLMClient Illegal input: Field 'candidates' is required for type with serial name 'ai.koog.prompt.executor.clients.google.models.GoogleResponse', but it was missing at path: $
ModelCapabilitiesIntegrationTest.[57] LLModel(provider=Google, id=gemini-2.5-pro, capabilities=[Temperature, Completion, MultipleChoices, Image, Video, Audio, Tools, ToolChoice, Basic, Standard], contextLength=1048576, maxOutputTokens=65536), Image[jvm]: integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/capabilities/ModelCapabilitiesIntegrationTest.kt#L180
ai.koog.prompt.executor.clients.LLMClientException: Error from client: GoogleLLMClient Illegal input: Field 'candidates' is required for type with serial name 'ai.koog.prompt.executor.clients.google.models.GoogleResponse', but it was missing at path: $
capabilities-tests
Process completed with exit code 1.
MultipleLLMPromptExecutorIntegrationTest.[8] LLModel(provider=Google, id=gemini-2.5-flash, capabilities=[Temperature, Completion, MultipleChoices, Image, Video, Audio, Tools, ToolChoice, Basic, Standard], contextLength=1048576, maxOutputTokens=65536)[jvm]: integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/MultipleLLMPromptExecutorIntegrationTest.kt#L197
ai.koog.prompt.executor.clients.LLMClientException: Error from client: GoogleLLMClient Illegal input: Field 'candidates' is required for type with serial name 'ai.koog.prompt.executor.clients.google.models.GoogleResponse', but it was missing at path: $
MultipleLLMPromptExecutorIntegrationTest.[7] LLModel(provider=Google, id=gemini-2.5-pro, capabilities=[Temperature, Completion, MultipleChoices, Image, Video, Audio, Tools, ToolChoice, Basic, Standard], contextLength=1048576, maxOutputTokens=65536)[jvm]: integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/MultipleLLMPromptExecutorIntegrationTest.kt#L197
ai.koog.prompt.executor.clients.LLMClientException: Error from client: GoogleLLMClient Illegal input: Field 'candidates' is required for type with serial name 'ai.koog.prompt.executor.clients.google.models.GoogleResponse', but it was missing at path: $
MultipleLLMPromptExecutorIntegrationTest.[2] BASIC_PNG, LLModel(provider=Google, id=gemini-2.5-pro, capabilities=[Temperature, Completion, MultipleChoices, Image, Video, Audio, Tools, ToolChoice, Basic, Standard], contextLength=1048576, maxOutputTokens=65536)[jvm]: integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/MultipleLLMPromptExecutorIntegrationTest.kt#L88
ai.koog.prompt.executor.clients.LLMClientException: Error from client: GoogleLLMClient Illegal input: Field 'candidates' is required for type with serial name 'ai.koog.prompt.executor.clients.google.models.GoogleResponse', but it was missing at path: $
MultipleLLMPromptExecutorIntegrationTest.[20] SMALL_IMAGE, LLModel(provider=Google, id=gemini-2.5-pro, capabilities=[Temperature, Completion, MultipleChoices, Image, Video, Audio, Tools, ToolChoice, Basic, Standard], contextLength=1048576, maxOutputTokens=65536)[jvm]: integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/MultipleLLMPromptExecutorIntegrationTest.kt#L88
ai.koog.prompt.executor.clients.LLMClientException: Error from client: GoogleLLMClient Illegal input: Field 'candidates' is required for type with serial name 'ai.koog.prompt.executor.clients.google.models.GoogleResponse', but it was missing at path: $
MultipleLLMPromptExecutorIntegrationTest.[10] LLModel(provider=OpenRouter, id=qwen/qwen3-vl-8b-instruct, capabilities=[Temperature, Speculation, Tools, Completion, Image, Standard, ToolChoice], contextLength=131072, maxOutputTokens=33000)[jvm]: integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/MultipleLLMPromptExecutorIntegrationTest.kt#L119
ai.koog.prompt.executor.clients.LLMClientException: Error from client: OpenRouterLLMClient Error from client: OpenRouterLLMClient Message: Field 'name' is required for type with serial name 'ai.koog.prompt.executor.clients.openai.base.models.OpenAIFunction', but it was missing at path: $.choices[***].delta.toolCalls[***].function Status code: 2***
MultipleLLMPromptExecutorIntegrationTest.[9] LLModel(provider=OpenRouter, id=deepseek/deepseek-chat-v3-0324, capabilities=[Temperature, Speculation, Tools, Completion], contextLength=163800, maxOutputTokens=163800)[jvm]: integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/MultipleLLMPromptExecutorIntegrationTest.kt#L119
ai.koog.prompt.executor.clients.LLMClientException: Error from client: OpenRouterLLMClient Error from client: OpenRouterLLMClient Message: Field 'name' is required for type with serial name 'ai.koog.prompt.executor.clients.openai.base.models.OpenAIFunction', but it was missing at path: $.choices[***].delta.toolCalls[***].function Status code: 2***
multiple-llm-executor-tests
Process completed with exit code 1.
SingleLLMPromptExecutorIntegrationTest.[8] LLModel(provider=Google, id=gemini-2.5-flash, capabilities=[Temperature, Completion, MultipleChoices, Image, Video, Audio, Tools, ToolChoice, Basic, Standard], contextLength=1048576, maxOutputTokens=65536)[jvm]: integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/SingleLLMPromptExecutorIntegrationTest.kt#L228
ai.koog.prompt.executor.clients.LLMClientException: Error from client: GoogleLLMClient Illegal input: Field 'candidates' is required for type with serial name 'ai.koog.prompt.executor.clients.google.models.GoogleResponse', but it was missing at path: $
SingleLLMPromptExecutorIntegrationTest.[7] LLModel(provider=Google, id=gemini-2.5-pro, capabilities=[Temperature, Completion, MultipleChoices, Image, Video, Audio, Tools, ToolChoice, Basic, Standard], contextLength=1048576, maxOutputTokens=65536)[jvm]: integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/SingleLLMPromptExecutorIntegrationTest.kt#L228
ai.koog.prompt.executor.clients.LLMClientException: Error from client: GoogleLLMClient Illegal input: Field 'candidates' is required for type with serial name 'ai.koog.prompt.executor.clients.google.models.GoogleResponse', but it was missing at path: $
SingleLLMPromptExecutorIntegrationTest.[2] BASIC_PNG, LLModel(provider=Google, id=gemini-2.5-pro, capabilities=[Temperature, Completion, MultipleChoices, Image, Video, Audio, Tools, ToolChoice, Basic, Standard], contextLength=1048576, maxOutputTokens=65536)[jvm]: integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/SingleLLMPromptExecutorIntegrationTest.kt#L207
ai.koog.prompt.executor.clients.LLMClientException: Error from client: GoogleLLMClient Illegal input: Field 'candidates' is required for type with serial name 'ai.koog.prompt.executor.clients.google.models.GoogleResponse', but it was missing at path: $
SingleLLMPromptExecutorIntegrationTest.[10] LLModel(provider=OpenRouter, id=qwen/qwen3-vl-8b-instruct, capabilities=[Temperature, Speculation, Tools, Completion, Image, Standard, ToolChoice], contextLength=131072, maxOutputTokens=33000)[jvm]: integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/SingleLLMPromptExecutorIntegrationTest.kt#L113
ai.koog.prompt.executor.clients.LLMClientException: Error from client: OpenRouterLLMClient Error from client: OpenRouterLLMClient Message: Field 'name' is required for type with serial name 'ai.koog.prompt.executor.clients.openai.base.models.OpenAIFunction', but it was missing at path: $.choices[***].delta.toolCalls[***].function Status code: 2***
SingleLLMPromptExecutorIntegrationTest.[9] LLModel(provider=OpenRouter, id=deepseek/deepseek-chat-v3-0324, capabilities=[Temperature, Speculation, Tools, Completion], contextLength=163800, maxOutputTokens=163800)[jvm]: integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/SingleLLMPromptExecutorIntegrationTest.kt#L113
ai.koog.prompt.executor.clients.LLMClientException: Error from client: OpenRouterLLMClient Error from client: OpenRouterLLMClient Message: Field 'name' is required for type with serial name 'ai.koog.prompt.executor.clients.openai.base.models.OpenAIFunction', but it was missing at path: $.choices[***].delta.toolCalls[***].function Status code: 2***
SingleLLMPromptExecutorIntegrationTest.[20] SMALL_IMAGE, LLModel(provider=Google, id=gemini-2.5-pro, capabilities=[Temperature, Completion, MultipleChoices, Image, Video, Audio, Tools, ToolChoice, Basic, Standard], contextLength=1048576, maxOutputTokens=65536)[jvm]: integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/SingleLLMPromptExecutorIntegrationTest.kt#L207
ai.koog.prompt.executor.clients.LLMClientException: Error from client: GoogleLLMClient Illegal input: Field 'candidates' is required for type with serial name 'ai.koog.prompt.executor.clients.google.models.GoogleResponse', but it was missing at path: $
SingleLLMPromptExecutorIntegrationTest.[13] LLModel(provider=MistralAI, id=mistral-medium-latest, capabilities=[Temperature, Completion, Tools, ToolChoice, Basic, Standard, Image, Document, MultipleChoices], contextLength=128000, maxOutputTokens=null)[jvm]: integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/SingleLLMPromptExecutorIntegrationTest.kt#L237
org.opentest4j.AssertionFailedError: "this image appears to be a png test file. png (portable network graphics) is a raster-graphics file format that supports lossless data compression. the content of the image itself is designed to test various aspects of png encoding and decoding. here are some observations: 1. **color and transparency**: the image includes a variety of colors and patterns, which are used to test how well the png format handles different color depths and transparency. 2. **text and graphics**: the text "png test" is prominently displayed, indicating the purpose of the image. the background and the man’s suit have a grid pattern with numbers and letters, which can be used to check for artifacts and compression quality. 3. **image quality**: the man in the image, who is holding an old-fashioned camera, is depicted with a mix of solid colors and gradients. this helps in testing how well the png format preserves image quality, especially in areas with fine details and gradients. 4. **artifacts and errors**: the colorful, pixelated overlay on the man and the background is likely used to test how the image handles complex patterns and potential artifacts during compression and decompression. overall, this image is a diagnostic tool to ensure that png files are being rendered correctly by software, maintaining color accuracy, transparency, and image quality." should include substring "test image" expected:<test image> but was:<this image appears to be a png test file. png (portable network graphics) is a raster-graphics file format that supports lossless data compression. the content of the image itself is designed to test various aspects of png encoding and decoding. here are some observations: 1. **color and transparency**: the image includes a variety of colors and patterns, which are used to test how well the png format handles different color depths and transparency. 2. **text and graphics**: the text "png test" is prominently displayed, indicating the purpose of the image. the background and the man’s suit have a grid pattern with numbers and letters, which can be used to check for artifacts and compression quality. 3. **image quality**: the man in the image, who is holding an old-fashioned camera, is depicted with a mix of solid colors and gradients. this helps in testing how well the png format preserves image quality, especially in areas with fine details and gradients. 4. **artifacts and errors**: the colorful, pixelated overlay on the man and the background is likely used to test how the image handles complex patterns and potential artifacts during compression and decompression. overall, this image is a diagnostic tool to ensure that png files are being rendered correctly by software, maintaining color accuracy, transparency, and image quality.>
SingleLLMPromptExecutorIntegrationTest.[6] LLModel(provider=Anthropic, id=claude-sonnet-4-5, capabilities=[Temperature, Tools, ToolChoice, Image, Document, Completion], contextLength=200000, maxOutputTokens=64000)[jvm]: integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/SingleLLMPromptExecutorIntegrationTest.kt#L237
org.opentest4j.AssertionFailedError: "this image appears to be a stylized, digitally-altered photograph with a glitchy, vaporwave aesthetic. here's what i can observe: **main subject:** - an elderly man with a white beard wearing a wide-brimmed hat and light-colored suit - he's positioned next to what appears to be an old-fashioned camera on a tripod - his pose and attire suggest he might be portraying a vintage photographer or historical figure **visual style:** - heavy digital distortion and glitch effects throughout - color palette dominated by purples, pinks, and blues - multiple overlapping elements creating a layered, corrupted appearance - asian characters (appears to be chinese/japanese text) scattered around the edges - "png test" text prominently displayed in yellow at the bottom **aesthetic:** - the overall style evokes internet/digital art movements like vaporwave or glitch art - intentionally distorted and corrupted appearance - mix of vintage subject matter with modern digital manipulation techniques the image seems to be an artistic experiment or test file, possibly exploring digital compression artifacts, glitch aesthetics, or file format characteristics (hence "png test")." should include substring "test image" expected:<test image> but was:<this image appears to be a stylized, digitally-altered photograph with a glitchy, vaporwave aesthetic. here's what i can observe: **main subject:** - an elderly man with a white beard wearing a wide-brimmed hat and light-colored suit - he's positioned next to what appears to be an old-fashioned camera on a tripod - his pose and attire suggest he might be portraying a vintage photographer or historical figure **visual style:** - heavy digital distortion and glitch effects throughout - color palette dominated by purples, pinks, and blues - multiple overlapping elements creating a layered, corrupted appearance - asian characters (appears to be chinese/japanese text) scattered around the edges - "png test" text prominently displayed in yellow at the bottom **aesthetic:** - the overall style evokes internet/digital art movements like vaporwave or glitch art - intentionally distorted and corrupted appearance - mix of vintage subject matter with modern digital manipulation techniques the image seems to be an artistic experiment or test file, possibly exploring digital compression artifacts, glitch aesthetics, or file format characteristics (hence "png test").>
single-llm-executor-tests
Process completed with exit code 1.

Artifacts

Produced during runtime
Name Size Digest
reports-ubuntu-latest-agent-tests
319 KB
sha256:6e19bfb0010191e03c06a8faee63e456973bd3d1184a6eb75dc206e99dd756e2
reports-ubuntu-latest-anthropic-schema-test
287 KB
sha256:ae7e36403080e798fbcecc035377144ad85fb3747dc63e11e3165314533cf651
reports-ubuntu-latest-bedrock-credentials-test
284 KB
sha256:b1af392ac7af26f09d53c8b080956881ef6a82831e6202761b6c713d0bc4e679
reports-ubuntu-latest-bedrock-tests
284 KB
sha256:ce5d40456ddcfc6c8f3275948df21e065323523d7f369ddde8f0674994318346
reports-ubuntu-latest-capabilities-tests
297 KB
sha256:a618a840c0200146fa1d0846f9baf3c3077106a0c65ab88f1fbcc3bd58c09008
reports-ubuntu-latest-embeddings-test
284 KB
sha256:80ac4cadac7c526c2d8485509475ef1f5f97ced54f87f458fdacf64343132b95
reports-ubuntu-latest-multiple-llm-executor-tests
311 KB
sha256:8582b7b39be9b665dd6b3a21847555b71b6d8fdb86ca746ab2df996172bcc6aa
reports-ubuntu-latest-other-executor-tests
284 KB
sha256:76397270b0920063f8ff92468d5fc73739db63302bfc3fb1881337fb3c42fe31
reports-ubuntu-latest-single-llm-executor-tests
312 KB
sha256:cb06236dd9cc3f2b11f3cf4089a702f33b0f8aaf8bd90cd4c4f8e2e996ddd3bb