Heavy Tests #900
Annotations
21 errors
|
ModelCapabilitiesIntegrationTest.[67] LLModel(provider=Google, id=gemini-2.5-flash, capabilities=[Temperature, Completion, MultipleChoices, Image, Video, Audio, Tools, ToolChoice, Basic, Standard], contextLength=1048576, maxOutputTokens=65536), Image[jvm]:
integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/capabilities/ModelCapabilitiesIntegrationTest.kt#L180
ai.koog.prompt.executor.clients.LLMClientException: Error from client: GoogleLLMClient
Illegal input: Field 'candidates' is required for type with serial name 'ai.koog.prompt.executor.clients.google.models.GoogleResponse', but it was missing at path: $
|
|
ModelCapabilitiesIntegrationTest.[57] LLModel(provider=Google, id=gemini-2.5-pro, capabilities=[Temperature, Completion, MultipleChoices, Image, Video, Audio, Tools, ToolChoice, Basic, Standard], contextLength=1048576, maxOutputTokens=65536), Image[jvm]:
integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/capabilities/ModelCapabilitiesIntegrationTest.kt#L180
ai.koog.prompt.executor.clients.LLMClientException: Error from client: GoogleLLMClient
Illegal input: Field 'candidates' is required for type with serial name 'ai.koog.prompt.executor.clients.google.models.GoogleResponse', but it was missing at path: $
|
|
capabilities-tests
Process completed with exit code 1.
|
|
MultipleLLMPromptExecutorIntegrationTest.[8] LLModel(provider=Google, id=gemini-2.5-flash, capabilities=[Temperature, Completion, MultipleChoices, Image, Video, Audio, Tools, ToolChoice, Basic, Standard], contextLength=1048576, maxOutputTokens=65536)[jvm]:
integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/MultipleLLMPromptExecutorIntegrationTest.kt#L197
ai.koog.prompt.executor.clients.LLMClientException: Error from client: GoogleLLMClient
Illegal input: Field 'candidates' is required for type with serial name 'ai.koog.prompt.executor.clients.google.models.GoogleResponse', but it was missing at path: $
|
|
MultipleLLMPromptExecutorIntegrationTest.[7] LLModel(provider=Google, id=gemini-2.5-pro, capabilities=[Temperature, Completion, MultipleChoices, Image, Video, Audio, Tools, ToolChoice, Basic, Standard], contextLength=1048576, maxOutputTokens=65536)[jvm]:
integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/MultipleLLMPromptExecutorIntegrationTest.kt#L197
ai.koog.prompt.executor.clients.LLMClientException: Error from client: GoogleLLMClient
Illegal input: Field 'candidates' is required for type with serial name 'ai.koog.prompt.executor.clients.google.models.GoogleResponse', but it was missing at path: $
|
|
MultipleLLMPromptExecutorIntegrationTest.[2] BASIC_PNG, LLModel(provider=Google, id=gemini-2.5-pro, capabilities=[Temperature, Completion, MultipleChoices, Image, Video, Audio, Tools, ToolChoice, Basic, Standard], contextLength=1048576, maxOutputTokens=65536)[jvm]:
integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/MultipleLLMPromptExecutorIntegrationTest.kt#L88
ai.koog.prompt.executor.clients.LLMClientException: Error from client: GoogleLLMClient
Illegal input: Field 'candidates' is required for type with serial name 'ai.koog.prompt.executor.clients.google.models.GoogleResponse', but it was missing at path: $
|
|
MultipleLLMPromptExecutorIntegrationTest.[12] LLModel(provider=Bedrock, id=us.anthropic.claude-sonnet-4-5-20250929-v1:0, capabilities=[Temperature, Tools, ToolChoice, Completion], contextLength=200000, maxOutputTokens=64000)[jvm]:
integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/MultipleLLMPromptExecutorIntegrationTest.kt#L119
org.opentest4j.AssertionFailedError: List should not be empty
|
|
MultipleLLMPromptExecutorIntegrationTest.[20] SMALL_IMAGE, LLModel(provider=Google, id=gemini-2.5-pro, capabilities=[Temperature, Completion, MultipleChoices, Image, Video, Audio, Tools, ToolChoice, Basic, Standard], contextLength=1048576, maxOutputTokens=65536)[jvm]:
integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/MultipleLLMPromptExecutorIntegrationTest.kt#L88
ai.koog.prompt.executor.clients.LLMClientException: Error from client: GoogleLLMClient
Illegal input: Field 'candidates' is required for type with serial name 'ai.koog.prompt.executor.clients.google.models.GoogleResponse', but it was missing at path: $
|
|
MultipleLLMPromptExecutorIntegrationTest.[10] LLModel(provider=OpenRouter, id=qwen/qwen3-vl-8b-instruct, capabilities=[Temperature, Speculation, Tools, Completion, Image, Standard, ToolChoice], contextLength=131072, maxOutputTokens=33000)[jvm]:
integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/MultipleLLMPromptExecutorIntegrationTest.kt#L119
ai.koog.prompt.executor.clients.LLMClientException: Error from client: OpenRouterLLMClient
Error from client: OpenRouterLLMClient
Message: Field 'name' is required for type with serial name 'ai.koog.prompt.executor.clients.openai.base.models.OpenAIFunction', but it was missing at path: $.choices[***].delta.toolCalls[***].function
Status code: 2***
|
|
MultipleLLMPromptExecutorIntegrationTest.[9] LLModel(provider=OpenRouter, id=deepseek/deepseek-chat-v3-0324, capabilities=[Temperature, Speculation, Tools, Completion], contextLength=163800, maxOutputTokens=163800)[jvm]:
integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/MultipleLLMPromptExecutorIntegrationTest.kt#L119
ai.koog.prompt.executor.clients.LLMClientException: Error from client: OpenRouterLLMClient
Error from client: OpenRouterLLMClient
Message: Field 'name' is required for type with serial name 'ai.koog.prompt.executor.clients.openai.base.models.OpenAIFunction', but it was missing at path: $.choices[***].delta.toolCalls[***].function
Status code: 2***
|
|
multiple-llm-executor-tests
Process completed with exit code 1.
|
|
SingleLLMPromptExecutorIntegrationTest.[8] LLModel(provider=Google, id=gemini-2.5-flash, capabilities=[Temperature, Completion, MultipleChoices, Image, Video, Audio, Tools, ToolChoice, Basic, Standard], contextLength=1048576, maxOutputTokens=65536)[jvm]:
integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/SingleLLMPromptExecutorIntegrationTest.kt#L228
ai.koog.prompt.executor.clients.LLMClientException: Error from client: GoogleLLMClient
Illegal input: Field 'candidates' is required for type with serial name 'ai.koog.prompt.executor.clients.google.models.GoogleResponse', but it was missing at path: $
|
|
SingleLLMPromptExecutorIntegrationTest.[7] LLModel(provider=Google, id=gemini-2.5-pro, capabilities=[Temperature, Completion, MultipleChoices, Image, Video, Audio, Tools, ToolChoice, Basic, Standard], contextLength=1048576, maxOutputTokens=65536)[jvm]:
integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/SingleLLMPromptExecutorIntegrationTest.kt#L228
ai.koog.prompt.executor.clients.LLMClientException: Error from client: GoogleLLMClient
Illegal input: Field 'candidates' is required for type with serial name 'ai.koog.prompt.executor.clients.google.models.GoogleResponse', but it was missing at path: $
|
|
SingleLLMPromptExecutorIntegrationTest.[2] BASIC_PNG, LLModel(provider=Google, id=gemini-2.5-pro, capabilities=[Temperature, Completion, MultipleChoices, Image, Video, Audio, Tools, ToolChoice, Basic, Standard], contextLength=1048576, maxOutputTokens=65536)[jvm]:
integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/SingleLLMPromptExecutorIntegrationTest.kt#L207
ai.koog.prompt.executor.clients.LLMClientException: Error from client: GoogleLLMClient
Illegal input: Field 'candidates' is required for type with serial name 'ai.koog.prompt.executor.clients.google.models.GoogleResponse', but it was missing at path: $
|
|
SingleLLMPromptExecutorIntegrationTest.[12] LLModel(provider=Bedrock, id=us.anthropic.claude-sonnet-4-5-20250929-v1:0, capabilities=[Temperature, Tools, ToolChoice, Completion], contextLength=200000, maxOutputTokens=64000)[jvm]:
integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/SingleLLMPromptExecutorIntegrationTest.kt#L113
org.opentest4j.AssertionFailedError: List should not be empty
|
|
SingleLLMPromptExecutorIntegrationTest.[10] LLModel(provider=OpenRouter, id=qwen/qwen3-vl-8b-instruct, capabilities=[Temperature, Speculation, Tools, Completion, Image, Standard, ToolChoice], contextLength=131072, maxOutputTokens=33000)[jvm]:
integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/SingleLLMPromptExecutorIntegrationTest.kt#L113
ai.koog.prompt.executor.clients.LLMClientException: Error from client: OpenRouterLLMClient
Error from client: OpenRouterLLMClient
Message: Field 'name' is required for type with serial name 'ai.koog.prompt.executor.clients.openai.base.models.OpenAIFunction', but it was missing at path: $.choices[***].delta.toolCalls[***].function
Status code: 2***
|
|
SingleLLMPromptExecutorIntegrationTest.[9] LLModel(provider=OpenRouter, id=deepseek/deepseek-chat-v3-0324, capabilities=[Temperature, Speculation, Tools, Completion], contextLength=163800, maxOutputTokens=163800)[jvm]:
integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/SingleLLMPromptExecutorIntegrationTest.kt#L113
ai.koog.prompt.executor.clients.LLMClientException: Error from client: OpenRouterLLMClient
Error from client: OpenRouterLLMClient
Message: Field 'name' is required for type with serial name 'ai.koog.prompt.executor.clients.openai.base.models.OpenAIFunction', but it was missing at path: $.choices[***].delta.toolCalls[***].function
Status code: 2***
|
|
SingleLLMPromptExecutorIntegrationTest.[20] SMALL_IMAGE, LLModel(provider=Google, id=gemini-2.5-pro, capabilities=[Temperature, Completion, MultipleChoices, Image, Video, Audio, Tools, ToolChoice, Basic, Standard], contextLength=1048576, maxOutputTokens=65536)[jvm]:
integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/SingleLLMPromptExecutorIntegrationTest.kt#L207
ai.koog.prompt.executor.clients.LLMClientException: Error from client: GoogleLLMClient
Illegal input: Field 'candidates' is required for type with serial name 'ai.koog.prompt.executor.clients.google.models.GoogleResponse', but it was missing at path: $
|
|
SingleLLMPromptExecutorIntegrationTest.[13] LLModel(provider=MistralAI, id=mistral-medium-latest, capabilities=[Temperature, Completion, Tools, ToolChoice, Basic, Standard, Image, Document, MultipleChoices], contextLength=128000, maxOutputTokens=null)[jvm]:
integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/SingleLLMPromptExecutorIntegrationTest.kt#L237
org.opentest4j.AssertionFailedError: "this image appears to be a png test file. png (portable network graphics) is a raster-graphics file format that supports lossless data compression.
the content of the image itself is designed to test various aspects of png encoding and decoding. here are some observations:
1. **color and transparency**: the image includes a variety of colors and patterns, which are used to test how well the png format handles different color depths and transparency.
2. **text and graphics**: the text "png test" is prominently displayed, indicating the purpose of the image. the background and the man’s suit have a grid pattern with numbers and letters, which can be used to check for artifacts and compression quality.
3. **image quality**: the man in the image, who is holding an old-fashioned camera, is depicted with a mix of solid colors and gradients. this helps in testing how well the png format preserves image quality, especially in areas with fine details and gradients.
4. **artifacts and errors**: the colorful, pixelated overlay on the man and the background is likely used to test how the image handles complex patterns and potential artifacts during compression and decompression.
overall, this image is a diagnostic tool to ensure that png files are being rendered correctly by software, maintaining color accuracy, transparency, and image quality." should include substring "test image"
expected:<test image> but was:<this image appears to be a png test file. png (portable network graphics) is a raster-graphics file format that supports lossless data compression.
the content of the image itself is designed to test various aspects of png encoding and decoding. here are some observations:
1. **color and transparency**: the image includes a variety of colors and patterns, which are used to test how well the png format handles different color depths and transparency.
2. **text and graphics**: the text "png test" is prominently displayed, indicating the purpose of the image. the background and the man’s suit have a grid pattern with numbers and letters, which can be used to check for artifacts and compression quality.
3. **image quality**: the man in the image, who is holding an old-fashioned camera, is depicted with a mix of solid colors and gradients. this helps in testing how well the png format preserves image quality, especially in areas with fine details and gradients.
4. **artifacts and errors**: the colorful, pixelated overlay on the man and the background is likely used to test how the image handles complex patterns and potential artifacts during compression and decompression.
overall, this image is a diagnostic tool to ensure that png files are being rendered correctly by software, maintaining color accuracy, transparency, and image quality.>
|
|
SingleLLMPromptExecutorIntegrationTest.[6] LLModel(provider=Anthropic, id=claude-sonnet-4-5, capabilities=[Temperature, Tools, ToolChoice, Image, Document, Completion], contextLength=200000, maxOutputTokens=64000)[jvm]:
integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/SingleLLMPromptExecutorIntegrationTest.kt#L237
org.opentest4j.AssertionFailedError: "this image appears to be a stylized, digitally-altered photograph with a glitchy, vaporwave aesthetic. here's what i can observe:
**main subject:**
- an elderly man with a white beard wearing a wide-brimmed hat and light-colored suit
- he's positioned next to what appears to be an old-fashioned camera on a tripod
- his pose and attire suggest he might be portraying a vintage photographer or historical figure
**visual style:**
- heavy digital distortion and glitch effects throughout
- color palette dominated by purples, pinks, and blues
- multiple overlapping elements creating a layered, corrupted appearance
- asian characters (appears to be chinese/japanese text) scattered around the edges
- "png test" text prominently displayed in yellow at the bottom
**aesthetic:**
- the overall style evokes internet/digital art movements like vaporwave or glitch art
- intentionally distorted and corrupted appearance
- mix of vintage subject matter with modern digital manipulation techniques
the image seems to be an artistic experiment or test file, possibly exploring digital compression artifacts, glitch aesthetics, or file format characteristics (hence "png test")." should include substring "test image"
expected:<test image> but was:<this image appears to be a stylized, digitally-altered photograph with a glitchy, vaporwave aesthetic. here's what i can observe:
**main subject:**
- an elderly man with a white beard wearing a wide-brimmed hat and light-colored suit
- he's positioned next to what appears to be an old-fashioned camera on a tripod
- his pose and attire suggest he might be portraying a vintage photographer or historical figure
**visual style:**
- heavy digital distortion and glitch effects throughout
- color palette dominated by purples, pinks, and blues
- multiple overlapping elements creating a layered, corrupted appearance
- asian characters (appears to be chinese/japanese text) scattered around the edges
- "png test" text prominently displayed in yellow at the bottom
**aesthetic:**
- the overall style evokes internet/digital art movements like vaporwave or glitch art
- intentionally distorted and corrupted appearance
- mix of vintage subject matter with modern digital manipulation techniques
the image seems to be an artistic experiment or test file, possibly exploring digital compression artifacts, glitch aesthetics, or file format characteristics (hence "png test").>
|
|
single-llm-executor-tests
Process completed with exit code 1.
|
Artifacts
Produced during runtime
| Name | Size | Digest | |
|---|---|---|---|
|
reports-ubuntu-latest-agent-tests
|
319 KB |
sha256:6e19bfb0010191e03c06a8faee63e456973bd3d1184a6eb75dc206e99dd756e2
|
|
|
reports-ubuntu-latest-anthropic-schema-test
|
287 KB |
sha256:ae7e36403080e798fbcecc035377144ad85fb3747dc63e11e3165314533cf651
|
|
|
reports-ubuntu-latest-bedrock-credentials-test
|
284 KB |
sha256:b1af392ac7af26f09d53c8b080956881ef6a82831e6202761b6c713d0bc4e679
|
|
|
reports-ubuntu-latest-bedrock-tests
|
284 KB |
sha256:ce5d40456ddcfc6c8f3275948df21e065323523d7f369ddde8f0674994318346
|
|
|
reports-ubuntu-latest-capabilities-tests
|
297 KB |
sha256:a618a840c0200146fa1d0846f9baf3c3077106a0c65ab88f1fbcc3bd58c09008
|
|
|
reports-ubuntu-latest-embeddings-test
|
284 KB |
sha256:80ac4cadac7c526c2d8485509475ef1f5f97ced54f87f458fdacf64343132b95
|
|
|
reports-ubuntu-latest-multiple-llm-executor-tests
|
311 KB |
sha256:8582b7b39be9b665dd6b3a21847555b71b6d8fdb86ca746ab2df996172bcc6aa
|
|
|
reports-ubuntu-latest-other-executor-tests
|
284 KB |
sha256:76397270b0920063f8ff92468d5fc73739db63302bfc3fb1881337fb3c42fe31
|
|
|
reports-ubuntu-latest-single-llm-executor-tests
|
312 KB |
sha256:cb06236dd9cc3f2b11f3cf4089a702f33b0f8aaf8bd90cd4c4f8e2e996ddd3bb
|
|