Skip to content
Heavy Tests

[examples] Add CI checks for code agent examples (#1280) #899

Sign in to view logs

[examples] Add CI checks for code agent examples (#1280)

[examples] Add CI checks for code agent examples (#1280) #899

Triggered via push December 17, 2025 15:05
@EugeneTheDevEugeneTheDev
pushed c8cdc17
develop
Status Failure
Total duration 10m 9s
Artifacts 9

heavy-tests.yml

on: push
Matrix: integration-tests
Fit to window
Zoom out
Zoom in

Annotations

30 errors
ModelCapabilitiesIntegrationTest.[67] LLModel(provider=Google, id=gemini-2.5-flash, capabilities=[Temperature, Completion, MultipleChoices, Image, Video, Audio, Tools, ToolChoice, Basic, Standard], contextLength=1048576, maxOutputTokens=65536), Image[jvm]: integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/capabilities/ModelCapabilitiesIntegrationTest.kt#L179
ai.koog.prompt.executor.clients.LLMClientException: Error from client: GoogleLLMClient Illegal input: Field 'candidates' is required for type with serial name 'ai.koog.prompt.executor.clients.google.models.GoogleResponse', but it was missing at path: $
ModelCapabilitiesIntegrationTest.[57] LLModel(provider=Google, id=gemini-2.5-pro, capabilities=[Temperature, Completion, MultipleChoices, Image, Video, Audio, Tools, ToolChoice, Basic, Standard], contextLength=1048576, maxOutputTokens=65536), Image[jvm]: integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/capabilities/ModelCapabilitiesIntegrationTest.kt#L179
ai.koog.prompt.executor.clients.LLMClientException: Error from client: GoogleLLMClient Illegal input: Field 'candidates' is required for type with serial name 'ai.koog.prompt.executor.clients.google.models.GoogleResponse', but it was missing at path: $
ModelCapabilitiesIntegrationTest.[34] LLModel(provider=OpenAI, id=gpt-5.1-codex, capabilities=[Completion, Temperature, Basic, Standard, Speculation, Tools, ToolChoice, Image, Document, MultipleChoices, Responses], contextLength=400000, maxOutputTokens=128000), MultipleChoices[jvm]: integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/capabilities/ModelCapabilitiesIntegrationTest.kt#L134
ai.koog.prompt.executor.clients.LLMClientException: Error from client: OpenAILLMClient Error from client: OpenAILLMClient Status code: 4***4 Error body: { "error": { "message": "This is not a chat model and thus not supported in the v1/chat/completions endpoint. Did you mean to use v1/completions?", "type": "invalid_request_error", "param": "model", "code": null } }
ModelCapabilitiesIntegrationTest.[33] LLModel(provider=OpenAI, id=gpt-5.1-codex, capabilities=[Completion, Temperature, Basic, Standard, Speculation, Tools, ToolChoice, Image, Document, MultipleChoices, Responses], contextLength=400000, maxOutputTokens=128000), Document[jvm]: integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/capabilities/ModelCapabilitiesIntegrationTest.kt#L1
ai.koog.http.client.KoogHttpClientException: Error from client: OpenAILLMClient Status code: 4*** Error body: { "error": { "message": "Missing required parameter: 'input[1].content[***]'.", "type": "invalid_request_error", "param": "input[1].content[***]", "code": "missing_required_parameter" } }
capabilities-tests
Process completed with exit code 1.
AIAgentIntegrationTest.[10] LLModel(provider=OpenRouter, id=qwen/qwen3-vl-8b-instruct, capabilities=[Temperature, Speculation, Tools, Completion, Image, Standard, ToolChoice], contextLength=131072, maxOutputTokens=33000)[jvm]: integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/agent/AIAgentIntegrationTest.kt#L445
kotlinx.serialization.json.internal.JsonDecodingException: Unexpected JSON token at offset 44: Expected EOF after parsing, but had { instead at path: $ JSON input: {"operation": "MULTIPLY", "a": 7, "b": 2} {"milliseconds": 5***}
AIAgentIntegrationTest.[2] LLModel(provider=OpenAI, id=gpt-4.1, capabilities=[Temperature, Basic, Standard, Speculation, Tools, ToolChoice, Image, Document, Completion, MultipleChoices, Completions, Responses], contextLength=1047576, maxOutputTokens=32768)[jvm]: integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/agent/AIAgentIntegrationTest.kt#L464
ai.koog.agents.core.agent.exception.AIAgentMaxNumberOfIterationsReachedException: AI Agent has run into a problem: Agent couldn't finish in given number of steps (1***). Please, consider increasing `maxAgentIterations` value in agent's configuration
agent-tests
Process completed with exit code 1.
MultipleLLMPromptExecutorIntegrationTest.[3] LLModel(provider=OpenAI, id=gpt-5.1-codex, capabilities=[Completion, Temperature, Basic, Standard, Speculation, Tools, ToolChoice, Image, Document, MultipleChoices, Responses], contextLength=400000, maxOutputTokens=128000)[jvm]: integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/MultipleLLMPromptExecutorIntegrationTest.kt#L233
java.lang.IllegalArgumentException: List has more than one element.
MultipleLLMPromptExecutorIntegrationTest.[8] LLModel(provider=Google, id=gemini-2.5-flash, capabilities=[Temperature, Completion, MultipleChoices, Image, Video, Audio, Tools, ToolChoice, Basic, Standard], contextLength=1048576, maxOutputTokens=65536)[jvm]: integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/MultipleLLMPromptExecutorIntegrationTest.kt#L197
ai.koog.prompt.executor.clients.LLMClientException: Error from client: GoogleLLMClient Illegal input: Field 'candidates' is required for type with serial name 'ai.koog.prompt.executor.clients.google.models.GoogleResponse', but it was missing at path: $
MultipleLLMPromptExecutorIntegrationTest.[2] BASIC_PNG, LLModel(provider=Google, id=gemini-2.5-pro, capabilities=[Temperature, Completion, MultipleChoices, Image, Video, Audio, Tools, ToolChoice, Basic, Standard], contextLength=1048576, maxOutputTokens=65536)[jvm]: integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/MultipleLLMPromptExecutorIntegrationTest.kt#L88
ai.koog.prompt.executor.clients.LLMClientException: Error from client: GoogleLLMClient Illegal input: Field 'candidates' is required for type with serial name 'ai.koog.prompt.executor.clients.google.models.GoogleResponse', but it was missing at path: $
MultipleLLMPromptExecutorIntegrationTest.[7] LLModel(provider=Google, id=gemini-2.5-pro, capabilities=[Temperature, Completion, MultipleChoices, Image, Video, Audio, Tools, ToolChoice, Basic, Standard], contextLength=1048576, maxOutputTokens=65536)[jvm]: integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/MultipleLLMPromptExecutorIntegrationTest.kt#L197
ai.koog.prompt.executor.clients.LLMClientException: Error from client: GoogleLLMClient Illegal input: Field 'candidates' is required for type with serial name 'ai.koog.prompt.executor.clients.google.models.GoogleResponse', but it was missing at path: $
MultipleLLMPromptExecutorIntegrationTest.[3] LLModel(provider=OpenAI, id=gpt-5.1-codex, capabilities=[Completion, Temperature, Basic, Standard, Speculation, Tools, ToolChoice, Image, Document, MultipleChoices, Responses], contextLength=400000, maxOutputTokens=128000)[jvm]: integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/MultipleLLMPromptExecutorIntegrationTest.kt#L197
java.lang.IllegalArgumentException: List has more than one element.
MultipleLLMPromptExecutorIntegrationTest.[20] SMALL_IMAGE, LLModel(provider=Google, id=gemini-2.5-pro, capabilities=[Temperature, Completion, MultipleChoices, Image, Video, Audio, Tools, ToolChoice, Basic, Standard], contextLength=1048576, maxOutputTokens=65536)[jvm]: integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/MultipleLLMPromptExecutorIntegrationTest.kt#L88
ai.koog.prompt.executor.clients.LLMClientException: Error from client: GoogleLLMClient Illegal input: Field 'candidates' is required for type with serial name 'ai.koog.prompt.executor.clients.google.models.GoogleResponse', but it was missing at path: $
MultipleLLMPromptExecutorIntegrationTest.[15] LARGE_IMAGE, LLModel(provider=OpenAI, id=gpt-5.2, capabilities=[Completion, Temperature, Basic, Standard, Speculation, Tools, ToolChoice, Image, Document, MultipleChoices, Completions, Responses], contextLength=400000, maxOutputTokens=128000)[jvm]: integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/MultipleLLMPromptExecutorIntegrationTest.kt#L88
org.opentest4j.AssertionFailedError: "Error from client: OpenAILLMClient Error from client: OpenAILLMClient Status code: 4*** Error body: { "error": { "message": "Invalid content type. image_url is only supported by certain models.", "type": "invalid_request_error", "param": "messages.[1].content.[1].type", "code": null } } " should include substring "image exceeds" Match[***]: part of slice with indexes [***..4] matched actual[152..156] Line[***] ="Error from client: OpenAILLMClient" Line[1] ="Error from client: OpenAILLMClient" Line[2] ="Status code: 4***" Line[3] ="Error body:" Line[4] ="{" Line[5] =" "error": {" Line[6] =" "message": "Invalid content type. image_url is only supported by certain models."," Match[***]= --------------------------------------+++++------------------------------------------- Line[7] =" "type": "invalid_request_error"," Line[8] =" "param": "messages.[1].content.[1].type"," Line[9] =" "code": null" Line[1***] =" }" Line[11] ="}" expected:<image exceeds> but was:<Error from client: OpenAILLMClient Error from client: OpenAILLMClient Status code: 4*** Error body: { "error": { "message": "Invalid content type. image_url is only supported by certain models.", "type": "invalid_request_error", "param": "messages.[1].content.[1].type", "code": null } } >
MultipleLLMPromptExecutorIntegrationTest.[12] LLModel(provider=Bedrock, id=us.anthropic.claude-sonnet-4-5-20250929-v1:0, capabilities=[Temperature, Tools, ToolChoice, Completion], contextLength=200000, maxOutputTokens=64000)[jvm]: integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/MultipleLLMPromptExecutorIntegrationTest.kt#L119
org.opentest4j.AssertionFailedError: List should not be empty
MultipleLLMPromptExecutorIntegrationTest.[10] LLModel(provider=OpenRouter, id=qwen/qwen3-vl-8b-instruct, capabilities=[Temperature, Speculation, Tools, Completion, Image, Standard, ToolChoice], contextLength=131072, maxOutputTokens=33000)[jvm]: integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/MultipleLLMPromptExecutorIntegrationTest.kt#L119
ai.koog.prompt.executor.clients.LLMClientException: Error from client: OpenRouterLLMClient Error from client: OpenRouterLLMClient Message: Field 'name' is required for type with serial name 'ai.koog.prompt.executor.clients.openai.base.models.OpenAIFunction', but it was missing at path: $.choices[***].delta.toolCalls[***].function Status code: 2***
MultipleLLMPromptExecutorIntegrationTest.[6] BASIC_JPG, LLModel(provider=OpenAI, id=gpt-5.2, capabilities=[Completion, Temperature, Basic, Standard, Speculation, Tools, ToolChoice, Image, Document, MultipleChoices, Completions, Responses], contextLength=400000, maxOutputTokens=128000)[jvm]: integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/MultipleLLMPromptExecutorIntegrationTest.kt#L88
ai.koog.prompt.executor.clients.LLMClientException: Error from client: OpenAILLMClient Error from client: OpenAILLMClient Status code: 4*** Error body: { "error": { "message": "Invalid content type. image_url is only supported by certain models.", "type": "invalid_request_error", "param": "messages.[1].content.[1].type", "code": null } }
multiple-llm-executor-tests
Process completed with exit code 1.
SingleLLMPromptExecutorIntegrationTest.[8] LLModel(provider=Google, id=gemini-2.5-flash, capabilities=[Temperature, Completion, MultipleChoices, Image, Video, Audio, Tools, ToolChoice, Basic, Standard], contextLength=1048576, maxOutputTokens=65536)[jvm]: integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/SingleLLMPromptExecutorIntegrationTest.kt#L228
ai.koog.prompt.executor.clients.LLMClientException: Error from client: GoogleLLMClient Illegal input: Field 'candidates' is required for type with serial name 'ai.koog.prompt.executor.clients.google.models.GoogleResponse', but it was missing at path: $
SingleLLMPromptExecutorIntegrationTest.[7] LLModel(provider=Google, id=gemini-2.5-pro, capabilities=[Temperature, Completion, MultipleChoices, Image, Video, Audio, Tools, ToolChoice, Basic, Standard], contextLength=1048576, maxOutputTokens=65536)[jvm]: integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/SingleLLMPromptExecutorIntegrationTest.kt#L228
ai.koog.prompt.executor.clients.LLMClientException: Error from client: GoogleLLMClient Illegal input: Field 'candidates' is required for type with serial name 'ai.koog.prompt.executor.clients.google.models.GoogleResponse', but it was missing at path: $
SingleLLMPromptExecutorIntegrationTest.[3] LLModel(provider=OpenAI, id=gpt-5.1-codex, capabilities=[Completion, Temperature, Basic, Standard, Speculation, Tools, ToolChoice, Image, Document, MultipleChoices, Responses], contextLength=400000, maxOutputTokens=128000)[jvm]: integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/SingleLLMPromptExecutorIntegrationTest.kt#L228
java.lang.IllegalArgumentException: List has more than one element.
SingleLLMPromptExecutorIntegrationTest.[2] BASIC_PNG, LLModel(provider=Google, id=gemini-2.5-pro, capabilities=[Temperature, Completion, MultipleChoices, Image, Video, Audio, Tools, ToolChoice, Basic, Standard], contextLength=1048576, maxOutputTokens=65536)[jvm]: integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/SingleLLMPromptExecutorIntegrationTest.kt#L207
ai.koog.prompt.executor.clients.LLMClientException: Error from client: GoogleLLMClient Illegal input: Field 'candidates' is required for type with serial name 'ai.koog.prompt.executor.clients.google.models.GoogleResponse', but it was missing at path: $
SingleLLMPromptExecutorIntegrationTest.[20] SMALL_IMAGE, LLModel(provider=Google, id=gemini-2.5-pro, capabilities=[Temperature, Completion, MultipleChoices, Image, Video, Audio, Tools, ToolChoice, Basic, Standard], contextLength=1048576, maxOutputTokens=65536)[jvm]: integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/SingleLLMPromptExecutorIntegrationTest.kt#L207
ai.koog.prompt.executor.clients.LLMClientException: Error from client: GoogleLLMClient Illegal input: Field 'candidates' is required for type with serial name 'ai.koog.prompt.executor.clients.google.models.GoogleResponse', but it was missing at path: $
SingleLLMPromptExecutorIntegrationTest.[12] LLModel(provider=Bedrock, id=us.anthropic.claude-sonnet-4-5-20250929-v1:0, capabilities=[Temperature, Tools, ToolChoice, Completion], contextLength=200000, maxOutputTokens=64000)[jvm]: integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/SingleLLMPromptExecutorIntegrationTest.kt#L113
org.opentest4j.AssertionFailedError: List should not be empty
SingleLLMPromptExecutorIntegrationTest.[10] LLModel(provider=OpenRouter, id=qwen/qwen3-vl-8b-instruct, capabilities=[Temperature, Speculation, Tools, Completion, Image, Standard, ToolChoice], contextLength=131072, maxOutputTokens=33000)[jvm]: integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/SingleLLMPromptExecutorIntegrationTest.kt#L113
ai.koog.prompt.executor.clients.LLMClientException: Error from client: OpenRouterLLMClient Error from client: OpenRouterLLMClient Message: Field 'name' is required for type with serial name 'ai.koog.prompt.executor.clients.openai.base.models.OpenAIFunction', but it was missing at path: $.choices[***].delta.toolCalls[***].function Status code: 2***
SingleLLMPromptExecutorIntegrationTest.[9] LLModel(provider=OpenRouter, id=deepseek/deepseek-chat-v3-0324, capabilities=[Temperature, Speculation, Tools, Completion], contextLength=163800, maxOutputTokens=163800)[jvm]: integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/SingleLLMPromptExecutorIntegrationTest.kt#L113
ai.koog.prompt.executor.clients.LLMClientException: Error from client: OpenRouterLLMClient Error from client: OpenRouterLLMClient Message: Field 'name' is required for type with serial name 'ai.koog.prompt.executor.clients.openai.base.models.OpenAIFunction', but it was missing at path: $.choices[***].delta.toolCalls[***].function Status code: 2***
SingleLLMPromptExecutorIntegrationTest.[15] LARGE_IMAGE, LLModel(provider=OpenAI, id=gpt-5.2, capabilities=[Completion, Temperature, Basic, Standard, Speculation, Tools, ToolChoice, Image, Document, MultipleChoices, Completions, Responses], contextLength=400000, maxOutputTokens=128000)[jvm]: integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/SingleLLMPromptExecutorIntegrationTest.kt#L207
org.opentest4j.AssertionFailedError: "Error from client: OpenAILLMClient Error from client: OpenAILLMClient Status code: 4*** Error body: { "error": { "message": "Invalid content type. image_url is only supported by certain models.", "type": "invalid_request_error", "param": "messages.[1].content.[1].type", "code": null } } " should include substring "image exceeds" Match[***]: part of slice with indexes [***..4] matched actual[152..156] Line[***] ="Error from client: OpenAILLMClient" Line[1] ="Error from client: OpenAILLMClient" Line[2] ="Status code: 4***" Line[3] ="Error body:" Line[4] ="{" Line[5] =" "error": {" Line[6] =" "message": "Invalid content type. image_url is only supported by certain models."," Match[***]= --------------------------------------+++++------------------------------------------- Line[7] =" "type": "invalid_request_error"," Line[8] =" "param": "messages.[1].content.[1].type"," Line[9] =" "code": null" Line[1***] =" }" Line[11] ="}" expected:<image exceeds> but was:<Error from client: OpenAILLMClient Error from client: OpenAILLMClient Status code: 4*** Error body: { "error": { "message": "Invalid content type. image_url is only supported by certain models.", "type": "invalid_request_error", "param": "messages.[1].content.[1].type", "code": null } } >
SingleLLMPromptExecutorIntegrationTest.[6] BASIC_JPG, LLModel(provider=OpenAI, id=gpt-5.2, capabilities=[Completion, Temperature, Basic, Standard, Speculation, Tools, ToolChoice, Image, Document, MultipleChoices, Completions, Responses], contextLength=400000, maxOutputTokens=128000)[jvm]: integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/executor/SingleLLMPromptExecutorIntegrationTest.kt#L207
ai.koog.prompt.executor.clients.LLMClientException: Error from client: OpenAILLMClient Error from client: OpenAILLMClient Status code: 4*** Error body: { "error": { "message": "Invalid content type. image_url is only supported by certain models.", "type": "invalid_request_error", "param": "messages.[1].content.[1].type", "code": null } }
single-llm-executor-tests
Process completed with exit code 1.

Artifacts

Produced during runtime
Name Size Digest
reports-ubuntu-latest-agent-tests
331 KB
sha256:c517a620b75bc0276318b50d86524fe14d612e286642c18f0afa494e4c30ac0d
reports-ubuntu-latest-anthropic-schema-test
287 KB
sha256:9839a721d2d5cdde25a6b8c3ba2e29043e351851206502b272b6319038e901c5
reports-ubuntu-latest-bedrock-credentials-test
284 KB
sha256:54947393a7e2236bb6913ea549bbb624a374440adada6a9a16f0edda491ba1b5
reports-ubuntu-latest-bedrock-tests
284 KB
sha256:822cffdcd08b88225581a2d5b105a0be11f79e04526c71e028500fddf8b2df9d
reports-ubuntu-latest-capabilities-tests
297 KB
sha256:983e7ba4001f4c75ae96ea9de091f009ad65bde850a85b7d79f889d78cacbb39
reports-ubuntu-latest-embeddings-test
284 KB
sha256:60cbb9ea91fc6c260fe170d7198ef9fb02d264251154d05c44468fe457448c46
reports-ubuntu-latest-multiple-llm-executor-tests
312 KB
sha256:95b7658b5d3780355240400f58e617b85581eb69ce2d4b6db8b2a58ffaca021f
reports-ubuntu-latest-other-executor-tests
284 KB
sha256:c16f0e675e63fa4e47ff444eb8a9df6bdc06ddb4fb67c77b9288c3796c8a3043
reports-ubuntu-latest-single-llm-executor-tests
312 KB
sha256:124130ae22f7a53549c007cf310dfabe36a0e44d423eeef565880d8a84a6e0b4