[Bug]: Bedrock-llm-provider bad response upon kubectl-ai trigger

[gilad-tayeb]

Environment (please complete the following):

• OS: MacOS
• kubectl-ai version (run kubectl-ai version): 0.0.23
• LLM provider: bedrock
• LLM model: us.anthropic.claude-sonnet-4-20250514-v1:0

Describe the bug
I've set the config.yaml at: ~/.config/kubectl-ai/config.yaml as said in the README. My configurations are as mentioned above (bedrock, us.anthropic.claude-sonnet-4-20250514-v1:0, kubeconfig path is set to default: ~/.kube/config). I've been trying to simply understand if i'm connected and set up properly using the following prompt: "how many namespaces are in the cluster?", or "what pods are currently live in my cluster?", and I keep getting the following answers:

>>> what namespaces are here?

  I'll check what namespaces are available in your cluster.                   

>>> 
>>> 
>>> exit

or even:

  Hey there, what can I help you with today?                                  

>>> what namespaces are here?

  I'll check what namespaces are available in your Kubernetes cluster.        

>>> exit

Meaning - nothing really happening.
If it make sense or assist with anything, I have multiple profiles in my ~/.aws/config file, but even when login to a specific profile (instead of using the regular aws sso login, i've tried aws sso login --profile <profile_name> - got the same responses.

To Reproduce
Steps to reproduce the behavior:

1. Open a terminal, after aws sso login successfully
2. run: kubectl-ai, then ask a simple question.
3. Nothing happens after the polite answer

Expected behavior
I was supposed to see any response, even an error code or something

[droot]

@augustintsang can you pl. help @gilad-tayeb since you are more familiar with the bedrock setup.

[gilad-tayeb]

@augustintsang Let me know if you need any additional info or testing, I really want to make this work

[deisenzimmer]

I found this thread on a google search because I wanted to try out kubectl-ai with bedrock and experienced something similar.

~/dev$ kubectl-ai --llm-provider=bedrock --model=us.anthropic.claude-sonnet-4-20250514-v1:0

  Hey there, what can I help you with today?                                  

>>> how many pods are running in the kong namespace?

  I'll check how many pods are running in the kong namespace for you.         

>>> 
>>> exit

Received signal, shutting down gracefully... (press Ctrl+C again to force)
~/dev$ 
~/dev$ kubectl get pods -n kong
NAME                                 READY   STATUS    RESTARTS        AGE
kong-release-kong-667c459b9d-265k4   2/2     Running   2 (4d22h ago)   4d22h
kong-release-kong-667c459b9d-2wldn   2/2     Running   2 (4d22h ago)   4d22h
kong-release-kong-667c459b9d-7s2lw   2/2     Running   2 (4d22h ago)   4d22h
kong-release-kong-667c459b9d-d7vvz   2/2     Running   2 (4d22h ago)   4d22h
kong-release-kong-667c459b9d-rjgb4   2/2     Running   2 (4d22h ago)   4d22h
kong-release-kong-667c459b9d-sszfz   2/2     Running   2 (4d22h ago)   4d22h

[gilad-tayeb]

Hi @deisenzimmer
Not sure what i'm seeing here, can you share a link to that thread? maybe the solution they might have suggested?

[deisenzimmer]

@gilad-tayeb In the example above I asked kubectl-ai to show me kong pods and it gave an initial reply and nothing beyond. Then I exited kubectl-ai and showed that my kubectl context works the old fashioned way.

[gilad-tayeb]

@deisenzimmer oh, got you.
I did the same thing (I knew my kubectl client works, but had to try to see if i'm missing something).
In my honest opinion, I believe that something with the response parsing isn't working correct, because it looks like it tries to pass the command - but doesn't show the response/doesn't pass it over to the LLM.
I've tried the same thing using OpenAI conf. and it went perfect.

[droot]

I see that some streaming related fixes were checked in recently, we cut a new release today https://github.com/GoogleCloudPlatform/kubectl-ai/releases/tag/v0.0.24

can you pl. try out the new release ? @gilad-tayeb @deisenzimmer

thank you.

[gilad-tayeb]

Hi @droot @deisenzimmer
First of all thanks for the support and responses here, it's highly appreciated! Kudos for the wonderful job here with the app.
I've downloaded the new release (v0.0.24) and installed it on my machine (first removed the previous one, then installed the new version).
It seems that now - the request is actually going through to Bedrock as LLM provider. But boy what is happening once sent?!
This is an example of a request inside the app using Azure OpenAI:
config file:

llmProvider: azopenai
model: gpt-4.1
kubeConfigPath: /Users/userA/.kube/config
skipPermissions: false

Inside the app:

$ kubectl-ai     

  Hey there, what can I help you with today?                                  

>>> What are the namespaces names that are in the cluster?

  Running: kubectl get namespaces -o custom-columns=NAME:.metadata.name --no-headers

  Here are the namespaces currently available in your cluster:  
  • ns-A                                                        
  • default                                                                   
  • ns-B                                                                   
  • ns-C
...

Now, using Bedrock as LLM provider:
config file:

llmProvider: bedrock
model: us.anthropic.claude-3-7-sonnet-20250219-v1:0
kubeConfigPath: /Users/userA/.kube/config
skipPermissions: false

Inside the app (same prompt):

$ kubectl-ai

  Hey there, what can I help you with today?                                  

>>> What are the namespaces names that are in the cluster?

  Running: kubectl get namespaces

  Running: kubectl get namespaces -o custom-columns=NAME:.metadata.name,LABELS:.metadata.labels

  Running: kubectl describe namespaces

  Running: kubectl get namespaces --show-labels

  Running: kubectl get namespaces -o name


  The following commands require your approval to run:                        
                                                                              
  • kubectl get namespaces -o json | jq '.items[].metadata.name'              
                                                                              
  Do you want to proceed ?                                                    


  1. Yes
  2. Yes, and don't ask me again
  3. No

Enter your choice:

You can see that both configs has the skipPermissions: false definition.
It seems like something bad is going on the app while using Bedrock.

[droot]

Thanks for trying out. Glad that it is now atleast working with bedrock :)

To debug sonnet w/ bedrock, can you invoke the command with --show-tool-output flag, I want to see what was the tool output because I am surprised to see so many toolcalls.

[gilad-tayeb]

Hi @droot,
I've invoked the tool using the flag you suggested, with the same prompt from my previous testings. Here's the output:

$ kubectl-ai --show-tool-output

  Hey there, what can I help you with today?                                  

>>> What namespaces are in this cluster?

  Running: kubectl get namespaces

  NAME                                   STATUS   AGE                         
  cilium-secrets                         Active   11d                         
  default                                Active   2y345d                      
  entitle                                Active   3d16h                       
  gitlab-agent-prod-dr-backend-cluster   Active   2y36d                       
  kong-system                            Active   2y345d                      
  kube-node-lease                        Active   2y345d                      
  kube-public                            Active   2y345d                      
  kube-system                            Active   2y345d                      
  kuma-system                            Active   2y345d                      
  walkme                                 Active   2y345d                      



  The following commands require your approval to run:                        
                                                                              
  • kubectl get namespaces --no-headers | wc -l                               
                                                                              
  Do you want to proceed ?                                                    


  1. Yes
  2. Yes, and don't ask me again
  3. No

Enter your choice:

It seems that the tool understand the command, execute it correctly, but then tries something else (not clear why)

Does it help in some way?

[gilad-tayeb]

@droot Is there any chance that someone is looking at it?
I would love to assist, but i'm not familier with golang unfortunately.

[droot]

@droot Is there any chance that someone is looking at it? I would love to assist, but i'm not familier with golang unfortunately.

I am afraid not. btw, you can use anthropic with their openapi compatible endpoint as well if that is an option. And If I remember correctly, gemini also has generous free tier.

About making progress on debugging, you can examine the kubectl-ai.log file that is created under /tmp or $TMPDIR/ to see if something of interest is logged.