Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ai-proxy nil error with bedrock misconfiguration #13666

Closed
1 task done
t-yuki opened this issue Sep 13, 2024 · 3 comments · May be fixed by #13676
Closed
1 task done

ai-proxy nil error with bedrock misconfiguration #13666

t-yuki opened this issue Sep 13, 2024 · 3 comments · May be fixed by #13676
Labels
bug pending author feedback Waiting for the issue author to get back to a maintainer with findings, more details, etc... plugins/ai-proxy stale

Comments

@t-yuki
Copy link

t-yuki commented Sep 13, 2024 •
edited
Loading

Is there an existing issue for this?

  • I have searched the existing issues

Kong version ($ kong version)

Kong 3.8.0.0

Current Behavior

nil error happen when aws_region is missing.
[kong] init.lua:427 [ai-proxy] /usr/local/share/lua/5.1/kong/llm/proxy/handler.lua:471: attempt to call field 'set_response_transformer_skipped' (a nil value)

Expected Behavior

Error detail on AWS auth config such as error authenticating with cloud-provider, AWS region not specified anywhere

Steps To Reproduce

  1. in docker environment

  2. with config of a route like plugins:
    - name: ai-proxy
    config:
    route_type: llm/v1/chat
    auth:
    aws_access_key_id: MYKEY
    aws_secret_access_key: MYSECRET
    model:
    provider: bedrock
    name: anthropic.claude-3-haiku-20240307-v1:0
    #options: # NO AWS_REGION CONFIG
    # bedrock: {"aws_region": "ap-northeast-1"}

  3. call the route

Anything else?

I think

kong/kong/llm/proxy/handler.lua

Line 461 in c983384

llm_state.set_response_transformer_skipped()
should use disable_ai_proxy_response_transform
@chobits
Copy link
Contributor

chobits commented Sep 13, 2024 •
edited
Loading

hi @t-yuki thanks for report, it seems a function typo here. I think your advice might be right.

  • not find the function definition after searching the source code:
xc kong $ grep 'set_response_transformer_skipped' -R kong
kong/llm/proxy/handler.lua:    llm_state.set_response_transformer_skipped()

Could you open a pull request for this? We warmly welcome community users to submit PRs to us, and we are also happy to help with reviews.

@chobits chobits added bug pending author feedback Waiting for the issue author to get back to a maintainer with findings, more details, etc... labels Sep 13, 2024
@t-yuki t-yuki mentioned this issue Sep 14, 2024
Open
3 tasks
@github-actions GitHub Actions
Copy link
Contributor

This issue is marked as stale because it has been open for 14 days with no activity.

@github-actions github-actions bot added the stale label Sep 28, 2024
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Oct 6, 2024

Dear contributor,

We are automatically closing this issue because it has not seen any activity for three weeks.
We're sorry that your issue could not be resolved. If any new information comes up that could
help resolving it, please feel free to reopen it.

Your contribution is greatly appreciated!

Please have a look
our pledge to the community
for more information.

Sincerely,
Your Kong Gateway team

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Oct 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug pending author feedback Waiting for the issue author to get back to a maintainer with findings, more details, etc... plugins/ai-proxy stale
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix(llm): fix nil error when identity_interface error t-yuki/kong
2 participants
@chobits @t-yuki