Can't access to Bedrock

[kawabata-mcl]

Issue

I can't access to Bedrock with Dify.
What could be the possible causes?

Steps Taken

1. Cloned this repository
2. Deployed without making any changes
3. Logged in after deployment was completed
4. Installed AWS (ver 0.0.5) Plugin from the Marketplace
5. Selected Oregon region from the model provider settings, chose the model (anthropic.claude-3-5-sonnet-20241022-v2:0) and saved

• Saved successfully
• Confirmed that the request was completed successfully in the Bedrock logs

6. Created a chatbot and selected the model (Claude 3.5 sonnet V2)
7. Executed the chatbot

Results

• The request to the model remains in the Running state indefinitely
• Checked the Bedrock logs, but no logs were generated

What I've Confirmed

• That anthropic.claude-3-5-sonnet-20241022-v2:0 is accessible in the Oregon region
• PluginDaemon container logs
  • The following logs seem to indicate the cause:
    • [ERROR]plugin langgenius/bedrock:0.0.5 is not active for 120 seconds, it may be dead, killing and restarting it
    • [INFO]plugin langgenius/bedrock:0.0.5 stopped
    • [ERROR]plugin langgenius/bedrock:0.0.5 exited with error: signal: killed
  • Is there an issue with the process running the Bedrock plugin?
• As a precaution, doubled the CPU and memory of the API service, but it had no effect

[tmokmss]

Hi, can you try to change this environment variable something greater e.g. 600 and see if the issue is gone? Thanks!

dify-self-hosted-on-aws/lib/constructs/dify-services/api.ts

Line 348 in 2bd2c69

PYTHON_ENV_INIT_TIMEOUT: '120',

[kawabata-mcl]

I changed PYTHON_ENV_INIT_TIMEOUT to 600, but I still have the same issue.

The same errors are still appearing:

• [ERROR]plugin langgenius/bedrock:0.0.5 is not active for 120 seconds, it may be dead, killing and restarting it
• [ERROR]plugin langgenius/bedrock:0.0.5 exited with error: signal: killed
• [INFO]plugin langgenius/bedrock:0.0.5 stopped

I also tried below:

• When I cloned and deployed the final version of v0, I was able to execute the model.
• Even after following the procedures to update from that state to v1, it still didn't work.

[tmokmss]

hmm I found the threshold is hardcoded here

https://github.com/langgenius/dify-plugin-daemon/blob/e0672c3c1a6451437e8f4b63b260c8b0863c9c80/internal/core/plugin_manager/local_runtime/stdio_handle.go#L190-L191

I'm not sure why it happens on your environment but not on mine. Generally, plugins should start up faster if there are more compute resources, so can you try to increase both cpu and ram here? (e.g. cpu:4096, ram: 16384 as an extreme case)

dify-self-hosted-on-aws/lib/constructs/dify-services/api.ts

Lines 53 to 55 in 2bd2c69

	const taskDefinition = new FargateTaskDefinition(this, 'Task', {
	cpu: 1024,
	memoryLimitMiB: 2048, // We got OOM frequently when RAM=512MB

[tmokmss]

Also, do you have any other suspicious log in plugin-daemon? Thanks and sorry for inconvenience 🙏

[kawabata-mcl]

Thanks for the all information.
I will try to deploy on a different environment and see how it goes.

If I have any other questions, I will post the comment again.

[kawabata-mcl]

I deployed to the different environment and still didn't work.
Increasing cpu and ram also didn't work.

But I might found a critical problem causing the issue.
The task is stopped consistently due to health check failure.

Suspicious logs below from Main container could be the reason of health check failure.

[2025-03-07 01:34:02 +0000] [1] [INFO] Starting gunicorn 23.0.0
--
[2025-03-07 01:34:02 +0000] [1] [INFO] Listening at: http://0.0.0.0:5001 (1)
[2025-03-07 01:34:02 +0000] [1] [INFO] Using worker: gevent
[2025-03-07 01:34:02 +0000] [36] [INFO] Booting worker with pid: 36
/app/api/.venv/lib/python3.12/site-packages/opik/evaluation/metrics/heuristics/regex_match.py:8: SyntaxWarning: invalid escape sequence '\d'
"""
[2025-03-07 01:35:51 +0000] [1] [ERROR] Worker (pid:43) exited with code 23
[2025-03-07 01:35:51 +0000] [1] [ERROR] Worker (pid:43) exited with code 23.
...

Plugin daemon records suspicious log as well.

2025/03/07 01:39:03 node.go:213: �[32m[INFO]node cc32e58a-43e4-497a-9cad-0d23a50c9ceb has been removed from the cluster due to being disconnected�[0m

CPU utilization is reaching to 100% sometimes, but looks like it only happens when creating a new task.

Do you have any thoughts for these?
Let me know if you need more information.

[tmokmss]

Thank you for the report. Actually afaik all of them are normal and can be seen in my environment as well. Can you export all of the plugin daemon log records and send here?

[kawabata-mcl]

Thanks for the quick response!!
Here is the logs from plugin daemon.
log-events-viewer-result.csv

[tmokmss]

@kawabata-mcl Thank you for sharing the log!

I'm not sure why this issue happens only in some environments. I still cannot reproduce it in my environment. Maybe can you try to deploy from CloudShell using simple-deploy script? https://github.com/aws-samples/dify-self-hosted-on-aws?tab=readme-ov-file#deploy-from-cloudshell This should totally eliminate environment-specific differences.

Also, I noticed that sometimes there is a time gap (15 minutes in the case below) between when the Bedrock plugin becomes active and when it is killed. Do you know if the plugin is working normally during this time? And do you have any idea what makes the plugin unresponsive? (e.g. did you send some prompt in 2025/03/07 01:52 ?)

1741311361218,2025/03/07 01:36:01 stdio_handle.go:119: �[32m[INFO]plugin langgenius/bedrock:0.0.5: Installed model: bedrock�[0m,log/PluginDaemon/02d9b0ae8d1f43ae896de24aab534f20

1741312376747,"2025/03/07 01:52:56 stdio_handle.go:198: �[33m[WARN]plugin langgenius/bedrock:0.0.5 is not active for %!d(float64=61.951740692) seconds, it may be dead�[0m",log/PluginDaemon/02d9b0ae8d1f43ae896de24aab534f20

Possibly related issue: langgenius/dify-plugin-daemon#66

[lcgash]

有一个特殊的点是，发现这个 plugin 是不允许启动在容器的 1 号进程的，代码里面有对 1 号进程的特殊判断，会导致 plugin 加载反复

[kawabata-mcl]

@tmokmss
Thanks a lot for all of your report!!
I was watching the same issue from dify-plugin-daemon and looks like it is fixed.

I will try to deploy from my local and see how it goes.
If it didn't work, I will deploy from cloudshell.

Do you know if the plugin is working normally during this time? And do you have any idea what makes the plugin unresponsive? (e.g. did you send some prompt in 2025/03/07 01:52 ?)

The plugin may worked only first request after the container was replaced.

@lcgash
Thanks for your information as well!!