Fail to create replica for clickhouse shard

[StuDDXT1610]

Hi, i deployed Langfuse using the official Helm guide: Self-Hosting with Kubernetes Helm, the Clickhouse pod shards 0-1 and 0-2 was failed to create and logged

<Information> StatusFile: Status file /bitnami/clickhouse/data/status already exists - unclean restart. Contents:
PID: 42
Started at: 2025-03-25 03:07:57
Revision: 54495
2025.03.25 03:26:53.212463 [ 41 ] {} <Information> Application: Shutting down storages.
2025.03.25 03:26:53.212763 [ 41 ] {} <Information> Application: Waiting for background threads
2025.03.25 03:26:53.212992 [ 41 ] {} <Information> Application: Background threads finished in 0 ms
2025.03.25 03:26:53.213497 [ 41 ] {} <Error> Application: Code: 76. DB::Exception: Cannot lock file /bitnami/clickhouse/data/status. Another server instance in same directory is already running. (CANNOT_OPEN_FILE), Stack trace (when copying this message, always include the lines below):
0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000d74fa3b
...

Steps to Reproduce:

Follow the steps in the Langfuse Helm installation guide.
Deploy using Helm.
Setting Clickhouse env vars:

-CLICKHOUSE_USER="default"
-CLICKHOUSE_DB="default"
-CLICKHOUSE_CLUSTER_ENABLED=true
-CLICKHOUSE_URL=http://langfuse-clickhouse:8123
-CLICKHOUSE_MIGRATION_URL=clickhouse://langfuse-clickhouse:9000

value:
clickhouse:
deploy: true
startupProbe:
  enabled: true
  failureThreshold: 5
  initialDelaySeconds: 20
  periodSeconds: 20
  successThreshold: 1
  timeoutSeconds: 2
shards: 1 # Fixed. Langfuse does not support multi-shard clusters.
replicaCount: 3
resourcesPreset: large
persistence:
  enabled: true
  existingClaim: clickhouse-pvc
  size: 20Gi  
  storageClass: "my-storage"

Results:
clickhouse shard error and langfuse web pod enter CrashLoopBackOff.

Expected Behavior:
All clickhouse shard are running successfully

Thank You !

[dosubot]

Hey @StuDDXT1610! I'm here to help you with any bugs, questions, or contributions you have. Let's tackle this issue together!

Langfuse does not support ClickHouse with multiple shards. To resolve the issue with Clickhouse shard creation, ensure that clickhouse.shards is set to 1 in your configuration ^[1]. Additionally, when using CLICKHOUSE_CLUSTER_ENABLED=true, ensure that the clickhouse.replicaCount is greater than 1, as a single replica setup with clustering enabled can cause the web pod to crash ^[2].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

[StuDDXT1610]

Update

After i changed the clickhouse persistence set up to

persistence:
    enabled: true
    size: 40Gi  
    accessModes:
    - ReadWriteMany
    storageClass: "my-storage"

all 3 shard replica has created sucessfully. However the langfuse web pod still logged this error

error: failed to open database: code: 999, message: Cannot use any of provided ZooKeeper nodes in line 0:
			CREATE TABLE schema_migrations ON CLUSTER default (
				version    Int64,
				dirty      UInt8,
				sequence   UInt64
			) Engine=ReplicatedMergeTree ORDER BY sequence

Do i have to turn the zoopkeepper down or what other solutions ?

Thank you !

[dosubot]

To resolve the error "failed to open database: code: 999, message: Cannot use any of provided ZooKeeper nodes," you should disable ZooKeeper in your configuration. This change will help prevent the ClickHouse and Web pods from restarting multiple times, leading to a faster and more stable deployment ^[1].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

[Steffen911]

Hey @StuDDXT1610 ,
Can you share the full values.yaml overwrite that you use? And can you confirm that you use the latest version of the helm chart (1.1.0)? There, the ClickHouse env vars should not be required anymore.