[BUG] [make compose up failed] panic: failed to start tenant "cozeloop"

[luxuze]

📋 CheckList

• I have searched existing issues to avoid duplicates
• I am using a recently maintained version of Coze Loop
• I have provided all required information
• I understand this is a bug report and not a feature request
• I have submitted this report in English (otherwise it will not be processed)

🐛 Bug Description

panic: failed to start tenant "cozeloop", cannot start pipelines, the topic=trace_ingestion_event route info not found, it may not exist

goroutine 137 [running]:

github.com/coze-dev/coze-loop/backend/modules/observability/domain/trace/service.(*IngestionServiceImpl).RunAsync.func1()

/coze-loop/src/backend/modules/observability/domain/trace/service/ingestion.go:74 +0x44

created by github.com/coze-dev/coze-loop/backend/modules/observability/domain/trace/service.(*IngestionServiceImpl).RunAsync in goroutine 1

/coze-loop/src/backend/modules/observability/domain/trace/service/ingestion.go:71 +0x7c

🔄 Steps to Reproduce

coze-loop-app | *************************************************************************
coze-loop-app | ****************************** Starting... ******************************
coze-loop-app | *************************************************************************
coze-loop-app | WARNING: dynamicgo only support amd64 && go1.17go1.24, your environment may cause performance degradation
coze-loop-app | INFO 2026-05-08 06:29:03,859 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:29:03,861 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:29:03,863 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:29:03,865 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:29:03,866 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:29:03,866 metric.go:156 107 metrics registered
coze-loop-app | INFO 2026-05-08 06:29:03,866 local_cache_refresh.go:38 Start syncing task cache...
coze-loop-app | INFO 2026-05-08 06:29:03,867 local_cache_refresh.go:46 Retrieved task information, taskCount:0, spaceCount:0, botCount:0
coze-loop-app | INFO 2026-05-08 06:29:03,867 local_cache.go:34 Store task list to cache, info={[] [] [] 2026-05-08 06:29:03.867697754 +0000 UTC m=+0.070639501}
coze-loop-app | INFO 2026-05-08 06:29:03,867 queue_processor.go:31 queue processor config: {default 2000 2000 1000 1000 4}
coze-loop-app | INFO 2026-05-08 06:29:03,867 clickhouse_exporter.go:21 ck exporter starting
coze-loop-app | INFO 2026-05-08 06:29:03,867 queue_processor.go:62 queue processor start
coze-loop-app | INFO 2026-05-08 06:29:03,867 rmq_receiver.go:29 rmq receiver starting
coze-loop-app | ERROR 2026-05-08 06:29:03,869 rmq_receiver.go:46 rmq receiver consumer start err: the topic=trace_ingestion_event route info not found, it may not exist
coze-loop-app | INFO 2026-05-08 06:29:03,869 rmq_receiver.go:56 rmq receiver shutting down
coze-loop-app | INFO 2026-05-08 06:29:03,869 queue_processor.go:71 queue processor shutting down
coze-loop-app | INFO 2026-05-08 06:29:03,869 queue_processor.go:156 shard 3 receive shutdown signal
coze-loop-app | INFO 2026-05-08 06:29:03,869 queue_processor.go:207 shutdown batch 3
coze-loop-app | INFO 2026-05-08 06:29:03,869 queue_processor.go:156 shard 0 receive shutdown signal
coze-loop-app | INFO 2026-05-08 06:29:03,869 queue_processor.go:207 shutdown batch 0
coze-loop-app | INFO 2026-05-08 06:29:03,869 queue_processor.go:156 shard 1 receive shutdown signal
coze-loop-app | INFO 2026-05-08 06:29:03,869 queue_processor.go:207 shutdown batch 1
coze-loop-app | INFO 2026-05-08 06:29:03,869 queue_processor.go:156 shard 2 receive shutdown signal
coze-loop-app | INFO 2026-05-08 06:29:03,869 queue_processor.go:207 shutdown batch 2
coze-loop-app | INFO 2026-05-08 06:29:03,869 queue_processor.go:78 queue processor shutted down
coze-loop-app | INFO 2026-05-08 06:29:03,869 clickhouse_exporter.go:26 ck exporter shutting down
coze-loop-app | panic: failed to start tenant "cozeloop", cannot start pipelines, the topic=trace_ingestion_event route info not found, it may not exist
coze-loop-app |
coze-loop-app | goroutine 156 [running]:
coze-loop-app | github.com/coze-dev/coze-loop/backend/modules/observability/domain/trace/service.(*IngestionServiceImpl).RunAsync.func1()
coze-loop-app | /coze-loop/src/backend/modules/observability/domain/trace/service/ingestion.go:74 +0x44
coze-loop-app | created by github.com/coze-dev/coze-loop/backend/modules/observability/domain/trace/service.(*IngestionServiceImpl).RunAsync in goroutine 1
coze-loop-app | /coze-loop/src/backend/modules/observability/domain/trace/service/ingestion.go:71 +0x7c
coze-loop-app | *************************************************************************
coze-loop-app | ****************************** Starting... ******************************
coze-loop-app | *************************************************************************
coze-loop-app | WARNING: dynamicgo only support amd64 && go1.17go1.24, your environment may cause performance degradation
coze-loop-app | INFO 2026-05-08 06:30:04,055 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:30:04,060 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:30:04,061 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:30:04,063 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:30:04,064 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:30:04,064 metric.go:156 107 metrics registered
coze-loop-app | INFO 2026-05-08 06:30:04,064 local_cache_refresh.go:38 Start syncing task cache...
coze-loop-app | INFO 2026-05-08 06:30:04,066 local_cache_refresh.go:46 Retrieved task information, taskCount:0, spaceCount:0, botCount:0
coze-loop-app | INFO 2026-05-08 06:30:04,066 local_cache.go:34 Store task list to cache, info={[] [] [] 2026-05-08 06:30:04.066025754 +0000 UTC m=+0.075108501}
coze-loop-app | INFO 2026-05-08 06:30:04,066 queue_processor.go:31 queue processor config: {default 2000 2000 1000 1000 4}
coze-loop-app | INFO 2026-05-08 06:30:04,066 clickhouse_exporter.go:21 ck exporter starting
coze-loop-app | INFO 2026-05-08 06:30:04,066 queue_processor.go:62 queue processor start
coze-loop-app | INFO 2026-05-08 06:30:04,066 rmq_receiver.go:29 rmq receiver starting
coze-loop-app | ERROR 2026-05-08 06:30:04,068 rmq_receiver.go:46 rmq receiver consumer start err: the topic=trace_ingestion_event route info not found, it may not exist
coze-loop-app | INFO 2026-05-08 06:30:04,068 rmq_receiver.go:56 rmq receiver shutting down
coze-loop-app | INFO 2026-05-08 06:30:04,068 queue_processor.go:71 queue processor shutting down
coze-loop-app | INFO 2026-05-08 06:30:04,068 queue_processor.go:156 shard 3 receive shutdown signal
coze-loop-app | INFO 2026-05-08 06:30:04,068 queue_processor.go:207 shutdown batch 3
coze-loop-app | INFO 2026-05-08 06:30:04,068 queue_processor.go:156 shard 0 receive shutdown signal
coze-loop-app | INFO 2026-05-08 06:30:04,068 queue_processor.go:207 shutdown batch 0
coze-loop-app | INFO 2026-05-08 06:30:04,068 queue_processor.go:156 shard 1 receive shutdown signal
coze-loop-app | INFO 2026-05-08 06:30:04,068 queue_processor.go:207 shutdown batch 1
coze-loop-app | INFO 2026-05-08 06:30:04,068 queue_processor.go:156 shard 2 receive shutdown signal
coze-loop-app | INFO 2026-05-08 06:30:04,068 queue_processor.go:207 shutdown batch 2
coze-loop-app | INFO 2026-05-08 06:30:04,068 queue_processor.go:78 queue processor shutted down
coze-loop-app | INFO 2026-05-08 06:30:04,068 clickhouse_exporter.go:26 ck exporter shutting down
coze-loop-app | panic: failed to start tenant "cozeloop", cannot start pipelines, the topic=trace_ingestion_event route info not found, it may not exist
coze-loop-app |
coze-loop-app | goroutine 92 [running]:
coze-loop-app | github.com/coze-dev/coze-loop/backend/modules/observability/domain/trace/service.(*IngestionServiceImpl).RunAsync.func1()
coze-loop-app | /coze-loop/src/backend/modules/observability/domain/trace/service/ingestion.go:74 +0x44
coze-loop-app | created by github.com/coze-dev/coze-loop/backend/modules/observability/domain/trace/service.(*IngestionServiceImpl).RunAsync in goroutine 1
coze-loop-app | /coze-loop/src/backend/modules/observability/domain/trace/service/ingestion.go:71 +0x7c
coze-loop-app | *************************************************************************
coze-loop-app | ****************************** Starting... ******************************
coze-loop-app | *************************************************************************
coze-loop-app | WARNING: dynamicgo only support amd64 && go1.17go1.24, your environment may cause performance degradation
coze-loop-app | INFO 2026-05-08 06:31:04,234 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:31:04,239 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:31:04,239 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:31:04,242 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:31:04,243 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:31:04,243 metric.go:156 107 metrics registered
coze-loop-app | INFO 2026-05-08 06:31:04,244 local_cache_refresh.go:38 Start syncing task cache...
coze-loop-app | INFO 2026-05-08 06:31:04,244 local_cache_refresh.go:46 Retrieved task information, taskCount:0, spaceCount:0, botCount:0
coze-loop-app | INFO 2026-05-08 06:31:04,244 local_cache.go:34 Store task list to cache, info={[] [] [] 2026-05-08 06:31:04.244924379 +0000 UTC m=+0.074230751}
coze-loop-app | INFO 2026-05-08 06:31:04,245 queue_processor.go:31 queue processor config: {default 2000 2000 1000 1000 4}
coze-loop-app | INFO 2026-05-08 06:31:04,245 clickhouse_exporter.go:21 ck exporter starting
coze-loop-app | INFO 2026-05-08 06:31:04,245 queue_processor.go:62 queue processor start
coze-loop-app | INFO 2026-05-08 06:31:04,245 rmq_receiver.go:29 rmq receiver starting
coze-loop-app | ERROR 2026-05-08 06:31:04,246 rmq_receiver.go:46 rmq receiver consumer start err: the topic=trace_ingestion_event route info not found, it may not exist
coze-loop-app | INFO 2026-05-08 06:31:04,246 rmq_receiver.go:56 rmq receiver shutting down
coze-loop-app | INFO 2026-05-08 06:31:04,246 queue_processor.go:71 queue processor shutting down
coze-loop-app | INFO 2026-05-08 06:31:04,246 queue_processor.go:156 shard 3 receive shutdown signal
coze-loop-app | INFO 2026-05-08 06:31:04,246 queue_processor.go:207 shutdown batch 3
coze-loop-app | INFO 2026-05-08 06:31:04,246 queue_processor.go:156 shard 0 receive shutdown signal
coze-loop-app | INFO 2026-05-08 06:31:04,246 queue_processor.go:207 shutdown batch 0
coze-loop-app | INFO 2026-05-08 06:31:04,246 queue_processor.go:156 shard 1 receive shutdown signal
coze-loop-app | INFO 2026-05-08 06:31:04,246 queue_processor.go:207 shutdown batch 1
coze-loop-app | INFO 2026-05-08 06:31:04,246 queue_processor.go:156 shard 2 receive shutdown signal
coze-loop-app | INFO 2026-05-08 06:31:04,246 queue_processor.go:207 shutdown batch 2
coze-loop-app | INFO 2026-05-08 06:31:04,246 queue_processor.go:78 queue processor shutted down
coze-loop-app | INFO 2026-05-08 06:31:04,246 clickhouse_exporter.go:26 ck exporter shutting down
coze-loop-app | panic: failed to start tenant "cozeloop", cannot start pipelines, the topic=trace_ingestion_event route info not found, it may not exist
coze-loop-app |
coze-loop-app | goroutine 133 [running]:
coze-loop-app | github.com/coze-dev/coze-loop/backend/modules/observability/domain/trace/service.(*IngestionServiceImpl).RunAsync.func1()
coze-loop-app | /coze-loop/src/backend/modules/observability/domain/trace/service/ingestion.go:74 +0x44
coze-loop-app | created by github.com/coze-dev/coze-loop/backend/modules/observability/domain/trace/service.(*IngestionServiceImpl).RunAsync in goroutine 1
coze-loop-app | /coze-loop/src/backend/modules/observability/domain/trace/service/ingestion.go:71 +0x7c
coze-loop-app | *************************************************************************
coze-loop-app | ****************************** Starting... ******************************
coze-loop-app | *************************************************************************
coze-loop-app | WARNING: dynamicgo only support amd64 && go1.17go1.24, your environment may cause performance degradation
coze-loop-app | INFO 2026-05-08 06:32:05,232 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:32:05,248 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:32:05,252 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:32:05,258 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:32:05,260 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:32:05,261 metric.go:156 107 metrics registered
coze-loop-app | INFO 2026-05-08 06:32:05,261 local_cache_refresh.go:38 Start syncing task cache...
coze-loop-app | INFO 2026-05-08 06:32:05,267 local_cache_refresh.go:46 Retrieved task information, taskCount:0, spaceCount:0, botCount:0
coze-loop-app | INFO 2026-05-08 06:32:05,267 local_cache.go:34 Store task list to cache, info={[] [] [] 2026-05-08 06:32:05.267303963 +0000 UTC m=+0.338376793}
coze-loop-app | INFO 2026-05-08 06:32:05,268 queue_processor.go:31 queue processor config: {default 2000 2000 1000 1000 4}
coze-loop-app | INFO 2026-05-08 06:32:05,269 clickhouse_exporter.go:21 ck exporter starting
coze-loop-app | INFO 2026-05-08 06:32:05,269 queue_processor.go:62 queue processor start
coze-loop-app | INFO 2026-05-08 06:32:05,269 rmq_receiver.go:29 rmq receiver starting
coze-loop-app | ERROR 2026-05-08 06:32:05,275 rmq_receiver.go:46 rmq receiver consumer start err: the topic=trace_ingestion_event route info not found, it may not exist
coze-loop-app | INFO 2026-05-08 06:32:05,275 rmq_receiver.go:56 rmq receiver shutting down
coze-loop-app | INFO 2026-05-08 06:32:05,275 queue_processor.go:71 queue processor shutting down
coze-loop-app | INFO 2026-05-08 06:32:05,275 queue_processor.go:156 shard 3 receive shutdown signal
coze-loop-app | INFO 2026-05-08 06:32:05,275 queue_processor.go:207 shutdown batch 3
coze-loop-app | INFO 2026-05-08 06:32:05,275 queue_processor.go:156 shard 0 receive shutdown signal
coze-loop-app | INFO 2026-05-08 06:32:05,275 queue_processor.go:207 shutdown batch 0
coze-loop-app | INFO 2026-05-08 06:32:05,275 queue_processor.go:156 shard 1 receive shutdown signal
coze-loop-app | INFO 2026-05-08 06:32:05,275 queue_processor.go:207 shutdown batch 1
coze-loop-app | INFO 2026-05-08 06:32:05,275 queue_processor.go:156 shard 2 receive shutdown signal
coze-loop-app | INFO 2026-05-08 06:32:05,275 queue_processor.go:207 shutdown batch 2
coze-loop-app | INFO 2026-05-08 06:32:05,275 queue_processor.go:78 queue processor shutted down
coze-loop-app | INFO 2026-05-08 06:32:05,275 clickhouse_exporter.go:26 ck exporter shutting down
coze-loop-app | panic: failed to start tenant "cozeloop", cannot start pipelines, the topic=trace_ingestion_event route info not found, it may not exist
coze-loop-app |
coze-loop-app | goroutine 153 [running]:
coze-loop-app | github.com/coze-dev/coze-loop/backend/modules/observability/domain/trace/service.(*IngestionServiceImpl).RunAsync.func1()
coze-loop-app | /coze-loop/src/backend/modules/observability/domain/trace/service/ingestion.go:74 +0x44
coze-loop-app | created by github.com/coze-dev/coze-loop/backend/modules/observability/domain/trace/service.(*IngestionServiceImpl).RunAsync in goroutine 1
coze-loop-app | /coze-loop/src/backend/modules/observability/domain/trace/service/ingestion.go:71 +0x7c
coze-loop-app | *************************************************************************
coze-loop-app | ****************************** Starting... ******************************
coze-loop-app | *************************************************************************
coze-loop-app | WARNING: dynamicgo only support amd64 && go1.17go1.24, your environment may cause performance degradation
coze-loop-app | INFO 2026-05-08 06:33:05,514 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:33:05,520 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:33:05,521 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:33:05,524 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:33:05,526 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:33:05,526 metric.go:156 107 metrics registered
coze-loop-app | INFO 2026-05-08 06:33:05,526 local_cache_refresh.go:38 Start syncing task cache...
coze-loop-app | INFO 2026-05-08 06:33:05,528 local_cache_refresh.go:46 Retrieved task information, taskCount:0, spaceCount:0, botCount:0
coze-loop-app | INFO 2026-05-08 06:33:05,528 local_cache.go:34 Store task list to cache, info={[] [] [] 2026-05-08 06:33:05.528104088 +0000 UTC m=+0.102572959}
coze-loop-app | INFO 2026-05-08 06:33:05,528 queue_processor.go:31 queue processor config: {default 2000 2000 1000 1000 4}
coze-loop-app | INFO 2026-05-08 06:33:05,528 clickhouse_exporter.go:21 ck exporter starting
coze-loop-app | INFO 2026-05-08 06:33:05,528 queue_processor.go:62 queue processor start
coze-loop-app | INFO 2026-05-08 06:33:05,529 rmq_receiver.go:29 rmq receiver starting
coze-loop-app | ERROR 2026-05-08 06:33:05,530 rmq_receiver.go:46 rmq receiver consumer start err: the topic=trace_ingestion_event route info not found, it may not exist
coze-loop-app | INFO 2026-05-08 06:33:05,530 rmq_receiver.go:56 rmq receiver shutting down
coze-loop-app | INFO 2026-05-08 06:33:05,530 queue_processor.go:71 queue processor shutting down
coze-loop-app | INFO 2026-05-08 06:33:05,530 queue_processor.go:156 shard 3 receive shutdown signal
coze-loop-app | INFO 2026-05-08 06:33:05,530 queue_processor.go:207 shutdown batch 3
coze-loop-app | INFO 2026-05-08 06:33:05,530 queue_processor.go:156 shard 0 receive shutdown signal
coze-loop-app | INFO 2026-05-08 06:33:05,530 queue_processor.go:207 shutdown batch 0
coze-loop-app | INFO 2026-05-08 06:33:05,530 queue_processor.go:156 shard 1 receive shutdown signal
coze-loop-app | INFO 2026-05-08 06:33:05,530 queue_processor.go:207 shutdown batch 1
coze-loop-app | INFO 2026-05-08 06:33:05,530 queue_processor.go:156 shard 2 receive shutdown signal
coze-loop-app | INFO 2026-05-08 06:33:05,530 queue_processor.go:207 shutdown batch 2
coze-loop-app | INFO 2026-05-08 06:33:05,530 queue_processor.go:78 queue processor shutted down
coze-loop-app | INFO 2026-05-08 06:33:05,530 clickhouse_exporter.go:26 ck exporter shutting down
coze-loop-app | panic: failed to start tenant "cozeloop", cannot start pipelines, the topic=trace_ingestion_event route info not found, it may not exist
coze-loop-app |
coze-loop-app | goroutine 139 [running]:
coze-loop-app | github.com/coze-dev/coze-loop/backend/modules/observability/domain/trace/service.(*IngestionServiceImpl).RunAsync.func1()
coze-loop-app | /coze-loop/src/backend/modules/observability/domain/trace/service/ingestion.go:74 +0x44
coze-loop-app | created by github.com/coze-dev/coze-loop/backend/modules/observability/domain/trace/service.(*IngestionServiceImpl).RunAsync in goroutine 1
coze-loop-app | /coze-loop/src/backend/modules/observability/domain/trace/service/ingestion.go:71 +0x7c
coze-loop-app | *************************************************************************
coze-loop-app | ****************************** Starting... ******************************
coze-loop-app | *************************************************************************
coze-loop-app | WARNING: dynamicgo only support amd64 && go1.17go1.24, your environment may cause performance degradation
coze-loop-app | INFO 2026-05-08 06:34:05,689 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:34:05,696 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:34:05,698 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:34:05,700 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:34:05,701 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:34:05,701 metric.go:156 107 metrics registered
coze-loop-app | INFO 2026-05-08 06:34:05,701 local_cache_refresh.go:38 Start syncing task cache...
coze-loop-app | INFO 2026-05-08 06:34:05,703 local_cache_refresh.go:46 Retrieved task information, taskCount:0, spaceCount:0, botCount:0
coze-loop-app | INFO 2026-05-08 06:34:05,703 local_cache.go:34 Store task list to cache, info={[] [] [] 2026-05-08 06:34:05.703535172 +0000 UTC m=+0.079121626}
coze-loop-app | INFO 2026-05-08 06:34:05,703 queue_processor.go:31 queue processor config: {default 2000 2000 1000 1000 4}
coze-loop-app | INFO 2026-05-08 06:34:05,703 clickhouse_exporter.go:21 ck exporter starting
coze-loop-app | INFO 2026-05-08 06:34:05,703 queue_processor.go:62 queue processor start
coze-loop-app | INFO 2026-05-08 06:34:05,703 rmq_receiver.go:29 rmq receiver starting
coze-loop-app | ERROR 2026-05-08 06:34:05,705 rmq_receiver.go:46 rmq receiver consumer start err: the topic=trace_ingestion_event route info not found, it may not exist
coze-loop-app | INFO 2026-05-08 06:34:05,705 rmq_receiver.go:56 rmq receiver shutting down
coze-loop-app | INFO 2026-05-08 06:34:05,705 queue_processor.go:71 queue processor shutting down
coze-loop-app | INFO 2026-05-08 06:34:05,705 queue_processor.go:156 shard 3 receive shutdown signal
coze-loop-app | INFO 2026-05-08 06:34:05,705 queue_processor.go:207 shutdown batch 3
coze-loop-app | INFO 2026-05-08 06:34:05,705 queue_processor.go:156 shard 0 receive shutdown signal
coze-loop-app | INFO 2026-05-08 06:34:05,705 queue_processor.go:207 shutdown batch 0
coze-loop-app | INFO 2026-05-08 06:34:05,705 queue_processor.go:156 shard 1 receive shutdown signal
coze-loop-app | INFO 2026-05-08 06:34:05,705 queue_processor.go:207 shutdown batch 1
coze-loop-app | INFO 2026-05-08 06:34:05,705 queue_processor.go:156 shard 2 receive shutdown signal
coze-loop-app | INFO 2026-05-08 06:34:05,705 queue_processor.go:207 shutdown batch 2
coze-loop-app | INFO 2026-05-08 06:34:05,705 queue_processor.go:78 queue processor shutted down
coze-loop-app | INFO 2026-05-08 06:34:05,705 clickhouse_exporter.go:26 ck exporter shutting down
coze-loop-app | panic: failed to start tenant "cozeloop", cannot start pipelines, the topic=trace_ingestion_event route info not found, it may not exist
coze-loop-app |
coze-loop-app | goroutine 145 [running]:
coze-loop-app | github.com/coze-dev/coze-loop/backend/modules/observability/domain/trace/service.(*IngestionServiceImpl).RunAsync.func1()
coze-loop-app | /coze-loop/src/backend/modules/observability/domain/trace/service/ingestion.go:74 +0x44
coze-loop-app | created by github.com/coze-dev/coze-loop/backend/modules/observability/domain/trace/service.(*IngestionServiceImpl).RunAsync in goroutine 1
coze-loop-app | /coze-loop/src/backend/modules/observability/domain/trace/service/ingestion.go:71 +0x7c
coze-loop-app | *************************************************************************
coze-loop-app | ****************************** Starting... ******************************
coze-loop-app | *************************************************************************
coze-loop-app | WARNING: dynamicgo only support amd64 && go1.17go1.24, your environment may cause performance degradation
coze-loop-app | INFO 2026-05-08 06:35:05,917 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:35:05,922 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:35:05,923 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:35:05,925 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:35:05,927 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:35:05,927 metric.go:156 107 metrics registered
coze-loop-app | INFO 2026-05-08 06:35:05,927 local_cache_refresh.go:38 Start syncing task cache...
coze-loop-app | INFO 2026-05-08 06:35:05,928 local_cache_refresh.go:46 Retrieved task information, taskCount:0, spaceCount:0, botCount:0
coze-loop-app | INFO 2026-05-08 06:35:05,928 local_cache.go:34 Store task list to cache, info={[] [] [] 2026-05-08 06:35:05.92836413 +0000 UTC m=+0.075470668}
coze-loop-app | INFO 2026-05-08 06:35:05,928 queue_processor.go:31 queue processor config: {default 2000 2000 1000 1000 4}
coze-loop-app | INFO 2026-05-08 06:35:05,928 clickhouse_exporter.go:21 ck exporter starting
coze-loop-app | INFO 2026-05-08 06:35:05,928 queue_processor.go:62 queue processor start
coze-loop-app | INFO 2026-05-08 06:35:05,928 rmq_receiver.go:29 rmq receiver starting
coze-loop-app | ERROR 2026-05-08 06:35:05,931 rmq_receiver.go:46 rmq receiver consumer start err: the topic=trace_ingestion_event route info not found, it may not exist
coze-loop-app | INFO 2026-05-08 06:35:05,931 rmq_receiver.go:56 rmq receiver shutting down
coze-loop-app | INFO 2026-05-08 06:35:05,931 queue_processor.go:71 queue processor shutting down
coze-loop-app | INFO 2026-05-08 06:35:05,931 queue_processor.go:156 shard 3 receive shutdown signal
coze-loop-app | INFO 2026-05-08 06:35:05,931 queue_processor.go:207 shutdown batch 3
coze-loop-app | INFO 2026-05-08 06:35:05,931 queue_processor.go:156 shard 0 receive shutdown signal
coze-loop-app | INFO 2026-05-08 06:35:05,931 queue_processor.go:207 shutdown batch 0
coze-loop-app | INFO 2026-05-08 06:35:05,931 queue_processor.go:156 shard 1 receive shutdown signal
coze-loop-app | INFO 2026-05-08 06:35:05,931 queue_processor.go:207 shutdown batch 1
coze-loop-app | INFO 2026-05-08 06:35:05,931 queue_processor.go:156 shard 2 receive shutdown signal
coze-loop-app | INFO 2026-05-08 06:35:05,931 queue_processor.go:207 shutdown batch 2
coze-loop-app | INFO 2026-05-08 06:35:05,931 queue_processor.go:78 queue processor shutted down
coze-loop-app | INFO 2026-05-08 06:35:05,931 clickhouse_exporter.go:26 ck exporter shutting down
coze-loop-app | panic: failed to start tenant "cozeloop", cannot start pipelines, the topic=trace_ingestion_event route info not found, it may not exist
coze-loop-app |
coze-loop-app | goroutine 146 [running]:
coze-loop-app | github.com/coze-dev/coze-loop/backend/modules/observability/domain/trace/service.(*IngestionServiceImpl).RunAsync.func1()
coze-loop-app | /coze-loop/src/backend/modules/observability/domain/trace/service/ingestion.go:74 +0x44
coze-loop-app | created by github.com/coze-dev/coze-loop/backend/modules/observability/domain/trace/service.(*IngestionServiceImpl).RunAsync in goroutine 1
coze-loop-app | /coze-loop/src/backend/modules/observability/domain/trace/service/ingestion.go:71 +0x7c
coze-loop-app | *************************************************************************
coze-loop-app | ****************************** Starting... ******************************
coze-loop-app | *************************************************************************
coze-loop-app | WARNING: dynamicgo only support amd64 && go1.17go1.24, your environment may cause performance degradation
coze-loop-app | INFO 2026-05-08 06:36:06,135 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:36:06,140 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:36:06,140 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:36:06,143 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:36:06,144 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:36:06,144 metric.go:156 107 metrics registered
coze-loop-app | INFO 2026-05-08 06:36:06,144 local_cache_refresh.go:38 Start syncing task cache...
coze-loop-app | INFO 2026-05-08 06:36:06,145 local_cache_refresh.go:46 Retrieved task information, taskCount:0, spaceCount:0, botCount:0
coze-loop-app | INFO 2026-05-08 06:36:06,145 local_cache.go:34 Store task list to cache, info={[] [] [] 2026-05-08 06:36:06.145545797 +0000 UTC m=+0.073833543}
coze-loop-app | INFO 2026-05-08 06:36:06,145 queue_processor.go:31 queue processor config: {default 2000 2000 1000 1000 4}
coze-loop-app | INFO 2026-05-08 06:36:06,145 clickhouse_exporter.go:21 ck exporter starting
coze-loop-app | INFO 2026-05-08 06:36:06,145 queue_processor.go:62 queue processor start
coze-loop-app | INFO 2026-05-08 06:36:06,145 rmq_receiver.go:29 rmq receiver starting
coze-loop-app | ERROR 2026-05-08 06:36:06,147 rmq_receiver.go:46 rmq receiver consumer start err: the topic=trace_ingestion_event route info not found, it may not exist
coze-loop-app | INFO 2026-05-08 06:36:06,147 rmq_receiver.go:56 rmq receiver shutting down
coze-loop-app | INFO 2026-05-08 06:36:06,147 queue_processor.go:71 queue processor shutting down
coze-loop-app | INFO 2026-05-08 06:36:06,147 queue_processor.go:156 shard 3 receive shutdown signal
coze-loop-app | INFO 2026-05-08 06:36:06,147 queue_processor.go:207 shutdown batch 3
coze-loop-app | INFO 2026-05-08 06:36:06,147 queue_processor.go:156 shard 0 receive shutdown signal
coze-loop-app | INFO 2026-05-08 06:36:06,147 queue_processor.go:207 shutdown batch 0
coze-loop-app | INFO 2026-05-08 06:36:06,147 queue_processor.go:156 shard 1 receive shutdown signal
coze-loop-app | INFO 2026-05-08 06:36:06,147 queue_processor.go:207 shutdown batch 1
coze-loop-app | INFO 2026-05-08 06:36:06,147 queue_processor.go:156 shard 2 receive shutdown signal
coze-loop-app | INFO 2026-05-08 06:36:06,147 queue_processor.go:207 shutdown batch 2
coze-loop-app | INFO 2026-05-08 06:36:06,147 queue_processor.go:78 queue processor shutted down
coze-loop-app | INFO 2026-05-08 06:36:06,147 clickhouse_exporter.go:26 ck exporter shutting down
coze-loop-app | panic: failed to start tenant "cozeloop", cannot start pipelines, the topic=trace_ingestion_event route info not found, it may not exist
coze-loop-app |
coze-loop-app | goroutine 131 [running]:
coze-loop-app | github.com/coze-dev/coze-loop/backend/modules/observability/domain/trace/service.(*IngestionServiceImpl).RunAsync.func1()
coze-loop-app | /coze-loop/src/backend/modules/observability/domain/trace/service/ingestion.go:74 +0x44
coze-loop-app | created by github.com/coze-dev/coze-loop/backend/modules/observability/domain/trace/service.(*IngestionServiceImpl).RunAsync in goroutine 1
coze-loop-app | /coze-loop/src/backend/modules/observability/domain/trace/service/ingestion.go:71 +0x7c
coze-loop-app | *************************************************************************
coze-loop-app | ****************************** Starting... ******************************
coze-loop-app | *************************************************************************
coze-loop-app | WARNING: dynamicgo only support amd64 && go1.17go1.24, your environment may cause performance degradation
coze-loop-app | INFO 2026-05-08 06:37:06,340 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:37:06,347 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:37:06,348 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:37:06,351 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:37:06,352 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:37:06,352 metric.go:156 107 metrics registered
coze-loop-app | INFO 2026-05-08 06:37:06,352 local_cache_refresh.go:38 Start syncing task cache...
coze-loop-app | INFO 2026-05-08 06:37:06,353 local_cache_refresh.go:46 Retrieved task information, taskCount:0, spaceCount:0, botCount:0
coze-loop-app | INFO 2026-05-08 06:37:06,353 local_cache.go:34 Store task list to cache, info={[] [] [] 2026-05-08 06:37:06.353256005 +0000 UTC m=+0.076822751}
coze-loop-app | INFO 2026-05-08 06:37:06,353 queue_processor.go:31 queue processor config: {default 2000 2000 1000 1000 4}
coze-loop-app | INFO 2026-05-08 06:37:06,353 clickhouse_exporter.go:21 ck exporter starting
coze-loop-app | INFO 2026-05-08 06:37:06,353 queue_processor.go:62 queue processor start
coze-loop-app | INFO 2026-05-08 06:37:06,353 rmq_receiver.go:29 rmq receiver starting
coze-loop-app | ERROR 2026-05-08 06:37:06,354 rmq_receiver.go:46 rmq receiver consumer start err: the topic=trace_ingestion_event route info not found, it may not exist
coze-loop-app | INFO 2026-05-08 06:37:06,354 rmq_receiver.go:56 rmq receiver shutting down
coze-loop-app | INFO 2026-05-08 06:37:06,354 queue_processor.go:71 queue processor shutting down
coze-loop-app | INFO 2026-05-08 06:37:06,354 queue_processor.go:156 shard 3 receive shutdown signal
coze-loop-app | INFO 2026-05-08 06:37:06,354 queue_processor.go:207 shutdown batch 3
coze-loop-app | INFO 2026-05-08 06:37:06,354 queue_processor.go:156 shard 0 receive shutdown signal
coze-loop-app | INFO 2026-05-08 06:37:06,354 queue_processor.go:207 shutdown batch 0
coze-loop-app | INFO 2026-05-08 06:37:06,354 queue_processor.go:156 shard 1 receive shutdown signal
coze-loop-app | INFO 2026-05-08 06:37:06,354 queue_processor.go:207 shutdown batch 1
coze-loop-app | INFO 2026-05-08 06:37:06,354 queue_processor.go:156 shard 2 receive shutdown signal
coze-loop-app | INFO 2026-05-08 06:37:06,354 queue_processor.go:207 shutdown batch 2
coze-loop-app | INFO 2026-05-08 06:37:06,355 queue_processor.go:78 queue processor shutted down
coze-loop-app | INFO 2026-05-08 06:37:06,355 clickhouse_exporter.go:26 ck exporter shutting down
coze-loop-app | panic: failed to start tenant "cozeloop", cannot start pipelines, the topic=trace_ingestion_event route info not found, it may not exist
coze-loop-app |
coze-loop-app | goroutine 146 [running]:
coze-loop-app | github.com/coze-dev/coze-loop/backend/modules/observability/domain/trace/service.(*IngestionServiceImpl).RunAsync.func1()
coze-loop-app | /coze-loop/src/backend/modules/observability/domain/trace/service/ingestion.go:74 +0x44
coze-loop-app | created by github.com/coze-dev/coze-loop/backend/modules/observability/domain/trace/service.(*IngestionServiceImpl).RunAsync in goroutine 1
coze-loop-app | /coze-loop/src/backend/modules/observability/domain/trace/service/ingestion.go:71 +0x7c
coze-loop-app | *************************************************************************
coze-loop-app | ****************************** Starting... ******************************
coze-loop-app | *************************************************************************
coze-loop-app | WARNING: dynamicgo only support amd64 && go1.17go1.24, your environment may cause performance degradation
coze-loop-app | INFO 2026-05-08 06:38:06,526 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:38:06,532 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:38:06,533 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:38:06,536 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:38:06,537 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:38:06,537 metric.go:156 107 metrics registered
coze-loop-app | INFO 2026-05-08 06:38:06,537 local_cache_refresh.go:38 Start syncing task cache...
coze-loop-app | INFO 2026-05-08 06:38:06,538 local_cache_refresh.go:46 Retrieved task information, taskCount:0, spaceCount:0, botCount:0
coze-loop-app | INFO 2026-05-08 06:38:06,538 local_cache.go:34 Store task list to cache, info={[] [] [] 2026-05-08 06:38:06.538043339 +0000 UTC m=+0.072830501}
coze-loop-app | INFO 2026-05-08 06:38:06,538 queue_processor.go:31 queue processor config: {default 2000 2000 1000 1000 4}
coze-loop-app | INFO 2026-05-08 06:38:06,538 clickhouse_exporter.go:21 ck exporter starting
coze-loop-app | INFO 2026-05-08 06:38:06,538 queue_processor.go:62 queue processor start
coze-loop-app | INFO 2026-05-08 06:38:06,538 rmq_receiver.go:29 rmq receiver starting
coze-loop-app | ERROR 2026-05-08 06:38:06,539 rmq_receiver.go:46 rmq receiver consumer start err: the topic=trace_ingestion_event route info not found, it may not exist
coze-loop-app | INFO 2026-05-08 06:38:06,540 rmq_receiver.go:56 rmq receiver shutting down
coze-loop-app | INFO 2026-05-08 06:38:06,540 queue_processor.go:71 queue processor shutting down
coze-loop-app | INFO 2026-05-08 06:38:06,540 queue_processor.go:156 shard 3 receive shutdown signal
coze-loop-app | INFO 2026-05-08 06:38:06,540 queue_processor.go:207 shutdown batch 3
coze-loop-app | INFO 2026-05-08 06:38:06,540 queue_processor.go:156 shard 0 receive shutdown signal
coze-loop-app | INFO 2026-05-08 06:38:06,540 queue_processor.go:207 shutdown batch 0
coze-loop-app | INFO 2026-05-08 06:38:06,540 queue_processor.go:156 shard 1 receive shutdown signal
coze-loop-app | INFO 2026-05-08 06:38:06,540 queue_processor.go:207 shutdown batch 1
coze-loop-app | INFO 2026-05-08 06:38:06,540 queue_processor.go:156 shard 2 receive shutdown signal
coze-loop-app | INFO 2026-05-08 06:38:06,540 queue_processor.go:207 shutdown batch 2
coze-loop-app | INFO 2026-05-08 06:38:06,540 queue_processor.go:78 queue processor shutted down
coze-loop-app | INFO 2026-05-08 06:38:06,540 clickhouse_exporter.go:26 ck exporter shutting down
coze-loop-app | panic: failed to start tenant "cozeloop", cannot start pipelines, the topic=trace_ingestion_event route info not found, it may not exist
coze-loop-app |
coze-loop-app | goroutine 140 [running]:
coze-loop-app | github.com/coze-dev/coze-loop/backend/modules/observability/domain/trace/service.(*IngestionServiceImpl).RunAsync.func1()
coze-loop-app | /coze-loop/src/backend/modules/observability/domain/trace/service/ingestion.go:74 +0x44
coze-loop-app | created by github.com/coze-dev/coze-loop/backend/modules/observability/domain/trace/service.(*IngestionServiceImpl).RunAsync in goroutine 1
coze-loop-app | /coze-loop/src/backend/modules/observability/domain/trace/service/ingestion.go:71 +0x7c
coze-loop-app | *************************************************************************
coze-loop-app | ****************************** Starting... ******************************
coze-loop-app | *************************************************************************
coze-loop-app | WARNING: dynamicgo only support amd64 && go1.17~go1.24, your environment may cause performance degradation
coze-loop-app | INFO 2026-05-08 06:39:06,734 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:39:06,739 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:39:06,740 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:39:06,742 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:39:06,744 trace.go:221 default trace ingest tenant is cozeloop
coze-loop-app | INFO 2026-05-08 06:39:06,744 metric.go:156 107 metrics registered
coze-loop-app | INFO 2026-05-08 06:39:06,744 local_cache_refresh.go:38 Start syncing task cache...
coze-loop-app | INFO 2026-05-08 06:39:06,745 local_cache_refresh.go:46 Retrieved task information, taskCount:0, spaceCount:0, botCount:0
coze-loop-app | INFO 2026-05-08 06:39:06,745 local_cache.go:34 Store task list to cache, info={[] [] [] 2026-05-08 06:39:06.745104005 +0000 UTC m=+0.070742626}
coze-loop-app | INFO 2026-05-08 06:39:06,745 queue_processor.go:31 queue processor config: {default 2000 2000 1000 1000 4}
coze-loop-app | INFO 2026-05-08 06:39:06,745 clickhouse_exporter.go:21 ck exporter starting
coze-loop-app | INFO 2026-05-08 06:39:06,745 queue_processor.go:62 queue processor start
coze-loop-app | INFO 2026-05-08 06:39:06,745 rmq_receiver.go:29 rmq receiver starting
coze-loop-app | ERROR 2026-05-08 06:39:06,746 rmq_receiver.go:46 rmq receiver consumer start err: the topic=trace_ingestion_event route info not found, it may not exist
coze-loop-app | INFO 2026-05-08 06:39:06,746 rmq_receiver.go:56 rmq receiver shutting down
coze-loop-app | INFO 2026-05-08 06:39:06,746 queue_processor.go:71 queue processor shutting down
coze-loop-app | INFO 2026-05-08 06:39:06,746 queue_processor.go:156 shard 3 receive shutdown signal
coze-loop-app | INFO 2026-05-08 06:39:06,746 queue_processor.go:207 shutdown batch 3
coze-loop-app | INFO 2026-05-08 06:39:06,746 queue_processor.go:156 shard 0 receive shutdown signal
coze-loop-app | INFO 2026-05-08 06:39:06,746 queue_processor.go:207 shutdown batch 0
coze-loop-app | INFO 2026-05-08 06:39:06,746 queue_processor.go:156 shard 1 receive shutdown signal
coze-loop-app | INFO 2026-05-08 06:39:06,746 queue_processor.go:207 shutdown batch 1
coze-loop-app | INFO 2026-05-08 06:39:06,746 queue_processor.go:156 shard 2 receive shutdown signal
coze-loop-app | INFO 2026-05-08 06:39:06,746 queue_processor.go:207 shutdown batch 2
coze-loop-app | INFO 2026-05-08 06:39:06,746 queue_processor.go:78 queue processor shutted down
coze-loop-app | INFO 2026-05-08 06:39:06,746 clickhouse_exporter.go:26 ck exporter shutting down
coze-loop-app | panic: failed to start tenant "cozeloop", cannot start pipelines, the topic=trace_ingestion_event route info not found, it may not exist
coze-loop-app |
coze-loop-app | goroutine 137 [running]:
coze-loop-app | github.com/coze-dev/coze-loop/backend/modules/observability/domain/trace/service.(*IngestionServiceImpl).RunAsync.func1()
coze-loop-app | /coze-loop/src/backend/modules/observability/domain/trace/service/ingestion.go:74 +0x44
coze-loop-app | created by github.com/coze-dev/coze-loop/backend/modules/observability/domain/trace/service.(*IngestionServiceImpl).RunAsync in goroutine 1
coze-loop-app | /coze-loop/src/backend/modules/observability/domain/trace/service/ingestion.go:71 +0x7c

✅ Expected Behavior

panic: failed to start tenant "cozeloop", cannot start pipelines, the topic=trace_ingestion_event route info not found, it may not exist

goroutine 137 [running]:

github.com/coze-dev/coze-loop/backend/modules/observability/domain/trace/service.(*IngestionServiceImpl).RunAsync.func1()

/coze-loop/src/backend/modules/observability/domain/trace/service/ingestion.go:74 +0x44

created by github.com/coze-dev/coze-loop/backend/modules/observability/domain/trace/service.(*IngestionServiceImpl).RunAsync in goroutine 1

/coze-loop/src/backend/modules/observability/domain/trace/service/ingestion.go:71 +0x7c

❌ Actual Behavior

panic: failed to start tenant "cozeloop"

🚨 Severity

None

🔧 Component

None

💻 Environment

No response

🔧 Go Environment

No response

📋 Logs

📝 Additional Context

No response