Skip to content

Commit ace6018

Browse files
authored
chore: Make Celery Queue Depth page less (#794)
companion to https://github.com/PostHog/charts/pull/773 We've been paging for this daily for more than 10 days. In all of those cases there hasn't been any action and it resolves itself. This creates alert fatigue, so let's make the alert less sensitive. We can additionally add a non-paging alert, if this is deemed useful.
1 parent 4ead4cb commit ace6018

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

charts/posthog/values.yaml

+2-2
Original file line numberDiff line numberDiff line change
@@ -2528,12 +2528,12 @@ prometheus:
25282528
25292529
- alert: CeleryQueueDepth
25302530
expr: (max (posthog_celery_queue_depth)) > 1000
2531-
for: 10m
2531+
for: 60m
25322532
labels:
25332533
rotation: common
25342534
severity: critical
25352535
annotations:
2536-
summary: Celery job execution delayed for more than 10 minutes.
2536+
summary: Celery job execution delayed for more than 60 minutes.
25372537
description: |
25382538
The Celery jobs queue (stored in Redis) is filling up faster than it is consumed. This impacts our
25392539
monitoring, as some paging monitors depend on metrics exported by Celery jobs.

0 commit comments

Comments
 (0)