Skip to content

[BUG] Notifcations are sent multiple times if there are multiple API Deployments #241

@zitrusgelb

Description

@zitrusgelb

Describe the bug
If Peekaping is deployed with multiple instances of the API notifications are sent by each instance of the API

Database

  • postgresql

To Reproduce
Steps to reproduce the behavior:

  1. Setup a Notification Channel
  2. Have multiple Instances of the API
  3. Force a downtime or uptime (create a new monitor), needs a notification channel selected
  4. See multiple notifications about the event, this is always equivalent to the count of the API instances

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
API Instance 1

{"level":"info","ts":1765296683.557561,"logger":"[monitor-event-listener]","caller":"monitor/listener.go:49","msg":"Monitor status changed event received for monitor: dc6b3bbd-2adc-4aa1-b303-87ca2285eaaf, new status: 0","svc":"peekaping:api"}
{"level":"info","ts":1765296683.5571902,"caller":"notification_channel/listener.go:89","msg":"Notification event received for monitor: dc6b3bbd-2adc-4aa1-b303-87ca2285eaaf","svc":"peekaping:api"}
{"level":"info","ts":1765296683.597729,"caller":"notification_channel/listener.go:100","msg":"Monitor notification: 63463c04-2892-4026-a612-0d10586bf64d","svc":"peekaping:api"}
{"level":"info","ts":1765296683.6793463,"caller":"providers/ntfy.go:163","msg":"Sending NTFY notification to https://ntfy.example.com with topic server-alerts","svc":"peekaping:api"}
{"level":"info","ts":1765296683.7353666,"caller":"providers/ntfy.go:181","msg":"NTFY notification sent successfully to https://ntfy.example.com","svc":"peekaping:api"}
{"level":"info","ts":1765296683.7354386,"caller":"notification_channel/listener.go:141","msg":"Notification sent to: NTFY for monitor: dc6b3bbd-2adc-4aa1-b303-87ca2285eaaf","svc":"peekaping:api"}
{"level":"info","ts":1765296683.8284128,"logger":"[monitor-event-listener]","caller":"monitor/listener.go:80","msg":"Successfully updated monitor dc6b3bbd-2adc-4aa1-b303-87ca2285eaaf status from 1 to 0","svc":"peekaping:api"}

API Instance 2

{"level":"info","ts":1765296683.5576427,"logger":"[monitor-event-listener]","caller":"monitor/listener.go:49","msg":"Monitor status changed event received for monitor: dc6b3bbd-2adc-4aa1-b303-87ca2285eaaf, new status: 0","svc":"peekaping:api"}
{"level":"info","ts":1765296683.557448,"caller":"notification_channel/listener.go:89","msg":"Notification event received for monitor: dc6b3bbd-2adc-4aa1-b303-87ca2285eaaf","svc":"peekaping:api"}
{"level":"info","ts":1765296683.61741,"caller":"notification_channel/listener.go:100","msg":"Monitor notification: 63463c04-2892-4026-a612-0d10586bf64d","svc":"peekaping:api"}
{"level":"info","ts":1765296683.772907,"caller":"providers/ntfy.go:163","msg":"Sending NTFY notification to https://ntfy.example.com with topic server-alerts","svc":"peekaping:api"}
{"level":"info","ts":1765296683.813862,"logger":"[monitor-event-listener]","caller":"monitor/listener.go:80","msg":"Successfully updated monitor dc6b3bbd-2adc-4aa1-b303-87ca2285eaaf status from 1 to 0","svc":"peekaping:api"}
{"level":"info","ts":1765296683.8185625,"caller":"providers/ntfy.go:181","msg":"NTFY notification sent successfully to https://ntfy.example.com","svc":"peekaping:api"}
{"level":"info","ts":1765296683.818652,"caller":"notification_channel/listener.go:141","msg":"Notification sent to: NTFY for monitor: dc6b3bbd-2adc-4aa1-b303-87ca2285eaaf","svc":"peekaping:api"}

API Instance 3

{"level":"info","ts":1765296683.558118,"logger":"[monitor-event-listener]","caller":"monitor/listener.go:49","msg":"Monitor status changed event received for monitor: dc6b3bbd-2adc-4aa1-b303-87ca2285eaaf, new status: 0","svc":"peekaping:api"}
{"level":"info","ts":1765296683.5587647,"caller":"notification_channel/listener.go:89","msg":"Notification event received for monitor: dc6b3bbd-2adc-4aa1-b303-87ca2285eaaf","svc":"peekaping:api"}
{"level":"info","ts":1765296683.5905206,"caller":"notification_channel/listener.go:100","msg":"Monitor notification: 63463c04-2892-4026-a612-0d10586bf64d","svc":"peekaping:api"}
{"level":"info","ts":1765296683.6671205,"caller":"providers/ntfy.go:163","msg":"Sending NTFY notification to https://ntfy.example.com with topic server-alerts","svc":"peekaping:api"}
{"level":"info","ts":1765296683.7791312,"logger":"[monitor-event-listener]","caller":"monitor/listener.go:80","msg":"Successfully updated monitor dc6b3bbd-2adc-4aa1-b303-87ca2285eaaf status from 1 to 0","svc":"peekaping:api"}
{"level":"info","ts":1765296683.8315582,"caller":"providers/ntfy.go:181","msg":"NTFY notification sent successfully to https://ntfy.example.com","svc":"peekaping:api"}
{"level":"info","ts":1765296683.8316314,"caller":"notification_channel/listener.go:141","msg":"Notification sent to: NTFY for monitor: dc6b3bbd-2adc-4aa1-b303-87ca2285eaaf","svc":"peekaping:api"}

Additional context
Doing a test notification does not help, as there it seems as the API Instance handling the Web request immediately sends the notification not going through the same notification event trigger as status changes

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions