[FEATURE] Write action message to the alert's comment section #1934
Description
Is your feature request related to a problem?
The type of alert varies and they serve different purposes, some of them require specific details in order to trace down the problem when triggered. For example, I have a monitor that checks for login attempt failure from firewall logs. When triggered, I want to know the source.ip. This information can be included in the message body by crafting extraction query and using ctx variables like {{#ctx.newAlerts}}{{bucket_keys}}{{/ctx.newAlerts}} or even {{#toJson}}ctx{{/toJson}} to print all details.
However, this information is then only sent to external systems and channels. When viewing alerts on OpenSearch Dashboard, I only get to know when and how many alerts are triggered, but not the actual details of each alert.
What solution would you like?
A checkbox for each action just like the existing "Enable action throttling" checkbox. When checked, not only the message is sent to the respective channel, but also written into the alert's comment.
A even better approach is to make comments a channel option. This could have been posible using the current custom webhook, if the webhook can read ctx variables and I can call the create comment API using {{ctx.alert.id}}. Unfortunately it can't really read each alert's ctx variable to craft it's webhook URL accordingly. If this is made possible, not only it benefits users who wish to save the message as a comment, but also facilitates sending alert to external system with the alert ID as part of the the URL / path / header / parameter.
What alternatives have you considered?
A simple external program could be developed, then it acts as a webhook destination and sends create comment API request back to OpenSearch with appropriate parameters. But this is not clean, this should be part of OpenSearch's integrated feature instead.
Another alternative is to write the alert details to a dedicated index using webhook. However, this would make OpenSearch's alerting feature seems incomplete, where I can only know when and how many alerts triggered from the Alerting plugin, and need to go somewhere else to see the details while these information are supposed to be directly linked.