Skip to content

Add notifications field to alert rule management - #96

Merged
ckuo-eq merged 3 commits into
mainfrom
ckuo-eq-patch-2
Aug 27, 2026
Merged

Add notifications field to alert rule management#96
ckuo-eq merged 3 commits into
mainfrom
ckuo-eq-patch-2

Conversation

@ckuo-eq

@ckuo-eq ckuo-eq commented Aug 26, 2026

Copy link
Copy Markdown
Collaborator

Checklist

  • I have read the CONTRIBUTING.md
  • I certify that my MD file is onboarded to the OnEvent.json or OnSchedule.json (not README or catalog.json)
  • I certify that my preview and released lists for OnEvent.json/OnSchedule.json are correct
  • I certify that all OnEvent/OnSchedule in released are properly tested and ready for production

@github-actions

github-actions Bot commented Aug 26, 2026

Copy link
Copy Markdown

🧪 Template Spec Check (static — not execution-tested)

Grades the template text only: structure + an LLM judge on spec quality. No agent was run and no tools were executed — a pass does not mean the agent works.

✅ All templates passed  ·  1/1 passed  ·  threshold: 0.6

Template Result Judge Min Issues
alert-rule-management.md ✅ PASS 0.70

⚠️ Warnings (non-blocking)

  • alert-rule-management.md: Tools not seen in any other template (may be new or mistyped): get_stream, create_stream_alert_rule
  • alert-rule-management.md: Judge advisory (completeness=0.70, below ideal 0.80): Step 4's 5-second wait is hardcoded with no polling or verification that attachment succeeded before proceeding; step 6 specifies no email content format, required fields, or handling of alert rule creation failures; no timeout or escalation path is defined if any step fails.

Judge Scores

alert-rule-management.md

Dimension Score Comment
clarity 0.85 Steps 1–6 are specific and tool-mapped, but step 6's instruction to use 'the outcome of the alert rule creation tool as the email body' is vague about what constitutes a valid email body and lacks guidance on handling malformed or oversized responses.
nonContradiction 0.90 Guidelines and instructions are largely aligned; however, the guideline 'Log all actions and decisions' contradicts the absence of a declared logging tool, forcing the agent to either ignore this mandate or improvise.
scope 0.95 The agent has a clear termination condition (email notification sent after alert rule creation), though no explicit success/failure criteria are defined for the overall workflow.
completeness 0.70 ⚠️ Step 4's 5-second wait is hardcoded with no polling or verification that attachment succeeded before proceeding; step 6 specifies no email content format, required fields, or handling of alert rule creation failures; no timeout or escalation path is defined if any step fails.
instructionToolAlignment 0.80 All six instruction steps map to declared tools (search_connections, get_stream, create_stream, attach_stream_asset, wait, create_stream_alert_rule, send_email_notification), but the declared tool 'get_stream' is used only conditionally and the guideline to 'log all actions' has no corresponding tool.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this tool was renamed to get_stream, can we update it?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes

@ckuo-eq
ckuo-eq merged commit 5bcccbf into main Aug 27, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants