Document the expected policy rule result format and result_helper functions in the authoring guide

[st3penta]

Problem

The Policy Authoring guide currently shows examples where policy rules produce results as simple objects with only a msg field (e.g. result := {"msg": "..."}). However, the actual result format consumed by the Conforma evaluation engine is richer and includes several fields that are not documented anywhere for external policy authors.

What is missing

The authoring docs should describe the result object structure that policy rules are expected to produce:

• code (string): A unique identifier for the rule violation, formatted as <package_path>.<short_name>. Used for skipping rules and in violation reports.
• msg (string): A human-readable message describing the violation, typically formatted via sprintf using custom.failure_msg from annotations.
• effective_on (string): An RFC3339 timestamp controlling when the rule becomes a hard failure vs. a warning.
• collections (list of strings, optional): Rule collections this violation belongs to, derived from custom.collections annotations.
• term (any, optional): An additional term for grouping or identifying the specific subject of the violation.
• severity (string, optional): Override severity level for the violation (e.g. "warning").

The docs should also document the helper functions available in data.lib.metadata (and their deprecated data.lib aliases) that produce correctly formatted results:

• result_helper(chain, failure_sprintf_params) - produces a result with code, msg, effective_on, and optionally collections
• result_helper_with_term(chain, failure_sprintf_params, term) - adds a term field
• result_helper_with_severity(chain, failure_sprintf_params, severity) - adds a severity field

These helpers use rego.metadata.chain() to automatically derive code and effective_on from annotations, which is the recommended pattern for all policy rules in this repository.

Why it matters

External users authoring custom policies need to understand:

1. What fields the result object must/can contain
2. How those fields interact with the evaluation engine (e.g. effective_on for grace periods, collections for grouping)
3. That helper functions exist to produce correctly formatted results
4. The relationship between rule annotations (custom.short_name, custom.failure_msg, custom.effective_on, custom.collections) and the generated result fields

Suggested approach

Update the authoring guide to:

1. Add a dedicated "Result format" section describing the result object schema
2. Document data.lib.metadata.result_helper and its variants with usage examples
3. Update the existing code examples to use result_helper instead of raw {"msg": "..."} objects
4. Explain how annotations map to result fields

[fullsend-ai-triage]

🤖 Finished Triage · ✅ Success · Started 10:18 AM UTC · Completed 10:21 AM UTC
Commit: 47d3320 · View workflow run →

[fullsend-ai-triage]

Triage Summary

Category: Documentation · Severity: Medium

Problem

The authoring guide shows policy rule results as simple {"msg": "..."} objects, but the actual result format includes additional fields (code, effective_on, collections, term, severity). The result_helper functions in data.lib.metadata are also undocumented for external policy authors.

Recommended Approach

1. Add a "Result format" section to the authoring guide documenting all result object fields with required/optional markers
2. Document result_helper, result_helper_with_term, and result_helper_with_severity with usage examples
3. Update existing code examples to use result_helper instead of raw objects
4. Explain how annotations (custom.short_name, custom.failure_msg, custom.effective_on, custom.collections) map to result fields
5. Run make generate-docs and commit any regenerated files

Related Issues

• Document how to use policy/lib functions in custom policies #1747 (Document how to use policy/lib functions in custom policies) — overlaps on documenting lib functions but has broader scope. Consider cross-referencing.
• Document how to use time-based data (effective_on) in policy rule data #1749 (Document time-based data / effective_on) — complementary; the result format docs should link to this for deeper effective_on coverage.

Proposed Test

- Verify the authoring guide renders correctly via Antora
- Verify all code examples are syntactically valid Rego
- Confirm documented result_helper signatures match policy/lib/metadata/metadata.rego
- Run `make ci` to ensure no conventions or formatting checks fail

---

Labels: Adding 'enhancement' label since repo conventions pair it with 'documentation' for doc improvement requests.

[fullsend-ai-coder]

🤖 Finished Code · ✅ Success · Started 10:23 AM UTC · Completed 10:28 AM UTC
Commit: 47d3320 · View workflow run →