[verifier_otel] Make policy_id, policy_name, package_policy_id multi:true by seanrathier · Pull Request #19965 · elastic/integrations

seanrathier · 2026-07-03T17:58:29Z

Update

Rebased [verifier_otel] Make policy_id, policy_name, package_policy_id multi:true #18786 onto current main to resolve merge conflicts (main had already shipped verifier_otel 0.1.0 for an unrelated change).
Bumps verifier_otel to 0.2.0 and consolidates the changelog entries for this change under that version.
Same functional change as [verifier_otel] Make policy_id, policy_name, package_policy_id multi:true #18786: makes policy_id, policy_name, and package_policy_id multi: true, and requires Kibana ^9.5.0.

Supersedes #18786 due to a fork push permission issue — original author is @Omolola-Akinleye.

Summary

Implements elastic/security-team#17149 (parent epic #16847).

Each verification log emitted by the OTel verifier must carry the integration's package_policy.id so Fleet background task can aggregate verifier logs by package-policy instance. Today, policy_id, policy_name, and package_policy_id are multi: false, which prevents the package manifest from multiple package policy id values when needed. We need to associate multiple permissions by package policy id(integration).

This PR flips those three fields to multi: true in manifest.yml. Per-target distribution itself is delivered by Kibana, which creates one verifier_otel package policy per target integration; so input.yml.hbs and the test fixtures are intentionally unchanged.

Changes

packages/verifier_otel/manifest.yml
- policy_id, policy_name, package_policy_id flipped to multi: true.
- Descriptions updated to note positional alignment with policy_templates.
- Version bumped 0.1.0 → 0.2.0.
- These vars stay multi: false: package_name, package_title, package_version, identity_federation_id, identity_federation_name, verification_id, verification_type, provider, account_type, all credentials_*.
packages/verifier_otel/changelog.yml — 0.2.0 entry added.

Test plan

elastic-package lint — passes.
elastic-package check — package builds (verifier_otel-0.1.0.zip).
elastic-package test static — passes.
elastic-package test policy — to be confirmed in CI.

Make `policy_id`, `policy_name`, and `package_policy_id` `multi: true` in `manifest.yml`, positionally aligned with `policy_templates`, so a single verifier_otel package policy can carry per-target-integration values. Update `input.yml.hbs` to iterate the parallel arrays by index and emit one `policies[]` entry per target integration, each with exactly one `integrations[]` entry. `policy_id` may repeat across entries when target integrations share an agent policy. Bumps version 0.0.1 -> 0.0.2 and updates policy/system test fixtures to the new array-valued shape (the policy fixture now exercises two targets sharing a `policy_id` with distinct `package_policy_id`s). Refs elastic/security-team#17149 Parent epic elastic/security-team#16847 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…hange Per-target `package_policy_id` is delivered by Kibana creating one verifier_otel package policy per target integration, not by carrying parallel arrays in a single package policy. So `input.yml.hbs` and the test fixtures stay unchanged. Only `manifest.yml` flips `policy_id`, `policy_name`, and `package_policy_id` to `multi: true`. Changelog collapsed to a single one-sentence entry. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Net diff vs main is now: three `multi: false` -> `multi: true` flips on `policy_id`, `policy_name`, and `package_policy_id`, plus the 0.0.1 -> 0.0.2 version bump and one-line changelog entry. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

`multi: false` -> `multi: true` is a value-set widening: every input the field previously accepted is still accepted, and Fleet auto-coerces saved scalar values to single-element arrays on upgrade. The template and fixtures are unchanged so rendered output is byte-identical for existing single-target deployments. No user action is required, which is the signature of `enhancement` rather than `breaking-change`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

… var Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

… receiver shape Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…t arrays Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

github-actions · 2026-07-03T17:59:41Z

✅ Elastic Docs Style Checker (Vale)

No issues found on modified lines!

The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale.

github-actions · 2026-07-03T18:26:31Z

TL;DR

The :scroll: Check changelog PR links step failed because the new packages/verifier_otel/changelog.yml entries link to PR 18786, but this build is for PR 19965. The Check integrations verifier_otel step is also marked failed, but the available log artifact only contains teardown/artifact-upload output, so the underlying verifier test error is not present in the prefetched data.

Remediation

In packages/verifier_otel/changelog.yml, change both new 0.2.0 changelog links from https://github.com/elastic/integrations/pull/18786 to https://github.com/elastic/integrations/pull/19965.
Re-run CI after that change; if Check integrations verifier_otel still fails, inspect the full Buildkite job log or the uploaded build/test-results/verifier_otel-policy-*.xml artifact because the provided log excerpt does not include the failing assertion/error.

Investigation details

Root Cause

This is a changelog-link configuration failure in the PR diff. The PR adds two packages/verifier_otel/changelog.yml entries under 0.2.0, and both use link: https://github.com/elastic/integrations/pull/18786. The Buildkite changelog checker expects every modified changelog PR link to point at the current PR, https://github.com/elastic/integrations/pull/19965.

The verifier package step is classified as inconclusive from available data: /tmp/gh-aw/buildkite-logs/integrations-check-integrations-verifier_otel.txt starts at stack teardown and only shows --- [verifier_otel] failed plus artifact upload lines, not the command output that caused elastic-package to return exit status 1.

Evidence

Build: https://buildkite.com/elastic/integrations/builds/45483
Job/step: :scroll: Check changelog PR links
Key log excerpt:

Expected PR link: https://github.com/elastic/integrations/pull/19965
Modified changelog files:
packages/verifier_otel/changelog.yml

ERROR: unexpected link: 'https://github.com/elastic/integrations/pull/18786'
       expected:         'https://github.com/elastic/integrations/pull/19965'
ERROR: unexpected link: 'https://github.com/elastic/integrations/pull/18786'
       expected:         'https://github.com/elastic/integrations/pull/19965'
--- 2 changelog link(s) do not match this PR

Job/step: Check integrations verifier_otel
Available log excerpt:

--- [verifier_otel] failed
🚨 Error: The command exited with status 1
user command error: exit status 1
...
Artifact uploads completed successfully

Verification

Not run locally. The local checkout is at base commit 8e358efc, while the failing build used PR commit e9564870dd4199a5790342bccf44247fbdcdbbe6; the prefetched verifier log does not contain the failure body needed to reproduce a specific assertion.

Follow-up

After fixing the changelog links, if verifier still fails, use the uploaded policy test XML (build/test-results/verifier_otel-policy-*.xml) or the full Buildkite raw job log to identify the exact elastic-package test policy failure.

What is this? | From workflow: PR Buildkite Detective

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

…olicy compiler output Bumping conditions.kibana.version to ^9.5.0 changed which Kibana version CI tests against (9.5.0-SNAPSHOT, resolved as the oldest version satisfying the constraint). That version's OTel agent-policy compiler assigns a component ID to the generic forward connector/pipeline too, shifting our receiver's pipeline from componentid-0 to componentid-1, and adds nil-guards to the transform processor's set() statements. The policies: payload itself is unchanged. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

The original PR (elastic#18786) was superseded by elastic#19965 after a rebase to resolve conflicts with main. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

elastic-vault-github-plugin-prod · 2026-07-03T18:54:12Z

✅ All changelog entries have the correct PR link.

infra-vault-gh-plugin-prod · 2026-07-03T19:10:11Z

💚 Build Succeeded

Buildkite Build
Commit: 9d47a75

History

💔 Build #45483 failed e956487

cc @seanrathier

Omolola-Akinleye and others added 10 commits July 3, 2026 13:54

[verifier_otel] Replace per-policy vars with a single policies yaml…

9e9b5af

… var Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

[verifier_otel] Keep legacy per-policy vars for backward compat

71c3334

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

[verifier_otel] Drop legacy scalar fallback in template

71d9d1d

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

[verifier_otel] Inline policies yaml var directly; producer now sends…

7257a86

… receiver shape Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

[verifier_otel] Switch all per-target vars to parallel multi:true tex…

88f4371

…t arrays Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

[verifier_otel] Bump to 0.1.0; require Kibana ^9.5.0

e956487

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

seanrathier requested a review from a team as a code owner July 3, 2026 17:58

seanrathier self-assigned this Jul 3, 2026

seanrathier added Team:Security-Cloud Services Security Data Experience - Cloud Services team [elastic/cloud-services] Integration:verifier_otel Permission Verifier labels Jul 3, 2026

seanrathier and others added 2 commits July 3, 2026 14:48

[verifier_otel] Point changelog links at PR elastic#19965

9d47a75

The original PR (elastic#18786) was superseded by elastic#19965 after a rebase to resolve conflicts with main. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[verifier_otel] Make policy_id, policy_name, package_policy_id multi:true#19965

[verifier_otel] Make policy_id, policy_name, package_policy_id multi:true#19965
seanrathier wants to merge 12 commits into
elastic:mainfrom
seanrathier:verifier_otel/multi-package-policy-id

seanrathier commented Jul 3, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jul 3, 2026

Uh oh!

github-actions Bot commented Jul 3, 2026

Root Cause

Evidence

Verification

Follow-up

Uh oh!

elastic-vault-github-plugin-prod Bot commented Jul 3, 2026

Uh oh!

infra-vault-gh-plugin-prod Bot commented Jul 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

seanrathier commented Jul 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Update

Summary

Changes

Test plan

Uh oh!

github-actions Bot commented Jul 3, 2026

✅ Elastic Docs Style Checker (Vale)

Uh oh!

github-actions Bot commented Jul 3, 2026

TL;DR

Remediation

Root Cause

Evidence

Verification

Follow-up

Uh oh!

elastic-vault-github-plugin-prod Bot commented Jul 3, 2026

Uh oh!

infra-vault-gh-plugin-prod Bot commented Jul 3, 2026

💚 Build Succeeded

History

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

seanrathier commented Jul 3, 2026 •

edited

Loading