[9.4](backport #51772) filebeat kafka input: add group_instance_id for static group membership

[mergify]

Proposed commit message

filebeat kafka input: add group_instance_id for static group membership

The Kafka input now has a `group_instance_id` option wiring it to Consumer.Group.InstanceId (KIP-345): a member that rejoins within 
`session_timeout` keeps its partitions instead of rebalancing.
Static membership needs Kafka `version` >= 2.3.0, so the input
fails fast with a clear error when it is set on the default 2.1.0.
Unset by default; existing configs are unaffected.

Assisted-By: Claude Code

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works.
• I have added an entry in ./changelog/fragments using the changelog tool.

Disruptive User Impact

None by default — the option is opt-in and unset out of the box; existing
configurations are unchanged. When enabled, operators should be aware:

• version must be >= 2.3.0; otherwise the input fails fast with a clear
  error.
• Each consumer instance sharing a group_id must use a unique
  group_instance_id.
• With static membership, a crashed/restarted member's partitions are not
  reassigned until session_timeout expires — that is the intended trade-off;
  tune session_timeout accordingly.

How to test this PR locally

Unit tests:

cd filebeat
go test -race -run TestNewSaramaConfig ./input/kafka/

Friendly version gate (no broker needed) — configure the input with
group_instance_id set but version left at the default and confirm startup
fails with group_instance_id requires 'version' >= 2.3.0 ....

End-to-end (optional, needs a broker >= 2.3.0): run two Filebeat instances that
share a group_id, each with a distinct group_instance_id and version: "2.3.0";
restart one instance and confirm the group does not rebalance within
session_timeout (no partition reassignment in the broker logs).

Related issues

• Closes [filebeat][kafka input] Expose Sarama static group membership (group_instance_id, KIP-345) #51768

Use cases

Multi-replica, containerized Filebeat consuming a high-throughput Kafka topic.
Setting a stable per-instance group_instance_id lets rolling restarts complete
without cascading rebalances, keeping consumption steady.

---

This is an automatic backport of pull request #51772 done by Mergify.

[github-actions]

🤖 GitHub comments

Just comment with:

• run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)
• /test : Run the Buildkite pipeline.

[infra-vault-gh-plugin-prod]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[github-actions]

🔍 Preview links for changed docs

• docs/reference/filebeat/filebeat-input-kafka.md

[github-actions]

✅ Elastic Docs Style Checker (Vale)

No issues found on modified lines!

---

The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale.