x-pack/winlogbeat/module: fix and enable ingest pipeline tests#51913
x-pack/winlogbeat/module: fix and enable ingest pipeline tests#51913efd6 wants to merge 1 commit into
Conversation
🤖 GitHub commentsJust comment with:
|
b9acf9b to
7763b54
Compare
This comment has been minimized.
This comment has been minimized.
7763b54 to
8a68e39
Compare
🔍 Preview links for changed docs |
✅ Elastic Docs Style Checker (Vale)No issues found on modified lines! The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale. |
TL;DRThe Linux Remediation
Investigation detailsRoot CauseInconclusive from the prefetched log. The job completed 125 tests and recorded four failures, but the available log ends with a serialized ingest-pipeline dump and the aggregate result. It does not include the failing subtest names, expected-versus-actual diffs, or pipeline execution errors needed to trace the failure to a changed source file. Evidence
VerificationNot run locally. These ingest tests require the test Elasticsearch container and the missing failing-case details would still be needed to validate a specific fix. Follow-upPlease provide the full test-collector output or the uploaded JUnit/JSON artifacts before making a source change; diagnosing from the aggregate count would risk attributing the failure to the wrong PR change. What is this? | From workflow: PR Buildkite Detective Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not. |
The module ingest pipeline tests (security, sysmon, powershell) were never run in CI. The test files are tagged //go:build !windows but all CI steps ran on Windows agents, so the tests were structurally excluded. This resulted in several issues accumulating undetected: - Pipeline count assertions were stale (5 pipelines, but security_standard was added as a 6th). - The ECS version in golden files drifted from 1.12.0 to 8.17.0. - The splitSidList Painless function in security_standard.yml used splitOnToken which returns a String[] incompatible with ES 9.x document context, causing event 4908 to fail with illegal_argument_exception. - The powershell.file.script_block_hash field produced by the fingerprint processor was undocumented in fields.yml. Fix the pipeline script to split on \s+ with regex and store results as ArrayList. Add the missing field definition. Update test assertions and regenerate golden files against the current ES. Add a Linux CI step to the x-pack/winlogbeat Buildkite pipeline so these tests run on every build. Assisted-By: Cursor
8a68e39 to
0c8cb76
Compare
|
Pinging @elastic/security-service-integrations (Team:Security-Service Integrations) |
|
Pinging @elastic/sec-windows-platform (Team:Security-Windows Platform) |
andrewkroh
left a comment
There was a problem hiding this comment.
LGTM. ![]()
The PR I mentioned when we were talking is #47395. I don't remember exactly what I was debugging but when I was looking at this so long ago, but I added error checking on the simulate pipeline API response in pipelineIngestedDocument of x-pack/winlogbeat/module/wintest/simulate.go (probably for when the Painless is bad and a pipeline won't even load). Just mentioning it in case it helps debugging failures.
Proposed commit message
Note
Not user-facing, so no changelog entry.
Checklist
stresstest.shscript to run them under stress conditions and race detector to verify their stability../changelog/fragmentsusing the changelog tool.Disruptive User Impact
How to test this PR locally
Related issues
Use cases
Screenshots
Logs