Skip to content

Commit b9acf9b

Browse files
efd6efd6
committed
x-pack/winlogbeat/module: fix and enable ingest pipeline tests
The module ingest pipeline tests (security, sysmon, powershell) were never run in CI. The test files are tagged //go:build !windows but all CI steps ran on Windows agents, so the tests were structurally excluded. This resulted in several issues accumulating undetected: - Pipeline count assertions were stale (5 pipelines, but security_standard was added as a 6th). - The ECS version in golden files drifted from 1.12.0 to 8.17.0. - The splitSidList Painless function in security_standard.yml used splitOnToken which returns a String[] incompatible with ES 9.x document context, causing event 4908 to fail with illegal_argument_exception. - The powershell.file.script_block_hash field produced by the fingerprint processor was undocumented in fields.yml. Fix the pipeline script to split on \s+ with regex and store results as ArrayList. Add the missing field definition. Update test assertions and regenerate golden files against the current ES. Add a Linux CI step to the x-pack/winlogbeat Buildkite pipeline so these tests run on every build. Assisted-By: Cursor
1 parent d884560 commit b9acf9b

120 files changed

Lines changed: 796 additions & 669 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.buildkite/x-pack/pipeline.xpack.winlogbeat.yml‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -80,6 +80,36 @@ steps:
8080
key: "x-pack-winlogbeat-mandatory-tests"
8181

8282
steps:
83+
- label: ":linux: x-pack/winlogbeat: Module Ingest Tests"
84+
key: "mandatory-module-ingest-tests"
85+
command: |
86+
set -eo pipefail
87+
cd x-pack/winlogbeat
88+
mage build goUnitTest
89+
retry:
90+
automatic:
91+
- limit: 1
92+
plugins:
93+
- *dockerhub_login_plugin
94+
- *docker_elastic_login_plugin
95+
- test-collector#v1.10.2:
96+
files: "x-pack/winlogbeat/build/TEST-*.xml"
97+
format: "junit"
98+
branches: "main"
99+
debug: true
100+
agents:
101+
provider: "gcp"
102+
image: "${IMAGE_UBUNTU_X86_64}"
103+
machineType: "${GCP_HI_PERF_MACHINE_TYPE}"
104+
disk_size: 100
105+
disk_type: "pd-ssd"
106+
artifact_paths:
107+
- "x-pack/winlogbeat/build/*.xml"
108+
- "x-pack/winlogbeat/build/*.json"
109+
notify:
110+
- github_commit_status:
111+
context: "x-pack/winlogbeat: Module Ingest Tests"
112+
83113
- label: ":windows: x-pack/winlogbeat Win 2019 Unit Tests"
84114
key: "mandatory-win-2019-unit-tests"
85115
command: |

‎x-pack/winlogbeat/module/powershell/_meta/fields.yml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -120,6 +120,10 @@
120120
type: group
121121
description: Data related to the executed script file.
122122
fields:
123+
- name: script_block_hash
124+
type: keyword
125+
description: SHA-256 hash of the executed script block.
126+
123127
- name: script_block_id
124128
type: keyword
125129
description: Id of the executed script block.

‎x-pack/winlogbeat/module/powershell/fields.go‎

Lines changed: 1 addition & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎x-pack/winlogbeat/module/powershell/test/testdata/ingest/400.golden.json‎

Lines changed: 32 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -2,18 +2,22 @@
22
{
33
"@timestamp": "2020-05-14T07:00:30.8914235Z",
44
"ecs": {
5-
"version": "1.12.0"
5+
"version": "8.17.0"
66
},
77
"event": {
88
"action": "Engine Lifecycle",
9-
"category": "process",
9+
"category": [
10+
"process"
11+
],
1012
"code": "400",
11-
"ingested": "2025-01-15T10:02:22.041691914Z",
13+
"ingested": "2026-07-14T03:51:16.442888256Z",
1214
"kind": "event",
1315
"module": "powershell",
1416
"provider": "PowerShell",
1517
"sequence": 13,
16-
"type": "start"
18+
"type": [
19+
"start"
20+
]
1721
},
1822
"host": {
1923
"name": "vagrant",
@@ -62,18 +66,22 @@
6266
{
6367
"@timestamp": "2020-05-14T07:01:14.3715076Z",
6468
"ecs": {
65-
"version": "1.12.0"
69+
"version": "8.17.0"
6670
},
6771
"event": {
6872
"action": "Engine Lifecycle",
69-
"category": "process",
73+
"category": [
74+
"process"
75+
],
7076
"code": "400",
71-
"ingested": "2025-01-15T10:02:22.042381068Z",
77+
"ingested": "2026-07-14T03:51:16.443386273Z",
7278
"kind": "event",
7379
"module": "powershell",
7480
"provider": "PowerShell",
7581
"sequence": 13,
76-
"type": "start"
82+
"type": [
83+
"start"
84+
]
7785
},
7886
"host": {
7987
"name": "vagrant",
@@ -124,18 +132,22 @@
124132
{
125133
"@timestamp": "2020-05-14T11:32:51.9892568Z",
126134
"ecs": {
127-
"version": "1.12.0"
135+
"version": "8.17.0"
128136
},
129137
"event": {
130138
"action": "Engine Lifecycle",
131-
"category": "process",
139+
"category": [
140+
"process"
141+
],
132142
"code": "400",
133-
"ingested": "2025-01-15T10:02:22.042395560Z",
143+
"ingested": "2026-07-14T03:51:16.443397663Z",
134144
"kind": "event",
135145
"module": "powershell",
136146
"provider": "PowerShell",
137147
"sequence": 13,
138-
"type": "start"
148+
"type": [
149+
"start"
150+
]
139151
},
140152
"host": {
141153
"name": "vagrant",
@@ -184,18 +196,22 @@
184196
{
185197
"@timestamp": "2020-06-04T07:20:27.7472275Z",
186198
"ecs": {
187-
"version": "1.12.0"
199+
"version": "8.17.0"
188200
},
189201
"event": {
190202
"action": "Engine Lifecycle",
191-
"category": "process",
203+
"category": [
204+
"process"
205+
],
192206
"code": "400",
193-
"ingested": "2025-01-15T10:02:22.042402302Z",
207+
"ingested": "2026-07-14T03:51:16.443403844Z",
194208
"kind": "event",
195209
"module": "powershell",
196210
"provider": "PowerShell",
197211
"sequence": 9,
198-
"type": "start"
212+
"type": [
213+
"start"
214+
]
199215
},
200216
"host": {
201217
"name": "vagrant",

‎x-pack/winlogbeat/module/powershell/test/testdata/ingest/403.golden.json‎

Lines changed: 32 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -2,18 +2,22 @@
22
{
33
"@timestamp": "2020-05-14T15:31:22.4269238Z",
44
"ecs": {
5-
"version": "1.12.0"
5+
"version": "8.17.0"
66
},
77
"event": {
88
"action": "Engine Lifecycle",
9-
"category": "process",
9+
"category": [
10+
"process"
11+
],
1012
"code": "403",
11-
"ingested": "2025-01-15T10:02:22.075700098Z",
13+
"ingested": "2026-07-14T03:51:16.481393912Z",
1214
"kind": "event",
1315
"module": "powershell",
1416
"provider": "PowerShell",
1517
"sequence": 33,
16-
"type": "end"
18+
"type": [
19+
"end"
20+
]
1721
},
1822
"host": {
1923
"name": "vagrant",
@@ -61,18 +65,22 @@
6165
{
6266
"@timestamp": "2020-05-15T08:11:47.932007Z",
6367
"ecs": {
64-
"version": "1.12.0"
68+
"version": "8.17.0"
6569
},
6670
"event": {
6771
"action": "Engine Lifecycle",
68-
"category": "process",
72+
"category": [
73+
"process"
74+
],
6975
"code": "403",
70-
"ingested": "2025-01-15T10:02:22.075721610Z",
76+
"ingested": "2026-07-14T03:51:16.481415877Z",
7177
"kind": "event",
7278
"module": "powershell",
7379
"provider": "PowerShell",
7480
"sequence": 37,
75-
"type": "end"
81+
"type": [
82+
"end"
83+
]
7684
},
7785
"host": {
7886
"name": "vagrant",
@@ -121,18 +129,22 @@
121129
{
122130
"@timestamp": "2020-05-15T08:28:53.6266982Z",
123131
"ecs": {
124-
"version": "1.12.0"
132+
"version": "8.17.0"
125133
},
126134
"event": {
127135
"action": "Engine Lifecycle",
128-
"category": "process",
136+
"category": [
137+
"process"
138+
],
129139
"code": "403",
130-
"ingested": "2025-01-15T10:02:22.075728323Z",
140+
"ingested": "2026-07-14T03:51:16.481422399Z",
131141
"kind": "event",
132142
"module": "powershell",
133143
"provider": "PowerShell",
134144
"sequence": 37,
135-
"type": "end"
145+
"type": [
146+
"end"
147+
]
136148
},
137149
"host": {
138150
"name": "vagrant",
@@ -188,18 +200,22 @@
188200
{
189201
"@timestamp": "2020-06-04T07:20:28.6861939Z",
190202
"ecs": {
191-
"version": "1.12.0"
203+
"version": "8.17.0"
192204
},
193205
"event": {
194206
"action": "Engine Lifecycle",
195-
"category": "process",
207+
"category": [
208+
"process"
209+
],
196210
"code": "403",
197-
"ingested": "2025-01-15T10:02:22.075733622Z",
211+
"ingested": "2026-07-14T03:51:16.481427357Z",
198212
"kind": "event",
199213
"module": "powershell",
200214
"provider": "PowerShell",
201215
"sequence": 10,
202-
"type": "end"
216+
"type": [
217+
"end"
218+
]
203219
},
204220
"host": {
205221
"name": "vagrant",

‎x-pack/winlogbeat/module/powershell/test/testdata/ingest/4103.golden.json‎

Lines changed: 16 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -8,18 +8,22 @@
88
}
99
},
1010
"ecs": {
11-
"version": "1.12.0"
11+
"version": "8.17.0"
1212
},
1313
"event": {
1414
"action": "Executing Pipeline",
15-
"category": "process",
15+
"category": [
16+
"process"
17+
],
1618
"code": "4103",
17-
"ingested": "2025-01-15T10:02:22.083664757Z",
19+
"ingested": "2026-07-14T03:51:16.489794412Z",
1820
"kind": "event",
1921
"module": "powershell",
2022
"provider": "Microsoft-Windows-PowerShell",
2123
"sequence": 34,
22-
"type": "info"
24+
"type": [
25+
"info"
26+
]
2327
},
2428
"host": {
2529
"name": "vagrant",
@@ -117,18 +121,22 @@
117121
{
118122
"@timestamp": "2020-05-15T08:13:06.7032939Z",
119123
"ecs": {
120-
"version": "1.12.0"
124+
"version": "8.17.0"
121125
},
122126
"event": {
123127
"action": "Executing Pipeline",
124-
"category": "process",
128+
"category": [
129+
"process"
130+
],
125131
"code": "4103",
126-
"ingested": "2025-01-15T10:02:22.083688734Z",
132+
"ingested": "2026-07-14T03:51:16.489826472Z",
127133
"kind": "event",
128134
"module": "powershell",
129135
"provider": "Microsoft-Windows-PowerShell",
130136
"sequence": 22,
131-
"type": "info"
137+
"type": [
138+
"info"
139+
]
132140
},
133141
"host": {
134142
"name": "vagrant",

‎x-pack/winlogbeat/module/powershell/test/testdata/ingest/4104.golden.json‎

Lines changed: 17 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -2,17 +2,21 @@
22
{
33
"@timestamp": "2020-05-14T11:33:51.3892662Z",
44
"ecs": {
5-
"version": "1.12.0"
5+
"version": "8.17.0"
66
},
77
"event": {
88
"action": "Execute a Remote Command",
9-
"category": "process",
9+
"category": [
10+
"process"
11+
],
1012
"code": "4104",
11-
"ingested": "2025-01-15T10:02:22.104487839Z",
13+
"ingested": "2026-07-14T03:51:16.507024201Z",
1214
"kind": "event",
1315
"module": "powershell",
1416
"provider": "Microsoft-Windows-PowerShell",
15-
"type": "info"
17+
"type": [
18+
"info"
19+
]
1620
},
1721
"host": {
1822
"name": "vagrant",
@@ -27,6 +31,7 @@
2731
"message": "Creating Scriptblock text (1 of 1):\n.\\patata.ps1\n\nScriptBlock ID: 50d2dbda-7361-4926-a94d-d9eadfdb43fa\nPath: ",
2832
"powershell": {
2933
"file": {
34+
"script_block_hash": "64TcviMSSJ/OdhiN8lVcBQeKWDU=",
3035
"script_block_id": "50d2dbda-7361-4926-a94d-d9eadfdb43fa",
3136
"script_block_text": ".\\patata.ps1"
3237
},
@@ -61,17 +66,21 @@
6166
{
6267
"@timestamp": "2020-05-14T11:33:51.3938848Z",
6368
"ecs": {
64-
"version": "1.12.0"
69+
"version": "8.17.0"
6570
},
6671
"event": {
6772
"action": "Execute a Remote Command",
68-
"category": "process",
73+
"category": [
74+
"process"
75+
],
6976
"code": "4104",
70-
"ingested": "2025-01-15T10:02:22.104615177Z",
77+
"ingested": "2026-07-14T03:51:16.507053493Z",
7178
"kind": "event",
7279
"module": "powershell",
7380
"provider": "Microsoft-Windows-PowerShell",
74-
"type": "info"
81+
"type": [
82+
"info"
83+
]
7584
},
7685
"file": {
7786
"directory": "C:\\Users\\vagrant\\Desktop",

0 commit comments

Comments
 (0)