Skip to content

Commit 1b58234

Browse files
committed
fix(security/scout): register LLM interceptors immediately before submit
The kbn-ftr-llm-proxy 30s timeout starts at intercept() call time. When interceptors were registered at test start, slow CI steps (waitForRuleAlert up to 60s, waitForAttachmentPillsRow up to 30s) could exhaust the timeout before the submit button was ever clicked, producing a spurious "Interceptor set_title timed out" failure. Moving the registration to immediately before submitButton.click() is safe because autoSendInitialMessage: false guarantees no LLM call fires before the user submits. Refs #17496
1 parent 80d7947 commit 1b58234

1 file changed

Lines changed: 35 additions & 34 deletions

File tree

x-pack/solutions/security/plugins/security_solution/test/scout/ui/parallel_tests/investigations/bulk_add_alerts_to_chat.spec.ts

Lines changed: 35 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -80,40 +80,6 @@ spaceTest.describe(
8080
async ({ pageObjects, llmProxy }) => {
8181
const { alertsTablePage, agentBuilderPage } = pageObjects;
8282

83-
// Set up LLM mock interceptors before triggering the chat so no call goes unanswered.
84-
// The agent builder makes three sequential calls: title generation, handover, and final answer.
85-
void llmProxy
86-
.intercept({
87-
name: 'set_title',
88-
when: (body) => {
89-
const sys = body.messages.find((m) => m.role === 'system');
90-
return String(sys?.content ?? '').includes('You are a title-generation utility');
91-
},
92-
responseMock: createToolCallMessage('set_title', { title: 'Alert triage' }),
93-
})
94-
.completeAfterIntercept();
95-
96-
void llmProxy
97-
.intercept({
98-
name: 'handover-to-answer',
99-
when: (body) => {
100-
const sys = body.messages.find((m) => m.role === 'system');
101-
return String(sys?.content ?? '').includes(
102-
'This response will serve as a handover note for the answering agent'
103-
);
104-
},
105-
responseMock: 'ready to answer',
106-
})
107-
.completeAfterIntercept();
108-
109-
void llmProxy
110-
.intercept({
111-
name: 'final-answer',
112-
when: () => true,
113-
responseMock: 'Here is the alert triage summary.',
114-
})
115-
.completeAfterIntercept();
116-
11783
await alertsTablePage.navigate();
11884

11985
// Wait for the first rule's alert to appear — this proves Sourcerer has initialized
@@ -143,6 +109,41 @@ spaceTest.describe(
143109
);
144110

145111
await spaceTest.step('send message and verify LLM received multiple alerts', async () => {
112+
// Register interceptors here — immediately before submit — so the 30s proxy
113+
// timeout doesn't expire during the earlier waitForRuleAlert / waitForAttachmentPillsRow
114+
// steps. autoSendInitialMessage: false guarantees no LLM call fires before this point.
115+
void llmProxy
116+
.intercept({
117+
name: 'set_title',
118+
when: (body) => {
119+
const sys = body.messages.find((m) => m.role === 'system');
120+
return String(sys?.content ?? '').includes('You are a title-generation utility');
121+
},
122+
responseMock: createToolCallMessage('set_title', { title: 'Alert triage' }),
123+
})
124+
.completeAfterIntercept();
125+
126+
void llmProxy
127+
.intercept({
128+
name: 'handover-to-answer',
129+
when: (body) => {
130+
const sys = body.messages.find((m) => m.role === 'system');
131+
return String(sys?.content ?? '').includes(
132+
'This response will serve as a handover note for the answering agent'
133+
);
134+
},
135+
responseMock: 'ready to answer',
136+
})
137+
.completeAfterIntercept();
138+
139+
void llmProxy
140+
.intercept({
141+
name: 'final-answer',
142+
when: () => true,
143+
responseMock: 'Here is the alert triage summary.',
144+
})
145+
.completeAfterIntercept();
146+
146147
await agentBuilderPage.submitButton.click();
147148
await llmProxy.waitForAllInterceptorsToHaveBeenCalled();
148149

0 commit comments

Comments
 (0)