Skip to content

Add Impersonation and BEC email hunting queries (with spoof and authentication enhancements) - #15027

Merged
v-atulyadav merged 5 commits into
Azure:masterfrom
damozes1:impersonation-bec-hunting-queries
Sep 3, 2026
Merged

Add Impersonation and BEC email hunting queries (with spoof and authentication enhancements)#15027
v-atulyadav merged 5 commits into
Azure:masterfrom
damozes1:impersonation-bec-hunting-queries

Conversation

@damozes1

Copy link
Copy Markdown
Contributor

Change(s)

Adds six Advanced Hunting queries for email impersonation and Business Email Compromise (BEC), each in both the Hunting Queries library and the Microsoft Defender XDR solution: Phishing Detected by LLM Content Analysis; First-Contact External Email Senders; New External Sender Domains Carrying Threats; Top Recipients Targeted by Impersonation or BEC; Impersonation and Spoof Detections by Sender Country; Inbound Sender Domains Failing Email Authentication.
Also enhances six existing queries (partner compromise; impersonation and spoof by detection technology; the three spoof-by-domain queries) with latest-record de-duplication, a robust set_has_element match on the parsed detection methods, and a bad-traffic percentage on the partner query. Versions bumped to 1.1.0.

Reason for Change(s)

Shares impersonation/BEC email threat-hunting queries from the Microsoft Defender for Office 365 Detections and Insights workbook, and improves the accuracy of the related existing queries.

Solution Version: N/A

Testing Completed: Yes

Checked that the validations are passing: Yes

…f and partner queries

New (Hunting + Solution copies): phishing detected by LLM content analysis; first-contact external senders; new external sender domains carrying threats; top recipients targeted by impersonation; impersonation and spoof detections by sender country; inbound sender domains failing email authentication. Enhances six existing queries (partner compromise; impersonation/spoof by detection technology; the three spoof-by-domain queries): latest-record de-duplication, set_has_element on parsed detection methods in place of a brittle string match, and a bad-traffic percentage on the partner query. Versions bumped to 1.1.0.
@damozes1
DanielM (damozes1) requested review from a team as code owners September 1, 2026 15:20
@v-maheshbh v-maheshbh (v-maheshbh) added the Hunting Hunting specialty review needed label Sep 2, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

Copilot was unable to run its full agentic suite in this review.

Pull request overview

Adds and updates Microsoft Defender Advanced Hunting query YAMLs to improve hunting for email impersonation/BEC and strengthen spoof/authentication analytics across both the standalone Hunting Queries library and the Microsoft Defender XDR solution.

Changes:

  • Added new hunting queries for LLM-detected phishing, first-contact external senders/domains with threats, top impersonation/BEC targets, sender-country breakdown, and auth-failure domains.
  • Enhanced multiple existing spoof/BEC queries with 30-day scoping, latest-record de-duplication, and more robust DetectionMethods parsing; bumped versions to 1.1.0 where updated.
  • Added “bad traffic percent” enrichment to the partner/BEC domain query.

Reviewed changes

Copilot reviewed 24 out of 24 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
Solutions/Microsoft Defender XDR/Hunting Queries/Email and Collaboration Queries/Spoof and Impersonation/Top Recipients Targeted by Impersonation or BEC.yaml New query to identify most-targeted recipients for impersonation/BEC.
Solutions/Microsoft Defender XDR/Hunting Queries/Email and Collaboration Queries/Spoof and Impersonation/Top Domains with BEC Threats inbound.yaml Updates query with time window, de-duplication, and bad-traffic percentage.
Solutions/Microsoft Defender XDR/Hunting Queries/Email and Collaboration Queries/Spoof and Impersonation/Spoof detections by Detection Technology.yaml Adds time scoping + de-duplication; bumps version.
Solutions/Microsoft Defender XDR/Hunting Queries/Email and Collaboration Queries/Spoof and Impersonation/Impersonation and Spoof Detections by Sender Country.yaml New country aggregation query for impersonation/spoof detections.
Solutions/Microsoft Defender XDR/Hunting Queries/Email and Collaboration Queries/Spoof and Impersonation/Impersonation Phishing detections by Detection Technology.yaml Adds time scoping + de-duplication; bumps version.
Solutions/Microsoft Defender XDR/Hunting Queries/Email and Collaboration Queries/Phish/Phishing Detected by LLM Content Analysis.yaml New query for phishing where LLM content analysis contributed.
Solutions/Microsoft Defender XDR/Hunting Queries/Email and Collaboration Queries/Mailflow/New External Sender Domains Carrying Threats.yaml New query to highlight newly seen external domains already carrying threats.
Solutions/Microsoft Defender XDR/Hunting Queries/Email and Collaboration Queries/Mailflow/First-Contact External Email Senders.yaml New query to surface new external sender relationships.
Solutions/Microsoft Defender XDR/Hunting Queries/Email and Collaboration Queries/Authentication/Top Spoof detections by Sender Domain.yaml Uses robust DetectionMethods parsing + de-dup; bumps version.
Solutions/Microsoft Defender XDR/Hunting Queries/Email and Collaboration Queries/Authentication/Top Spoof Intra-Org detections by SenderDomain.yaml Uses robust DetectionMethods parsing + de-dup; bumps version.
Solutions/Microsoft Defender XDR/Hunting Queries/Email and Collaboration Queries/Authentication/Top Spoof DMARC detections by Sender Domain.yaml Uses robust DetectionMethods parsing + de-dup; bumps version.
Solutions/Microsoft Defender XDR/Hunting Queries/Email and Collaboration Queries/Authentication/Inbound Sender Domains Failing Email Authentication.yaml New query to rank inbound sender domains by authentication failures.
Hunting Queries/Microsoft 365 Defender/Email and Collaboration Queries/Spoof and Impersonation/Top Recipients Targeted by Impersonation or BEC.yaml New query to identify most-targeted recipients for impersonation/BEC.
Hunting Queries/Microsoft 365 Defender/Email and Collaboration Queries/Spoof and Impersonation/Top Domains with BEC Threats inbound.yaml Updates query with time window, de-duplication, and bad-traffic percentage.
Hunting Queries/Microsoft 365 Defender/Email and Collaboration Queries/Spoof and Impersonation/Spoof detections by Detection Technology.yaml Adds time scoping + de-duplication; bumps version.
Hunting Queries/Microsoft 365 Defender/Email and Collaboration Queries/Spoof and Impersonation/Impersonation and Spoof Detections by Sender Country.yaml New country aggregation query for impersonation/spoof detections.
Hunting Queries/Microsoft 365 Defender/Email and Collaboration Queries/Spoof and Impersonation/Impersonation Phishing detections by Detection Technology.yaml Adds time scoping + de-duplication; bumps version.
Hunting Queries/Microsoft 365 Defender/Email and Collaboration Queries/Phish/Phishing Detected by LLM Content Analysis.yaml New query for phishing where LLM content analysis contributed.
Hunting Queries/Microsoft 365 Defender/Email and Collaboration Queries/Mailflow/New External Sender Domains Carrying Threats.yaml New query to highlight newly seen external domains already carrying threats.
Hunting Queries/Microsoft 365 Defender/Email and Collaboration Queries/Mailflow/First-Contact External Email Senders.yaml New query to surface new external sender relationships.
Hunting Queries/Microsoft 365 Defender/Email and Collaboration Queries/Authentication/Top Spoof detections by Sender Domain.yaml Uses robust DetectionMethods parsing + de-dup; bumps version.
Hunting Queries/Microsoft 365 Defender/Email and Collaboration Queries/Authentication/Top Spoof Intra-Org detections by SenderDomain.yaml Uses robust DetectionMethods parsing + de-dup; bumps version.
Hunting Queries/Microsoft 365 Defender/Email and Collaboration Queries/Authentication/Top Spoof DMARC detections by Sender Domain.yaml Uses robust DetectionMethods parsing + de-dup; bumps version.
Hunting Queries/Microsoft 365 Defender/Email and Collaboration Queries/Authentication/Inbound Sender Domains Failing Email Authentication.yaml New query to rank inbound sender domains by authentication failures.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

- Inbound Sender Domains Failing Email Authentication: rank domains by the number of messages that failed any authentication check instead of summing the per-check counts, which double-counted messages failing more than one check. The per-check counts remain as columns.

- Top Recipients Targeted by Impersonation or BEC: rename the sampled sender-domain set to SampleSenderDomains and clarify the description, since it is a sample and not a ranked list.

- Top Domains with BEC Threats inbound: read the deduplicated email set once via a let statement, and rename the ratio column to InboundThreatsPer100Outbound. Results are unchanged.
@damozes1

Copy link
Copy Markdown
Contributor Author

I looked into the KqlValidations result. All 24 files in this PR are hunting queries under Hunting Queries/Microsoft 365 Defender/Email and Collaboration Queries/ and Solutions/Microsoft Defender XDR/Hunting Queries/Email and Collaboration Queries/. I extracted every one of their template GUIDs and cross-checked them against the full CI test log, and none of them appear among the failing templates.
The run reports Failed: 158, Passed: 12051, Skipped: 0. Every failing template I checked belongs to an unrelated content area (ASIM parsers, third-party data connectors, identity analytics, and various analytic-rule / detection templates), and they reproduce on a current master baseline, so they look pre-existing rather than introduced here.
I have also just pushed an update applying the review feedback on this PR: ranking the email authentication query by the number of messages failing any check instead of summing the per-check counts, naming the sampled sender-domain set and the partner-BEC ratio column more clearly, and reading the deduplicated set once via a let statement. The detection-technology ordering was intentionally left as is (dedup before filter, to reflect the final verdict on a re-classifiable field), and I have replied on that thread.
Could you validate on your side and, if you agree, re-run or treat the failing checks as unrelated pre-existing failures so this PR can proceed? Happy to rebase on the latest master if that helps clear any transient or unrelated results. Thanks!

New: Phish Detections by Threat Classification (LLM intent), Top Attacked Users by Phish Threat Classification, Phish Detection IP and Geo Position, Phish Detections by Sender Country. Updated 9 existing Phish queries to de-duplicate messages with arg_max(Timestamp,*) per NetworkMessageId and recipient, enable the phishing-simulation and SecOps-mailbox exclusion, replace bag_unpack/union detection-technology logic with a single mv-expand, use robust parse_json(ConfidenceLevel) instead of substring matching, use Timestamp (not TimeGenerated), and add explicit render types, a phish confidence split and a bad-traffic percentage. Mirrored to both the Hunting and Solutions locations with unique ids per file. The two ThreatClassification queries are added to the KqlValidations skip list because ThreatClassification is a valid EmailEvents column not yet present in the offline validator schema.
@damozes1
DanielM (damozes1) requested a review from a team as a code owner September 2, 2026 21:29
v-rusraut
v-rusraut previously approved these changes Sep 3, 2026
# Conflicts:
#	.script/tests/KqlvalidationsTests/SkipValidationsTemplates.json
# Conflicts:
#	.script/tests/KqlvalidationsTests/SkipValidationsTemplates.json
@damozes1

Copy link
Copy Markdown
Contributor Author

hey v-maheshbh (@v-maheshbh) v-atulyadav the merge conflict here is now resolved and the branch is synced with the latest master (The PR is mergeable again). Can you kick off to re-run the checks and merge if all comes back clean now?

@v-atulyadav
v-atulyadav merged commit 849b69e into Azure:master Sep 3, 2026
31 of 32 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Hunting Hunting specialty review needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants