Fix GitHubAuditData dropping rows when source timestamp is missing - #15015
Fix GitHubAuditData dropping rows when source timestamp is missing#15015d4rk pr10r (d4rk-pri0r) wants to merge 2 commits into
Conversation
Unguarded unixtime_milliseconds_todatetime on created_at_d/CreatedAt nulled TimeGenerated and dropped rows. Use coalesce with column_ifexists fallback on all three views. V3-repackage GitHub solution 3.5.1 -> 3.5.2. Closes Azure#14659
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
This PR prevents GitHub audit rows from disappearing when the source timestamp field is missing by falling back to ingestion TimeGenerated, and bumps the parser/solution versions accordingly.
Changes:
- Update
GitHubAuditDataparser to usecoalesce(unixtime_milliseconds_todatetime(column_ifexists(...)), TimeGenerated)forTimeGenerated. - Bump parser version
1.1.0→1.1.1and solution/package version3.5.1→3.5.2. - Document the change in release notes.
Reviewed changes
Copilot reviewed 3 out of 5 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| Solutions/GitHub/data/Solution_GitHub.json | Bumps GitHub solution version to 3.5.2. |
| Solutions/GitHub/ReleaseNotes.md | Adds a 3.5.2 entry describing the parser fix. |
| Solutions/GitHub/Parsers/GitHubAuditData.yaml | Guards source timestamp conversion to preserve ingestion TimeGenerated. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
|
Kindly verify the Copilot comments, make the necessary changes, and mark the conversations as resolved. Thanks! |
Copilot: unixtime_milliseconds_todatetime expects integer milliseconds. column_ifexists default real(null) could widen the type. Use long(null). Parser 1.1.2. V3 package 3.5.3.
|
Hi v-maheshbh (@v-maheshbh) — Copilot comments addressed. All three |

Summary
Solutions/GitHub/Parsers/GitHubAuditData.yaml: all three views (GitHubAuditLogPolling_CL,GitHubAuditLogsV2_CL,GitHubAuditLogsV3_CL) used unguardedunixtime_milliseconds_todatetime(...). Missing source timestamp columns produced nullTimeGeneratedand those rows vanished from every query using the parser.coalesce(unixtime_milliseconds_todatetime(column_ifexists(...)), TimeGenerated).1.1.0→1.1.1.3.5.1→3.5.2(Package/mainTemplate.json,Package/3.5.2.zip,data/Solution_GitHub.json,ReleaseNotes.md) so this does not bounce the way Replaced the direct unixtime_milliseconds_todatetime() calls with a safe fallback pattern #14842 did.Closes #14659
Test Plan
TimeGenerated=lines in the parser YAML usecoalesce+column_ifexistsPackage/mainTemplate.jsonembedscoalesce(unixtime(not the old unguarded calls only)