Skip to content

fd-knowledge-graph: live Databricks scouts show little query-history SELECT signal in current scope #236

@ronsse

Description

@ronsse

Live bounded Databricks smoke on 2026-06-03 using warehouse cd8046ca523abdd9 and profile fdg-data-eng-dev produced these graph-safe artifacts under build/live-databricks: amplitude 1d and 7d query-history scouts returned zero non-pipeline SELECT groups; snowplow 7d returned zero; application_events 7d returned zero; customer360 7d returned 7 non-pipeline rows but all statement_type OTHER; broad scope-only 7d returned zero non-pipeline SELECT groups. BI saved-query scope-only scout over 500 saved queries found one SELECT candidate for landing.application_events.events, and review-first BI curation produced one non-promoted decision with zero mutations. Recommendation: do not start direct query-history graph writes for these domains yet; prioritize BI metadata, documentation, and pipeline/static-code enrichment, then revisit query-history with 30/90 day windows or refined predicates only after scout evidence improves.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions