Skip to content

Documentation Dataset Validation #5683

Description

@SajidAlamQB

Description

User-facing documentation for dataset validation: a new docs page, a rewrite of the pandera integration page, and release notes.

Context

Part of [Implementation] Dataset Validation (KEP-10) — parent issue #5677. Depends on sub-issues 1–3.

Includes the explicit ask from tech design to document how node signatures differ from parameter validation.

Possible Implementation

  • New data validation page: quick start, validator: long form, schemas/ convention (convention, not enforcement), per-backend install matrix (pandas + Polars advertised; pyspark experimental)
  • Rewrite docs/integrations-and-plugins/pandera.md to lead with validator:; demote the hook recipe; credit kedro-pandera as inspiration
  • Signature note: node signatures stay pd.DataFrame; annotations are documentation; pd.DataFrame[Schema] is the recommended visible form; unlike parameter validation no model instance is injected, and why (datasets are data, not config)
  • Release notes: coercion changes data identity; validator documented as a reserved catalog key
  • Interactive validation recipe (the documented pandera path from tech design)

Metadata

Metadata

Assignees

No one assigned

    Labels

    Issue: Feature RequestNew feature or improvement to existing feature

    Type

    Projects

    Status
    No status

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions