Skip to content

Add spatial and grouped validation with regional error analysis #4

Description

@unes07

Problem

A random row split does not demonstrate that recommendation models generalize to unseen sites or regions. Geographic autocorrelation and uneven crop coverage can make headline metrics overly optimistic.

Proposed work

  • Group records by site and add commune-, province-, or region-held-out evaluation.
  • Compare random-split and spatial-generalization results using identical features and targets.
  • Report errors by crop, nutrient target, region, soil-data completeness, and zero-output frequency.
  • Record split assignments so evaluations are reproducible.

Acceptance criteria

  • No site appears in both training and evaluation groups for grouped tests.
  • Spatial split definitions and coverage are published.
  • Reports compare random and spatial results side by side.
  • Sparse or unsupported regions are clearly identified.
  • Documentation avoids broad reliability claims not supported by held-out geographic results.

Depends on #2 and #3. Uses cleaned and versioned data from open-turba/turba-data#5, open-turba/turba-data#6, and open-turba/turba-data#7.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions