Skip to content

Fix fold split toggles and live train toggle visibility - #866

Merged
cristian-tamblay merged 3 commits into
developfrom
fix/fold-charts-scored-splits
Sep 8, 2026
Merged

Fix fold split toggles and live train toggle visibility#866
cristian-tamblay merged 3 commits into
developfrom
fix/fold-charts-scored-splits

Conversation

@Irozuku

@Irozuku Irozuku commented Sep 7, 2026

Copy link
Copy Markdown
Collaborator

Summary

The metric charts assumed every evaluation strategy scores a training partition. A forecasting strategy does not: fitting on the training rows and then scoring the model on those same rows would report a fit as a forecast, so ForecastingHoldoutEvaluationStrategy and ForecastingCrossValidationEvaluationStrategy write validation and test metrics only.

Both charts opened on a TRAIN toggle regardless. The fold chart asked the backend for fold metrics that were never written, and the live chart opened on an empty panel. Neither had any way to know, because the split buttons were hardcoded.

Evaluation strategies now report which partitions they score, and both charts build their toggles from that instead of from a fixed list. A third fix rides along, same class of bug: the comparison table assumed every session is cross-validated and crashed on a nested run when it was not.


Type of Change

Check all that apply like this [x]:

  • Backend change
  • Frontend change
  • CI / Workflow change
  • Build / Packaging change
  • Bug fix
  • Documentation

Changes (by file)

  • DashAI/back/evaluation/base_evaluation_strategy.py: get_metadata now reports scored_splits beside kind, taken from the strategy's own SCORED_SPLITS. A screen that offers one control per partition reads this rather than assuming all three exist.
  • DashAI/front/src/hooks/useStrategyKind.js: the hook was renamed and widened to useStrategyMetadata, returning the whole metadata object instead of just kind, with the same cross component cache. useStrategyKind stays as a thin wrapper over it, so every existing caller is untouched.
  • DashAI/front/src/components/models/FoldMetricsChart.jsx: the TRAIN/VALIDATION toggles are built from scored_splits, and the selected split starts as null and resolves to the first partition the strategy actually scores. The fetch is held until the strategy metadata arrives, so no request goes out for metrics that were never written, and the spinner covers that wait.
  • DashAI/front/src/components/models/LiveMetricsChart.jsx: same treatment for the TRAIN/VALIDATION/TEST toggles. The previous effect only corrected a stale TEST selection; it now keeps the selection on any partition still being offered, which folds in the old behaviour.
  • DashAI/front/src/components/models/ModelComparisonTable.jsx: nestedCv is always defined in runTypeStyles and the legend does the cross-validation filtering, with a fallback on the row lookup so no run type can crash the table over a missing style.
  • tests/back/evaluation/test_forecasting_strategies.py: asserts each strategy reports the partitions it scores, forecasting strategies naming validation and test, the ordinary ones naming all three.

Testing (Optional)

  • On the forecasting session's fold chart, only a Validation toggle is offered and it is selected on open. No Train toggle, no empty chart, no failed request in the network tab.
  • On the ordinary session's fold chart, Train and Validation are both offered and Train is still the default, as before.
  • On the forecasting session's live chart during a run, Train is absent and the panel opens on Validation with data in it.
  • On the ordinary session's live chart, all three toggles behave as before, and Test still appears only once test metrics exist.
  • Switch between the two sessions without reloading. The toggles must follow the session rather than keeping the previous one's set.
  • Open a session that has nested cross-validation runs. The comparison table renders rather than throwing, and the nested rows carry their colour.
  • Open a session with nested runs that is not cross-validated. The table renders and the CV anidado entry stays out of the legend.

A forecasting run scores no train partition, so the chart's train default asked for fold metrics that were never written.
A forecasting strategy writes no train metrics, so the toggle opened on an empty panel.
The style set only defined nestedCv for a cross-validation session, but
getRunType returns it for any run flagged nested. The strategy kind
arrives asynchronously, so on the first render of a session with nested
runs the key was missing and destructuring undefined took the whole
table down.

The style is now always defined and the legend does the filtering, with
a fallback on the lookup so no row can crash the table over a style.
@Irozuku Irozuku added bug Something isn't working front Frontend work back Backend work labels Sep 7, 2026
@cristian-tamblay
cristian-tamblay merged commit fe5f510 into develop Sep 8, 2026
21 checks passed
@cristian-tamblay
cristian-tamblay deleted the fix/fold-charts-scored-splits branch September 8, 2026 20:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

back Backend work bug Something isn't working front Frontend work

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants