Skip to content

fix: validate dataset mixture fractions before loading - #7142

Open
behroozazarkhalili wants to merge 3 commits into
huggingface:mainfrom
behroozazarkhalili:reopen/6982
Open

fix: validate dataset mixture fractions before loading#7142
behroozazarkhalili wants to merge 3 commits into
huggingface:mainfrom
behroozazarkhalili:reopen/6982

Conversation

@behroozazarkhalili

@behroozazarkhalili behroozazarkhalili commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator

What does this PR do?

get_dataset normalizes mixture fraction values by their sum, so an all-zero mixture raises ZeroDivisionError after every dataset has already been downloaded, and a negative fraction is silently turned into a negative weight. The validation now runs before any load_dataset call: fractions must all be set or all unset (moved up from the post-load site), non-negative, and sum to a positive value. Individual zero fractions with a positive total stay allowed. The streaming guard moves up with it. A parametrized regression test pins each rejection and asserts that datasets.load_dataset was never called.

Fixes #6981

Credit

The approach comes from #6982 by @cananoo, which was closed on 2026-09-09; this PR carries it as a single commit of mine, rebased onto current main.

Verification

  • tests/test_cli_utils.py -k fraction: 6 passed on the rebased branch (3 new cases, 3 existing).
  • ruff check and ruff format clean on both files.

Before submitting


Note

Low Risk
Behavior change is limited to invalid fraction configs; valid mixtures behave the same, with earlier, clearer errors and no accidental downloads.

Overview
get_dataset now validates mixture fraction settings before any Hub/local load, so bad configs fail fast with clear ValueErrors instead of downloading everything and then hitting ZeroDivisionError (all zeros) or odd negative weights.

The pre-load checks cover: fractions must be set on all datasets or none (moved up from after load), each value must be non-negative, the sum must be > 0, and fraction + streaming is still rejected. Weight normalization reuses the precomputed total_fraction. The DatasetConfig.fraction docstring documents the non-negative / positive-sum rule.

Tests add a parametrized case for zero-sum and negative fractions and assert datasets.load_dataset is never called when validation fails.

Reviewed by Cursor Bugbot for commit ebf4e9c. Bugbot is set up for automated code reviews on this repo. Configure here.

@bot-ci-comment

bot-ci-comment Bot commented Sep 9, 2026

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Fractions are checked before any load_dataset call: all set or all unset,
non-negative, and summing to a positive value, so an all-zero mixture no
longer raises ZeroDivisionError after downloading every dataset and a
negative fraction no longer becomes a negative weight. Based on the
approach proposed in huggingface#6982. Fixes huggingface#6981.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

🐛 [Bug] get_dataset raises ZeroDivisionError for zero-sum fractions

1 participant