Skip to content

[ENH] Add preprocessing (scalers and normalizers) to tslib datamodule - #2368

Open
echo-xiao wants to merge 7 commits into
sktime:mainfrom
echo-xiao:enh/tslib-preprocessing
Open

[ENH] Add preprocessing (scalers and normalizers) to tslib datamodule#2368
echo-xiao wants to merge 7 commits into
sktime:mainfrom
echo-xiao:enh/tslib-preprocessing

Conversation

@echo-xiao

Copy link
Copy Markdown
Contributor

Reference Issues/PRs

Fixes #2330.

What does this implement/fix? Explain your changes.

TslibDataModule accepted scalers / target_normalizer but never used them
(stored and ignored). This makes them functional, following #2302 and reusing
ScalerAdapter:

  • __init__ wraps them in ScalerAdapter, adds _*_fitted flags and a
    per-series _preprocess_cache.
  • _fit_scalers / _fit_target_normalizer fit on the train split only.
  • _normalize_features / _normalize_target transform per series (no-op until
    fitted; only configured continuous columns).
  • _preprocess_data applies the transforms and caches per series;
    setup("fit") fits on _train_indices.

What should a reviewer concentrate their feedback on?

  1. no deepcopy of scalers
  2. deferring target_scale** to a follow-up
  3. only continuous features are scaled, the models don't consume them yet
  4. target_scale, collate_fn and the models are untouched

Did you add any tests for the change?

Yes — 7 tests in pytorch_forecasting/data/tests/test_tslib_data_module.py:

  • adapter wrapping + flags;
  • fit-on-train (StandardScaler → ~0 mean / ~1 std);
  • target-normalizer fit flag;
  • feature transform (only configured columns);
  • target no-op-until-fitted; _preprocess_data scaling + cache identity;
  • setup("fit") produces scaled samples;
  • a no-scaler backward-compat guard (byte-identical output, no target_scale).

PR checklist

  • The PR title starts with [ENH].
  • Added/modified tests.
  • Used pre-commit hooks (pre-commit run --files on both changed files, clean).

@echo-xiao

echo-xiao commented Aug 3, 2026

Copy link
Copy Markdown
Contributor Author

hi, @phoeenniixx please review these and let me know if these decisions are right:

  • no deepcopy of scalers
  • deferring target_scale to a follow-up, collate_fn and the models are untouched
  • only continuous features are scaled, the models don't consume them yet

TODO:

@codecov

codecov Bot commented Aug 4, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 26.15385% with 96 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (main@5a35991). Learn more about missing BASE report.

Files with missing lines Patch % Lines
...h_forecasting/data/tests/test_tslib_data_module.py 0.00% 77 Missing ⚠️
...forecasting/data/data_module/_tslib_data_module.py 64.15% 19 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2368   +/-   ##
=======================================
  Coverage        ?   86.69%           
=======================================
  Files           ?      175           
  Lines           ?    10292           
  Branches        ?        0           
=======================================
  Hits            ?     8923           
  Misses          ?     1369           
  Partials        ?        0           
Flag Coverage Δ
cpu 86.69% <26.15%> (?)
pytest 86.69% <26.15%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.



def _make_ts(n_series: int = 20, length: int = 40, offset: float = 100.0) -> TimeSeries:
"""合成数据集:连续特征 ``x`` 远离 0(~offset),便于看出标准化;目标 ``y`` 是正弦。"""

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please use english docstrings

@phoeenniixx phoeenniixx left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!
I have a few questions:

  • Did you look at the thuml implementation? I think we should try to keep it as close to this package as possible - when it comes to the preprocessing logic as this dm is based on this package
    • Currently it supports all the scalers and normalizers that EncoderDecoderDataModule supports, but is this true for the tslib implementation as well?

Also, please use english docstrings, to maintain uniformity across the documentation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request module:datasets&dataloaders ptf-v2 Related to `pytorch-forecasting` v2

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[ENH] Add preprocessing to tslib datamodule

2 participants