Is your feature request related to a problem? Please describe.
When building machine learning models for NIR and other vibrational spectroscopy data, checking for model robustness is a real pain point. The chemotools.augmentation module already handles things like random noise and flat baseline shifts for stress-testing models, but it has no way to simulate actual physical light scattering.
In NIR spectroscopy, scattering isn't a flat offset or a simple multiplier, it's a wavelength-dependent effect. Without a tool to synthetically inject this specific type of scatter into a testing pipeline, it's hard to evaluate whether a chemometric model is genuinely robust against real-world instrument and sample variation.
Describe the solution you'd like
I'd love to see a new augmentation transformer, ScatterShift, that implements the forward Extended Multiplicative Scatter Correction (EMSC) model to deliberately add scatter artifacts to spectra:
$$x_{\text{aug}} = m \cdot x + \sum_{i=0}^{N} c_i \cdot \lambda^i$$
It would reuse the exact same normalized-index polynomial basis that ExtendedMultiplicativeScatterCorrection already builds internally, so it acts as the forward counterpart of that corrector, the same model and the same basis, run in the inject direction rather than the remove direction.
This gives users a concrete way to generate EMSC-style augmented test sets and check whether their downstream predictive models hold up against realistic physical scatter, rather than only against noise and flat offsets.
To keep things tidy, a single ScatterShift class can cover both MSC and EMSC behavior via an order parameter:
-
MSC (
order=0): a constant offset ($c_0$) alongside the multiplicative factor ($m$).
-
EMSC (
order >= 1): linear, quadratic, or higher-order baseline curves to simulate wavelength-dependent scatter.
Like the other augmenters, setting the scales to 0.0 makes the transformer return the spectra unchanged (identity transform), so it passes scikit-learn's check_estimator() the same way the existing augmenters do.
Describe alternatives you've considered
I tried chaining SpectrumScale and BaselineShift, but it doesn't cover the case that matters. Because BaselineShift only adds a flat offset, there's no way to produce the higher-order baseline curvature ($i \ge 1$) that physical scattering causes in NIR data. Since EMSC was specifically designed to estimate and remove that non-linear behavior, a dedicated forward tool is the natural way to generate it. (At order=0 the effect is reproducible by chaining the two existing transformers, the new value is order >= 1.)
Additional context
I currently have barebones of the implementation ready, but wanted to open an issue first to check on scope and naming. I can put together the implementation that includes:
- API: follows the scikit-learn transformer API (
BaseEstimator/TransformerMixin), uses _parameter_constraints, and passes check_estimator() at default parameters.
- Tests: sklearn compliance, identity-at-defaults, multiplicative-only and polynomial-baseline behavior, reproducibility, and a test asserting the internal polynomial basis is identical to the existing EMSC design matrix.
- Docs: Numpydoc docstring with the model equation, the EMSC reference (Afseth & Kohler, 2012), and a runnable example.
In a controlled round-trip test, applying ExtendedMultiplicativeScatterCorrection after ScatterShift recovered the original signal almost entirely, which is consistent with the two sharing the same model.
ScatterShift is just a proposed name, happy to rename to whatever fits your conventions. If this sounds like a good fit, I'll open the PR.
Is your feature request related to a problem? Please describe.
When building machine learning models for NIR and other vibrational spectroscopy data, checking for model robustness is a real pain point. The
chemotools.augmentationmodule already handles things like random noise and flat baseline shifts for stress-testing models, but it has no way to simulate actual physical light scattering.In NIR spectroscopy, scattering isn't a flat offset or a simple multiplier, it's a wavelength-dependent effect. Without a tool to synthetically inject this specific type of scatter into a testing pipeline, it's hard to evaluate whether a chemometric model is genuinely robust against real-world instrument and sample variation.
Describe the solution you'd like
I'd love to see a new augmentation transformer,
ScatterShift, that implements the forward Extended Multiplicative Scatter Correction (EMSC) model to deliberately add scatter artifacts to spectra:It would reuse the exact same normalized-index polynomial basis that
ExtendedMultiplicativeScatterCorrectionalready builds internally, so it acts as the forward counterpart of that corrector, the same model and the same basis, run in the inject direction rather than the remove direction.This gives users a concrete way to generate EMSC-style augmented test sets and check whether their downstream predictive models hold up against realistic physical scatter, rather than only against noise and flat offsets.
To keep things tidy, a single
ScatterShiftclass can cover both MSC and EMSC behavior via anorderparameter:order=0): a constant offset (order >= 1): linear, quadratic, or higher-order baseline curves to simulate wavelength-dependent scatter.Like the other augmenters, setting the scales to
0.0makes the transformer return the spectra unchanged (identity transform), so it passes scikit-learn'scheck_estimator()the same way the existing augmenters do.Describe alternatives you've considered
I tried chaining$i \ge 1$ ) that physical scattering causes in NIR data. Since EMSC was specifically designed to estimate and remove that non-linear behavior, a dedicated forward tool is the natural way to generate it. (At
SpectrumScaleandBaselineShift, but it doesn't cover the case that matters. BecauseBaselineShiftonly adds a flat offset, there's no way to produce the higher-order baseline curvature (order=0the effect is reproducible by chaining the two existing transformers, the new value isorder >= 1.)Additional context
I currently have barebones of the implementation ready, but wanted to open an issue first to check on scope and naming. I can put together the implementation that includes:
BaseEstimator/TransformerMixin), uses_parameter_constraints, and passescheck_estimator()at default parameters.In a controlled round-trip test, applying
ExtendedMultiplicativeScatterCorrectionafterScatterShiftrecovered the original signal almost entirely, which is consistent with the two sharing the same model.ScatterShiftis just a proposed name, happy to rename to whatever fits your conventions. If this sounds like a good fit, I'll open the PR.