[BUG] Updated sbd_distance() to handle multivariate series (#2674)

[pvprajwal]

Reference Issues/PRs

Fixes #2674

What does this implement/fix? Explain your changes.

• Updated sbd_distance() to handle multivariate data consistently with tslearn and other implementations
• added _multivariate_sbd_distance() which finds the correlations for each of the channels and then normalizes using the norm of the multivariate series.

The earlier implementation found the average of the distance after calculating normalized_cc for each channel independenly.

if x.shape[0] == 1 and y.shape[0] == 1:
    _x = x.ravel()
    _y = y.ravel()
    return _univariate_sbd_distance(_x, _y, standardize)
else:
    # independent (time series should have the same number of channels!)
    nchannels = min(x.shape[0], y.shape[0])
    distance = 0.0
    for i in range(nchannels):
        distance += _univariate_sbd_distance(x[i], y[i], standardize)
    return distance / nchannels

The is replaced by a new _multivariate_sbd_distance() method which normalizes using the norm of the multivariate series, as is the case with tslearn and kshape-python.

@njit(cache=True, fastmath=True)
def _multivariate_sbd_distance(x: np.ndarray, y: np.ndarray, standardize: bool) -> float:
    x = x.astype(np.float64)
    y = y.astype(np.float64)

    x = np.transpose(x, (1, 0))
    y = np.transpose(y, (1, 0))

    if standardize:
        if x.size == 1 or y.size == 1:
            return 0.0

        x = (x - np.mean(x)) / np.std(x)
        y = (y - np.mean(y)) / np.std(y)

    norm1 = np.linalg.norm(x)
    norm2 = np.linalg.norm(y)
    
    denom = norm1 * norm2
    if denom < 1e-9:  # Avoid NaNs
        denom = np.inf

    with objmode(cc="float64[:, :]"):
        cc = np.array([correlate(x[:, i], y[:, i], mode="full", method="fft") for i in range(x.shape[1])]).T

    sz = x.shape[0]
    cc = np.vstack((cc[-(sz - 1):], cc[:sz]))  # Reorganize correlation values
    norm_cc = np.real(cc).sum(axis=-1) / denom

    return np.abs(1.0 - np.max(norm_cc))

Does your contribution introduce a new dependency? If yes, which one?

Nil

Any other comments?

Tests need to be modified. Now the values are consistent for tslearn and kshape-python

PR checklist

For all contributions

• I've added myself to the list of contributors. Alternatively, you can use the @all-contributors bot to do this for you after the PR has been merged.
• The PR title starts with either [ENH], [MNT], [DOC], [BUG], [REF], [DEP] or [GOV] indicating whether the PR topic is related to enhancement, maintenance, documentation, bugs, refactoring, deprecation or governance.

For new estimators and functions

• I've added the estimator/function to the online API documentation.
• (OPTIONAL) I've added myself as a __maintainer__ at the top of relevant files and want to be contacted regarding its maintenance. Unmaintained files may be removed. This is for the full file, and you should not add yourself if you are just making minor changes or do not want to help maintain its contents.

For developers with write access

• (OPTIONAL) I've updated aeon's CODEOWNERS to receive notifications about future changes to these files.

[aeon-actions-bot]

Thank you for contributing to aeon

I have added the following labels to this PR based on the title: [ $\color{#d73a4a}{\textsf{bug}}$ ].
I have added the following labels to this PR based on the changes made: [ $\color{#5209C9}{\textsf{distances}}$ ]. Feel free to change these if they do not properly represent the PR.

The Checks tab will show the status of our automated tests. You can click on individual test runs in the tab or "Details" in the panel below to see more information if there is a failure.

If our pre-commit code quality check fails, any trivial fixes will automatically be pushed to your PR unless it is a draft.

Don't hesitate to ask questions on the aeon Slack channel if you have any.

PR CI actions

These checkboxes will add labels to enable/disable CI functionality for this PR. This may not take effect immediately, and a new commit may be required to run the new configuration.

• Run pre-commit checks for all files
• Run mypy typecheck tests
• Run all pytest tests and configurations
• Run all notebook example tests
• Run numba-disabled codecov tests
• Stop automatic pre-commit fixes (always disabled for drafts)
• Disable numba cache loading
• Push an empty commit to re-run CI checks

[pvprajwal]

I think the tests have to be modified to be consistent with the new sbd_distance calculation. The same approach is used in the original k-shapes paper at https://dl.acm.org/doi/pdf/10.1145/2723372.2737793.

[SebastianSchmidl]

You transpose the output in L270 again. Is this really necessary?

[SebastianSchmidl]

Please keep this branch for now.

[SebastianSchmidl]

After transposing, this is standardizing along the wrong axis, isn't it?

[SebastianSchmidl]

Python loops are expensive. This is the reason, we use Numba a lot in the distance module. Is this loop really necessary? Maybe the correlate function can be used in a vectorized way?

[SebastianSchmidl]

Slicing with [:n] should be equal to slicing with [:-1], which is simpler to understand, or are you working around a limitation in Numba? Then, please add a comment.

[SebastianSchmidl]

We should benchmark the new implementation against the existing one for univariate series. If it is faster, we could get rid of the existing code.

Please also verify that the pairwise_distance-function supports variable-length inputs.