Two new y-transformation approaches

[mlindauer]

• bilog (log transformations above 0 and below 0)
• Gaussian Copula (ECDF -> quantiles -> Inverse Gaussian CDF)

[dengdifan]

If everyone is happy with the implementation, I will merge this branch

[mfeurer]

I'm not sure if we want to merge this PR at the moment:

1. We don't have a method that uses quantile transformations
2. I think the quantile transformation should be improved
3. We don't have a method that uses bilog transformations at the moment

[mfeurer]

I believe this is incorrect. I reimplemented this according to Salinas et al., which appears to give better, and most importantly, symmetric outputs:

import numpy as np
import scipy.stats

values = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9, 10])
VERY_SMALL_NUMBER = 1e-10

# This PR
quants = [scipy.stats.percentileofscore(values, v)/100 - VERY_SMALL_NUMBER for v in values]
output = np.array([scipy.stats.norm.ppf(q) for q in quants]).reshape((-1, 1))
print(output)

# Correct
quants = (scipy.stats.rankdata(values.flatten()) - 1) / (len(values) - 1)
cutoff = 1 / (4 * np.power(len(values), 0.25) * np.sqrt(np.pi * np.log(len(values))))
quants = np.clip(quants, a_min=cutoff, a_max=1 - cutoff)
# Inverse Gaussian CDF
rval = np.array([scipy.stats.norm.ppf(q) for q in quants]).reshape((-1, 1))
print(rval)

output:

[-1.28155157e+00 -8.41621234e-01 -5.24400513e-01 -2.53347103e-01
 -2.50662848e-10  2.53347103e-01  5.24400512e-01  8.41621233e-01
  1.28155156e+00  6.36134089e+00]
[-1.62322583 -1.22064035 -0.76470967 -0.4307273  -0.1397103   0.1397103
  0.4307273   0.76470967  1.22064035  1.62322583]

[alexandertornede]

We will have a look at how these methods perform once we have the new benchmarking fully in place.

[mfeurer]

The recent HEBO suggests using a PowerTransform from scikit-learn. If you plan to benchmark these two, could you also throw this one in the mix?

[alexandertornede]

Thanks for the pointer. Sure!

[mfeurer]

Was this, by any chance, implemented in #1276 ?