Drift resilient Bayesian linear regression

[MaxHalford]

• http://www.iotbds.org/Templates.aspx
• https://github.com/MaxHalford/bayes-lin-reg-drift
• http://www.wikicfp.com/cfp/servlet/event.showcfp?eventid=95866&copyownerid=93970
• So we use an EWM, but what other decaying schemes?
• First things first: implement formulas for logistic regression
• Handle multi-class with online one-vs-rest
• Need to see if other weighting schemes are possible
• See if alpha can be determined from the data

class RobustBayesLinReg(BayesLinReg):

    def __init__(self, smoothing, n_features, alpha, beta):
        super().__init__(n_features, alpha, beta)
        self.smoothing = smoothing

    def learn(self, x, y):

        # Update the inverse covariance matrix, with smoothing
        cov_inv = (
            self.smoothing * self.cov_inv +
            (1 - self.smoothing) * self.beta * np.outer(x, x)
        )

        # Update the mean vector, with smoothing
        cov = np.linalg.inv(cov_inv)
        mean = cov @ (
            self.smoothing * self.cov_inv @ self.mean +
            (1 - self.smoothing) * self.beta * y * x
        )

        self.cov_inv = cov_inv
        self.mean = mean

        return self