Implementation of Random Fourier Features using only numpy.
paper: Random Features for Large-Scale Kernel Machines (NIPS 2008)
Random Fourier Features (RFF) is a finite set of feature maps that approximate the inner product of a continuous and stationary positive definite kernel.
The existence of this distribution is guaranteed by Bochner's theorem.
In this part, we approximate GP using RBF kernel by Bayesian Linear Regression (BLR) with RFF.
- Number of features : 1000
- Training points : 3, 10, 100, 3000
In RFF, the predictions in extrapolation is unstable and cause variance starvation. (see https://arxiv.org/pdf/1706.01445.pdf)
| Gaussian process regression | Bayesian linear regression with RFF | |
|---|---|---|
| n=3 |  |
 |
| n=10 |  |
 |
| n=100 |  |
 |
| n=3000 |  |
 |