megaman: Manifold Learning for Millions of Points

megaman is a scalable manifold learning package implemented in python. It has a front-end API designed to be familiar to scikit-learn but harnesses the C++ Fast Library for Approximate Nearest Neighbors (FLANN) and the Sparse Symmetric Positive Definite (SSPD) solver Locally Optimal Block Precodition Gradient (LOBPCG) method to scale manifold learning algorithms to large data sets. On a personal computer megaman can embed 1 million data points with hundreds of dimensions in 10 minutes. megaman is designed for researchers and as such caches intermediary steps and indices to allow for fast re-computation with new parameters.

Package documentation can be found at http://mmp2.github.io/megaman/

If you use our software please cite the following JMLR paper:

McQueen, Meila, VanderPlas, & Zhang, "Megaman: Scalable Manifold Learning in Python", Journal of Machine Learning Research, Vol 17 no. 14, 2016. http://jmlr.org/papers/v17/16-109.html

You can also find our arXiv paper at http://arxiv.org/abs/1603.02763

Examples

Tutorial Notebook

Installation with Conda

The easiest way to install megaman and its dependencies is with conda, the cross-platform package manager for the scientific Python ecosystem.

To install megaman and its dependencies, run

$ conda install megaman --channel=conda-forge

Currently builds are available for OSX and Linux, on Python 2.7, 3.4, and 3.5. For other operating systems, see the full install instructions below.

Installation from source

To install megaman from source requires the following:

python tested with versions 2.7, 3.4, and 3.5
numpy version 1.8 or higher
scipy version 0.16.0 or higher
scikit-learn
FLANN
cython
a C++ compiler such as gcc/g++

Optional requirements include

pyamg, which allows for faster decompositions of large matrices
pysamg, which allows for even faster decompositions of large matrices. This needs and is included in the commercial software Fraunhofer SAMG. For licensing (including test or educational licenses) contact samg@scai.fraunhofer.de
pyflann which offers another method of computing distance matrices (this is bundled with the FLANN source code)
nose for running the unit tests

These requirements(except for SAMG) can be installed on Linux and MacOSX using the following conda command:

$ conda install --channel=conda-forge pip nose coverage gcc cython numpy scipy scikit-learn pyflann pyamg

Finally, within the source repository, run this command to install the megaman package itself:

$ python setup.py install

Unit Tests

megaman uses nose for unit tests. With nose installed, type

$ make test

to run the unit tests. megaman is tested on Python versions 2.7, 3.4, and 3.5.

Authors

Other Contributors

Xiao Wang: lazy rmetric, Nystrom Extension

Future Work

See this issues list for what we have planned for upcoming releases:

Future Work

License

jrsassen/megaman

ErrorLooks like something went wrong!

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Languages

Error
Looks like something went wrong!

Packages