translate coordinate descent to cython

[jasmainak]

I just chatted with @rvraghav93 and he is willing to help us with cython code.

This might be a good starting point: https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/linear_model/cd_fast.pyx

it might also be easier to benchmark with R / scikit-learn that way.

Also, he mentioned this to check scikit-learn compatibility: https://github.com/scikit-learn-contrib/scikit-learn-contrib

[pavanramkumar]

sounds great! thanks @rvraghav93

I'm also dropping here is alex gramfort's PR for gap safe screening rules in case we add it to our coordinate descent implementation: https://github.com/scikit-learn/scikit-learn/pull/5075/files

[pavanramkumar]

Just fyi, I have implemented and tested newton coordinate descent in pure python for the poisson case. It works well, by 5-10x slower than current batch gradient descent code. I'll make a WIP PR of the pure python version once I've done it for other distr. Soon!

Meanwhile, I've been trying to accelerate by jit compiling with numba but it seems less straightforward than I thought. Could use an extra pair of brains and eyes for this once the pure python version is ready.

[jasmainak]

we're going to experiment with cython in the coming weekend hopefully. It's a 4 day weekend here ;) Neither me or @raghavrv have experience with numba though ... WIP PRs are always welcome :)

[pavanramkumar]

Sounds great! My WIP PR will give you a pure python code so that you can focus on translating to cython or numba. I'll aim to get it in to you guys for Bastille day.

[pavanramkumar]

Ok, PR is now updated to solve the multinomial case with the cdfast method.

[jasmainak]

Thanks @pavanramkumar ! We almost sat down with the intention of hacking cython today but ended up enjoying food in a delicious asian veg restaurant instead. I blame it on Paris! We'll try to get down to it soon ... hopefully tomorrow :)

[pavanramkumar]

Hey guys! @jasmainak @raghavrv I haven't been able to carve out time for cython experiments myself, but I ran into SWIG today: http://www.swig.org/tutorial.html

A useful alternative to cython could be:

• write the cdfast() functions directly in C
• wrap using SWIG

In any case, if you guys want to take a stab at Cython let me know!

[jasmainak]

@pavanramkumar sorry I haven't been able to look into cython yet. But maybe it's a good thing that we don't rush premature optimization.

Do we already have comparable performance (in terms of accuracy) to scikit-learn and R?

[themantalope]

I can start working on this during the weekend.