[BUG] Missing dual_gap_ attribute in Lasso/ElasticNet implementation

[csadorf]

Problem

Two scikit-learn tests are failing in cuML's Lasso implementation:

• test_lasso_zero
• test_lasso_toy

The failures occur because these tests check for the dual_gap_ attribute, which is present in scikit-learn's implementation but missing in cuML's.

Impact

• Test failures in scikit-learn compatibility tests
• Missing functionality that could be useful for users who rely on this metric

Additional Context

• cuML uses different solvers (CD and QN) than scikit-learn
• The duality gap computation may not be feasible or meaningful with cuML's solver implementations
• This is a scikit-learn specific implementation detail that may not align with cuML's optimization approach

[jcrist]

We can compute this outside of the solvers (but still at fit time), though it would definitely be more efficient to do so within the solver (and I think should be fairly straightforward at least in the CD solver).

Doing it post-solve would be easy to do with a few cupy calls and shouldn't add too much overhead. I can push something up, if only for discussion.

[jcrist]

Ok, I've done a bit more inspection here.

cd.cuh doesn't currently compute the dual gap as part of its stopping criteria, it instead stops when the largest update along a coordinate is smaller than a tolerance. In contrast, sklearn uses that check to see if the stopping criteria might be hit, and in that case then computes the dual gap to use as the actual check. This means in practice that our cd solver might stop earlier than sklearn's for the same tolerance.

A few options:

1. Leave things alone - this is a linear model, which while not bad is also not what cuml is best at. This may be low priority, and dual_gap_ is of limited use here anyway.
2. Leave things alone in cuml proper, and only handle dual_gap_ in the cuml.accel proxy layer. This would be blocked on us refactoring the proxy layer, but that's something we want to do anyway.
3. Leave the solver alone, compute the dual_gap_ afterwards in fit. Easy to do with a few cupy calls.
4. Fix the cd solver stopping criteria to be based on the dual gap (and plumb that through as an output of the solve). This takes the most effort, but it shouldn't be that bad. The downside is that for the qn solver we'd still need to compute dual_gap_ afterwards for peak compatibility.

No strong thoughts. I do think that if we're going to compute dual_gap_ we might as well do it in cuml proper (so not in favor of 2).

[csadorf]

My preference would be 3, then 1.