Skip to content

Compute and use dual_gap in coordinate descent solver #6759

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
jcrist opened this issue May 19, 2025 · 0 comments
Open

Compute and use dual_gap in coordinate descent solver #6759

jcrist opened this issue May 19, 2025 · 0 comments
Labels
cuml-accel Issues related to cuml.accel improvement Improvement / enhancement to an existing function Linear Models

Comments

@jcrist
Copy link
Member

jcrist commented May 19, 2025

Sklearn's coordinate descent solver computes and uses the dual_gap as part of its stopping criteria, while ours does not. This causes a few problems:

  • In the cuml.accel layer, our exposed estimators lack a dual_gap_ fitted attribute, exposing the difference between the primary and the dual as a result of the fit. We tried to patch around this in Support dual_gap_ on ElasticNet & Lasso #6714, but it came with a performance cost and didn't make sense if the computed value wasn't being used in the solver.
  • The meaning of the tol parameter differs between solvers. Currently we work around that by scaling the tol parameter when converting to/from sklearn's parameters. If our solver's had similar stopping criteria we wouldn't need to do this, and could better guarantee functionally equivalent results.

Since we'll need to look into the solver anyway in #6736, I think we should consider making this change to better improve compatibility with sklearn's ElasticNet/Lasso implementations.

@jcrist jcrist added improvement Improvement / enhancement to an existing function Linear Models cuml-accel Issues related to cuml.accel labels May 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cuml-accel Issues related to cuml.accel improvement Improvement / enhancement to an existing function Linear Models
Projects
None yet
Development

No branches or pull requests

1 participant