Disciplined nonlinear programming (DNLP)#3108
Conversation
initial attempts at adding a smooth canon for maximum
Ipopt interface prototype
adding more smooth canonicalizers
* adds oracles and bounds class to ipopt interface * adds some settings and solver lists changes for IPOPT * adds nlp solver option and can call ipopt * adds more experiments for integrating ipopt as a solver interface * passing the problem through the inversion * add some more extra changes * adding nlmatrixstuffing --------- Co-authored-by: William Zijie Zhang <william@gridmatic.com>
Co-authored-by: William Zijie Zhang <william@gridmatic.com>
* adding many tests, new smoothcanon for min, and improvements to ipopt_nlpif * fixing last two tests * add another example, qcp * adding example for acopf * add control of a car example done --------- Co-authored-by: William Zijie Zhang <william@gridmatic.com>
* update solution statuses thanks to odow * removes unusued solver information --------- Co-authored-by: William Zijie Zhang <william@gridmatic.com>
* getting rocket landing example to work * add changes to the jacobian computation --------- Co-authored-by: William Zijie Zhang <william@gridmatic.com>
* adding many more example of non-convex functions * making lots of progress on understanding good canonicalizations --------- Co-authored-by: William Zijie Zhang <william@gridmatic.com>
Co-authored-by: William Zijie Zhang <william@gridmatic.com>
|
Benchmarks that have improved:
Benchmarks that have stayed the same: |
* uses new coo handling * added comment * fix pre-commit * fix error message if sample bounds are not set * docs for DNLP * remove whitespace * split table up into two
* uses new coo handling * added comment * fix pre-commit * fix error message if sample bounds are not set * docs for DNLP * remove whitespace * split table up into two * cache C problem in best of
- Remove `submodules: recursive` from CI workflows (build, test_nlp_solvers, test_backends) - Revert `_grad` in affine_atom.py to upstream version - Remove NaN-allowance block in leaf.py added for NLP structural jacobian - Update is_dnlp() docstrings to use linearizable terminology - Add normalize_shape() helper in converters.py, replacing 7 inline occurrences - Remove unnecessary comment in nlp_solver.py - Export UNO in cvxpy/__init__.py - Revert README.md to upstream CVXPY version Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
#168) * remove unnecessary point in dom + clean up C_problem + clean up oracle * fix william's comments
…nology (#170) - Revert whitespace-only changes in huber, abs, ceil, affine_atom, binary_operators - Revert cosmetic rename and _jacobian_operator addition in index.py - Revert isnan check in leaf.py back to original error handling - Restore removed blank line in quad_form.py - Update DNLP constraint docstrings to use "linearizable convex/concave" terminology Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
|
@PTNobel @SteveDiamond @rileyjmurray could you have another look at this PR please?
@dance858 and I also did a thorough scan of the changes and we are quite happy about the state of the PR/fork. One big question is moving the non-convex atoms to a new namespace. Would it be fine to do that in another PR? It would be nice if we could discuss this as well more. I agree that if we create many new atoms it would be good to put them under a new namespace, but maybe we could just keep the trigonometric ones for now? The code is certainly not perfect, and we will definitely keep making many improvements over time. But I think we were able to integrate most of the big necessary changes for the project. It is also quite tedious to keep the fork synced (especially with all the recent PRs we have been doing). So to summarize, I think it is ready to be merged into CVXPY master! |
|
Thanks for all your work on this PR! I agree we should focus on the code as is and defer further design decisions like the nlp namespace. There will be many rounds of revision of all new features in 1.9 before we release so we should take an iterative development approach. |
- Remove unused `self.broadcast_type` attribute from `broadcast_to.__init__` (was previously used for derivative computation, now handled elsewhere) - Revert CLAUDE.md to upstream master version (DNLP-specific changes were broader than intended for the PR) Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
PTNobel
left a comment
There was a problem hiding this comment.
GitHub is breaking on me so I'm only halfway done.
I'm really confused why we're not using the bounds propagation infrastructure; I think it's perfect for this
| from cvxpy.expressions.variable import Variable | ||
|
|
||
|
|
||
| def sinh_canon(expr, args): |
There was a problem hiding this comment.
They seem to still be duplicated
PTNobel
left a comment
There was a problem hiding this comment.
All tests seem to be missing copyright headers
| verbose=verbose, use_hessian=use_hessian) | ||
|
|
||
| nlp = cyipopt.Problem( | ||
| n=len(data["x0"]), |
There was a problem hiding this comment.
Why is this at the same indentation as the nlp =
| elif isinstance(constraint, NonPos): | ||
| lower.extend([0.0] * constraint.size) | ||
| upper.extend([np.inf] * constraint.size) | ||
| new_constr.append(nonpos2nonneg(constraint)) |
There was a problem hiding this comment.
That's fine with me for now then; I think this could go in ExactCone2Cone in the future
|
Regarding your comments, I started my review before I saw it. I don't think this PR is ready to merge. Notably, many files are messing copyright headers. I'm pretty sure the domain property of tan always errors when accessed. Also, I think some edits made in response to my previous review never made it onto this branch possibly? There's a handful of comments you replied to saying something was done that wasn't. I think for all of my code quality changes (of which there are 25 small changes or so), as long as they're listed on an open issue somewhere (could be here with a check list; could be on cvxgrp/DNLP) I'll be satisfied. I'm worried if they're only on this PR it'll be very hard to track them after merge. Regarding my bigger concerns, (1. using the get_bounds infrastructure to narrow bounds, 2. deduplicating canon functions and having consistent implementation for the different smooth canons) I'm happy to just have an issue filed for each and for them to be addressed later. |
Done |
Co-authored-by: Parth Nobel <parthnobel@berkeley.edu>
|
I fixed most of the comments but tagged @Transurgeon in a few that he'll fix later today. Then we'll update the PR. I also implemented a common canonicalizer for smooth atoms with full domain. Great suggestion. Many thanks for the review. Please check if my answers above make sense. |
* Centralize sparsediffpy import into _bindings.py Move the duplicated try/except import of sparsediffpy from c_problem.py and converters.py into a single _bindings.py module. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * cleanup fix * fix import issue * fix import for c_problem as well --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
|
Thanks Daniel! I looked at the PR in the other repo and it looks great. I think there's maybe 3 or 4 things to file issues about, merge your PR there, and port all the changes from master to cvxpy-merge on the other repo and then we'll be all set! Really excited to get this all merged. It is super exciting. |
Thanks Parth! I'll go ahead and update this branch then. I'll then move over the issues to cvxpy master (there must be a way to do this somehow?). Then you guys could have a final look.. pinging @SteveDiamond and @rileyjmurray for approval as well. |
* many fixes to PR feedback * domain tan non-strict inequalities * div canon raise warning nonneg + document it * common canonicalizers
|
@PTNobel sorry for the ping.. but I just moved all issues from DNLP to cvxpy (with a new tag for them). |
|
I think get_bounds is the main thing missing. I don't see an issue for it yet? |
Just opened one now.. feel free to add more info if needed. |
Add Disciplined Nonlinear Programming (DNLP) extension to CVXPY
Summary
This PR adds Disciplined Nonlinear Programming (DNLP) support to CVXPY, enabling users to formulate and solve general nonlinear programs by freely mixing smooth functions with nonsmooth convex and concave functions under disciplined composition rules.
For theoretical background, see: Disciplined Nonlinear Programming
Usage
What's included
DNLP expression system and composition rules
is_smooth(),is_linearizable_convex(),is_linearizable_concave()onExpression,Atom,Leaf,Objective, and constraint classesis_dnlp()validation onProblemto check DNLP complianceis_atom_smooth()method on atoms to classify smooth vs. non-smooth atomsNew atoms
sin,cos,tan(cvxpy/atoms/elementwise/trig.py)sinh,tanh,asinh,atanh(cvxpy/atoms/elementwise/hyperbolic.py)Dnlp2Smooth reduction
Dnlp2Smoothreduction (cvxpy/reductions/dnlp2smooth/) that rewrites non-smooth atoms into smooth equivalents using auxiliary variablesabs,max,maximum,norm_inf,sum_largest,div,entr,exp,geo_mean,huber,kl_div,log,log_sum_exp,logistic,multiply,pnorm,power,prod,quad_over_lin,rel_entr, and trig/hyperbolic atomsNLP solver infrastructure
NLPsolverclass (cvxpy/reductions/solvers/nlp_solvers/nlp_solver.py) with:Bounds: extracts variable and constraint bounds from the reduced problemOracles: provides function/derivative oracles (objective, gradient, constraints, Jacobian, Hessian of the Lagrangian)cvxpy/reductions/solvers/nlp_solving_chain.py): orchestratesFlipObjective → CvxAttr2Constr → Dnlp2Smooth → NLPSolver, handles initial point construction (_set_nlp_initial_point), andbest_ofmulti-start logicNLP_SOLVER_VARIANTS) for algorithm selection (e.g.,"knitro_sqp")Differentiation engine integration
C_problemwrapper (cvxpy/reductions/solvers/nlp_solvers/diff_engine/c_problem.py) around the SparseDiffPy C library, providingobjective_forward(),gradient(),jacobian(),hessian()oraclesconverters.py) withATOM_CONVERTERSdict mapping ~40 atom types to C constructors, including sparse parameter matmul fusionParametersupport: parameters are registered with the C engine and values can be updated without rebuilding the expression treeFour NLP solver interfaces
ipopt_nlpif.pyknitro_nlpif.pycopt_nlpif.pyuno_nlpif.pyCI
test_nlp_solversGitHub Actions workflow (.github/workflows/test_nlp_solvers.yml) that installs system IPOPT viauvand runs NLP solver testsDocumentation
doc/source/tutorial/dnlp/index.rst)Comprehensive test suite (29 test files, ~3,800 lines)
DerivativeChecker(finite-difference checks of gradients, Jacobians, and Hessians)Diff stats
Test plan
pytest cvxpy/tests/nlp_tests/passes with IPOPT installedtest_nlp_solvers) passes