Wrappers for solvers of general nonconvex/nonlinear constrained problems (NLP) for Matlab

Question

I have a general nonconvex function with general nonconvex inequality constraints. I have a feasible starting point, and I'd like to minimize an energy under the constraints. The solver should never leave the feasible domain (i.e. a barrier method) and should never increase the energy. So far I used fmincon which failed on both accounts, and I'd like an easy way to try other solvers such as IPOPT, KNITRO, and SNOPT. Speaking of which, I won't mind a recommendation for a specific solver that would accomplish what I'm looking for (nonincreasing and stays in the feasible area).

I'd like to try other solvers, but I'm looking for modeling the problem once at some wrapper (e.g. like yalmip and cvx for convex optimization; what about AMPL?), which would translate it to the other solvers (or just call my functions and convert their output to what is required by each solver - i.e. I'd like to work with a single interface). To be flexible as possible in terms of code, I'd prefer to supply callback functions (written in Matlab) for the objective and constraints functions and their gradient, which would return real values. Of course, if there would be an option to use internal variables that would provide things like auto-diff without compromising the code flexibility, it would be nice (I have a complex code, and making it friendly to some special variable type would be a hassle). I didn't try it, but if it's flexible enough, I won't mind using a .nl file, though I'll need some tips on the pipeline to interface with my matlab code (running some app that would request to solve a problem - i.e. it provides an input and expects a solution from some solver function like fmincon).

BTW, my problem with fmincon() was that at some point it starts to increase the function and can end with a worse point (larger objective), as well as compromise the constraints if terminated prematurely (it doesn't seem to converge).

Answer 1

I played with some of the solvers and the modeling packages, with an emphasis on Matlab.

• First, it seems that the definition of the interior point method is quite relaxed. All the solvers allow themselves to enter the infeasible area (i.e. behaving like penalty methods), while wiki describes a barrier method quite clearly with a log function wrapping the constraints:

https://en.wikipedia.org/wiki/Interior_point_method

Therefore, When I wanted to be strict with my problem (to stay withing the feasible domain), I defined my own f_aggregate() wrapping my constraints with a log, and used the solvers only to bound the variables range (or just used an unconstrained solver).

• Yalmip is quite convenient. You define an NLP problem the same way you define a convex problem using the sdpvar, which tracks the operations, and I assume it creates a model similar the auto-diff packages. Drawbacks:

  1. The code needs to be compatible with sdpvar, so some effort is needed, for example, replacing the bsxfun() calls or replacing assignments to a double matrix. On the other hand, no need to supply derivatives.
  2. It doesn't scale well. On a tiny problem it worked fine, but on a small problem it ran almost an hour, where half of the time was spent on formulation (e.g. processing a power operation seems costly). This implies to me that unlike supplying a model to mosek, it supplies a slow callback (similar to auto-diff) to IPOPT (but I might be wrong). I couldn't see the iterations IPOPT did. My related post: https://groups.google.com/forum/?fromgroups#!searchin/yalmip/zohar/yalmip/eJidaJgI9ec/0KqI-KP4LwAJ

• OptiToolbox is more low level, allowing callbacks, and it's more what I was looking for. But it seems that the only interesting supported solver for NLP is IPOPT. To be fair, other interesting solvers are commercial. It can also write a .gms file using SCIP that can be executed by GAMS or exported by GAMS to AMPL. Unfortunately, my specific problem wasn't written correctly, and both packages produced the same wrong result on the exported problem.

• I played with AMPL a bit (and GAMS looks similar). It's not that intuitive, the problem formulation is rather low level (e.g. no norm function or other sweeteners), and it seems like a pain to convert my matlab code or generate a problem based on it for AMPL (by myself). Also, the demo version limits the problem to 10 variables for the interesting solvers.

• There are a couple of commercial solvers out there, such as Knitro and Snopt, that I would like to try out. They both have a trial period of 6 months, but they are limited to 300 variables. I'm not sure how can I test a solver on a large scale problem limited to 300 variables. Update: Knitro also has a full month trial which I may try.

• Tomlab looks interesting. It's a commercial package similar to OptiToolbox that can interface with the commercial solvers. I'm waiting for them to reply with a license, but it would be limited for 21 days.

• I think there are online servers, where one could try the commercial solvers. But I'll need a good exporter for (e.g.) GAMS or AMPL formats.

• All in all, Matlab's fmincon doesn't seem that bad. While only the interior point can handle large scale problems, it seems at least as good as IPOPT (which seems to currently have a bug when my objective function returns Inf to reject a solution). Since NLP problems in general are hard, I think I shouldn't expect a magic solver like Mosek.

UPDATE: I happen to try Knitro and Snopt (full versions). Knitro's matlab interface is superb. It's exactly like fmincon, implementing the sparse more robust version of the same algorithms. Snopt's fmincon like interface is clunky. I had to patch it to add Jacobian sparsity, and the returned Jacobian was transposed, and it took me time to get the grips on it since it crashed matlab for every little mistake. In terms of performance on my problem, Knitro surprisingly failed once on one of my tiny problems, and both solvers in general didn't do much better than the others (running for hours without converging).

UPDATE2: I tried Erwin Kalvelagen below to add a variable for each inequality constraint, replace the inequality constraint with an equality constraint (ineq = new var), and bound the var. fmincon respected these bounds, but it wasn't a magic solution and the solver got stuck early; moreover IPOPT and Knitro didn't respect the eq bounds and entered the infeasible area.