CODEWITHSUNDEEP
pythonscipymathematical-optimization
JonathanK
JonathanK

Reputation: 867

Scipy optimize: Set maximum error

I'm trying to optimize a 4 dimensional function with scipy. Everything works so far, except that I'm not satisfied with the quality of the solution. Right now I have ground truth data, which I use to verify my code. What I get so far is:

End error:  1.52606896507e-05
End Gradient:  [ -1.17291295e-05   2.60362493e-05   5.15347856e-06  -2.72388430e-05]

Ground Truth:   [0.07999999..., 0.0178329..., 0.9372903878..., 1.7756283966...]
Reconstructed:  [ 0.08375729  0.01226504  1.13730592  0.21389899]

The error itself sounds good, but as the values are totally wrong I want to force the optimization algorithm (BFGS) to do more steps.

In the documentation I found the options 'gtol' and 'norm' and I tried to set both to pretty small values (like 0.0000001) but it did not seem to change anything.

Background: The problem is, that I try to demodulate waves, so I have sin and cos terms and potentially many local (or global) minima. I use bruteforce search to find a good starting point, witch helps a lot, but it currently seems that the most work is done by that brute force search, as the optimization uses often only one iteration step. So I'm trying to improve that part of the calculation somehow.

Upvotes: 1

Views: 466

Answers (2)

JonathanK
JonathanK

Reputation: 867

I found out that the gradient at the starting point is already very flat (values in 10^-5), so I tried to scale the gradient function witch I already provided. This seemed to be pretty effective, I could force the Algorithm to do much more steps and my results are far better now. They are not perfect though, but a complete discussion of this is outside of the bounds of this question, so I might start a new one, where I describe the whole problem from bottom up.

Upvotes: 3

ev-br
ev-br

Reputation: 26040

Many local minima + hardly any improvement after brute search, that sounds bad. It's hard to say something very specific with the level of detail you provide in the question, so here are vague ideas to try (basically, what I'd do if I suspect my minimizer gets stuck):

  • try manually starting the minimizer from a bunch of different initial guesses.
  • try using a stochastic minimizer. You're tagging a question scipy, so try basinhopping
  • if worst comes to worst, just throw random points in a loop, leave it to work over the lunch break (or overnight)

Also, waves, sines and cosines --- it might be useful to think if you can reformulate your problem in the Fourier space.

Upvotes: 4

Related Questions