CODEWITHSUNDEEP
pythonoptimizationscipy
Krishna Kalyan
Krishna Kalyan

Reputation: 1702

Minimising a function using scipy.optimize

enter image description here

I need to minimise the above function using scipy.

My Input Data

np.random.seed(1234)
m = 500    #500
n = 100    #100
A = np.asmatrix(np.random.randint(low=-10,high=1,size=(n,m)))     
b = np.asmatrix(np.random.randint(low=500,high=1000,size=(m,1)))  
c = np.asmatrix(np.random.randint(low=0.1,high=2,size=(n,1)))
x = np.asmatrix(np.random.randint(low=1,high=10,size=(n,1)))

function

My Function and Gradient:

def func(x, A, b, c):
    fx = np.dot(c.T, x) - np.sum(np.log10((b - np.dot(A.T, x))))
    return fx

def grad(x, A, b, c):
    gradient = A * (1.0/(b - A.transpose()*x)) + c
    return gradient

This is what i have tried to run

scipy.optimize.fmin_cg(func(x + t*grad(x, A, b, c),A, b, c), x, args=(A,b,c,x))

Upvotes: 0

Views: 373

Answers (1)

ali_m
ali_m

Reputation: 74172

I'm not sure what you're trying to do with

func(x + t*grad(x, A, b, c), A, b, c)

What is t?

At any rate, your call to fmin_cg is incorrect - the signature of fmin_cg is

fmin_cg(f, x0, fprime=None, args=(), ...)

The first argument needs to be your objective function, func, the second needs to be your initial guess at x, the third (optional) argument is your gradient function, grad, and the fourth is the set of additional arguments to f and fprime (excluding x).

The call should look like:

scipy.optimize.fmin_cg(func, x, fprime=grad, args=(A, b, c))

However, this still won't work because of a problem with your array dimensions:

<ipython-input-49-bf5fa71345fe> in grad(x, A, b, c)
      1 def grad(x, A, b, c):
----> 2         gradient = A * (1.0/(b - A.transpose()*x)) + c
      3         return gradient
      4 

/home/alistair/.venvs/core/local/lib/python2.7/site-packages/numpy/matrixlib/defmatrix.pyc in __mul__(self, other)
    341         if isinstance(other, (N.ndarray, list, tuple)) :
    342             # This promotes 1-D vectors to row vectors
--> 343             return N.dot(self, asmatrix(other))
    344         if isscalar(other) or not hasattr(other, '__rmul__') :
    345             return N.dot(self, other)

ValueError: shapes (500,100) and (1,100) not aligned: 100 (dim 1) != 1 (dim 0)

To figure out why this is happening, we can set a breakpoint in grad:

import pdb

def grad(x, A, b, c):
    pdb.set_trace()
    gradient = A * (1.0/(b - A.transpose()*x)) + c
    return gradient

On the first call to grad, we see:

(Pdb) type(x)
<type 'numpy.ndarray'>
(Pdb) !x.shape
(100,)

Somewhere inside fmin_cg, x is being converted from a (100, 1) np.matrix to a (100,) 1D np.ndarray. For an np.ndarray, the * operator does elementwise multiplication rather than matrix multiplication, which will fail because x and A.transpose() have incompatible dimensions.

Basically you are running up against the fact that np.matrix is not fully supported by many functions in numpy and scipy that expect an np.ndarray. I highly recommend you switch from using np.matrix to np.ndarray - use of np.matrix is officially discouraged, and will probably be deprecated in the not-too-distant future.

Your grad function can be rewritten as:

def grad(x, A, b, c):
    gradient = A.dot(1.0/(b - A.T.dot(x))) + c
    return gradient

... and your initial arguments as:

np.random.seed(1234)
m = 500    #500
n = 100    #100
A = np.random.randint(low=-10, high=1, size=(n, m))
b = np.random.randint(low=500, high=1000, size=m) 
c = np.random.randint(low=0.1, high=2, size=n)
x = np.random.randint(low=1, high=10, size=n)

... and now your call to fmin_cg should work:

res = scipy.optimize.fmin_cg(func, x, fprime=grad, args=(A, b, c))

Upvotes: 2

Related Questions