Reputation: 201
I have a function that takes in numpy arrays [1], which I want to optimise using Tensorflow [2]. Here is a minimal example, where I also artificially force a function to take numpy arrays. Hopefully it illustrates what I am trying to do.
import tensorflow as tf
import numpy as np
def minimize_function(f, initial_vector, learning_rate=0.1, num_iterations=100, tol=1.e-5):
vector = tf.Variable(initial_vector, dtype=tf.float64)
optimizer = tf.optimizers.SGD(learning_rate)
previous_value = float('inf')
for iteration in range(num_iterations):
with tf.GradientTape() as tape:
value = f(vector)
gradients = tape.gradient(value, vector)
optimizer.apply_gradients([(gradients, vector)])
if iteration > 0 and abs(value - previous_value) < tol:
break
previous_value = value
print("Iteration: ", iteration, "; value: ", value.numpy())
return value.numpy(), vector.numpy()
def example_function_np(x):
assert isinstance(x, np.ndarray), "Input must be numpy array"
assert x.dtype == np.float64, "Input must be float64"
return np.sum(np.square(x))
N = 100
initial_vector = np.ones(N, dtype=np.float64)
minimum_value, minimum_vector = minimize_function(example_function_np, initial_vector)
As written, this will fail since the input to f
won't be a numpy array. If I modify the minimize_function
function to use numpy arrays and then cast to tf.Variable
when necessary, then I find that tape.gradient
gives me None
.
Instead optimising the function
def example_function_wrapper(x):
assert isinstance(x, tf.Variable), "Input must be tensorflow variable"
return example_function_np(x.numpy())
returns the error AttributeError: 'numpy.float64' object has no attribute '_id'
which indicates to me that numpy can't be used anywhere in the function to be optimised, though maybe I am mistaken.
Is there any way around this?
[1]: Actually, the function comes from a much larger code base written in Fortran which I am interfacing into Python using Cython with code that looks like this
cimport numpy as np
cdef extern from "functions_to_minimize.h":
void example_function_interface(double* x, int* n, double* result)
def example_function_wrapper(np.ndarray[np.float64_t, ndim=1] x):
cdef double result
cdef int n = x.size
example_function_interface(<double*> x.data, &n, &result)
return result
but as far as I know you cannot rewrite this to use TensorFlow instead (something to do with the includes when doing cimport tensorflow
). If there's a way to do that, that would be a useful answer too.
[2]: If there is another automatic differentiation library in Python (pytorch, autograd?) that can do this just as well, showing how to do that would also be a useful answer.
Upvotes: 0
Views: 65