tmph
tmph

Reputation: 201

Running TensorFlow on a Numpy-only function

I have a function that takes in numpy arrays [1], which I want to optimise using Tensorflow [2]. Here is a minimal example, where I also artificially force a function to take numpy arrays. Hopefully it illustrates what I am trying to do.

import tensorflow as tf
import numpy as np
def minimize_function(f, initial_vector, learning_rate=0.1, num_iterations=100, tol=1.e-5):
    vector = tf.Variable(initial_vector, dtype=tf.float64)
    optimizer = tf.optimizers.SGD(learning_rate)

    previous_value = float('inf')

    for iteration in range(num_iterations):
        with tf.GradientTape() as tape:
            value = f(vector)
        gradients = tape.gradient(value, vector)
        optimizer.apply_gradients([(gradients, vector)])

        if iteration > 0 and abs(value - previous_value) < tol:
            break

        previous_value = value

        print("Iteration: ", iteration, "; value: ", value.numpy())

    return value.numpy(), vector.numpy()

def example_function_np(x):
    assert isinstance(x, np.ndarray), "Input must be numpy array"
    assert x.dtype == np.float64, "Input must be float64"
    return np.sum(np.square(x))

N = 100
initial_vector = np.ones(N, dtype=np.float64)
minimum_value, minimum_vector = minimize_function(example_function_np, initial_vector)

As written, this will fail since the input to f won't be a numpy array. If I modify the minimize_function function to use numpy arrays and then cast to tf.Variable when necessary, then I find that tape.gradient gives me None.

Instead optimising the function

def example_function_wrapper(x):
    assert isinstance(x, tf.Variable), "Input must be tensorflow variable"
    return example_function_np(x.numpy())

returns the error AttributeError: 'numpy.float64' object has no attribute '_id' which indicates to me that numpy can't be used anywhere in the function to be optimised, though maybe I am mistaken.

Is there any way around this?


[1]: Actually, the function comes from a much larger code base written in Fortran which I am interfacing into Python using Cython with code that looks like this

cimport numpy as np

cdef extern from "functions_to_minimize.h":
    void example_function_interface(double* x, int* n, double* result)

def example_function_wrapper(np.ndarray[np.float64_t, ndim=1] x):
    cdef double result
    cdef int n = x.size
    example_function_interface(<double*> x.data, &n, &result)
    return result

but as far as I know you cannot rewrite this to use TensorFlow instead (something to do with the includes when doing cimport tensorflow). If there's a way to do that, that would be a useful answer too.

[2]: If there is another automatic differentiation library in Python (pytorch, autograd?) that can do this just as well, showing how to do that would also be a useful answer.

Upvotes: 0

Views: 65

Answers (0)

Related Questions