How to use sympy codegen with expressions that contain implemented functions

Question

Im trying to compile an expression that contains an UndefinedFunction which has an implementation provided. (Alternatively: an expression which contains a Symbol which represents a call to an external numerical function)

Is there a way to do this? Either with autowrap or codegen or perhaps with some manual editing of the generated files?

The following naive example does not work:

import sympy as sp
import numpy as np
from sympy.abc import *
from sympy.utilities.lambdify import implemented_function
from sympy.utilities.autowrap import autowrap, ufuncify

def g_implementation(a):
    """represents some numerical function"""
    return a*5

# sympy wrapper around the above function
g = implemented_function('g', g_implementation)

# some random expression using the above function
e = (x+g(a))**2+100

# try to compile said expression
f = autowrap(e, backend='cython')
# fails with "undefined reference to `g'"

EDIT:

I have several large Sympy expressions

The expressions are generated automatically (via differentiation and such)

The expressions contain some "implemented UndefinedFunctions" that call some numerical functions (i.e. NOT Sympy expressions)

The final script/program that has to evaluate the expressions for some input will be called quite often. That means that evaluating the expression in Sympy (via evalf) is definitely not feasible. Even compiling just in time (lambdify, autowrap, ufuncify, numba.jit) produces too much of an overhead.

Basically I want to create a binary python extension for those expressions without having to implement them by hand in C, which I consider too error prone.

OS is Windows 7 64bit

Answer 1

You may want to take a look at this answer about serialization of SymPy lambdas (generated by lambdify).

It's not exactly what you asked but may alleviate your problem with startup performance. Then lambdify-ied functions will mostly count only execution time.

You may also take a look at Theano. It has nice integration with SymPy.

Answer 2

Ok, this could do the trick, I hope, if not let me know and I'll try again.

I compare a compiled version of an expression using Cython against the lambdified expression.

from sympy.utilities.autowrap import autowrap
from sympy import symbols, lambdify

def wraping(expression):
    return autowrap(expression, backend='cython')

def lamFunc(expression, x, y):
    return lambdify([x,y], expr)

x, y = symbols('x y')
expr = ((x - y)**(25)).expand()
print expr


binary_callable = wraping(expr)
print binary_callable(1, 2)

lamb = lamFunc(expr, x, y)
print lamb(1,2)

which outputs:

x**25 - 25*x**24*y + 300*x**23*y**2 - 2300*x**22*y**3 + 12650*x**21*y**4 - 53130*x**20*y**5 + 177100*x**19*y**6 - 480700*x**18*y**7 + 1081575*x**17*y**8 - 2042975*x**16*y**9 + 3268760*x**15*y**10 - 4457400*x**14*y**11 + 5200300*x**13*y**12 - 5200300*x**12*y**13 + 4457400*x**11*y**14 - 3268760*x**10*y**15 + 2042975*x**9*y**16 - 1081575*x**8*y**17 + 480700*x**7*y**18 - 177100*x**6*y**19 + 53130*x**5*y**20 - 12650*x**4*y**21 + 2300*x**3*y**22 - 300*x**2*y**23 + 25*x*y**24 - y**25

-1.0
-1

If I time the execution times the autowraped function is 10x faster (depending on the problem I have also observed cases where the factor was as little as two):

%timeit binary_callable(12, 21)
100000 loops, best of 3: 2.87 µs per loop

%timeit lamb(12, 21)
100000  loops, best of 3: 28.7 µs per loop

So here wraping(expr) wraps your expression expr and returns a wrapped object binary_callable. This you can use at any time to do numerical evaluation.

EDIT: I have done this on Linux/Ubuntu and Windows OS, both seem to work fine!