CODEWITHSUNDEEP
pythonpython-3.xnumpylist-comprehensionnumba
Post169
Post169

Reputation: 708

Numba giving error for NumPy

After watching a SciPy talk on Numba, my understanding is that if it is done with NumPy tools, it can be done with Numba (with a good chance of speeding it up).

However, as I experiment with Numba-fying some code, it looks like the simple JIT (1) has no trouble with list comprehension, but (2) can't make head or tail of numpy.asarray().

When I run

import numba
@numba.jit
def squareTest(xlist):
    y = [x**2 for x in xlist]
    return y

it runs fine. But when I run

import numba
import numpy as np
@numba.jit
def squareTest(xlist):
    y = [x**2 for x in xlist]
    y1 = np.asarray(y)
    return y1

the IPython console gives me a long traceback, says there was a Lowering Error, and ends with

def squareTest(xlist):
    y = [x**2 for x in xlist]`  
    ^

I don't know the structure of the Numba module, but the very last part tells me that when it's going to turn the produced list into a NumPy array afterward, it then has trouble doing list comprehensions.

I checked

@numba.jit
def squareTest(xlist):
    y = list(range(20))
    y1 = np.asarray(y)
    return y1

and it ran with no trouble, so it looks like it can handle either list comprehension or np.asarray(), but not both.

My first guess was that the njit mode can handle everything NumPy, and the mode that is turned off by adding the n to the jit is what handles list comprehension. But no, the first code sample still works when it is changed to njit.

What is the problem here, and how can I Numba-fy functions that involve doing NumPy array operations to arrays that are most pythonically made with list comprehensions?

Upvotes: 0

Views: 767

Answers (1)

Post169
Post169

Reputation: 708

Numba might be able to handle list comprehensions sometimes, but the speaker in the "Numba - Tell Those C++ Bullies to Get Lost" talk that you linked made the point that for it to reach its potential, you need to write the code that you JIT in an un-pythonic manner.

It is very likely that when Numba JITs the code, it changes the order of operations, or even combines the operations, so that even though it can do a list comprehension on a list, and it can also turn a list into a NumPy array, it can't do both.

You need to replace the list comprehension command with an un-pythonic sequence of commands to do the same thing.

Upvotes: 1

Related Questions