CODEWITHSUNDEEP
pythonlistfunctiondictionary
rprog
rprog

Reputation: 2130

Efficiently passing arguments to indexed functions stored in a list or a dicitonary

Say for example I have 100 unique functions (for demonstrative purposes I'll just have one)

def calcs(x):
     return x**2

and these one hundred unique functions are stored in a list or a dictionary (open to suggestions on ways to store these functions)

import numpy as np
store_funcs = []
for i in np.arange(100):
     store_funcs.append(calcs)

now I have an array of 100 numbers

nums = np.random.uniform(0,1, 100)

I would like to pass the first element of nums to the first function in store_funcs and have it return the answer, the second element of nums to the second function in store_funcs etc... My question is, is there a cleaner (and faster?) way to accomplish this without a loop?

for i in np.arange(100):
     print store_funcs[i](nums[i])

Also open to other ideas on how to accomplish this. Thanks!

Upvotes: 0

Views: 59

Answers (2)

skrrgwasme
skrrgwasme

Reputation: 9633

It looks like your problem could benefit from some parallelism, via the multiprocessing module:

import multiprocessing as mp

# create a function that will execute our function + argument pair
def executor(*args):

    # assume args[0] is our function object and args[1] is the number
    return args[0](args[1])

# assume we have a list of functions called "funcs" and a list of numbers
# called "nums", and they are of equal length - use zip to pair them off
pairs = zip(funcs, nums) 

# create a multiprocessing pool
pool = mp.Pool()

# submit our list of function+argument pairs to the pool for execution
result_list = pool.map(executor, pairs)

# clean up
pool.join()
pool.close()

Some useful links:

Multiprocessing Pools
The zip builtin

It's difficult to answer your original question of which data structure makes more sense, since we don't know how your actual functions and number arrays are generated (I assume [and hope] that they're not all hard-coded) or reused. If what you've described here is really the only use case for both, then the dictionary may make more sense, as it clearly expresses the 1:1 relationship between each function and number.

However, both dictionaries and lists will be equally capable of allowing you to execute the function+argument pairs in an iterative fashion, so if either the list of functions or the array of numbers is reused elsewhere in the code, I would stick with the two lists so their existence as separate entities remains clear.

Upvotes: 1

Thtu
Thtu

Reputation: 2032

In general, it's very hard to optimize code without knowing the nature of the code that is being executed. That said, here's a one liner:

map(lambda x : x[0](x[1]), zip(store_funcs, nums))

This probably wouldn't be faster, but it's one line?

On the other hand, if your functions are the bottleneck, then you may want to look into something such as to celery or another async execution library. That way, you would still loop through, but on each iteration, you only spend the time it takes to send a message to some queue for a pool of workers to consume.

Upvotes: 1

Related Questions