CODEWITHSUNDEEP
pythonfunctionpartialcurrying
lazytype
lazytype

Reputation: 925

Why doesn't functools.partial return a real function (and how to create one that does)?

So I was playing around with currying functions in Python and one of the things that I noticed was that functools.partial returns a partial object rather than an actual function. One of the things that annoyed me about this was that if I did something along the lines of:

five = partial(len, 'hello')
five('something')

then we get

TypeError: len() takes exactly 1 argument (2 given)

but what I want to happen is

TypeError: five() takes no arguments (1 given)

Is there a clean way to make it work like this? I wrote a workaround, but it's too hacky for my taste (doesn't work yet for functions with varargs):

def mypartial(f, *args):
  argcount = f.func_code.co_argcount - len(args)
  params = ''.join('a' + str(i) + ',' for i in xrange(argcount))
  code = '''
def func(f, args):
  def %s(%s):
    return f(*(args+(%s)))
  return %s
  ''' % (f.func_name, params, params, f.func_name)

  exec code in locals()
  return func(f, args)

Edit: I think it might be helpful if I added more context. I'm writing a decorator that will automatically curry a function like so:

@curry
def add(a, b, c):
  return a + b + c

f = add(1, 2) # f is a function
assert f(5) == 8

I want to hide the fact that f was created from a partial (maybe a bad idea :P). The message that the TypeError message above gives is one example of where whether something is a partial can be revealed. I want to change that.

This needs to be generalizable so EnricoGiampieri's and mgilson's suggestions only work in that specific case.

Upvotes: 13

Views: 3961

Answers (2)

abarnert
abarnert

Reputation: 365707

You definitely don't want to do this with exec.

You can find recipes for partial in pure Python, such as this one—many of them are mislabeled as curry recipes, so look for that as well. At any rate, these will show you the proper way to do it without exec, and you can just pick one and modify it to do what you want.

Or you could just wrap partial…

However, whatever you do, there's no way the wrapper can know that it's defining a function named "five"; that's just the name of the variable you store the function in. So if you want a custom name, you'll have to pass it in to the function:

five = my_partial('five', len, 'hello')

At that point, you have to wonder why this is any better than just defining a new function.

However, I don't think this is what you actually want anyway. Your ultimate goal is to define a @curry decorator that creates a curried version of the decorated function, with the same name (and docstring, arg list, etc.) as the decorated function. The whole idea of replacing the name of the intermediate partial is a red herring; use functools.wraps properly inside your curry function, and it won't matter how you define the curried function, it'll preserve the name of the original.

In some cases, functools.wraps doesn't work. And in fact, this may be one of those times—you need to modify the arg list, for example, so curry(len) can take either 0 or 1 parameter instead of requiring 1 parameter, right? See update_wrapper, and the (very simple) source code for wraps and update_wrapper to see how the basics work, and build from there.

Expanding on the previous: To curry a function, you pretty much have to return something that takes (*args) or (*args, **kw) and parse the args explicitly, and possibly raise TypeError and other appropriate exceptions explicitly. Why? Well, if foo takes 3 params, curry(foo) takes 0, 1, 2, or 3 params, and if given 0-2 params it returns a function that takes 0 through n-1 params.

The reason you might want **kw is that it allows callers to specify params by name—although then it gets much more complicated to check when you're done accumulating arguments, and arguably this is an odd thing to do with currying—it may be better to first bind the named params with partial, then curry the result and pass in all remaining params in curried style…

If foo has default-value or keyword args, it gets even more complicated, but even without those problems, you already need to deal with this problem.

For example, let's say you implement curry as a class that holds the function and all already-curried parameters as instance members. Then you'll have something like this:

def __call__(self, *args):
    if len(args) + len(self.curried_args) > self.fn.func_code.co_argcount:
        raise TypeError('%s() takes exactly %d arguments (%d given)' %
                        (self.fn.func_name, self.fn.func_code.co_argcount,
                         len(args) + len(self.curried_args)))
    self.curried_args += args
    if len(self.curried_args) == self.fn.func_code.co_argcount:
        return self.fn(*self.curried_args)
    else:
        return self

This is horribly oversimplified, but it shows how to handle the basics.

Upvotes: 6

EnricoGiampieri
EnricoGiampieri

Reputation: 6085

My guess is that the partial function just delay the execution of the function, do not create a whole new function out of it.

My guess is that is just easier to define directly a new function in place:

def five(): return len('hello')

This is a very simple line, won't clutter your code and is quite clear, so i wouldn't bother writing a function to replace it, especially if you don't need this situation in a large number of cases

Upvotes: 0

Related Questions