Function composition, tuples and unpacking

Question

(disclaimed: not a Python kid, so please be gentle)

I am trying to compose functions using the following:

def compose(*functions):
  return functools.reduce(lambda acc, f: lambda x: acc(f(x)), functions, lambda x: x)

which works as expected for scalar functions. I'd like to work with functions returning tuples and others taking multiple arguments, eg.

def dummy(name):
  return (name, len(name), name.upper())

def transform(name, size, upper):
  return (upper, -size, name)

# What I want to achieve using composition, 
# ie. f = compose(transform, dummy)
transform(*dummy('Australia'))
=> ('AUSTRALIA', -9, 'Australia')

Since dummy returns a tuple and transform takes three arguments, I need to unpack the value.

How can I achieve this using my compose function above? If I try like this, I get:

f = compose(transform, dummy)
f('Australia')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 2, in <lambda>
  File "<stdin>", line 2, in <lambda>
TypeError: transform() takes exactly 3 arguments (1 given)

Is there a way to change compose such that it will unpack where needed?

Answer 1

You might consider inserting a "function" (really, a class constructor) in your compose chain to signal the unpacking of the prior/inner function's results. You would then adjust your composer function to check for that class to determine if the prior result should be unpacked. (You actually end up doing the reverse: tuple-wrap all function results except those signaled to be unpacked -- and then have the composer unpack everything.) It adds overhead, it's not at all Pythonic, it's written in a terse lambda style, but it does accomplish the goal of being able to properly signal in a function chain when the composer should unpack a result. Consider the following generic code, which you can then adapt to your specific composition chain:

from functools import reduce
from operator import add

class upk:  #class constructor signals composer to unpack prior result
  def __init__(s,r):  s.r = r  #hold function's return for wrapper function

idt = lambda x: x  #identity
wrp = lambda x: x.r if isinstance(x, upk) else (x,)  #wrap all but unpackables
com = lambda *fs: (  #unpackable compose, unpacking whenever upk is encountered
  reduce(lambda a,f: lambda *x: a(*wrp(f(*x))), fs, idt) )

foo = com(add, upk, divmod)  #upk signals divmod's results should be unpacked
print(foo(6,4))

This circumvents the problem, as called out by prior answers/comments, of requiring your composer to guess which types of iterables should be unpacked. Of course, the cost is that you must explicitly insert upk into the callable chain whenever unpacking is required. In that sense, it is by no means "automatic", but it is still a fairly simple/terse way of achieving the intended result while avoiding unintended wraps/unwraps in many corner cases.

Answer 2

This one works for your example but it wont handle just any arbitrary function - it will only works with positional arguments and (of course) the signature of any function must match the return value of the previous (wrt/ application order) one.

def compose(*functions):
    return functools.reduce(
        lambda f, g: lambda *args: f(*g(*args)), 
        functions, 
        lambda *args: args
        )

Note that using reduce here, while certainly idiomatic in functional programming, is rather unpythonic. The "obvious" pythonic implementation would use iteration instead:

def itercompose(*functions):
    def composed(*args):
        for func in reversed(functions):
            args = func(*args)
        return args    
    return composed

Edit:

You ask "Is there a way to make have a compose function which will work in both cases" - "both cases" here meaning wether the functions returns an iterable or not (what you call "scalar functions", a concept that has no meaning in Python).

Using the iteration-based implementation, you could just test if the return value is iterable and wrap it in a tuple ie:

import collections

def itercompose(*functions):
    def composed(*args):            
        for func in reversed(functions):
            if not isinstance(args, collections.Iterable):
                args = (args,)
            args = func(*args)
        return args    
    return composed

but this is not garanteed to work as expected - actually this is even garanteed to NOT work as expected for most use cases. There are a lot of builtin iterable types in Python (and even more user-defined ones) and just knowing an object is iterable doesn't say much about it's semantic.

For example a dict or str are iterable but in this case should obviously be considered a "scalar". A list is iterable too, and how it should be interpreted in this case is actually just undecidable without knowing exactly what it contains and what the "next" function in composition order expects - in some cases you will want to treat it as a single argument, in other cases ase a list of args.

IOW only the caller of the compose() function can really tell how each function result should be considered - actually you might even have cases where you want a tuple to be considered as a "scalar" value by the next function. So to make a long story short: no, there's no one-size-fits-all generic solution in Python. The best I could think of requires a combination of result inspection and manual wrapping of composed functions so the result is properly interpreted by the "composed" function but at this point manually composing the functions will be both way simpler and much more robust.

FWIW remember that Python is first and mostly a dynamically typed object oriented language so while it does have a decent support for functional programming idioms it's obviously not the best tool for real functional programming.

Answer 3

The compose function in the answer contributed by Bruno did do the job for functions with multiple arguments but didn't work any more for scalar ones unfortunately.

Using the fact that Python `unpacks' tuples into positional arguments, this is how I solved it:

import functools

def compose(*functions):
  def pack(x): return x if type(x) is tuple else (x,)

  return functools.reduce(
    lambda acc, f: lambda *y: f(*pack(acc(*pack(y)))), reversed(functions), lambda *x: x)

which now works just as expected, eg.

#########################
# scalar-valued functions
#########################

def a(x): return x + 1
def b(x): return -x

# explicit
> a(b(b(a(15))))
# => 17

# compose
> compose(a, b, b, a)(15)
=> 17


########################
# tuple-valued functions
########################

def dummy(x):
  return (x.upper(), len(x), x)
def trans(a, b, c):
  return (b, c, a)

# explicit
> trans(*dummy('Australia'))
# => ('AUSTRALIA', 9, 'Australia')

# compose
> compose(trans, dummy)('Australia')
# => ('AUSTRALIA', 9, 'Australia')

And this also works with multiple arguments:

def add(x, y): return x + y

# explicit
> b(a(add(5, 3)))
=> -9

# compose
> compose(b, a, add)(5, 3)
=> -9