getting a python function of functionType for a nested function with recursion

Question

I am trying to do unit testing for nested functions (function inside a function), I am using code("nested" is function name) from below link which would supply closures and returns a valid function that is callable from tests. It works for simple functions. I am trying to make it work for recursive functions.

As an example: I am trying to get a valid function for "innerfunction" which has an entry in co_freevars as "innerfunction".

I want to get a function (FunctionType I believe) for "innerfunction" as a callable. To get this I need to call FunctionType with a tuple with a FunctionType for "innerfunction". This becomes a recursive "dependency". How can I resolve this dependency for parameters to be sent for "closures"

Function for

def toplevelfunction():
    def innerfunction(a):
        print ('val of a is ', a)
        if a > 0:
            innerfunction(a -1)
    innerfunction(10)

Original code that I am using:

def freeVar(val):
  def nested():
    return val
  return nested.__closure__[0]

codeAttribute = '__code__' if sys.version_info[0] == 3 else 'func_code'

def nested(outer, innerName, **freeVars):
  if isinstance(outer, (types.FunctionType, types.MethodType)):
    outer = outer.__getattribute__(codeAttribute)
  for const in outer.co_consts:
    if isinstance(const, types.CodeType) and const.co_name == innerName:
      return types.FunctionType(const, globals(), None, None, tuple(
          freeVar(freeVars[name]) for name in const.co_freevars))

https://code.activestate.com/recipes/580716-unit-testing-nested-functions/

How to add support for closures so below works:

func = nested(toplevelfunction, 'innerfunction')

func(5)

would return error need a closure of length 1. Adding a closure referring to "const" shows up that it is of CodeType and not FunctionType. Adding a closure value to refer itself seems tricky after reading through the documentation.

I do find innerfunction as:

 {code} <code object innerfunction at 0x104b9bc00, file "/<filedirectory>/handle_results.py", line 44>
 co_argcount = {int} 1
 co_cellvars = {tuple} <class 'tuple'>: ()
 co_code = {bytes} b"t\x00\x00d\x01\x00|\x00\x00\x83\x02\x00\x01|\x00\x00d\x02\x00k\x04\x00r'\x00\x88\x00\x00|\x00\x00d\x03\x00\x18\x83\x01\x00\x01d\x00\x00S"
 co_consts = {tuple} <class 'tuple'>: (None, 'val of a is ', 0, 1)
 co_filename = {str} '/<fildirectory>/handle_results.py'
 co_firstlineno = {int} 44
 co_flags = {int} 19
 co_freevars = {tuple} <class 'tuple'>: ('innerfunction ',)
 co_kwonlyargcount = {int} 0
 co_lnotab = {bytes} b'\x00\x01\r\x01\x0c\x01'
 co_name = {str} 'innerfunction '
 co_names = {tuple} <class 'tuple'>: ('print',)
 co_nlocals = {int} 1
 co_stacksize = {int} 3
 co_varnames = {tuple} <class 'tuple'>: ('a',)

Answer 1

Thank you for hinting me on this limitation on my ActiveState recipe you linked to in your question!

You tried to unit test a local recursive function; my minimal example for this looks like this:

def f(x):
  def fac(n):
    return fac(n-1) * n if n > 1 else 1
  print "Faculty of", x, "is", fac(x)

So in a unit test you want to test the inner function fac(). Simply applying my recipe leads to this error:

nestedFac = nested(f, 'fac')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 7, in nested
  File "<stdin>", line 7, in <genexpr>
KeyError: 'fac'

This is because fac is an identifier used within the inner function, and you must specify the value of each identifier used within the inner function. Callable identifiers are no exception:

nestedFac = nested(f, 'fac', fac=None)

This sets fac to the value None and the call to nested does not fail anymore.

Unfortunately, the resulting function will try to call None(...) when you call it:

nestedFac(5)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in fac
TypeError: 'NoneType' object is not callable

You would need to pass the function itself as the value for fac to avoid this. Unfortunately, you do not have it yet when you call nested(). A repeated wrapping could be done like this:

nestedFac = nested(f, 'fac', fac=None)
nestedFac = nested(f, 'fac', fac=nestedFac)
nestedFac = nested(f, 'fac', fac=nestedFac)
nestedFac = nested(f, 'fac', fac=nestedFac)

This would limit your recursion depth to the number of wraps you applied:

nestedFac(2)
2
nestedFac(3)
6
nestedFac(4)
24
nestedFac(5)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in fac
  File "<stdin>", line 3, in fac
  File "<stdin>", line 3, in fac
  File "<stdin>", line 3, in fac
TypeError: 'NoneType' object is not callable

To avoid this, you can pass a local function as proxy:

def q(*args, **kwargs):
  return nestedFac(*args, **kwargs)
nestedFac = nested(f, 'fac', fac=q)

Or as a lambda:

nestedFac = nested(f, 'fac', fac=lambda *args, **kwargs:
    nestedFac(*args, **kwargs))

Or for the special case of just one parameter:

nestedFac = nested(f, 'fac', fac=lambda n: nestedFac(n))

An extension to the nested() recipe which does this automatically would be the better approach, though. Feel free to fork the recipe and add this aspect! :-)