How to extend/overload range() function from Python 2.7 to accept floats?

Question

Our core application changed from Python 2.6 to Python 2.7 maybe to Python 3 in later release.

Range function from python is changed (quote from the Python 2.7 changelog).

The range() function processes its arguments more consistently; it will now call __int__() on non-float.

We allow users to add expressions / Python code based on results we process further.

Now how to change range function? Some of them are using float argument that are failing now in Python 2.7.

As code is written by users we cannot change in Python code/expression. There are 1000s of files. Some users may have their own files.

1. Is there is a way to extend the range() function from Python, such that it will take float arguments?

2. Another alternative is to parse python code and change float to int. It is very time consuming as it requires lot of sting manipulation and some range calls have formulas as parameter.

Our application build in C++, and we evaluate python expression using C++ python APIS

Answer 1

The change is not that calling __int__ on non-float. The change affecting you is that float arguments are not accepted any more in Python 2.7:

Python 2.6.9 (default, Sep 15 2015, 14:14:54) 
[GCC 5.2.1 20150911] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> range(10.0)
__main__:1: DeprecationWarning: integer argument expected, got float
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

See the DeprecationWarning - all this time your code has emitted warnings but you chose to ignore them. In Python 2.7:

Python 2.7.12 (default, Jul  1 2016, 15:12:24) 
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> range(10.0)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: range() integer end argument expected, got float.

The solution is to wrap the range() into a new function:

def py26_range(*args):
    args = [int(i) if isinstance(i, float) else i for i in args]
    return range(*args)

This function coerces floats into ints retaining the Python 2.6 behaviour. You then need to replace all usages of range with py26_range in those parts of code where the argument might be a float.

---

If you're desperate enough, you can install this version of range into __builtin__:

from functools import wraps
import __builtin__

@wraps(range)
def py26_range(*args):
    args = [int(i) if isinstance(i, float) else i for i in args]
    return range(*args)    

__builtin__.range = py26_range

Execute this before the other modules are even imported and it should work as before.

Answer 2

You have a serious migration issue. If you want to switch to Python 2.7 and even Python 3, there is basically no easy way around refactoring the existing (user) code base eventually. IMHO, you have the following options.

1. Provide a permanent (non-optional) 2.6 compatible interface to your users. That means they do not have to change anything, but it kind of defeats the purpose of upgrading to Python 2.7 since the users still have to satisfy Python 2.6 semantics. Verdict: Not recommended.

2. Provide a temporary (non-optional) 2.6 compatible interface for a limited amount of time. In that case the existing code needs to be refactored eventually. Verdict: Not recommended.

3. Make users include a flag in their code (e.g. a magic comment that can be identified safely without executing the file like # *$$ supertool-pythonversion: 2.7 $$*), which Python version the code expects to be run with and use 2.6 compatibility only for the files that have not been flagged with Python 2.7. That way, you can do whatever compatibility hacks are needed to run old files and run new files the way they are. Verdict: Increases complexity, but helps you doing the migration. Recommended.

However, you are in the convenient position of calling Python from C++. So you can control the environment that scripts are run with via the globals and locals dictionary passed to PyEval_EvalCode. In order to implement scenario 3, after checking the compatibility flag from the file, you can put a custom range function which supports float arguments into the gloabls dictionary before calling PyEval_EvalCode to "enable" the compatibility mode.

I am not proficient with Python's C API, but in Python this would look like this (and it is possible to do the same via the C API):

range27 = range

def range26(start=None, stop=None, step=None):
    if start is not None and not isinstance(start, int):
        start = int(start)
    if stop is not None and not isinstance(stop, int):
        stop = int(stop)
    if step is not None and not isinstance(step, int):
        step = int(step)
    return range27(start, stop, step)

def execute_user_code(user_file):
    ...
    src = read(user_file)
    global_dict = {}
    local_dict = {}
    ...

    if check_magic_version_comment(src) in (None, '2.6'):
        global_dict['range'] = range26
        global_dict['range27'] = range27
        # the last line is needed because the call
        # of range27 will be resolved against global_dict
        # when the user code is executed

    eval_code(src, global_dict, local_dict)
    ...