CODEWITHSUNDEEP
pythonmigrationportabilitypython-unicode
bignose
bignose

Reputation: 32279

Portable code: __import__ parameter string type between Python 2 and Python 3

What should I do, in a world where all text literals are Unicode by default, to make __import__ work in both Python 2 and 3?

I'm slowly learning about making Python code that will run under both Python 2 (version 2.6 or above) and Python 3 (version 3.2 or above).

This entails, I believe, the admonition to ensure text literals are Unicode by default:

from __future__ import unicode_literals

and to specify bytes literals explicitly with b'wibble' if needed.

The __import__ built-in function, though, is tripping up.

A contrived, trivial project layout:

$ mkdir fooproject/
$ cd fooproject/

$ mkdir foo/
$ printf "" > foo/__init__.py
$ mkdir foo/bar/
$ printf "" > foo/bar/__init__.py

Here's a simple fooproject/setup.py for that project:

from __future__ import unicode_literals

main_module_name = 'foo'
main_module = __import__(main_module_name, fromlist=['bar'])

assert main_module.bar

That fails under Python 2, but runs fine under Python 3:

$ python2 ./setup.py
Traceback (most recent call last):
  File "./setup.py", line 4, in <module>
    main_module = __import__(main_module_name, fromlist=['bar'])
TypeError: Item in ``from list'' not a string

$ python3 ./setup.py

We've deliberately made unadorned strings Unicode by default. By “not a string”, I presume Python 2 means “not a ‘bytes’ object”.

Okay, so we'll explicitly set that to a bytes literal:

from __future__ import unicode_literals

main_module_name = 'foo'
main_module = __import__(main_module_name, fromlist=[b'bar'])

assert main_module.bar

Now Python 2 is satisfied, but Python 3 complains:

$ python2 ./setup.py

$ python3 ./setup.py
Traceback (most recent call last):
  File "./setup.py", line 4, in <module>
    main_module = __import__(main_module_name, fromlist=[b'bar'])
  File "<frozen importlib._bootstrap>", line 2281, in
    _handle_fromlist
TypeError: hasattr(): attribute name must be string

So I've deliberately set unadorned strings to be Unicode by default, just as I'm supposed to; but that's apparently breaking the expectations of __import__ between Python 2 and Python 3.

How can I get that __import__ call, complete with its fromlist parameter, working correctly under both Python 2 and Python 3, keeping the unicode_literals setting?

Upvotes: 1

Views: 330

Answers (2)

John La Rooy
John La Rooy

Reputation: 304117

recall str works differently for python2 and python3 (see @BrenBarn's comment), so:

main_module = __import__(main_module_name, fromlist=[str('bar')])

or more generally

main_module = __import__(main_module_name, fromlist=list(map(str, ['bar'])))

Upvotes: 2

bignose
bignose

Reputation: 32279

The best I can come up with so far is a wrapper function to convert the type depending on Python version:

from __future__ import unicode_literals

import sys

fromlist_expects_type = str
if sys.version_info < (3, 0):
    fromlist_expects_type = bytes

def import_module(
        name, globals=None, locals=None, fromlist=(), level=0):
    """ Import specified module, together with options used by __import__.

        :param module_name: Text string of the module name to import.
        :param fromlist: List of names of attributes in the module to
            also import.
        :return: The module object.

        The built-in ``__import__`` function accepts a ``fromlist``
        parameter, but expects different string types between Python 2
        and Python 3. Python 2 only allows text string items; Python 3
        only allows byte string items.

        This function's ``fromlist`` parameter must have items of text
        string (Unicode) type only; the items are converted depending
        on the running Python version.

        """
    module_fromlist = ()
    if fromlist:
        module_fromlist = [
                fromlist_expects_type(attr_name) for attr_name in fromlist]
    module = __import__(
            name=name,
            globals=globals,
            locals=locals,
            fromlist=module_fromlist,
            level=level)

    return module

main_module_name = 'foo'
main_module = import_module(main_module_name, fromlist=['bar'])

That's pretty unwieldy, and I've probably made several mistakes. Surely this is a bug in Python that I need to do this?

Upvotes: 1

Related Questions