CODEWITHSUNDEEP
pythonunicodedoctest
saffsd
saffsd

Reputation: 24292

How do I include unicode strings in Python doctests?

I am working on some code that has to manipulate unicode strings. I am trying to write doctests for it, but am having trouble. The following is a minimal example that illustrates the problem:

# -*- coding: utf-8 -*-
def mylen(word):
  """
  >>> mylen(u"áéíóú")
  5
  """
  return len(word)

print mylen(u"áéíóú")

First we run the code to see the expected output of print mylen(u"áéíóú").

$ python mylen.py
5

Next, we run doctest on it to see the problem.

$ python -m
5
**********************************************************************
File "mylen.py", line 4, in mylen.mylen
Failed example:
    mylen(u"áéíóú")
Expected:
    5
Got:
    10
**********************************************************************
1 items had failures:
   1 of   1 in mylen.mylen
***Test Failed*** 1 failures.

How then can I test that mylen(u"áéíóú") evaluates to 5?

Upvotes: 21

Views: 4782

Answers (5)

Pieter Ennes
Pieter Ennes

Reputation: 2409

As already mentioned, you need to ensure your docstrings are Unicode.

If you can switch to Python 3, then it would work automatically there, as both the source encoding is already utf-8 and the default string type is Unicode.

To achieve the same in Python 2, you need to keep the coding: utf-8 next to which you can either prefix all docstrings with u, or simply add

from __future__ import unicode_literals

Upvotes: 1

dmitry_romanov
dmitry_romanov

Reputation: 5425

Python 2.6.6 doesn't understand unicode output very well, but this can be fixed using:

  • already described hack with sys.setdefaultencoding("UTF-8")
  • unicode docstring (already mentioned above too, thanks a lot)
  • AND print statement.

In my case this docstring tells that test is broken:

def beatiful_units(*units):
    u'''Returns nice string like 'erg/(cm² sec)'.

    >>> beatiful_units(('erg', 1), ('cm', -2), ('sec', -1))
    u'erg/(cm² sec)'
    '''

with "error" message

Failed example:
    beatiful_units(('erg', 1), ('cm', -2), ('sec', -1))
Expected:
    u'erg/(cm² sec)'
Got:
    u'erg/(cm\xb2 sec)'

Using print we can fix that:

def beatiful_units(*units):
    u'''Returns nice string like 'erg/(cm² sec)'.

    >>> print beatiful_units(('erg', 1), ('cm', -2), ('sec', -1))
    erg/(cm² sec)
    '''

Upvotes: 6

Andrew Dalke
Andrew Dalke

Reputation: 15305

My solution was to escape the unicode characters, like u'\xe1\xe9\xed\xf3\xfa'. Wasn't as easy to read though, but my tests only had a few non-ASCII characters so in those cases I put the description to the side as a comment, like "# n with tilde".

Upvotes: 1

u0b34a0f6ae
u0b34a0f6ae

Reputation: 49803

If you want unicode strings, you have to use unicode docstrings! Mind the u!

# -*- coding: utf-8 -*-
def mylen(word):
  u"""        <----- SEE 'u' HERE
  >>> mylen(u"áéíóú")
  5
  """
  return len(word)

print mylen(u"áéíóú")

This will work -- as long as the tests pass. For Python 2.x you need yet another hack to make verbose doctest mode work or get correct tracebacks when tests fail:

if __name__ == "__main__":
    import sys
    reload(sys)
    sys.setdefaultencoding("UTF-8")
    import doctest
    doctest.testmod()

NB! Only ever use setdefaultencoding for debug purposes. I'd accept it for doctest use, but not anywhere in your production code.

Upvotes: 21

Ned Deily
Ned Deily

Reputation: 85045

This appears to be a known and as yet unresolved issue in Python. See open issues here and here.

Not surprisingly, it can be modified to work OK in Python 3 since all strings are Unicode there:

def mylen(word):
  """
  >>> mylen("áéíóú")
  5
  """
  return len(word)

print(mylen("áéíóú"))

Upvotes: 2

Related Questions