python hypothesis unit test using faker

Question

I have a problem with python unit test by using hypothesis additional package, Faker. I want to test login process of my website, I already have the unit test scenario, but I want to automated the scenario with hypothesis. This is my simple code for email and password validation.

import unittest

from Environtments.ParametrizedTestCase import ParametrizedTestCase
from hypothesis import given
from faker import Faker

class TestLogin(ParametrizedTestCase):
    fake = Faker('en_US')
    @given(email = fake.email(),password = fake.password()
    def test_login(self,email,password):
        assert '@' in email
        assert len(password) >= 6

if __name__ == '__main__':
    suite = unittest.TestSuite()
    suite.addTest(ParametrizedTestCase.parametrize(TestLogin, param=""))
    unittest.TextTestRunner(verbosity=2).run(suite)

I always find this error. Is there something I missed?

test_login (__main__.TestLogin) ... You can add @seed(311936867547523412638507752560457398354) to this test to reproduce this failure.
ERROR

======================================================================
ERROR: test_login (__main__.TestLogin)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tester_uf_exmp_hypo.py", line 10, in test_login
    def test_login(self,email,password):
  File "/Library/Python/2.7/site-packages/hypothesis/core.py", line 1001, in wrapped_test
    state.run()
  File "/Library/Python/2.7/site-packages/hypothesis/core.py", line 725, in run
    runner.run()
  File "/Library/Python/2.7/site-packages/hypothesis/internal/conjecture/engine.py", line 435, in run
    self._run()
  File "/Library/Python/2.7/site-packages/hypothesis/internal/conjecture/engine.py", line 771, in _run
    self.reuse_existing_examples()
  File "/Library/Python/2.7/site-packages/hypothesis/internal/conjecture/engine.py", line 626, in reuse_existing_examples
    self.test_function(last_data)
  File "/Library/Python/2.7/site-packages/hypothesis/internal/conjecture/engine.py", line 153, in test_function
    self._test_function(data)
  File "/Library/Python/2.7/site-packages/hypothesis/core.py", line 691, in evaluate_test_data
    escalate_hypothesis_internal_error()
  File "/Library/Python/2.7/site-packages/hypothesis/core.py", line 663, in evaluate_test_data
    result = self.execute(data, collect=True)
  File "/Library/Python/2.7/site-packages/hypothesis/core.py", line 578, in execute
    result = self.test_runner(data, run)
  File "/Library/Python/2.7/site-packages/hypothesis/executors.py", line 58, in default_new_style_executor
    return function(data)
  File "/Library/Python/2.7/site-packages/hypothesis/core.py", line 552, in run
    args, kwargs = data.draw(self.search_strategy)
  File "/Library/Python/2.7/site-packages/hypothesis/internal/conjecture/data.py", line 158, in draw
    return self.__draw(strategy, label=label)
  File "/Library/Python/2.7/site-packages/hypothesis/internal/conjecture/data.py", line 175, in __draw
    return strategy.do_draw(self)
  File "/Library/Python/2.7/site-packages/hypothesis/core.py", line 180, in do_draw
    return self.base.do_draw(data)
  File "/Library/Python/2.7/site-packages/hypothesis/searchstrategy/lazy.py", line 157, in do_draw
    return data.draw(self.wrapped_strategy)
  File "/Library/Python/2.7/site-packages/hypothesis/internal/conjecture/data.py", line 148, in draw
    if strategy.is_empty:
  File "/Library/Python/2.7/site-packages/hypothesis/searchstrategy/strategies.py", line 144, in accept
    recur(self)
  File "/Library/Python/2.7/site-packages/hypothesis/searchstrategy/strategies.py", line 141, in recur
    mapping[strat] = getattr(strat, calculation)(recur)
  File "/Library/Python/2.7/site-packages/hypothesis/searchstrategy/collections.py", line 67, in calc_is_empty
    return any(recur(e) for e in self.element_strategies)
  File "/Library/Python/2.7/site-packages/hypothesis/searchstrategy/collections.py", line 67, in <genexpr>
    return any(recur(e) for e in self.element_strategies)
  File "/Library/Python/2.7/site-packages/hypothesis/searchstrategy/strategies.py", line 141, in recur
    mapping[strat] = getattr(strat, calculation)(recur)
  File "/Library/Python/2.7/site-packages/hypothesis/searchstrategy/strategies.py", line 541, in calc_is_empty
    return recur(self.mapped_strategy)
  File "/Library/Python/2.7/site-packages/hypothesis/searchstrategy/strategies.py", line 141, in recur
    mapping[strat] = getattr(strat, calculation)(recur)
  File "/Library/Python/2.7/site-packages/hypothesis/searchstrategy/lazy.py", line 87, in calc_is_empty
    return recur(self.wrapped_strategy)
  File "/Library/Python/2.7/site-packages/hypothesis/searchstrategy/lazy.py", line 110, in wrapped_strategy
    *self.__args, **self.__kwargs
  File "/Library/Python/2.7/site-packages/hypothesis/strategies.py", line 651, in fixed_dictionaries
    check_strategy(v)
  File "/Library/Python/2.7/site-packages/hypothesis/internal/validation.py", line 45, in check_strategy
    check_type(SearchStrategy, arg, name)
  File "/Library/Python/2.7/site-packages/hypothesis/internal/validation.py", line 39, in check_type
    % (typ_string, name, arg, type(arg).__name__))
InvalidArgument: Expected SearchStrategy but got u'2)Y4+APqu@' (type=unicode)

----------------------------------------------------------------------
Ran 1 test in 2.013s

FAILED (errors=1)

I change the code to this.

from hypothesis.extra.fakefactory import fake_factory    
@given(email = fake_factory('email'),password = fake_factory('password'))

This is the warn. But the faker works. Is this will be a problem?

tester_uf_exmp_hypo.py:9: HypothesisDeprecationWarning: hypothesis.extra.fakefactory has been deprecated, because it does not support example discovery or shrinking.  Consider using a lower-level strategy (such as st.text()) instead.
  @given(email = fake_factory('email'),password = fake_factory('password'))
test_login (__main__.TestLogin) ... kaareberge@pedersen.no
!l3QCGDd8I
myslrshd@bn.com
(@s(RwUM0z

Answer 1

Your first case is calling @given with arguments of the wrong type - Hypothesis requires them to be "strategies" for generating example values, rather than an value itself. You can see this at the bottom of the traceback - got u'2)Y4+APqu@' (type=unicode) is the result of fake.password(), rather than (eg) hypothesis.strategies.text().

The second code snippet does work - hurray!

However - and here's why I wrote that deprecation warning - it's giving up most of the advantages of Hypothesis: because this is just a wrapper around fake-factory, examples can't be minimised and the testing process doesn't have enough information to do coverage-guided optimisation.

Instead, you can think about what a password is - in this case, perhaps "a unicode string with between 6 and 40 characters". In Hypothesis, that would be:

import hypothesis.strategies as st

@given(password=st.text(min_size=6, max_size=40))
def test(password): 
    ...

And then you get shrinking, profile-guided example selection, and a whole lot more :D

"Email addresses" are also strings, but with more structure - luckily, Hypothesis has that covered too. You can compose strategies from parts with .map(), or use the from_regex() strategy, for example, or just copy our internal implementation from hypothesis.provisional.emails (or even just import it, if you don't mind unstable APIs that could be removed at any time).

TLDR: you can mix Hypothesis and fake-factory, but I'd just use Hypothesis.