CODEWITHSUNDEEP
pythonpython-hypothesisproperty-based-testing
tommsch
tommsch

Reputation: 688

How does python-hypothesis shrink a deferred strategy

I am at the moment implementing PBB for Matlab and am somehow influenced by hypothesis.

I do not understand how hypothesis handles the shrinking of deferred strategies. In the documentation there is the code snippet

import hypothesis.strategies as st
x = st.deferred(lambda: st.booleans() | st.tuples(x, x))
x.example()
>>> (((False, (True, True)), (False, True)), (True, True))

Now, this example should most likely shrink by reducing the depth of recursion. But, how does hypothesis know how to manipulate the lambda such that the example gets shrunk?

Question after answer of DRMacIver:

Upvotes: 0

Views: 315

Answers (1)

DRMacIver
DRMacIver

Reputation: 2315

Shrinking a deferred strategy works the same as shrinking any other strategy, because shrinking in Hypothesis works uniformly on an underlying representation rather than needing to know much of anything about the strategies being used.

Instead of manipulating the generated values, Hypothesis modifies the choices used to construct them. You can think of this as making a series of coin flips. e.g. (True, False) might be generated by the sequence 10100 which is 1 (pick the second branch of the |), followed by 01 which picks the first branch then generates a True and 00 which picks the first branch then generates a False. We could reduce (True, False) to True by replacing 10100 with 01 or to False by replacing it with 00 both of which we can find as subsequences of the original sequence of choices.

If you want to know more about this you can read the paper about it (or watch the talk): https://2020.ecoop.org/details/ecoop-2020-papers/13/Test-Case-Reduction-via-Test-Case-Generation-Insights-From-the-Hypothesis-Reducer

Is hypothesis storing which choices belong to which strategy? E.g.: (False,(False,False)) can be constructed as 10010000 (depth first). If we take the subsequence 01 (where the first 0 belongs to the strategy booleans instead to tuples now) we would get the example True, which probably does not count as a shrinked version of the former.

No, Hypothesis doesn't have any particular notion of "ownership" of choices and considered True to be a perfectly valid shrink of (False, (False, False))! It is after all considerably smaller and simpler.

Upvotes: 3

Related Questions