CODEWITHSUNDEEP
testinghaskellquickcheckhtf
Koterpillar
Koterpillar

Reputation: 8188

Test.QuickCheck: speed up testing multiple properties for the same type

I am testing a random generator generating instances of my own type. For that I have a custom instance of Arbitrary:

complexGenerator :: (RandomGen g) => g -> (MyType, g)

instance Arbitrary MyType where
    arbitrary = liftM (fst . complexGenerator . mkStdGen) arbitrary

This works well with Test.QuickCheck (actually, Test.Framework) for testing that the generated values hold certain properties. However, there are quite a few properties I want to check, and the more I add, the more time it takes to verify them all.

Is there a way to use the same generated values for testing every property, instead of generating them anew each time? I obviously still want to see, on failures, which property did not hold, so making one giant property with and is not optimal.

Upvotes: 4

Views: 1173

Answers (2)

John P. Feltz
John P. Feltz

Reputation: 51

In combination with the voted answer, what I've found helpful is using a Reader transformer with the Writer monad:

type Predicate r = ReaderT r (Writer String) Bool

The Reader "shared environment" is the tested input in this case. Then you can compose properties like this:

inv_even :: Predicate Int
inv_even = do
  lift . tell $ "this is the even invariant"
  (==) 0 . flip mod 2 <$> ask 

toLabeledProp :: r -> Predicate r -> Property
toLabeledProp cause r =
  let (effect, msg) = runWriter . (runReaderT r) $ cause in
    printTestCase ("inv: " ++ msg) . property $ effect

and combining:

fromPredicates :: [Predicate r] -> r -> Property
fromPredicates predicates cause =
  conjoin . map (toLabeledProp cause) $ predicates

I suspect there is another approach involving something similar to Either or a WriterT here- which would concisely compose predicates on different types into one result. But at the least, this allows for documenting properties which impose different post-conditions dependent on the the value of the input.

Edit: This idea spawned a library: http://github.com/jfeltz/quickcheck-property-comb

Upvotes: 1

dave4420
dave4420

Reputation: 47062

I obviously still want to see, on failures, which property did not hold, so making one giant property with and is not optimal.

You could label each property using printTestCase before making a giant property with conjoin.

e.g. you were thinking this would be a bad idea:

prop_giant :: MyType -> Bool
prop_giant x = and [prop_one x, prop_two x, prop_three x]

this would be as efficient yet give you better output:

prop_giant :: MyType -> Property
prop_giant x = conjoin [printTestCase "one" $ prop_one x,
                        printTestCase "two" $ prop_two x,
                        printTestCase "three" $ prop_three x]

(Having said that, I've never used this method myself and am only assuming it will work; conjoin is probably marked as experimental in the documentation for a reason.)

Upvotes: 3

Related Questions