Generating custom data in FsCheck

Question

I’ve got an FsCheck question:
I have the following record type (and I say upfront, I have been told that my Single-Case DUs are maybe an overkill, but I find them descriptive of the domain, and therefore necessary, and I will not remove them unless must):

type Name = Name of string
type Quality = Quality of int
type ShelfLife = Days of int
type Style = Plain | Aged | Legendary

type Item = {
    Name: Name
    Quality: Quality
    ShelfLife: ShelfLife
    Style: Style
}

Assuming I already have defined the functions repeat: Int -> ('a -> 'a) -> 'a and decreaseQuality: Item -> Item, I want to code an FsCheck test that checks the invariant: Any item with style OTHER THAN Legendary, after 100 days have passed, has quality of 0.

My problem is I don’t know the following things about FsCheck:
1. How do I define a custom generator that generates items whose style is not Legendary? And in contrast, how do I define items only of type Legendary (to test both types)?

I’ve looked into:

 let itemGenerator = Arb.generate<Item>
 Gen.sample 80 5 itemGenerator

but that creates Items that are plain weird, as the size controller, in the example, 80, controls also the length of the Name of string (due to the of string) and also produces Quality and ShelfLife values that are unacceptable to my domain (i.e. negative) as they are both defined as ... of int which the size also controls.
(I’ve also looked into Gen.oneof..., but that turned out to also be a dud).

2. How do I even define a test that only tests the Quality property of the record, even assuming I found out a way to generate custom data?.

Thanks!

Answer 1

After resolving all the issues I had with understanding FsCheck usage (for the moment, I'm sure I'll have more issues in the future), this is the repo with my entire solution.

Obviously, the tests code is in the (I think aptly named) GildedRoseTests folder.

I have used the gen computation expression method, suggested by rmunn above, but a different experiment I did, using Tarmil's Arb method works just as well (and you get shrinking "for free").

Answer 2

Most of what you want becomes easy once you know how to use the gen { } computation expression to maximum effect.

First, I'll tackle how to generate a Style that isn't Legendary. You could use Gen.oneOf, but in this case I think it's simpler to use Gen.elements, since oneOf takes a sequence of generators to use, but elements just takes a list of items and generates one item from that list. So to generate a Style that isn't Legendary, I'd use Gen.elements [Plain; Aged]. (And to generate a Style that is Legendary, I'd just not use a generator and simply assign Legendary to the appropriate record field, but more on that later.)

As for names being too long, to limit the size of the strings produced to, say, a maximum of 15 characters, I'd use:

let genString15 = Gen.sized (fun s -> Gen.resize (min s 15) Arb.generate<string>)
// Note: "min" is not a typo. We want either s or 15, whichever is SMALLER
Gen.sample 80 5 genString15
// Never produces any strings longer than 15 characters

But this can still generate null strings, so I'd probably use this for my final version:

let genString15 =
    Gen.sized (fun s -> Gen.resize (min s 15) Arb.generate<NonNull<string>>)
    |> Gen.map (fun (NonNull x) -> x)  // Unwrap
Gen.sample 80 5 genString15
// Never produces any strings longer than 15 characters, AND never produces null

Now, since Quality and ShelfLife both can't be negative, I'd use either PositiveInt (where 0 isn't allowed either) or NonNegativeInt (which allows 0). Neither one is well documented in the FsCheck documentation, but they work like this:

let x = Arb.generate<NonNegativeInt>
Gen.sample 80 5 x
// Produces [NonNegativeInt 79; NonNegativeInt 75; NonNegativeInt 0;
//           NonNegativeInt 69; NonNegativeInt 16] which is hard to deal with
let y = Arb.generate<NonNegativeInt> |> Gen.map (fun (NonNegativeInt n) -> n)
Gen.sample 80 5 y
// Much better: [79; 75; 0; 69; 16]

To avoid duplicating code between generators for Quality and Days, I'd write something like the following:

let genNonNegativeOf (f : int -> 'a) = gen {
    let! (NonNegativeInt n) = Arb.generate<NonNegativeInt>
    return (f n)
}
Gen.sample 80 5 (genNonNegativeOf Quality)
// Produces: [Quality 79; Quality 35; Quality 2; Quality 42; Quality 73]
Gen.sample 80 5 (genNonNegativeOf Days)
// Produces: [Days 60; Days 27; Days 50; Days 22; Days 23]

And finally, let's tie that all together in a nice, elegant fashion with a gen { } CE:

let genNonLegendaryItem = gen {
    let! name = genString15 |> Gen.map Name
    let! quality = genNonNegativeOf Quality
    let! shelfLife = genNonNegativeOf Days
    let! style = Gen.elements [Plain; Aged]
    return {
        Name = name
        Quality = quality
        ShelfLife = shelfLife
        Style = style
    }
}
let genLegendaryItem =
    // This is the simplest way to avoid code duplication
    genNonLegendaryItem
    |> Gen.map (fun item -> { item with Style = Legendary })

Then once you've done that, to actually use this in your tests, you'll need to register the generators, as Tarmil mentioned in his answer. I'd probably use single-case DUs here so that the tests are easy to write, like this:

type LegendaryItem = LegendaryItem of Item
type NonLegendaryItem = NonLegendaryItem of Item

Then you'd register the genLegendaryItem and genNonLegendaryItem generators as producing (Non)LegendaryItem types by passing them through a Gen.map. And then your test cases would look like this (I'll use Expecto for my example here):

[<Tests>]
let tests =
    testList "Item expiration" [
        testProperty "Non-legendary items expire after 100 days" <| fun (NonLegendaryItem item) ->
            let itemAfter100Days = item |> repeat 100 decreaseQuality
            itemAfter100Days.Quality = Quality 0
        testProperty "Legendary items never expire" <| fun (LegendaryItem item) ->
            let itemAfter100Days = item |> repeat 100 decreaseQuality
            itemAfter100Days.Quality > Quality 0
    ]

Note that with this approach, you'd basically have to write the shrinkers yourself, whereas using Arb.convert as Tarmil suggested would get you shrinkers "for free". Don't underestimate the value of shrinkers, but if you find you can live without them, I like the nice, clean nature of the gen { } computation expression, and how easy it is to read the resulting code.

Answer 3

To always generate valid Quality and ShelfLife, you need to register Arbitrary instances:

type Arbs =
    static member Quality() =
        Arb.Default.NonNegativeInt()
        |> Arb.convert (fun (NonNegativeInt x) -> Quality x) (fun (Quality x) -> NonNegativeInt x)

    static member ShelfLife() =
        Arb.Default.NonNegativeInt()
        |> Arb.convert (fun (NonNegativeInt x) -> ShelfLife x) (fun (ShelfLife x) -> NonNegativeInt x)

Arb.register<Arbs>()

For the actual property you want to check, here's a rewording that will help translate it to FsCheck: IF Style is not Legendary, THEN after 100 days the quality is 0. In code:

let ``Non-legendary item breaks after 100 days`` (item: Item) =
    (item.Style <> Legendary) ==>
        let agedItem = item |> repeat 100 decreaseQuality
        agedItem.Quality = Quality 0