CODEWITHSUNDEEP
scalascalacheck
Chris Stewart
Chris Stewart

Reputation: 1689

ScalaCheck specificy minimum successful tests for property

I'm trying to make sure that my ScalaCheck property runs 500 times instead of the default 100 times. I'm having trouble configuring this though. 

class BlockSpec extends Properties("BlockSpec") with BitcoinSLogger {

  val myParams = Parameters.default.withMinSuccessfulTests(500)
  override def overrideParameters(p: Test.Parameters) = myParams

  property("Serialization symmetry") =
  Prop.forAll(BlockchainElementsGenerator.block) { block =>
    logger.warn("Hex:" + block.hex)
    Block(block.hex) == block
  }
}

However when I actually run this test it only says 100 tests passed successfully

EDIT: 

$ sbt
[info] Loading project definition from /home/chris/dev/bitcoins-core/project
[info] Set current project to bitcoin-s-core (in build file:/home/chris/dev/bitcoins-core/)
> test-only *BlockSpec*
[info] + BlockSpec.Serialization symmetry: OK, passed 100 tests.
[info] Elapsed time: 1 min 59.775 sec 
[info] ScalaCheck
[info] Passed: Total 1, Failed 0, Errors 0, Passed 1
[info] ScalaTest
[info] Run completed in 2 minutes.
[info] Total number of tests run: 0
[info] Suites: completed 0, aborted 0
[info] Tests: succeeded 0, failed 0, canceled 0, ignored 0, pending 0
[info] No tests were executed.
[info] Passed: Total 1, Failed 0, Errors 0, Passed 1
[success] Total time: 123 s, completed Aug 1, 2016 11:36:17 AM
>

How do I actually pass this to my property?

Upvotes: 5

Views: 1951

Answers (3)

Aaron Mast
Aaron Mast

Reputation: 267

It looks a little different for me using munit. Here is what worked for me,

import munit.ScalaCheckSuite
import munit.FunSuite

class FoobarTest extends FunSuite with ScalaCheckSuite {
  property(
    "Foo does the bar"
  ) {
    forAll { (intValue: Int) =>
      assert(intValue == intValue)
    }
  }.check(_.withMinSuccessfulTests(5000))
}

Upvotes: 0

jopasserat
jopasserat

Reputation: 5930

As far as I understand you can specify the test parameters at two levels and they don't seem to communicate.

The first option is within the property as you're trying to do:

import org.scalacheck.Properties
import org.scalacheck.Test.{ TestCallback, Parameters }
import org.scalacheck.Prop.{ forAll, BooleanOperators }
import org.scalacheck.Test

class TestFoo extends Properties("BlockSpec") {

  override def overrideParameters(p: Parameters) = 
    p.withMinSuccessfulTests(1000000)

  property("Serialization symmetry") = forAll { n: Int =>
    (n > 0) ==> (math.abs(n) == n)
  }

}

This will have no impact as long as you don't call .check on the property. Can be from the sbt shell or directly within the class.

Now if you want to impact the number of tests run when calling the sbt:test target, it seems you have to play with options build.sbt (taken from here):

name := "scalacheck-demo"

scalaVersion := "2.11.5"

libraryDependencies += "org.scalacheck" %% "scalacheck" % "1.12.2" % "test"

testOptions in Test += Tests.Argument(TestFrameworks.ScalaCheck, "-maxSize", "5", "-minSuccessfulTests", "33", "-workers", "1", "-verbosity", "1")

Upvotes: 3

flavian
flavian

Reputation: 28511

There's definitely an easier way to achieve that than overriding any kind of global test config:

class SampleTest extends FlatSpec
  with Matchers with GeneratorDrivenPropertyChecks {

  it should "work for a basic scenario" in {
    // This will require 500 successful tests to succeed
    forAll(minSuccessful(500)) { (d: String) =>
      whenever (d.nonEmpty) {
        d.length shouldBe > 0
      }
    }
  }
}

Upvotes: 1

Related Questions