CODEWITHSUNDEEP
testingjulia
Alec
Alec

Reputation: 4482

How to create dynamically named testsets in Julia?

Using Julia's Test package, how can I create @testsets that reference a value at runtime?

E.g. I have

test_data = [
   (model="a",datapath="models/a.json"),
   (model="b",datapath="models/b.json"),
   ...
]

And I would like to do this:

@testset "models" begin
    for test in test_data
        @testset test.model begin
            
            # test range of values/functions here
        end
    end

end

But I get an error like this, which I assume is because I'm trying to define the testset name at runtime.

ERROR: LoadError: LoadError: LoadError: Unexpected argument test.model to @testset
Stacktrace:
 [1] error(::String) at .\error.jl:33
 [2] parse_testset_args(::Tuple{Expr}) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Test\src\Test.jl:1244
 [3] testset_beginend(::Tuple{Expr,Expr}, ::Expr, ::LineNumberNode) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Test\src\Test.jl:1087
 [4] @testset(::LineNumberNode, ::Module, ::Vararg{Any,N} where N) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Test\src\Test.jl:1079
 [5] include(::String) at .\client.jl:457
 [6] top-level scope at C:\MyPackage\runtests.jl:7
 [7] include(::String) at .\client.jl:457
 [8] top-level scope at none:6

Because I'm running hundereds of tests in each inner @testset it would be really nice to have the results grouped when running the tests. Also, when I get an error it tends to be in many results and my terminal loses the history above where the error starts, so printing the model name before each set of tests doesn't work very nicely either.

Upvotes: 3

Views: 228

Answers (1)

François Févotte
François Févotte

Reputation: 20248

You can interpolate the value of an expression in the test name using the standard $ syntax:


julia> using Test

julia> i = 1
1

julia> @testset "Model $i" begin
           @test true
       end
Test Summary: | Pass  Total
Model 1       |    1      1

But in your case it is probably better to use the @testset for ... end syntax rather than the more common @testset begin ... end (see the documentation for more detailed explanations):

julia> @testset "Model $i" for i in 1:2
           @test i == i
       end
Test Summary: | Pass  Total
Model 1       |    1      1
Test Summary: | Pass  Total
Model 2       |    1      1

In your specific use-case, it would look like:

julia> test_data = [
          (model="a",datapath="models/a.json"),
          (model="b",datapath="models/b.json"),
       ]
2-element Array{NamedTuple{(:model, :datapath),Tuple{String,String}},1}:
 (model = "a", datapath = "models/a.json")
 (model = "b", datapath = "models/b.json")

julia> @testset "Model $(test.model)" for test in test_data
           @test test.datapath == "models/$(test.model).json"
       end
Test Summary: | Pass  Total
Model a       |    1      1
Test Summary: | Pass  Total
Model b       |    1      1

Upvotes: 6

Related Questions