CODEWITHSUNDEEP
timejulia
jboy
jboy

Reputation: 209

Difference between the built-in @time macros and the ones from the benchmark module

In the Julia package BenchmarkTools, there are macros like @btime, @belapse that seem redundant to me since Julia has built-in @time, @elapse macros. And it seems to me that these macros serve the same purpose. So what's the difference between @time and @btime, and @elapse and @belapsed?

Upvotes: 2

Views: 582

Answers (1)

Przemyslaw Szufel
Przemyslaw Szufel

Reputation: 42214

TLDR ;)

@time and @elapsed just run the code once and measure the time. This measurement may or may not include the compile time (depending whether @time is run for the first or second time) and includes time to resolve global variables.

On the the other hand @btime and @belapsed perform warm up so you know that compile time and global variable resolve time (if $ is used) do not affect the time measurement.

Details

For further understand how this works lets used the @macroexpand (I am also stripping comment lines for readability):

julia> using MacroTools, BenchmarkTools

julia> MacroTools.striplines(@macroexpand1 @elapsed sin(x))
quote
    Experimental.@force_compile
    local var"#28#t0" = Base.time_ns()
    sin(x)
    (Base.time_ns() - var"#28#t0") / 1.0e9
end

Compilation if sin is not forced and you get different results when running for the first time and subsequent times. For an example:

julia> @time cos(x);
  0.110512 seconds (261.97 k allocations: 12.991 MiB, 99.95% compilation time)

julia> @time cos(x);
  0.000008 seconds (1 allocation: 16 bytes)

julia> @time cos(x);
  0.000006 seconds (1 allocation: : 16 bytes)

The situation is different with @belapsed:

julia> MacroTools.striplines(@macroexpand @belapsed sin($x))
quote
    (BenchmarkTools).time((BenchmarkTools).minimum(begin
                    local var"##314" = begin
                                BenchmarkTools.generate_benchmark_definition(Main, Symbol[], Any[], [Symbol("##x#315")], (x,), $(Expr(:copyast, :($(QuoteNode(:(sin(var"##x#315"))))))), $(Expr(:copyast, :($(QuoteNode(nothing))))), $(Expr(:copyast, :($(QuoteNode(nothing))))), BenchmarkTools.Parameters())
                            end
                    (BenchmarkTools).warmup(var"##314")
                    (BenchmarkTools).tune!(var"##314")
                    (BenchmarkTools).run(var"##314")
                end)) / 1.0e9
end

You can see that a minimum value is taken (the code is run several times). Basically most time you should use BenchmarkTools for measuring times when designing your application.

Last but not least try @benchamrk:

julia> @benchmark sin($x)
BenchmarkTools.Trial: 10000 samples with 999 evaluations.
 Range (min … max):  13.714 ns … 51.151 ns  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     13.814 ns              ┊ GC (median):    0.00%
 Time  (mean ± σ):   14.089 ns ±  1.121 ns  ┊ GC (mean ± σ):  0.00% ± 0.00%

  █▇  ▂▄   ▁▂   ▃          ▁                                  ▂
  ██▆▅██▇▅▄██▃▁▃█▄▃▁▅█▆▁▄▃▅█▅▃▁▄▇▆▁▁▁▁▁▆▄▄▁▁▃▄▇▃▁▃▁▁▁▆▅▁▁▁▆▅▅ █
  13.7 ns      Histogram: log(frequency) by time        20 ns <

 Memory estimate: 0 bytes, allocs estimate: 0.

Upvotes: 6

Related Questions