CODEWITHSUNDEEP
f#
Marko Grdinić
Marko Grdinić

Reputation: 4062

Is F# Interactive supposed to be much slower than compiled?

I've been using F# for nearly six months and have been so sure that F# Interactive should have the same performance as compiled, that when I bothered to benchmark it, I was convinced it was some kind of compiler bug. Though now it occurs to me that I should have checked here first before opening an issue.

For me it is roughly 3x slower and the optimization switch does not seem to be doing anything at all.

Is this supposed to be standard behavior? If so, I really got trolled by the #time directive. I have the timings for how long it takes to sum 100M elements on this Reddit thread.

Update:

Thanks to FuleSnabel, I uncovered some things.

I tried running the example script from both fsianycpu.exe (which is the default F# Interactive) and fsi.exe and I am getting different timings for two runs. 134ms for the first and 78ms for the later. Those two timings also correspond to the timings from unoptimized and optimized binaries respectively.

What makes the matter even more confusing is that the first project I used to compile the thing is a part of the game library (in script form) I am making and it refuses to compile the optimized binary, instead switching to the unoptimized one without informing me. I had to start a fresh project to get it to compile properly. It is a wonder the other test compiled properly.

So basically, something funky is going on here and I should look into switching fsianycpu.exe to fsi.exe as the default interpreter.

Upvotes: 1

Views: 407

Answers (1)

Just another metaprogrammer
Just another metaprogrammer

Reputation: 11577

I tried the example code in pastebin I don't see the behavior you describe. This is the result from my performance run:

.\bin\Release\ConsoleApplication3.exe
Total iterations: 300000000, Outer: 10000, Inner: 30000
reduce sequence of list, result 450015000, time 2836 ms
reduce array, result 450015000, time 594 ms
for loop array, result 450015000, time 180 ms
reduce list, result 450015000, time 593 ms
fsi -O --exec .\Interactive.fsx
Total iterations: 300000000, Outer: 10000, Inner: 30000
reduce sequence of list, result 450015000, time 2617 ms
reduce array, result 450015000, time 589 ms
for loop array, result 450015000, time 168 ms
reduce list, result 450015000, time 603 ms

It's expected that Seq.reduce would be the slowest, the for loop the fastest and that the reduce on list/array is roughly similar (this assumes locality of list elements which isn't guaranteed).

I rewrote your code to allow for longer runs w/o running out of memory and to improve cache locality of data. With short runs the uncertainity of measurements makes it hard to compare the data.

Program.fs:

module fs

let stopWatch = 
  let sw = new System.Diagnostics.Stopwatch()
  sw.Start ()
  sw

let total = 300000000
let outer = 10000
let inner = total / outer

let timeIt (name : string) (a : unit -> 'T) : unit =
  let t = stopWatch.ElapsedMilliseconds
  let v = a ()
  for i = 2 to outer do
    a () |> ignore
  let d = stopWatch.ElapsedMilliseconds - t
  printfn "%s, result %A, time %d ms" name v d

[<EntryPoint>]
let sumTest(args) = 
  let numsList = [1..inner]
  let numsArray = [|1..inner|]

  printfn "Total iterations: %d, Outer: %d, Inner: %d" total outer inner

  let sumsSeqReduce () = Seq.reduce (+) numsList
  timeIt "reduce sequence of list" sumsSeqReduce

  let sumsArray () = Array.reduce (+) numsArray
  timeIt "reduce array" sumsArray

  let sumsLoop () =
    let mutable total = 0
    for i in 0 .. inner - 1 do
      total <- total + numsArray.[i]
    total

  timeIt "for loop array" sumsLoop

  let sumsListReduce () = List.reduce (+) numsList

  timeIt "reduce list" sumsListReduce

  0

Interactive.fsx:

#load "Program.fs"
fs.sumTest [||]

PS. I am running on Windows with Visual Studio 2015. 32bit or 64bit seemed to make only marginal difference

Upvotes: 2

Related Questions