CODEWITHSUNDEEP
common-lispperformance-testingsbcl
claudio
claudio

Reputation: 168

Why is this lisp benchmark (in sbcl) so slow?

Since I’m interested in C++ as well as in Lisp, I tried to reproduce the benchmark presented here written by Guillaume Michel. The benchmark is basically a DAXPY BLAS Level 1 operation performed multiple times on big arrays. The full code was thankfully posted on github and is in both languages roughly one page.

Sadly, I discovered that it was not possible for me to reach the velocity of his calculations for lisp.

For Linux, he got similar results for C++ as well as Lisp:

Size | C++ | Common Lisp

100,000,000 | 181.73 | 183.9

The numbers differ (naturally) both on my PC:

Size | C++ | Common Lisp

100,000,000 | 195.41 | 3544.3

Since I wanted an additional measurement, I started both programs with the time command and got (shortened up):

$ time ./saxpy
real    0m7.381s
$ time ./saxpy_lisp
real    0m40.661s

I assumed different reasons for this blatant difference. I scanned through both code samples, but found no big algorithmic or numeric difference between the C++ and Lisp Version. Then I thought, the usage of buildapp created the delay, so I started the benchmark directly in the REPL. My last resort was to try another version of sbcl, so I downloaded the newest sbcl-1.3.11 and evaluated it there – still the (optimized) Lisp version needs much longer than it’s C++ counterpart.

What am I missing?

Upvotes: 3

Views: 2074

Answers (1)

Sylwester
Sylwester

Reputation: 48745

I cannot replicate your findings:

sylwester@pus:~/a/bench-saxpy:master$ sbcl --dynamic-space-size 14000
This is SBCL 1.3.1.debian, an implementation of ANSI Common Lisp.
More information about SBCL is available at <http://www.sbcl.org/>.

SBCL is free software, provided as is, with absolutely no warranty.
It is mostly in the public domain; some portions are provided under
BSD-style licenses.  See the CREDITS and COPYING files in the
distribution for more information.
* (compile-file "saxpy.lisp")

; compiling file "/pussycat/natty-home/westerp/apps/bench-saxpy/saxpy.lisp" (written 06 NOV 2016 12:04:30 AM):
; compiling (DEFUN SAXPY ...)
; compiling (DEFUN DAXPY ...)
; compiling (DEFMACRO TIMING ...)
; compiling (DEFUN BENCH ...)
; compiling (DEFUN MAIN ...)

; /pussycat/natty-home/westerp/apps/bench-saxpy/saxpy.fasl written
; compilation finished in 0:00:00.038
#P"/pussycat/natty-home/westerp/apps/bench-saxpy/saxpy.fasl"
NIL
NIL
* (load "saxpy.fasl")

T
* (main t)
Size, Execution Time (ms)
10, 0.0
100, 0.0
1000, 0.0
10000, 0.1
100000, 0.49999997
1000000, 3.6
10000000, 39.2
100000000, 346.30002
(NIL NIL NIL NIL NIL NIL NIL NIL)

So on my machine it took 346ms (relatively old machine). The whole test took about 5 seconds but that is a series of many tests. Interesting that running from sbcl was faster than making an image and running it:

sylwester@pus:~/a/bench-saxpy:master$ make lisp
buildapp --output saxpy_lisp --entry main --load saxpy.lisp --dynamic-space-size 14000
;; loading file #P"/pussycat/natty-home/westerp/apps/bench-saxpy/saxpy.lisp"
[undoing binding stack and other enclosing state... done]
[saving current Lisp image into saxpy_lisp:
writing 4944 bytes from the read-only space at 0x20000000
writing 3168 bytes from the static space at 0x20100000
writing 60522496 bytes from the dynamic space at 0x1000000000
done]
sylwester@pus:~/a/bench-saxpy:master$ ./saxpy_lisp 
Size, Execution Time (ms)
10, 0.0
100, 0.0
1000, 0.0
10000, 0.0
100000, 0.4
1000000, 3.5
10000000, 40.2
100000000, 369.49997

In the C version I get:

100000000, 296.693634, 295.762695, 340.574860

It seems like the C version actually calculates the same in 3 different ways and reports the time it took. Using time wouldn't do it justice.

Upvotes: 4

Related Questions