Understanding How the Compile Function Works in SBCL Common Lisp

Question

I was hoping someone can explain why the compile function is not working as I would expect.

First question:

* (compile 'square (lambda (x) (* x x)))
SQUARE
NIL
NIL

But then:

* (square 3)
; in: SQUARE 3
;     (SQUARE 3)
;
; caught STYLE-WARNING:
;   undefined function: COMMON-LISP-USER::SQUARE
;
; compilation unit finished
;   Undefined function:
;     SQUARE
;   caught 1 STYLE-WARNING condition

* (describe 'square)
COMMON-LISP-USER::SQUARE
  [symbol]

SQUARE names an undefined function
  Assumed type: FUNCTION

What does this mean?

Second question, starting over:

* (defun square (x) (* x x x))  ;faulty definition
    SQUARE
 * (compile 'square (lambda (x) (* x x)))  ;attempted correction
    SQUARE
    NIL
    NIL
    * (square 3)
    27

Here the old definition is still in place. However, the hyperspec says about compile: "If a non-nil name is given, then the resulting compiled function replaces the existing function definition of name and the name is returned as the primary value;".

(ps: my goal is to define (or update) a function programmatically, but have it be recognized by the sbcl static profiler--otherwise, it is ignored. An alternative is to use the statistical profiler, but it doesn't seem to work in windows-64.)

Edit 5/26/24 & 5/27/24: Thanks for the enlightening discussions. I tend to agree with ignis volens that sbcl's implementation of compile could be improved. Maybe someone with a deeper understanding of the details could discuss this with the maintainers. Re my initial objective of building lambda expressions that are recognized by the sbcl static profiler, I have a simplified example that for better or worse seems to work and readily integrates into existing structures, so I'll use this unless there's a downside:

* (defmacro init-fns (name1 name2)
  `(progn (defun ,name1 ())  ;define dummy fns at top-level
          (defun ,name2 ())))
INIT-FNS
* (init-fns square cube)
CUBE
* (defun build-fns ()
  (setf (symbol-function 'square)
        (compile nil '(lambda (x) (* x x))))
  (setf (symbol-function 'cube)
        (compile nil '(lambda (x) (* x x x)))))
BUILD-FNS
* (build-fns)
#<FUNCTION (LAMBDA (X)) {2479F75B}>
* (square 3)
9
* (cube 3)
27

Answer 1

I think this is a bug in SBCL. In particular it seems that if you pass compile a compiled function as its second argument, things don't work properly.

This works the way you would expect (each example run in a cold post-2.4.5 SBCL):

> (fboundp 'square)
nil
> (compile 'square '(lambda (x) (* x x)))
square
nil
nil
> (fboundp 'square)
#<function square>
> (square 10)
100

But

> (fboundp 'square)
nil
> (compile 'square (lambda (x) (* x x)))
square
nil
nil
> (fboundp 'square)
nil
> (type-of (lambda (x) (* x x)))
compiled-function

Equivalent things apply in your redefinition case.

SBCL normally does not really have interpreted functions, but the standard is prepared for this case:

compile produces a compiled function from definition. If the definition is a lambda expression, it is coerced to a function. If the definition is already a compiled function, compile either produces that function itself (i.e., is an identity operation) or an equivalent function.

If the name is nil, the resulting compiled function is returned directly as the primary value. If a non-nil name is given, then the resulting compiled function replaces the existing function definition of name and the name is returned as the primary value; if name is a symbol that names a macro, its macro function is updated and the name is returned as the primary value.

From COMPILE, my emphasis.

So SBCL is not dealing properly with the case where the definition (a) is not already the definition associated with name and (b) is already a compiled function. I would report this as a bug.

It does work properly if the definition is an interpreted function. You can force this to happen by setting *evaluator-mode* to :interpret:

> (setf *evaluator-mode* :interpret)
:interpret
> (compile 'square (lambda (x) (* x x)))
square
nil
nil
> (fboundp 'square)
#<function (lambda (x) nil) {7007117D6B}>
> (square 100)
10000
> (type-of #'square)
compiled-function
> (type-of (lambda (x) (* x x)))
sb-kernel:interpreted-function

So it looks like the bug is just in the compiled function case.

In practice, if what you want to install is a bit of source code in the form of a lambda expression, then you should be fine in this case, because SBCL does handle that case properly, as you can see above. or you could write something like this:

(defun compile-and-install (name definition)
  ;; Does not handle the macro case that COMPILE does
  (setf (fdefinition name)
        (typecase definition
          (compiled-function definition)
          (t (compile nil definition))))
  name)

Answer 2

Compiling an already compiled function in SBCL

The problem is very subtle:

(compile 'square (lambda (x) (* x x)))

COMPILE is a function. Now look at (lambda (x) (* x x)). What is it evaluated?

> (compiled-function-p (lambda (x) (* x x)))
T

It is a compiled function!

In SBCL all functions are by default compiled.

Now we call COMPILE on an already compiled function.

What's going on?

• we call COMPILE with a name and a compiled function
• SBCL does not compile the function, it is already compiled
• SBCL does not set the name, since it hasn't compiled the already compiled function

One can argue that it nevertheless should set the function. Best to discuss that with the maintainers

Compiling source code

This works:

CL-USER> (compile 'square '(lambda (x) (* x x)))
SQUARE
NIL
NIL

CL-USER> #'square
#<FUNCTION SQUARE>

It works because '(lambda (x) (* x x)) evaluates to a list, which is a valid LAMBDA macro form. The compiler will compile this source successfully and then set the symbol function of SQUARE to the resulting compiled function.

Compare with another CL implementation, here LispWorks

Compare with LispWorks, which does not compile functions by default.

CL-USER 134 > (let ((f (compile nil '(lambda (x) (* x x)))))
                (compile 'square f))
;;;*** Warning in SQUARE: The definition supplied for SQUARE is already compiled.
SQUARE
((SQUARE #<CONDITIONS::SIMPLE-STYLE-WARNING 80101F2C9B>))
NIL

CL-USER 135 > (symbol-function 'square)
#<Function 14 8020000CB9>

LispWorks warns that the function is already compiled and sets the symbol function of SQUARE anyway.