Compile expansion mystery of macro in Elisp

Question

I always have confusion about macro in Emacs. There has been many documents about how to use macro. But the documents only mention the surface and examples are often too simple. In addition, it is quite hard to search macro itself. It will come up with keyboard macro results.

People always say macro is expanded at compile time and it is as fast as writing the same copy of code directly. Is this always true?

I start with the example when.

(pp-macroexpand-expression
 '(when t
    (print "t")))

;; (if t
;;     (progn
;;       (print "t")))

When we use when during compiling, the (if t .....) is inserted to our code directly, right?

Then, I find a more complicated example dotimes from subr.el. I simplified the code a bit:

(defmacro dotimes (spec &rest body)

  (declare (indent 1) (debug dolist))

  (let ((temp '--dotimes-limit--)
        (start 0)
        (end (nth 1 spec)))

    `(let ((,temp ,end)
           (,(car spec) ,start))
       (while (< ,(car spec) ,temp)
         ,@body
         (setq ,(car spec) (1+ ,(car spec))))
       ,@(cdr (cdr spec)))))

What drew my attention is the (end (nth 1 spec)). I think this part must be done at runtime. When this is done at runtime, it implies that the code expansion cannot be done at compile time. To test with it, I modified the dotimes a bit and byte-compile the file.

(defmacro my-dotimes (spec &rest body)
  (declare (indent 1) (debug dolist))
  (let ((temp '--dotimes-limit--)
        (start 0)
        ;; This is my test
        (end (and (print "test: ")(print (nth 1 spec)) (nth 1 spec))))
    `(let ((,temp ,end)
           (,(car spec) ,start))
       (while (< ,(car spec) ,temp)
         ,@body
         (setq ,(car spec) (1+ ,(car spec))))
       ,@(cdr (cdr spec)))))

(provide 'my)

Result

(require 'my)
(my-dotimes (var 3)
  (print "dotimes"))

;; "test: "
;; 3
;; "dotimes"
;; "dotimes"
;; "dotimes"

Indeed, my test statement is done at runtime. When it comes to expansion:

(pp-macroexpand-expression
 '(my-dotimes (var 3)
    (print "dotimes")))

;; (let
;;     ((--dotimes-limit-- 3)
;;      (var 0))
;;   (while
;;       (< var --dotimes-limit--)
;;     (print "dotimes")
;;     (setq var
;;           (1+ var))))

Surprisingly my test part lost.

So is dotimes expanded at runtime?

If yes, does it mean it losses the advantage of a generic macro that macro is as fast as writing the same code directly?

What does the interpreter do when it meets macro that have runtime components and?

Answer 1

Just my two cents...

People always say macro is expanded at compile time and it is as fast as writing the same copy of code directly. Is this always true?

It's not exactly always true.

If you compile a function whose code contains a call to a macro, it's true.

But, if you don't compile it, it's almost true.

Suppose you define a function using your modified version of dotimes, like:

(defun foo (x)
  (my-dotimes (var x)
    (print "dotimes")))

and you don't compile it.

If you call only once (foo 5), the interpreter will have to expand the macro before evaluating the expanded code, so you will see the prints and it will be slower than if you had written yourself the expanded code inside the function.

But now, the symbol my-times does not appear anymore in the definition of foo. The list that contained it has been replaced by the result of its expansion.

So, if you call again foo, like (foo 3), now you don't see anymore the prints and it will be as fast as if you had written yourself the expanded code.

Answer 2

I get the impression that you byte-compiled the file containing the macro definition, but did not byte-compile the file which calls the macro?

It is the calls to a macro which get expanded.

Expansion happens at compile-time if the calling code is compiled; at load-time ("eager" macro expansion, only in recent Emacs versions) if the loaded code is not compiled; and at run-time if eager expansion was not possible or not available.

Obviously if a call to a macro is evaluated at run-time, there was never any opportunity to expand it in advance, so expansion happens right away, at run-time.

Macro expansion at compile time (or load time) is possible because macro arguments are not evaluated. There is nothing problematic about (nth 1 spec) being evaluated at expansion time, because the value of spec is the unevaluated argument which appeared in the original call to the macro.

i.e. When expanding (dotimes (var 3)) the spec argument is the list (var 3) and (nth 1 spec) is therefore evaluated at expansion-time to 3.

For clarity (as numbers could confuse matters here), if it had been (nth 0 spec) then it would have evaluated to the symbol var, and in particular not var's value as a variable (which cannot be established at expansion time).

So (nth 1 spec) -- and indeed any manipulation of the unevaluated arguments passed to the macro -- can absolutely be established at expansion time.

(edit: Wait, we covered this in Can I put condition in emacs lisp macro? )

If you are asking what happens if the macro does something at expansion-time with a value that is dynamic at run-time, the answer is simply that it sees the expansion-time value, and consequently the expanded code might vary depending on when the expansion occurred. There's nothing preventing you from writing macros which behave this way, but it's generally not advisable.