How to create configurable macros

Question

Suppose that we want to write a macro whose behavior depends on some kind of configuration. Specifically, in my case the configuration is just a boolean value indicating some optimization in the output code. Even more specifically, the transformation looks like this (denoting macroexpansion by ==>):

(transform ...) ==> (transform'-opt ...) ==> (transform' ...) ==> ...

when the optimization takes place, otherwise transform'-opt is omitted and transform' is used in its place.

Using a parameter for the configuration would result in too much repetition, so it would be preferable to configure the macro in a local/with-... style (and possibly in a global style too). By this I mean that, using something like (with-opt value expr*), during the macroexpansion of each expr value is used as the configuration of transform unless this is changed by another with-opt. The solution I have chosen so far is to use a dynamic var and binding in the following way:

(require '[clojure.tools.analyzer.jvm :refer [macroexpand-all]])

(def ^:dynamic *opt* true)

(defmacro with-opt [value & body]
  `(do ~@(binding [*opt* value] (mapv macroexpand-all body))))

(defmacro transform [...]
  `(~(if *opt* `transform'-opt `transform') ...))

However, the use of dynamic vars and macroexpand-all at macroexpansion looks somewhat unconventional to me. Other (also unconventional) options I have considered include using a regular var and with-redefs, an atom, a volatile and also hiding the configuration inside closures like this:

(let [opt ...]
  (defmacro with-opt ...)
  (defmacro transform ...))

[Perhaps an imaginary generic way to avoid mutation and manual macroexpansion in such cases could be to introduce another implicit &config parameter in macros and a new special form (with-macro-configs [(macro config)*] expr*) that would specify the value used for this parameter for various macros.]

Is there any common practice for such a situation? What would your approach be and why? Thanks in advance.

Answer 1

A macro transforms code, represented as a datastructure, to another datastructure. I would write a macro that transforms the code using postwalk and inserts the parameter for a specified form as last argument every time it encounters it. You could have a macro with-arg that will do the transformation:

(require '[clojure.walk :refer [postwalk]])

(defmacro with-arg [[sym arg] & body]
  `(do ~@(postwalk #(if (and (seq? %)
                             (= sym (first %)))
                      (concat % [arg])
                      %)
                   body)))

Suppose we have some macro that will do the code transformation and that we can pass it options. Just to demonstrate, here is dummy macro for that to test things out:

(defmacro transform [x & optimization-options]
  `(println ~x ~(into [] cat optimization-options)))

and here is an example using it:

(do
  (println "No optimizations enabled")
  (transform :X)
  (with-arg [transform [:unroll]]
    (println "With unrolling")
    (transform :Y)
    (with-arg [transform [:inline :O3]]
      (println "With some extra options")
      (transform :Z))))

which will print

No optimizations enabled
:X []
With unrolling
:Y [:unroll]
With some extra options
:Z [:unroll :inline :O3]

In the code example, we only call transform with a single argument, but the macro transformations make it receive the options from all the surrounding with-arg forms. We can then decide how we combine all the options, e.g. using (into [] cat optimization-options). Or we could do (last optimization-options) if we want the innermost with-arg to shadow the outer ones.

One issue with this approach is that it may not compose well with other macros, such as ->. For example, this code:

(with-arg [transform [:unroll]]
  (-> :X0
      transform))

(with-arg [transform [:unroll]]
  (-> :X1
      (transform)))

will print this:

:X0 []
:X1 [:unroll]

But this might not be a problem in practice, as long as you are aware of it.

Answer 2

This is the kind of thing that is difficult to do with Clojure's macro system, but would be easy if we had macrolet. I recall reading, I think in Let Over Lambda, that most code-walking macros that attempt to walk code themselves would be made much simpler and more robust by using macrolet (which does exist in other Lisp dialects). That way, the compiler does all the difficult work for you, including handling shadowing and quoting correctly, etc.

For a simple example, suppose you have some macro that needs a boolean parameter, and you wish within a lexical context to have that always be true. With macrolet, you could write:

(defmacro change* [code toggle]
  (if toggle
    `(not ~code)
    code))

(defmacro with-toggle [toggle & body]
  `(macrolet [(~'change [code#]
                `(change* ~code# ~~toggle))]
     ~@body))

(with-toggle true
  (change (= 1 2)))

While macrolet does not exist in the Clojure standard library, there is an implementation in clojure.tools.macro that is almost as good as one built into the compiler would be. There are a few edge cases it does not support, but for every reasonable macro I've ever wanted to write it has been correct. Here is evidence that it works for our example case:

user=> (macroexpand '(with-toggle false (change (= 1 2))))
(do (= 1 2))
user=> (macroexpand '(with-toggle true (change (= 1 2))))
(do (clojure.core/not (= 1 2)))

Of course the difficult part is writing the body of the macrolet, because you have to deal with two levels of syntax quoting. I supposedly know what I'm doing, but I still took three or four tries to get this simple macro right. Just remember that each ~ escapes a single level of syntax-quoting, after which you have to decide what to do next: escape again, use a gensym# variable, use a normal quote, or just stay a single level deep.