Clojure macros: quoting, unquoting and evaluation

Question

I have the following code:

(ns macroo)

(def primitives #{::byte ::short ::int})

(defn primitive? [type]
  (contains? primitives type))

(def pp clojure.pprint/pprint)

(defn foo [buffer data schema]
  (println schema))

(defmacro write-fn [buffer schema schemas]
  (let [data (gensym)]
    `(fn [~data]
       ~(cond
          (primitive? schema) `(foo ~buffer ~data ~schema)
          (vector? schema) (if (= ::some (first schema))
                             `(do (foo ~buffer (count ~data) ::short)
                                 (map #((write-fn ~buffer ~(second schema) ~schemas) %)
                                       ~data))
                             `(do ~@(for [[i s] (map-indexed vector schema)]
                                      ((write-fn buffer s schemas) `(get ~data ~i)))))
          :else [schema `(primitive? ~schema) (primitive? schema)])))) ; for debugging

(pp (clojure.walk/macroexpand-all '(write-fn 0 [::int ::int] 0)))

The problem is, upon evaluating the last expression, I get

=>
(fn*
 ([G__6506]
  (do
   [:macroo/int :macroo/int true false]
   [:macroo/int :macroo/int true false])))

I'll explain the code if necessary, but for now i'll just state the problem (it might be just a newbie error I'm making):

`(primitive? ~schema)

and

(primitive? schema)

in the :else branch return true and false respectively, and since i'm using the second version in the cond expression, it fails where it shouldn't (I'd prefer the second version as it would be evaluated at compile time if i'm not mistaken).

I suspect it might have something to do with symbols being namespace qualified?

Answer 1

After some investigations (see edits), here is a working Clojure alternative. Basically, you rarely need recursive macros. If you need to build forms recursively, delegate to auxiliary functions and call them from the macro (also, write-fn is not a good name).

(defmacro write-fn [buffer schemas fun]
  ;; we will evaluate "buffer" and "fun" only once
  ;; and we need gensym for intermediate variables.
  (let [fsym (gensym)
        bsym (gensym)]

    ;; define two mutually recursive function
    ;; to parse and build a map consisting of two keys
    ;;
    ;; - args is the argument list of the generated function
    ;; - body is a list of generated forms 
    ;;
    (letfn [(transformer [schema]
              (cond
                (primitive? schema)
                (let [g (gensym)]
                  {:args g
                   :body `(~fsym ~schema ~bsym ~g)})

                (sequential? schema)
                (if (and(= (count schema) 2)
                        (= (first schema) ::some)
                        (primitive? (second schema)))
                  (let [g (gensym)]
                    {:args ['& g]
                     :body
                     `(doseq [i# ~g]
                        (~fsym ~(second schema) ~bsym i#))})
                  (reduce reducer {:args [] :body []} schema))
                :else (throw (Exception. "Bad input"))))

            (reducer [{:keys [args body]} schema]
              (let [{arg :args code :body} (transformer schema)]
                {:args (conj args arg)
                 :body (conj body code)}))]

      (let [{:keys [args body]} (transformer schemas)]
        `(let [~fsym ~fun
               ~bsym ~buffer]
           (fn [~args] ~@body))))))

The macro takes a buffer (whatever it is), a schema as defined by your language and a function to be called for each value being visited by the generated function.

Example

(pp (macroexpand
      '(write-fn 0 
                 [::int [::some ::short] [::int ::short ::int]] 
                 (fn [& more] (apply println more)))))

... produces the following:

(let*
 [G__1178 (fn [& more] (apply println more)) G__1179 0]
 (clojure.core/fn
  [[G__1180 [& G__1181] [G__1182 G__1183 G__1184]]]
  (G__1178 :macroo/int G__1179 G__1180)
  (clojure.core/doseq
   [i__1110__auto__ G__1181]
   (G__1178 :macroo/short G__1179 i__1110__auto__))
  [(G__1178 :macroo/int G__1179 G__1182)
   (G__1178 :macroo/short G__1179 G__1183)
   (G__1178 :macroo/int G__1179 G__1184)]))

• First, evaluate buffer and fun and bind them to local variables
• Return a closure which accept one argument and destructures it according to the given schema, thanks to Clojure's destructuring capabilities.
• For each value, call fun with the appropriate arguments.
• When the schema is [::some x], accept zero or more values as a vector and call the function fun for each of those values. This needs to be done with a loop, since the size is only know when calling the function.

If we pass the vector [32 [1 3 4 5 6 7] [2 55 1]] to the function generated by the above macroexpansion, the following is printed:

:macroo/int 0 32
:macroo/short 0 1
:macroo/short 0 3
:macroo/short 0 4
:macroo/short 0 5
:macroo/short 0 6
:macroo/short 0 7
:macroo/int 0 2
:macroo/short 0 55
:macroo/int 0 1

Answer 2

In this line:

`(do ~@(for [[i s] (map-indexed vector schema)]
         ((write-fn buffer s schemas) `(get ~data ~i)))))

you are calling write-fn, the macro, in your current scope, where s is just a symbol, not one of the entries in schema. Instead, you want to emit code that will run in the caller's scope:

`(do ~@(for [[i s] (map-indexed vector schema)]
         `((write-fn ~buffer ~s ~schemas) (get ~data ~i)))))

And make a similar change to the other branch of the if, as well.

As an aside, it looks to me at first glance like this doesn't really need to be a macro, but could be a higher-order function instead: take in a schema or whatever, and return a function of data. My guess is you're doing it as a macro for performance, in which case I would counsel you to try it out the slow, easy way first; once you have that working you can make it a macro if necessary. Or, maybe I'm wrong and there's something in here that fundamentally has to be a macro.