Returning callable functions/closures from inside a Common Lisp macro

Question

I want to define a macro of the following form, where each of the rules (nested in the parameter list) are recorded into a hash-table:

(proc-rule
 ((100 ((+ w 10) (- h 25)))
  ((+ ip 12) ((* w .2) (* h .1)))
  ((* ip 2) ((+ ix (* 2 w)) iy))
  (45.5 ((+ ix (* 2 w)) iy)))  
  table)

These rules can contain references to specific argument names. The first list (which is also the only obligatory one!):

(100 ((+ w 10) (- h 25)))

has a head which is a value, and a tail consisting of two other expressions (which could refer to w y or not) which i add to the hash-table this way:

(setq table (make-hash-table :test #'equalp))

(defmacro proc-rule (rule table)
  (destructuring-bind (ip (ix iy)) (car rule)
    `(progn
       ;; Record the initial forms
       (setf (gethash ,ip ,table) #'(lambda (w h) (list ,ix ,iy)))
       ;; 
  )))

Till now works as expected, when looking for the value 100 i get the function which i can call with arguments as w and h:

(funcall (gethash 100 table) 100 100)   ; (110 75) 

Now i want to iterate over the rest of the rules and add them to the table. The head of each of these rules could be an expression having reference to the head of the very first rule (called ip) or be just another fresh value (which evaluates to itself. Here is the complete macro definition again):

(defmacro proc-rule (rule table)
  (destructuring-bind (ip (ix iy)) (car rule)
    `(progn
       ;; Record the initial forms
       (setf (gethash ,ip ,table) #'(lambda (w h) (list ,ix ,iy)))

       ;; Add the rest of the rules

       (dolist (pattern (cdr rule))
         (setf (gethash (car pattern) ,table)
           #'(lambda (w h) (list (caadr pattern) (cadadr pattern)))))
       )))

The value to this key is also a closure with again two parameters W H which also now can contain references to the passed in arguments which i have labeled as ix iy. Compiling this expansion:

(PROGN
 (SETF (GETHASH 100 TABLE) #'(LAMBDA (W H) (LIST (+ W 10) (- H 25))))
 (DOLIST (PATTERN (CDR RULE))
   (SETF (GETHASH (CAR PATTERN) TABLE)
           #'(LAMBDA (W H) (LIST (CAADR PATTERN) (CADADR PATTERN))))))

leads to a funcall error because of the unquoted ,(cdr rule):

(((+ IP 12) ((* W 0.2) (* H 0.1))) ((* IP 2) ((+ IX (* 2 W)) IY))
   (45.5 ((+ IX (* 2 W)) IY))) 

Changing that part to (cdr ',rule) results of course in recording quoted conses as values to the keys so that:

(funcall (gethash 45.5 table) 100 100)  ;(((+ IX (* 2 W)) IY) NIL)

How could i get tails of these rules to be saved as function bodies and not conses so that calling them computes the supplied expressions?

Second question: is this all in all a good design, and if not please explain why not? (I wanted the user to supply the expressions in a more convenient form like ((+ ip 12) ((* w .2) (* h .1))).

Answer 1

Allow me to format it a bit differently:

(proc-rule ((100 ((+ w 10) (- h 25)))
            ((+ ip 12) ((* w .2) (* h .1)))
            ((* ip 2) ((+ ix (* 2 w)) iy))
            (45.5 ((+ ix (* 2 w)) iy)))  
           table)

It seems that:

• the 100 must be a literal value
• the first form of each rule should be evaluated with ip bound to that first key (100)
• the second form of each rule is a list of two expressions such that these forms describe a function that returns a list of evaluating these two expressions
• the two parameters of these functions are always named w and h
• they can also refer to ix and iy which are the two elements of the return list of the first rule function

I see two ways of accomplishing that last part:

• either ix and iy are symbol macros that expand to the forms given in the first rule at macro expansion time. This would maybe be a bit hairy.

• or each subsequent rule function should call the first rule function and bind ix and iy to its return list; something like this (untested sketch):

  (defmacro proc-rule (rules table)
    (let ((ip (first (first rules))))
      `(setf ,@(loop :for (keyform expr) :in rules
                     :collect `(gethash (let ((ip ,ip)) ,keyform) ,table)
                     :collect `(lambda (w h)
                                 (destructuring-bind (ix iy)
                                     (funcall (gethash ,ip ,table) w h)
                                   (declare (ignorable ix iy))
                                   (list ,@expr)))))))
  

However, from personal convictions, I'd advise against these implicit bindings and try to find a better way to express these rules.

Answer 2

Basic Rule for writing Macros

Write down the code you want to generate. Then write the code transforming code which generates this code.

Example

See this example for generated code - not specifically for your example, but similar - I'm also using LOOP instead of DOLIST, because it does destructuring:

(loop for ((one two)) in '((((+ a b) (- a b)))
                           (((- a b) (+ a b))))
      collect (lambda (a b) (list one two)))

Above does not work as intended, because forms like (+ a b) are treated as lists and a variable like one does just return such a list. It also does not work because of using the iteration variables.

To address the later we could rebind them:

(loop for ((one two)) in '((((+ a b) (- a b)))
                           (((- a b) (+ a b))))
      collect (let ((one one)
                    (two two))
                (lambda (a b) (list one two))))

Still in above code we have lists and not code for the expressions.

If you want to create functions from source code you need to call EVAL or COMPILE:

(loop for ((one two)) in '((((+ a b) (- a b)))
                           (((- a b) (+ a b))))
      collect (compile nil `(lambda (a b) (list ,one ,two))))

Above creates code and compiles it at runtime.

That would be code to generate. But you would generate code which explicitly calls EVAL or COMPILE. This is a typical anti-pattern. A macro creates code which then gets automatically evaluated. One rarely needs another step of evaluation - so always think whether it's possible to get rid of that added evaluation step.

But what you really want is to generate this code:

(list (lambda (a b) (list (+ a b) (- a b)))
      (lambda (a b) (list (- a b) (+ a b))))

Think about how to change your macro to create fully expanded code like above.

Macro Syntax

I would name the macro differently, change the argument order and get rid of the list:

(define-proc-rules table
 (100       ((+ w 10)        (- h 25)))
 ((+ ip 12) ((* w .2)        (* h .1)))
 ((* ip 2)  ((+ ix (* 2 w))  iy))
 (45.5      ((+ ix (* 2 w))  iy)))

The macro would then be defined with:

(defmacro define-proc-rules (table &body rules) ...)