Lisp Webtales Chapter 5 - how to get it working

Question

Following on from my solution to getting the code in chapter 4 working as here, I need some help debugging the application "linkdemo" in Chapter 5 of lisp webtales.

I've carefully typed in all the code & eliminated one typo contained in the book.

In CCL if I issue (ql:quickload "linkdemo") I get this:

CCL is free software.  It is distributed under the terms of the Apache
Licence, Version 2.0.
? (ql:quickload "linkdemo")
To load "linkdemo":
  Load 1 ASDF system:
    linkdemo
; Loading "linkdemo"
.
;;; Checking for wide character support... yes, using code points.
;;; Checking for wide character support... yes, using code points.
;;; Building Closure with CHARACTER RUNES
.
> Error: The value NIL is not of the expected type STRING.
> While executing: ENSURE-SIMPLE-STRING, in process listener(1).
> Type :POP to abort, :R for a list of available restarts.
> Type :? for other options.
1 > 

If I type :R I get this:

1 > :R
>   Type (:C <n>) to invoke one of the following restarts:
0. Return to break level 1.
1. #<RESTART ABORT-BREAK #x133AEDD>
2. Retry compiling #P"/Users/m/quicklisp/local-projects/linkdemo/defmodule.lisp"
3. Skip compiling #P"/Users/m/quicklisp/local-projects/linkdemo/defmodule.lisp"
4. Retry compiling #<CL-SOURCE-FILE "linkdemo" "defmodule">.
5. Continue, treating compiling #<CL-SOURCE-FILE "linkdemo" "defmodule"> as having been successful.
6. Retry ASDF operation.
7. Retry ASDF operation after resetting the configuration.
8. Give up on "linkdemo"
9. Return to toplevel.
10. #<RESTART ABORT-BREAK #x133C53D>
11. Reset this thread
12. Kill this thread
1 > 

from which I infer the error is in defmodule.lisp.

Then, I found that if I comment out the following entire block in defmodule.lisp:

(restas:define-policy datastore
  (:interface-package #:linkdemo.policy.datastore)
  (:interface-method-template "DATASTORE-~A")
  (:internal-package #:linkdemo.datastore)

  (define-method init ()                                    
    "Initiate the datastore")

  (define-method find-user (username)                       
    "Find the user by username")

  (define-method auth-user (username password)
    "Check if a user exists and has the supplied password")

  (define-method register-user (username password)
    "Register a new user")

  (define-method upvoted-p (link-id username)
    "Check if a user has upvoted a link")

  (define-method upvote (link-id user)
    "Upvote a link")

  (define-method post-link (url title user)
    "Post a new link")

  (define-method get-all-links (&optional user)
    "Get all of the links in the datastore")

  (define-method upvote-count (link-id)
    "Get the number of upvotes for a given link"))

then re-issue (ql:quickload "linkdemo"), I get no error.

(Also, if I then comment it in again, and reload with quickload, I also get no error. I'm not sure what's happening there or the reason for that.)

Whether I've quickloaded with or without the offending section above, I am able to explore a function within pg-datastore.lisp, like so:

1 > (linkdemo.pg-datastore::hash-password "42")
(:PASSWORD-HASH "71a8c8f54475acb5afee9eae061fa5f7ba838215f280259855e59a7c0ac768f8" :SALT "a283c5328f67c36595dd7277c54342f3")

It doesn't seem possible to try to boot up the server yet, as code from Chapter 6 is needed before it's fully functional (we have no routes yet, just the DAO layer with postmodern).

So not sure how to proceed with testing the code in Ch.5 / debugging this. The error seems to be something in restas itself - maybe.

Seeking any educated guesses on how to proceed.

The error can be reproduced by simply doing (ql:quickload "restas") followed by evaluating the block given above. This gives Error: The value NIL is not of the expected type STRING. I am new to using the code of others in LISP, so I'm a bit stuck on how to proceed.

More info: it is the calls to define-method that are giving this error, and the source is the file policy.lisp here, which is a bit beyond me at the moment.

I tried sbcl too:

* (ql:quickload "linkdemo")
To load "linkdemo":
  Load 1 ASDF system:
    linkdemo
; Loading "linkdemo"
.
;;; Checking for wide character support... WARNING: Lisp implementation doesn't use UTF-16, but accepts surrogate code points.
 yes, using code points.
;;; Building Closure with CHARACTER RUNES
.........
debugger invoked on a SB-KERNEL:CASE-FAILURE in thread
#<THREAD "main thread" RUNNING {10004F04C3}>:
  NIL fell through ETYPECASE expression.
  Wanted one of (SIMPLE-STRING STRING SB-FORMAT::FMT-CONTROL).

...

(SB-FORMAT::%FORMAT #<SB-IMPL::CHARACTER-STRING-OSTREAM {1003BD1673}> NIL ("UPVOTE-COUNT") ("UPVOTE-COUNT"))
0] 

Answer 1

I can reproduce the test case when defining only one method:

(restas:define-policy datastore
  (:interface-package #:linkdemo.policy.datastore)
  (:interface-method-template "DATASTORE-~A")
  (:internal-package #:linkdemo.datastore)
  (define-method init ()                                    
    "Initiate the datastore"))

The problem here is that when you expand the above macro, you obtain a function call to RESTAS::%DEFINE-POLICY where :INTERNAL-FUNCTION-TEMPLATE is given NIL, which is not an appropriate format string. This is because of the way the define-policy macro parses options.

Here is a reduced example:

(defun my-function (&key (x 0))
  (print x))

MY-FUNCTION has a default value 0 for :x. If you call (my-function), it prints 0. But the macro does something like this:

(defmacro my-macro (args)
  (let ((x-arg))
    (loop for (k v) in args 
       do (case k (:x (setf x-arg v))))
    `(my-function :x ,x-arg)))

It loops over the argument list to check whether some keyword for the :x argument exist, and then sets a local variable (initialized to nil) to the corresponding value. In all cases, it expands into a call to my-function and explicitly binds x with the value of x-arg, even when it is nil. And indeed, if we macroexpand (my-macro ()), if gives (my-function :x nil).

Since the function already defines default values, we would like to avoid repeating this default value elsewhere, and so the macro should take care not to give an :x when its corresponding value is nil. Or more precisely, when the keyword argument is not present (here I make no distinction between both cases).Typically, this is done as follows:

(defmacro my-macro (args)
  (flet ((maybe (k v) (and v (list k v))))
    (let ((x-arg))
      (loop for (k v) in args do (case k (:x (setf x-arg v))))
      `(my-function ,@(maybe :x x-arg)))))

The ,@ splice operator is used to inject keyword arguments into the call, based on whether the value is nil or not. When it is nil, the empty list is spliced. Otherwise, the property list (list k v) is spliced. And thus, a call to (my-macro ()) now expands as (my-function), and thus x will take its default value.

This is not well tested, but I guess you can redefine define-policy as follows:

(defmacro define-policy (name &body body)
  (let (methods
        internal-package  internal-function-template
        interface-package interface-method-template)
    (iter (for item in body)
          (case (car item)
            (:internal-package
             (setf internal-package (second item)))
            (:internal-function-template
             (setf internal-function-template (second item)))
            (:interface-package
             (setf interface-package (second item)))
            (:interface-method-template
             (setf interface-method-template (second item)))
            (otherwise
             (cond
               ((string= (car item) "DEFINE-METHOD")
                (push (cdr item) methods))
               (t
                (error "Unknown DEFINE-POLICY option: ~A" item))))))
    (flet ((maybe (key value) (and value (list key value)))
           (maybe/quote (key value) (and value (list key `(quote ,value)))))
      `(eval-when (:compile-toplevel :load-toplevel :execute)
         (%define-policy ',name ',methods
                         ,@(maybe/quote :interface-package interface-package)
                         ,@(maybe :interface-method-template interface-method-template)
                         ,@(maybe/quote :internal-package internal-package)
                         ,@(maybe :internal-function-template internal-function-template))))))

Alternatively, just add the :internal-function-template argument (e.g. "~A") to your macro.

---

Patched: https://github.com/archimag/restas/commit/81bbbab6b36f81f846f78e71232e9d3d15f6d952