Why in this example calling (f arg) and calling the body of f explicitly yields different results?

Question

First, I have no experience with CS and Clojure is my first language, so pardon if the following problem has a solution, that is immediately apparent for a programmer.

The summary of the question is as follows: one needs to create atoms at will with unknown yet symbols at unknown times. My approach revolves around a) storing temporarily the names of the atoms as strings in an atom itself; b) changing those strings to symbols with a function; c) using a function to add and create new atoms. The problem pertains to step "c": calling the function does not create new atoms, but using its body does create them.

All steps taken in the REPL are below (comments follow code blocks):

user=> (def atom-pool
  #_=>   (atom ["a1" "a2"]))
#'user/atom-pool

'atom-pool is the atom that stores intermediate to-be atoms as strings.

user=> (defn atom-symbols []
  #_=>   (mapv symbol (deref atom-pool)))
#'user/atom-symbols
user=> (defmacro populate-atoms []
  #_=>   (let [qs (vec (remove #(resolve %) (atom-symbols)))]
  #_=>     `(do ~@(for [s qs]
  #_=>              `(def ~s (atom #{}))))))
#'user/populate-atoms

'populate-atoms is the macro, that defines those atoms. Note, the purpose of (remove #(resolve %) (atom-symbols)) is to create only yet non-existing atoms. 'atom-symbols reads 'atom-pool and turns its content to symbols.

user=> (for [s ['a1 'a2 'a-new]]
  #_=>   (resolve s))
(nil nil nil)

Here it is confirmed that there are no 'a1', 'a2', 'a-new' atoms as of yet.

user=> (defn new-atom [a]
  #_=>   (do
  #_=>     (swap! atom-pool conj a)
  #_=>     (populate-atoms)))
#'user/new-atom

'new-atom is the function, that first adds new to-be atom as string to `atom-pool. Then 'populate-atoms creates all the atoms from 'atom-symbols function.

user=> (for [s ['a1 'a2 'a-new]]
  #_=>   (resolve s))
(#'user/a1 #'user/a2 nil)

Here we see that 'a1 'a2 were created as clojure.lang.Var$Unbound just by defining a function, why?

user=> (new-atom "a-new")
#'user/a2
user=> (for [s ['a1 'a2 'a-new]]
  #_=>   (resolve s))
(#'user/a1 #'user/a2 nil)

Calling (new-atom "a-new") did not create the 'a-new atom!

user=> (do
  #_=>   (swap! atom-pool conj "a-new")
  #_=>   (populate-atoms))
#'user/a-new
user=> (for [s ['a1 'a2 'a-new]]
  #_=>   (resolve s))
(#'user/a1 #'user/a2 #'user/a-new)
user=> 

Here we see that resorting explicitly to 'new-atom's body did create the 'a-new atom. 'a-new is a type of clojure.lang.Atom, but 'a1 and 'a2 were skipped due to already being present in the namespace as clojure.lang.Var$Unbound.

Appreciate any help how to make it work!

EDIT: Note, this is an example. In my project the 'atom-pool is actually a collection of maps (atom with maps). Those maps have keys {:name val}. If a new map is added, then I create a corresponding atom for this map by parsing its :name key.

Answer 1

I've already submitted one answer to this question, and I think that that answer is better, but here is a radically different approach based on eval:

(def atom-pool (atom ["a1" "a2"]))

(defn new-atom! [name]
  (load-string (format "(def %s (atom #{}))" name)))

(defn populate-atoms! []
  (doseq [x atom-pool]
    (new-atom x)))

format builds up a string where %s is substituted with the name you're passing in. load-string reads the resulting string (def "name" (atom #{})) in as a data structure and evals it (this is equivalent to (eval (read-string "(def ...)

Of course, then we're stuck with the problem of only defining atoms that don't already exist. We could change the our new-atom! function to make it so that we only create an atom if it doesn't already exist:

(defn new-atom! [name]
  (when-not (resolve (symbol name))
    (load-string (format "(def %s (atom #{}))" name name))))

The Clojure community seems to be against using eval in most cases, as it is usually not needed (macros or functions will do what you want in 99% of cases*), and eval can be potentially unsafe, especially if user input is involved -- see Brian Carper's answer to this question.

*After attempting to solve this particular problem using macros, I came to the conclusion that it either cannot be done without relying on eval, or my macro-writing skills just aren't good enough to get the job done with a macro!

At any rate, I still think my other answer is a better solution here -- generally when you're getting way down into the nuts & bolts of writing macros or using eval, there is probably a simpler approach that doesn't involve metaprogramming.

Answer 2

@JoostDiepenmaat is right about why populate-atoms is not behaving as expected. You simply cannot do this using macros, and it is generally best to avoid generating vars at runtime. A better solution would be to define your atom-pool as a map of keywords to atoms:

(def atom-pool
  (atom {:a1 (atom #{}) :a2 (atom #{})}))

Then you don't need atom-symbols or populate-atoms because you're not dealing with vars at compile-time, but typical data structures at run-time. Your new-atom function could look like this:

(defn new-atom [kw]
  (swap! atom-pool assoc kw (atom #{})))

EDIT: If you don't want your new-atom function to override existing atoms which might contain actual data instead of just #{}, you can check first to see if the atom exists in the atom-pool:

(defn new-atom [kw]
  (when-not (kw @atom-pool)
    (swap! atom-pool assoc kw (atom #{}))))

Answer 3

"The summary of the question is as follows: one needs to create atoms at will with unknown yet symbols at unknown times. "

This sounds like a solution looking for a problem. I would generally suggest you try another way of achieving whatever the actual functionality is without generating vars at runtime, but if you must, you should use intern and leave out the macro stuff.

You cannot solve this with macros since macros are expanded at compile time, meaning that in

(defn new-atom [a]
 (do
   (swap! atom-pool conj a)
   (populate-atoms)))

populate-atoms is expanded only once; when the (defn new-atom ...) form is compiled, but you're attempting to change its expansion when new-atom is called (which necessarily happens later).