convert to clojure threading operator

Question

I have the following code within a let

; clause in a let
lessons-full (into []
                   (map #(dissoc (merge % (first (lesson-detail %)))
                                 :id :series_id :lesson_id :num))
                   lessons)

bizarrely I ended up using the transducer form of into by accident, by getting a paren in the wrong place. That's the version shown above. For comparison, version without the transducer is:

; clause in a let
lessons-full (into []
                   (map #(dissoc (merge % (first (lesson-detail %)))
                                 :id :series_id :lesson_id :num)
                        lessons))

which also works.

I have a couple more things I want to do in this transform, including convert the value of a key called :type which is currently a string, with keyword. However this is becoming high cognitive load. Not yet skilled with the threading operators. Can anyone assist with first steps/thinking process for that?

lessons is a list of maps from a jdbc query.

---

update: draft answer - thinking process to convert to thread last operator

Step 1

Prepare for juggling. 1. Start with thread last ->>, 2. put the argument, lessons up front, 3. put the final transform into at the end. This version works, but we're not finished yet:

; clause in a let
lessons-full (->> lessons
                  (map #(dissoc (merge % (first (lesson-detail %)))
                                         :id :series_id :lesson_id :num) ,,,)
                  (into [] ,,,))

Note, the triple commas ,,, are ignored by clojure and we add them to help visualise where the argument is injected by the thread-last ->> macro.

Step 2

We pull out the dissoc but since we used it within a call to map, when we pull it out we need to wrap it in another call to map.

; clause in a let
lessons-full (->> lessons
                  (map #(merge % (first (lesson-detail %))) ,,,)
                  (map #(dissoc % :id :series_id :lesson_id :num) ,,,)
                  (into [] ,,,))

So that works too. Let it sink in.

---

update 2

Finally, here's code that achieves my original goal:

; clause in a let
lessons-full (->> lessons
                  (map #(merge % (lesson-detail %)) ,,,)
                  (map #(dissoc % :id :series_id :lesson_id :num) ,,,)
                  (map #(assoc % :type (keyword (:type %))) ,,,)
                  (into [] ,,,))

It appears that list comps are not easy to use inside of the thread last macro, unless I'm mistaken. Also, I'm mapping over the map three times here. In my current use case that might not matter, but is there anything to be said here regarding performance, or any other improvements possible?

Answer 1

I would write like that, using transducers

In theory, you have more flexibility to define computing and use it regardless of data structures

(let [xf (comp (map #(merge % (first (lesson-detail %))))
               (map #(dissoc % :id :series_id :lession_id :num))
               (map #(update % :type keyword)))]
  ;; into array, eager
  (into [] xf lessons)
  ;; lazy sequence
  (sequence xf lessons))

Answer 2

If the cognitive load of the code that you are suggesting is the problem here, you can factor out the operation you do on each lesson into its own function, such as this one:

(defn preprocess-lesson [lesson]
  (dissoc (merge lesson (first (lesson-detail lesson)))
          :id :series_id :lesson_id :num))

This has several benefits:

• You can write a test case for preprocess-lesson
• It makes it obvious that this operation of preprocessing a lesson only needs a lesson as argument and does not depend on some value from the surrounding scope.
• The transformation on the sequence of lessons looks cleaner

If you want to use the threading operator, you would write

(->> lessons
     (map preprocess-lesson))

If you want to use a transducer, you could write

(into []  (map preprocess-lesson) lessons)

Suppose that you want to do some more operations with a lesson, you can even use the threading operator inside preprocess-lesson:

(defn preprocess-lesson [lesson common-lesson-data]
  (-> lesson
      (merge (first (lesson-detail lesson)))
      (dissoc :id :series_id :lesson_id :num)
      (assoc :type (get lesson "type"))
      (dissoc "type")
      (merge common-lesson-data)))

and then call it like

(->> lessons
     (map #(preprocess-lesson % {:teacher "Rudolf"})))

Not yet skilled with the threading operators. Can anyone assist with first steps/thinking process for that

Threading macros are there to make your code more readable by clarifying the flow of data through your computation. Instead of writing

(filter a? (map b (filter c? X)))

you can write

(->> X
     (filter c?)
     (map b)
     (filter a?))

which may be more readable and it may be easier to understand the intention. But apart from that, they don't add any extra expressive power over just nesting expressions like (filter a? (map b (filter c? X))). In short, use the threading macros to make your code more readable and not for the sake of using them. If factoring out some piece of code in a separate function makes your code more readable, then do that.