CODEWITHSUNDEEP
clojuretransformation
omue
omue

Reputation: 68

How to transform a list of maps to a nested map of maps?

Getting data from the database as a list of maps (LazySeq) leaves me in need of transforming it into a map of maps.

I tried to 'assoc' and 'merge', but that didn't bring the desired result because of the nesting.

This is the form of my data:

(def data (list {:structure 1 :cat "A" :item "item1" :val 0.1}
                {:structure 1 :cat "A" :item "item2" :val 0.2}
                {:structure 1 :cat "B" :item "item3" :val 0.4}
                {:structure 2 :cat "A" :item "item1" :val 0.3}
                {:structure 2 :cat "B" :item "item3" :val 0.5}))

I would like to get it in the form

=> {1 {"A" {"item1" 0.1}
            "item2" 0.2}}
      {"B" {"item3" 0.4}}
    2 {"A" {"item1" 0.3}}
      {"B" {"item3" 0.5}}}

I tried

(->> data
     (map #(assoc {} (:structure %) {(:cat %) {(:item %) (:val %)}}))
     (apply merge-with into))

This gives

{1 {"A" {"item2" 0.2}, "B" {"item3" 0.4}},
 2 {"A" {"item1" 0.3}, "B" {"item3" 0.5}}}

By merging I lose some entries, but I can't think of any other way. Is there a simple way? I was even about to try to use specter.

Any thoughts would be appreciated.

Upvotes: 2

Views: 402

Answers (3)

rmcv
rmcv

Reputation: 1976

You may continue to use your existing code. Only the final merge has to change:

(defn deep-merge [& xs]
  (if (every? map? xs)
    (apply merge-with deep-merge xs)
    (apply merge xs)))

(->> data
     (map #(assoc {} (:structure %) {(:cat %) {(:item %) (:val %)}}))
     (apply deep-merge))

;; => 
{1
 {"A" {"item1" 0.1, "item2" 0.2},
  "B" {"item3" 0.4}},
 2
 {"A" {"item1" 0.3},
  "B" {"item3" 0.5}}}

Explanation: your original (apply merge-with into) only merge one level down. deep-merge from above will recurse into all nested maps to do the merge.

Addendum: @pete23 - one use of juxt I can think of is to make the function reusable. For example, we can extract arbitrary fields with juxt, then convert them to nested maps (with yet another function ->nested) and finally do a deep-merge:

(->> data
     (map (juxt :structure :cat :item :val))
     (map ->nested)
     (apply deep-merge))

where ->nested can be implemented like:

(defn ->nested [[k & [v & r :as t]]]
  {k (if (seq r) (->nested t) v)})

(->nested [1 "A" "item1" 0.1])
;; => {1 {"A" {"item1" 0.1}}}

One sample application (sum val by category):

(let [ks [:cat :val]]
  (->> data
       (map (apply juxt ks))
       (map ->nested)
       (apply (partial deep-merge-with +))))
;; => {"A" 0.6000000000000001, "B" 0.9}

Note deep-merge-with is left as an exercise for our readers :)

Upvotes: 1

pete23
pete23

Reputation: 2280

If I'm dealing with nested maps, first stop is usually to think about update-in or assoc-in - these take a sequence of the nested keys. For a problem like this where the data is very regular, it's straightforward.

(assoc-in {} [1 "A" "item1"] 0.1)
;; =>
{1 {"A" {"item1" 0.1}}}

To consume a sequence into something else, reduce is the idiomatic choice. The reducing function is right on the edge of the complexity level I'd consider an anonymous fn for, so I'll pull it out instead for clarity.

(defn- add-val [acc line]
   (assoc-in acc [(:structure line) (:cat line) (:item line)] (:val line)))

(reduce add-val {} data)
;; =>
{1 {"A" {"item1" 0.1, "item2" 0.2}, "B" {"item3" 0.4}},
 2 {"A" {"item1" 0.3}, "B" {"item3" 0.5}}}

Which I think was the effect you were looking for.

Roads less travelled:

As your sequence is coming from a database, I wouldn't worry about using a transient collection to speed the aggregation up. Also, now I think about it, dealing with nested transient maps is a pain anyway.

update-in would be handy if you wanted to add up any values with the same key, for example, but the implication of your question is that structure/cat/item tuples are unique and so you just need the grouping.

juxt could be used to generate the key structure - i.e.

((juxt :structure :cat :item) (first data))
[1 "A" "item1"]

but it's not clear to me that there's any way to use this to make the add-val fn more readable.

Upvotes: 3

Lee
Lee

Reputation: 144206

(defn map-values [f m]
  (into {} (map (fn [[k v]] [k (f v)])) m))

(defn- transform-structures [ss]
  (map-values (fn [cs]
                (into {} (map (juxt :item :val) cs))) (group-by :cat ss)))

(defn transform [data]
  (map-values transform-structures (group-by :structure data)))

then

(transform data)

Upvotes: 0

Related Questions