Update nested structure of maps and vectors

Question

I have a map with a vector of map like this:

{:tags    ["type:something" "gw:somethingelse"],
 :sources [{:tags    ["s:my:tags"],
            :metrics [{:tags    ["a tag"]}
                      {:tags    ["a noether tag" "aegn"]}
                      {:tags    ["eare" "rh"]}]}]}

Note that there can be multiple sources, and multiple metrics.

Now I want to update the :metrics with an id by looking at the value of the tags.

Example: if ["a tag"] matches with for example id 1, and ["a noether tag" "aegn"] with id 2 I want the updated structure to look like this:

{:tags    ["type:something" "gw:somethingelse"],
 :sources [{:tags    ["s:my:tags"],
            :metrics [{:tags    ["a tag"]
                       :id      1}
                      {:tags    ["a noether tag" "aegn"]
                       :id      2}
                      {:tags    ["eare" "rh"]}]}]}

I made a function transform that can convert a tag to an id. E.g, (transform "a tag") returns 1.

Now, when I try do add the ids with a for-comprehension I miss the old structure (only the inner ones get returned) and with assoc-in I have to know the indices upfront.

How can I perform this transformation elegantly?

Answer 1

(use 'clojure.walk)

(def transform {["a tag"]                1
                ["a noether tag" "aegn"] 2})

(postwalk #(if-let [id (transform (:tags %))]
            (assoc % :id id)
            %)
          data)

Output:

{:tags ["type:something" "gw:somethingelse"],
:sources
[{:tags ["s:my:tags"],
  :metrics
  [{:tags ["a tag"], :id 1}
    {:tags ["a noether tag" "aegn"], :id 2}
    {:tags ["eare" "rh"]}]}]}

Answer 2

i would start bottom up, making transformation function for :tags entry, then for :metrics and then for :sources.

let's say our transform function produces ids just by counting tags (just for illustration, it could be easily changed later):

(defn transform [tags] (count tags))

user> (transform ["asd" "dsf"])
;;=> 2

then apply transformation the metric entry:

(defn transform-metric [{:keys [tags] :as m}]
  (assoc m :id (transform tags)))

user> (transform-metric {:tags    ["a noether tag" "aegn"]})
;;=> {:tags ["a noether tag" "aegn"], :id 2}

now use transform-metric to update source entry:

(defn transform-source [s]
  (update s :metrics #(mapv transform-metric %)))

user> (transform-source {:tags    ["s:my:tags"],
                         :metrics [{:tags    ["a tag"]}
                                   {:tags    ["a noether tag" "aegn"]}
                                   {:tags    ["eare" "rh"]}]})

;;=> {:tags ["s:my:tags"], 
;;    :metrics [{:tags ["a tag"], :id 1} 
;;              {:tags ["a noether tag" "aegn"], :id 2} 
;;              {:tags ["eare" "rh"], :id 2}]}

and the last step is to transform the whole data:

(defn transform-data [d]
  (update d :sources #(mapv transform-source %)))

user> (transform-data data)
;;=> {:tags ["type:something" "gw:somethingelse"], 
;;    :sources [{:tags ["s:my:tags"], 
;;               :metrics [{:tags ["a tag"], :id 1} 
;;                         {:tags ["a noether tag" "aegn"], :id 2}
;;                         {:tags ["eare" "rh"], :id 2}]}]}

so, we're done here.

Now notice that transform-data and transform-source are almost identical, so we can make an utility function that generates such updating functions:

(defn make-vec-updater [field transformer]
  (fn [data] (update data field (partial mapv transformer))))

with this function we can define deep transformations like this:

(def transformer
  (make-vec-updater
   :sources
   (make-vec-updater
    :metrics
    (fn [{:keys [tags] :as m}]
      (assoc m :id (transform tags))))))

user> (transformer data)
;;=> {:tags ["type:something" "gw:somethingelse"], 
;;    :sources [{:tags ["s:my:tags"], 
;;               :metrics [{:tags ["a tag"], :id 1} 
;;                         {:tags ["a noether tag" "aegn"], :id 2}
;;                         {:tags ["eare" "rh"], :id 2}]}]}

and based on this transformer construction approach we can make a nice function to update the values in vectors-of-maps-of-vectors-of-maps-of-vectors... structures, with arbitrary nesting level:

(defn update-in-v [data ks f]
  ((reduce #(make-vec-updater %2 %1) f (reverse ks)) data))

user> (update-in-v data [:sources :metrics]
                   (fn [{:keys [tags] :as m}] 
                     (assoc m :id (transform tags))))

;;=> {:tags ["type:something" "gw:somethingelse"], 
;;    :sources [{:tags ["s:my:tags"], 
;;               :metrics [{:tags ["a tag"], :id 1} 
;;                         {:tags ["a noether tag" "aegn"], :id 2} 
;;                         {:tags ["eare" "rh"], :id 2}]}]}

UPDATE

in addition to this manual approach, there is a fantastic lib called specter out there, that does exactly the same thing (and much more), but is obviously more universal and usable:

(require '[com.rpl.specter :as sp])

(sp/transform [:sources sp/ALL :metrics sp/ALL]
              (fn [{:keys [tags] :as m}] 
                (assoc m :id (transform tags)))
              data)

;;=> {:tags ["type:something" "gw:somethingelse"], 
;;    :sources [{:tags ["s:my:tags"], 
;;               :metrics [{:tags ["a tag"], :id 1} 
;;                         {:tags ["a noether tag" "aegn"], :id 2} 
;;                         {:tags ["eare" "rh"], :id 2}]}]}