How to calculate the sum of all depths in a tree using basic `recur`?

Question

For a given tree I would like to sum the depth of each node and calculate that recursively (so not with map/flatten/sum).

Is there a way to do that with recur or do I need to use a zipper in this case?

Answer 1

Using normal stack consuming recursion you can accomplish this pretty easily, by doing a depth-first traversal and summing the depth on the way back out.

(defn sum-depths
  ([tree]
   (sum-depths tree 0))
  ([node depth]
   (if-not (vector? node)
     depth
     (do
       (apply
        +
        (for [child-node (second node)]
          (sum-depths child-node (inc depth))))))))

(sum-depths [:root
             [:a1
              [:b1
               [:a2 :b2]]
              :c1]])
;; => 6

(sum-depths ["COM"
             [["B"
               [["C"
                 [["D"
                   [["E"
                     ["F"
                      ["J"
                       [["K"
                         ["L"]]]]]]
                    "I"]]]]
                ["G"
                 ["H"]]]]]])
;; => 19

The details depend a little bit on how you model your tree, so the above assumes that a node is either a vector pair where the first element is the value and the second element is a vector of children nodes, or if it is a leaf node then it's not a vector.

So a leaf node is anything that's not a vector. And a node with children is a vector of form: [value [child1 child2 ...]

And here I assumed you wanted to sum the depth of all leaf nodes. Since I see from your answer, that your example gives 42, I'm now thinking you meant the sum of the depth of every node, not just leaves, if so it only takes one extra line of code to do so:

(defn sum-depths
  ([tree]
   (sum-depths tree 0))
  ([node depth]
   (if-not (vector? node)
     depth
     (do
       (apply
        +
        depth
        (for [child-node (second node)]
          (sum-depths child-node (inc depth))))))))

(sum-depths [:root
             [:a1
              [:b1
               [:a2 :b2]]
              :c1]])
;; => 7

(sum-depths ["COM"
             [["B"
               [["C"
                 [["D"
                   [["E"
                     ["F"
                      ["J"
                       [["K"
                         ["L"]]]]]]
                    "I"]]]]
                ["G"
                 ["H"]]]]]])
;; => 42

And like your own answer showed, this particular algorithm can be solved without a stack as well, by doing a level order traversal (aka breadth-first traversal) of the tree. Here it is working on my tree data-structure (similar strategy then your own answer otherwise):

(defn sum-depths [tree]
  (loop [children (second tree) depth 0 total 0]
    (if (empty? children)
      total
      (let [child-depth (inc depth)
            level-total (* (count children) child-depth)]
        (recur (into [] (comp (filter vector?) (mapcat second)) children)
               child-depth
               (+ total level-total))))))

(sum-depths [:root
             [:a1
              [:b1
               [:a2 :b2]]
              :c1]])
;; => 7

(sum-depths ["COM"
             [["B"
               [["C"
                 [["D"
                   [["E"
                     ["F"
                      ["J"
                       [["K"
                         ["L"]]]]]]
                    "I"]]]]
                ["G"
                 ["H"]]]]]])
;; => 42

And for completeness, I also want to show how you can do a depth-first recursive traversal using core.async instead of the function call stack in order to be able to traverse trees that would cause a StackOverFlow otherwise, but still using a stack based recursive depth-first traversal instead of an iterative one. As an aside, there exists some non stack consuming O(1) space depth-first traversals as well, using threaded trees (Morris algorithm) or tree transformations, but I won't show those as I'm not super familiar with them and I believe they only work on binary trees.

First, let's construct a degenerate tree of depth 10000 which causes a StackOverFlow when run against our original stack-recursive sum-depths:

(def tree
  (loop [i 0 t [:a [:b]]]
    (if (< i 10000)
      (recur (inc i)
             [:a [t]])
      t)))

(defn sum-depths
  ([tree]
   (sum-depths tree 0))
  ([node depth]
   (if-not (vector? node)
     depth
     (do
       (apply
        +
        depth
        (for [child-node (second node)]
          (sum-depths child-node (inc depth))))))))

(sum-depths tree)
;; => java.lang.StackOverflowError

If it works on your machine, try increasing 10000 to something even bigger.

Now we rewrite it to use core.async instead:

(require '[clojure.core.async :as async])

(defmacro for* [[element-sym coll] & body]
  `(loop [acc# [] coll# ~coll]
     (if-let [~element-sym (first coll#)]
       (recur (conj acc# (do ~@body)) (next coll#))
       acc#)))

(def tree
  (loop [i 0 t [:a [:b]]]
    (if (< i 10000)
      (recur (inc i)
             [:a [t]])
      t)))

(defn sum-depths
  ([tree]
   (async/<!! (sum-depths tree 0)))
  ([node depth]
   (async/go
     (if-not (vector? node)
       depth
       (do
         (apply
          +
          depth
          (for* [child-node (second node)]
            (async/<!
             (sum-depths child-node (inc depth))))))))))

;; => (sum-depths tree)
50015001

It is relatively easy to rewrite a stack-recursive algorithm to use core.async instead of the call stack, and thus make it so it isn't at risk of causing a StackOverFlow in the case of large inputs. Just wrap it in a go block, and wrap the recursive calls in a <! and the whole algorithm in a <!!. The only tricky part is that core.async cannot cross function boundaries, which is why the for* macro is used above. The normal Clojure for macro crosses function boundaries internally, and thus we can't use <! inside it. By rewriting it to not do so, we can use <! inside it.

Now for this particular algorithm, the tail-recursive variant using loop/recur is probably best, but I wanted to show this technique of using core.async for posterity, since it can be useful in other cases where there isn't a trivial tail-recursive implementation.

Answer 2

You can use map/mapcat to walk a tree recursively to produce a lazy-seq (of leaf nodes). If you need depth information, just add it along the way.

(defn leaf-seq
  [branch? children root]
  (let [walk (fn walk [lvl node]
               (if (branch? node)
                 (->> node
                      children
                      (mapcat (partial walk (inc lvl))))
                 [{:lvl  lvl
                   :leaf node}]))]
    (walk 0 root)))

To run:

(->> '((1 2 ((3))) (4))
     (leaf-seq seq? identity)
     (map :lvl)
     (reduce +))
;; => 10

where the depths of each node are:

(->> '((1 2 ((3))) (4))
     (leaf-seq seq? identity)
     (map :lvl))
;; => (2 2 4 2)

---

Updates - sum all nodes instead of just leaf nodes

I misread the original requirement and was assuming leaf nodes only. To add the branch node back is easy, we just need to cons it before its child sequence.

(defn node-seq
  "Returns all the nodes marked with depth/level"
  [branch? children root]
  (let [walk (fn walk [lvl node]
               (lazy-seq
                (cons {:lvl lvl
                       :node node}
                      (when (branch? node)
                        (->> node
                             children
                             (mapcat (partial walk (inc lvl))))))))]
    (walk 0 root)))

Then we can walk on the hiccup-like tree as before:

(->> ["COM" [["B" [["C" [["D" [["E" [["F"] ["J" [["K" [["L"]]]]]]] ["I"]]]]] ["G" [["H"]]]]]]]
     (node-seq #(s/valid? ::branch %) second)
     (map :lvl)
     (reduce +))
;; => 42

Note: above function uses below helper specs to identify the branch/leaf:

(s/def ::leaf (s/coll-of string? :min-count 1 :max-count 1))
(s/def ::branch (s/cat :tag string? :children (s/coll-of (s/or :leaf ::leaf
                                                               :branch ::branch))))

Answer 3

i would also propose this one, which is kinda straightforward: it uses more or less the same approach, as tail recursive flatten does:

(defn sum-depth 
  ([data] (sum-depth data 1 0))  
  ([[x & xs :as data] curr res]
   (cond (empty? data) res
         (coll? x) (recur (concat x [:local/up] xs) (inc curr) res)
         (= :local/up x) (recur xs (dec curr) res)
         :else (recur xs curr (+ res curr)))))

the trick is that when you encounter the collection at the head of the sequence, you concat it to the rest, adding special indicator that signals the end of branch and level up. It allows you to track the current depth value. Quite simple, and also using one pass.

user> (sum-depth [1 [2 7] [3]])
;;=> 7

user> (sum-depth [1 2 3 [[[[[4]]]]]])
;;=> 9

Answer 4

You can do this using the Tupelo Forest library. Here is a function to extract information about a tree in Hiccup format. First, think about how we want to use the information for a simple tree with 3 nodes:

(dotest
  (hid-count-reset)
  (let [td (tree-data [:a
                       [:b 21]
                       [:c 39]])]
    (is= (grab :paths td) [[1003]
                           [1003 1001]
                           [1003 1002]])
    (is= (grab :node-hids td) [1003 1001 1002])
    (is= (grab :tags td) [:a :b :c])
    (is= (grab :depths td) [1 2 2])
    (is= (grab :total-depth td) 5) ))

Here is how we calculate the above information:

(ns tst.demo.core
  (:use tupelo.forest tupelo.core tupelo.test)
  (:require
    [schema.core :as s]
    [tupelo.schema :as tsk]))

(s/defn tree-data :- tsk/KeyMap
  "Returns data about a hiccup tree"
  [hiccup :- tsk/Vec]
  (with-forest (new-forest)
    (let [root-hid    (add-tree-hiccup hiccup)
          paths       (find-paths root-hid [:** :*])
          node-hids   (mapv xlast paths)
          tags        (mapv #(grab :tag (hid->node %)) node-hids)
          depths      (mapv count paths)
          total-depth (apply + depths)]
      (vals->map paths node-hids tags depths total-depth))))

and an example on a larger Hiccup-format tree:

(dotest
  (let [td (tree-data [:a 
                       [:b 21]
                       [:b 22]
                       [:b
                        [:c
                         [:d
                          [:e
                           [:f
                            [:g 7]
                            [:h
                             [:i 9]]]]]]
                        [:c 32]]
                       [:c 39]])]
    (is= (grab :tags td) [:a :b :b :b :c :d :e :f :g :h :i :c :c])
    (is= (grab :depths td) [1 2 2 2 3 4 5 6 7 7 8 3 2])
    (is= (grab :total-depth td) 52)))

Don't be afraid of stack size for normal processing. On my computer, the default stack doesn't overflow until you get to a stack depth of over 3900 recursive calls. For a binary tree, just 2^30 is over a billion nodes, and 2^300 is more nodes than the number of protons in the universe (approx).

Answer 5

Here's my alternative approach that does use recur:

(defn sum-of-depths
  [branches]
  (loop [branches branches
         cur-depth 0
         total-depth 0]
    (cond
      (empty? branches) total-depth
      :else (recur
             (mapcat (fn [node] (second node)) branches)
             (inc cur-depth)
             (+ total-depth (* (count branches) cur-depth))))))

(def tree ["COM" (["B" (["C" (["D" (["E" (["F"] ["J" (["K" (["L"])])])] ["I"])])] ["G" (["H"])])])])
(sum-of-depths [tree]) ; For the first call we have to wrap the tree in a list.
=> 42

Answer 6

recur is for tail recursion, meaning if you could do it with normal recursion, where the return value is exactly what a single recursive call would return, then you can use it.

Most functions on trees cannot be written in a straightforward way when restricted to using only tail recursion. Normal recursive calls are much more straightforward, and as long as the tree depth is not thousands of levels deep, then normal recursive calls are just fine in Clojure.

The reason you may have found recommendations against using normal recursive calls in Clojure is for cases when the call stack could grow to tens or hundreds of thousands of calls deep, e.g. a recursive call one level deep for each element of a sequence that could be tens or hundreds of thousands of elements long. That would exceed the default maximum call stack depth limits of many run-time systems.