Reagent not rendering as expected when adding a new item at end of reactive vector

Question

I'm working on a tree control in ClojureScript and Reagent. It can be used as a file system navigator, topic navigator, outliner, etc.

When a headline in an outline is selected and being edited, the traditional behavior when Return is tapped is to create a new headline (a child or sibling depending on the expansion state of the headline and whether it already has children or not) then focus it leaving it ready to edit. My control does that all correctly except in the case of editing the last sibling in a group.

In the problematic case, the headline is created as expected, but focusing the new control fails.

I created an MCVE using the figwheel template.

lein new figwheel test-reagent-vector -- --reagent

Here's a listing exhibiting the problem.

(ns test-reagent-vector.core
  (:require [clojure.string :as s]
            [reagent.core :as r]))

(def ^{:constant true} topic-separator \u02D1)
(def empty-test-topic {:topic "Empty Test Topic"})

(defonce global-state-with-hierarchy
         (r/atom {:name "Global Application State, Inc."
                  :data {:one "one" :two 2 :three [3]}
                  :tree [{:topic "First Headline"}
                         {:topic "Middle Headline"}
                         {:topic "Last Headline"}]}))

(defn get-element-by-id
  [id]
  (.getElementById js/document id))

(defn event->target-element
  [evt]
  (.-target evt))

(defn event->target-value
  [evt]
  (.-value (event->target-element evt)))

(defn swap-style-property
  "Swap the specified style settings for the two elements."
  [first-id second-id property]
  (let [style-declaration-of-first (.-style (get-element-by-id first-id))
        style-declaration-of-second (.-style (get-element-by-id second-id))
        value-of-first (.getPropertyValue style-declaration-of-first property)
        value-of-second (.getPropertyValue style-declaration-of-second property)]
    (.setProperty style-declaration-of-first property value-of-second)
    (.setProperty style-declaration-of-second property value-of-first)))

(defn swap-display-properties
  "Swap the display style properties for the two elements."
  [first-id second-id]
  (swap-style-property first-id second-id "display"))

;;------------------------------------------------------------------------------
;; Vector-related manipulations.

(defn delete-at
  "Remove the nth element from the vector and return the result."
  [v n]
  (vec (concat (subvec v 0 n) (subvec v (inc n)))))

(defn remove-last
  "Remove the last element in the vector and return the result."
  [v]
  (subvec v 0 (dec (count v))))

(defn remove-last-two
  "Remove the last two elements in the vector and return the result."
  [v]
  (subvec v 0 (- (count v) 2)))

(defn insert-at
  "Return a copy of the vector with new-item inserted at the given n. If
  n is less than zero, the new item will be inserted at the beginning of
  the vector. If n is greater than the length of the vector, the new item
  will be inserted at the end of the vector."
  [v n new-item]
  (cond (< n 0) (into [new-item] v)
        (>= n (count v)) (conj v new-item)
        :default (vec (concat (conj (subvec v 0 n) new-item) (subvec v n)))))

(defn replace-at
  "Replace the current element in the vector at index with the new-element
  and return it."
  [v index new-element]
  (insert-at (delete-at v index) index new-element))

;;------------------------------------------------------------------------------
;; Tree id manipulation functions.

(defn tree-id->tree-id-parts
  "Split a DOM id string (as used in this program) into its parts and return
  a vector of the parts"
  [id]
  (s/split id topic-separator))

(defn tree-id-parts->tree-id-string
  "Return a string formed by interposing the topic-separator between the
  elements of the input vector."
  [v]
  (str (s/join topic-separator v)))

(defn increment-leaf-index
  "Given the tree id of a leaf node, return an id with the node index
  incremented."
  [tree-id]
  (let [parts (tree-id->tree-id-parts tree-id)
        index-in-vector (- (count parts) 2)
        leaf-index (int (nth parts index-in-vector))
        new-parts (replace-at parts index-in-vector (inc leaf-index))]
    (tree-id-parts->tree-id-string new-parts)))

(defn change-tree-id-type
  "Change the 'type' of a tree DOM element id to something else."
  [id new-type]
  (let [parts (tree-id->tree-id-parts id)
        shortened (remove-last parts)]
    (str (tree-id-parts->tree-id-string shortened) (str topic-separator new-type))))

(defn tree-id->nav-vector-and-index
  "Parse the id into a navigation path vector to the parent of the node and an
  index within the vector of children. Return a map containing the two pieces
  of data. Basically, parse the id into a vector of information to navigate
  to the parent (a la get-n) and the index of the child encoded in the id."
  [tree-id]
  (let [string-vec (tree-id->tree-id-parts tree-id)
        idx (int (nth string-vec (- (count string-vec) 2)))
        without-last-2 (remove-last-two string-vec)
        without-first (delete-at without-last-2 0)
        index-vector (mapv int without-first)
        interposed (interpose :children index-vector)]
    {:path-to-parent (vec interposed) :child-index idx}))

;;------------------------------------------------------------------------------
;; Functions to manipulate the tree and subtrees.

(defn add-child!
  "Insert the given topic at the specified index in the parents vector of
  children. No data is deleted."
  [parent-topic-ratom index topic-to-add]
  (swap! parent-topic-ratom insert-at index topic-to-add))

(defn graft-topic!
  "Add a new topic at the specified location in the tree. The topic is inserted
  into the tree. No data is removed. Any existing information after the graft
  is pushed down in the tree."
  [root-ratom id-of-desired-node topic-to-graft]
  (let [path-and-index (tree-id->nav-vector-and-index id-of-desired-node)]
    (add-child! (r/cursor root-ratom (:path-to-parent path-and-index))
                (:child-index path-and-index) topic-to-graft)))

;;;-----------------------------------------------------------------------------
;;; Functions to handle keystroke events.

(defn handle-enter-key-down!
  "Handle a key-down event for the Enter/Return key. Insert a new headline
  in the tree and focus it, ready for editing."
  [root-ratom span-id]
  (let [id-of-new-child (increment-leaf-index span-id)]
    (graft-topic! root-ratom id-of-new-child empty-test-topic)
    (let [id-of-new-editor (change-tree-id-type id-of-new-child "editor")
          id-of-new-label (change-tree-id-type id-of-new-child "label")]
      (swap-display-properties id-of-new-label id-of-new-editor)
      (.focus (get-element-by-id id-of-new-editor)))))

(defn handle-key-down
  "Detect key-down events and dispatch them to the appropriate handlers."
  [evt root-ratom span-id]
  (when
    (= (.-key evt) "Enter") (handle-enter-key-down! root-ratom span-id)))

;;;-----------------------------------------------------------------------------
;;; Functions to build the control.

(defn build-topic-span
  "Build the textual part of a topic/headline."
  [root-ratom topic-ratom span-id]
  (let [label-id (change-tree-id-type span-id "label")
        editor-id (change-tree-id-type span-id "editor")]
    [:span
     [:label {:id      label-id
              :style   {:display :initial}
              :onClick (fn [e]
                         (swap-display-properties label-id editor-id)
                         (.focus (get-element-by-id editor-id))
                         (.stopPropagation e))}
      @topic-ratom]
     [:input {:type      "text"
              :id        editor-id
              :style     {:display :none}
              :onKeyDown #(handle-key-down % root-ratom span-id)
              :onFocus   #(.stopPropagation %)
              :onBlur    #(swap-display-properties label-id editor-id)
              :onChange  #(reset! topic-ratom (event->target-value %))
              :value     @topic-ratom}]]))

(defn tree->hiccup
  "Given a data structure containing a hierarchical tree of topics, generate
  hiccup to represent that tree. Also generates a unique, structure-based
  id that is included in the hiccup so that the correct element in the
  application state can be located when its corresponding HTML element is
  clicked."
  ([root-ratom]
   (tree->hiccup root-ratom root-ratom "root"))
  ([root-ratom sub-tree-ratom path-so-far]
   [:ul
    (doall
      (for
        [index (range (count @sub-tree-ratom))]
        (let [t (r/cursor sub-tree-ratom [index])
              topic-ratom (r/cursor t [:topic])
              id-prefix (str path-so-far topic-separator index)
              topic-id (str id-prefix topic-separator "topic")
              span-id (str id-prefix topic-separator "span")]
          ^{:key topic-id}
          [:li {:id topic-id}
           [:div (build-topic-span root-ratom topic-ratom span-id)]])))]))

(defn home
  "Return a function to layout the home (only) page."
  [app-state-atom]
  (fn [app-state-ratom]
    [:div (tree->hiccup (r/cursor app-state-ratom [:tree]))]))

(r/render-component [home global-state-with-hierarchy]
                    (get-element-by-id "app"))

(I assume some of this is not relevant to the problem, like the tree id manipulation functions. They are just here to make building the example easier.)

The control uses a vector to contain siblings, something about inserting a new element at the end of the vector seems to cause the timing of rendering to change.

When the user has the last item selected and clicks Return, an error message appears in the browser console about a null argument being passed to get-element-by-id. This is triggered by the keyboard handling function handle-enter-key-down!.

The items in the list of headlines are really two HTML elements: a label that displays when the user is not editing it, and a text input that is shown during editing. When a new headline is created, the swap-display-properties function is called to make the editor visible, then it is focused.

When a headline is created at the end of a vector of siblings, the DOM identifiers for the new label and text input are not available to switch the visibility of the two elements. Thus the error message about a null argument to get-element-by-id.

But it works correctly for all of the other positions.

I've reproduced this

• on a Mac
• with OpenJDK 9 and 11
• with the original versions of the dependencies used in the template and after updating them to current
• on Safari and Firefox

I can force it to work by delaying the call to swap-display-properties by 25ms or longer.

;; Wait for rendering to catch up.
   (js/setTimeout #(do (swap-display-properties id-of-new-label id-of-new-editor)
                       (.focus (get-element-by-id id-of-new-editor))) 25)

I assume I could do something with Reacts componentDidMount method, but I don't understand why there is a failure only when inserting a new headline at the end of a vector of siblings.

So...

• Is it just a coincidence that the other cases work as expected?
• Am I misunderstanding something about the Reagent workflow?
• Is there some problem with Reagent?

Any ideas would be appreciated.

Answer 1

Browser focus is difficult and confusing to maintain.

Here's what I think is happening when you hit enter

The keydown event fires

• You add the new topic via graft-topic!

• You switch the styles such that the input is showing and the label is hidden

• You focus on the next element in the list

then, after the keydown event is done, reagent rerenders

• When this happens the element that is focused is replaced by the new element you created

In the case that the element you are hitting enter from is not the final element in the list

• You are able to focus on the already existing element and then later reagent replaces this element with the new element created in graft-topic!

In the case that the element you are hitting enter from is the final element in the list

• Focusing fails to happen because no element with that id exists yet

• therefore no element is in focus when reagent rerenders with the newly created element

What the browser is doing

The new element you created is in the same place as the old focused element, so the browser keeps the focus in that place

You can test this with the following code snippet which switches the two inputs on every keydown.

Despite having different ids and being different components, focus remains in the same place it was even when swapping the two components

(defn test-comp []
  (r/with-let [*test? (r/atom true)]
    [:div
     (if @*test?
       [:div
        [:input
         {:value "test"
          :id "test"
          :on-key-down #(swap! *test? not)}]
        [:input
         {:value "not test"
          :id "not test"
          :on-key-down #(swap! *test? not)}]]
       [:div
        [:input
         {:value "not test"
          :id "not test"
          :on-key-down #(swap! *test? not)}]
        [:input
         {:value "test"
          :id "test"
          :on-key-down #(swap! *test? not)}]])]))

(note: this will give you a warning about not having an on-change handler, but that's not important for this demo, just wanted to specify the value so you could see the two inputs swap places, but the focus remain in the same place)

As to how to fix this...

Don't rely on waiting a cycle or using js timeout to fix this, that just wastes precious time

I would recommend not using the browser to retain focus

The simple answer is to keep what index is focused in app-state, and then decide whether the label or the input is rendered based on what

Then add an auto-focus attribute to the input so that when it renders it will come into focus

Some pointers for how to use reagent

In your code you are resolving the reagent component with (), but what you should do is use []

This relates to how reagent decides when to rerender components, but since you resolve the whole tree, every time you change an atom that you derefed it will rerender your whole tree, not just the place you derefed the atom. (test this by adding a println to your code in the build-topic-span component)

Define cursors in a form-2 component (or use with-let), they only need to be defined once per component, so no need to have them be redefined on every subsequent render (not sure if this will lead to bugs, but it's good practice)

also you can use cursor like get-in, so instead of

t           (r/cursor sub-tree-ratom [index])
topic-ratom (r/cursor t [:topic])

you can do

topic-ratom (r/cursor t [index :topic])

Some other notes

The swapping styles thing you're doing is confusing, if you keep track of what is focused, you can just render a different component depending on what is focused, no need to have both the label and the input in the dom at the same time.

passing around a bunch of string ids is very confusing, especially when calling graft-topic! you destructure the string back into the path. Data is much easier to work with, keep the path in a vector and make it a string only when it needs to be

This example refactored with these things in mind

(ns test-reagent-vector.core
  (:require [clojure.string :as s]
            [reagent.core :as r]))

(def ^{:constant true} topic-separator \u02D1)
(def empty-test-topic {:topic "Empty Test Topic"})

(defonce global-state-with-hierarchy
  (r/atom {:name          "Global Application State, Inc."
           :focused-index nil
           :data          {:one "one" :two 2 :three [3]}
           :tree          [{:topic "First Headline"}
                           {:topic "Middle Headline"}
                           {:topic "Last Headline"}]}))

(defn get-element-by-id
  [id]
  (.getElementById js/document id))

(defn event->target-element
  [evt]
  (.-target evt))

(defn event->target-value
  [evt]
  (.-value (event->target-element evt)))

(defn swap-style-property
  "Swap the specified style settings for the two elements."
  [first-id second-id property]
  (let [style-declaration-of-first (.-style (get-element-by-id first-id))
        style-declaration-of-second (.-style (get-element-by-id second-id))
        value-of-first (.getPropertyValue style-declaration-of-first property)
        value-of-second (.getPropertyValue style-declaration-of-second property)]
    (.setProperty style-declaration-of-first property value-of-second)
    (.setProperty style-declaration-of-second property value-of-first)))

(defn swap-display-properties
  "Swap the display style properties for the two elements."
  [first-id second-id]
  (swap-style-property first-id second-id "display"))

;;------------------------------------------------------------------------------
;; Vector-related manipulations.

(defn delete-at
  "Remove the nth element from the vector and return the result."
  [v n]
  (vec (concat (subvec v 0 n) (subvec v (inc n)))))

(defn remove-last
  "Remove the last element in the vector and return the result."
  [v]
  (subvec v 0 (dec (count v))))

(defn remove-last-two
  "Remove the last two elements in the vector and return the result."
  [v]
  (subvec v 0 (- (count v) 2)))

(defn insert-at
  "Return a copy of the vector with new-item inserted at the given n. If
  n is less than zero, the new item will be inserted at the beginning of
  the vector. If n is greater than the length of the vector, the new item
  will be inserted at the end of the vector."
  [v n new-item]
  (cond (< n 0) (into [new-item] v)
        (>= n (count v)) (conj v new-item)
        :default (vec (concat (conj (subvec v 0 n) new-item) (subvec v n)))))

(defn replace-at
  "Replace the current element in the vector at index with the new-element
  and return it."
  [v index new-element]
  (insert-at (delete-at v index) index new-element))

;;------------------------------------------------------------------------------
;; Tree id manipulation functions.

(defn tree-id->tree-id-parts
  "Split a DOM id string (as used in this program) into its parts and return
  a vector of the parts"
  [id]
  (s/split id topic-separator))

(defn tree-id-parts->tree-id-string
  "Return a string formed by interposing the topic-separator between the
  elements of the input vector."
  [v]
  (str (s/join topic-separator v)))

(defn increment-leaf-index
  "Given the tree id of a leaf node, return an id with the node index
  incremented."
  [tree-id]
  (let [parts (tree-id->tree-id-parts tree-id)
        index-in-vector (- (count parts) 2)
        leaf-index (int (nth parts index-in-vector))
        new-parts (replace-at parts index-in-vector (inc leaf-index))]
    (tree-id-parts->tree-id-string new-parts)))

(defn change-tree-id-type
  "Change the 'type' of a tree DOM element id to something else."
  [id new-type]
  (let [parts (tree-id->tree-id-parts id)
        shortened (remove-last parts)]
    (str (tree-id-parts->tree-id-string shortened) (str topic-separator new-type))))

(defn tree-id->nav-vector-and-index
  "Parse the id into a navigation path vector to the parent of the node and an
  index within the vector of children. Return a map containing the two pieces
  of data. Basically, parse the id into a vector of information to navigate
  to the parent (a la get-n) and the index of the child encoded in the id."
  [tree-id]
  (let [string-vec (tree-id->tree-id-parts tree-id)
        idx (int (nth string-vec (- (count string-vec) 2)))
        without-last-2 (remove-last-two string-vec)
        without-first (delete-at without-last-2 0)
        index-vector (mapv int without-first)
        interposed (interpose :children index-vector)]
    {:path-to-parent (vec interposed) :child-index idx}))

;;------------------------------------------------------------------------------
;; Functions to manipulate the tree and subtrees.

(defn add-child!
  "Insert the given topic at the specified index in the parents vector of
  children. No data is deleted."
  [parent-topic-ratom index topic-to-add]
  (swap! parent-topic-ratom insert-at index topic-to-add))

(defn graft-topic!
  "Add a new topic at the specified location in the tree. The topic is inserted
  into the tree. No data is removed. Any existing information after the graft
  is pushed down in the tree."
  [root-ratom id-of-desired-node topic-to-graft]
  (let [path-and-index (tree-id->nav-vector-and-index id-of-desired-node)]
    (add-child! (r/cursor root-ratom (:path-to-parent path-and-index))
                (:child-index path-and-index) topic-to-graft)))

;;;-----------------------------------------------------------------------------
;;; Functions to handle keystroke events.

(defn handle-enter-key-down!
  "Handle a key-down event for the Enter/Return key. Insert a new headline
  in the tree and focus it, ready for editing."
  [app-state root-ratom index]
  (add-child! root-ratom (inc index) empty-test-topic)
  (swap! app-state update :focused-index inc)
  )

(defn handle-key-down
  "Detect key-down events and dispatch them to the appropriate handlers."
  [evt app-state root-ratom index]
  (when (= (.-key evt) "Enter")
    (handle-enter-key-down! app-state root-ratom index)))

;;;-----------------------------------------------------------------------------
;;; Functions to build the control.

(defn build-topic-span
  "Build the textual part of a topic/headline."
  [root-ratom index]
  (r/with-let [topic-ratom   (r/cursor root-ratom [index :topic])
               focused-index (r/cursor global-state-with-hierarchy [:focused-index])]
    (if-not (= index @focused-index)
      [:label
       {:onClick #(reset! focused-index index)}
       @topic-ratom]
      [:input {:type      "text"
               :auto-focus true
               :onKeyDown #(handle-key-down % global-state-with-hierarchy root-ratom index)
               :onChange  #(reset! topic-ratom (event->target-value %))
               :on-blur #(when (= index @focused-index)
                           (reset! focused-index nil))
               :value     @topic-ratom}])))


(defn tree->hiccup
  "Given a data structure containing a hierarchical tree of topics, generate
  hiccup to represent that tree. Also generates a unique, structure-based
  id that is included in the hiccup so that the correct element in the
  application state can be located when its corresponding HTML element is
  clicked."
  ([root-ratom]
   [tree->hiccup root-ratom root-ratom "root"])
  ([root-ratom sub-tree-ratom path-so-far]
   [:ul
    (doall
     (for [index (range (count @sub-tree-ratom))]
       ^{:key (str index)}
       [:li
        [:div
         [build-topic-span root-ratom index]]]
       ))]))

(defn home
  "Return a function to layout the home (only) page."
  [app-state-ratom]
  (r/with-let [tree-ratom (r/cursor app-state-ratom [:tree])]
    [:div
     [tree->hiccup tree-ratom]]))


(r/render
 [home global-state-with-hierarchy]
 (get-element-by-id "app"))

I only changed home, tree→hiccup, build topic span and handle keydown.

In the future

The example I wrote up assumes this is a flat list, but seems like you're planning on making this a nested list in the future and if that's true I would recommend changing some things

associate a unique id to every topic and use that id to determine if that element is in focus

specify the path-so-far as a vector of the ids up to that point in the tree

don't specify the the key as a function of the index, what if the element switches places with another element in the tree? we don't want to rerender it. Base this off a unique id

investigate the reagent track! function to cut down on rerenders when asking if the current element is focused

Hope this helps

Feel free to message me if you have any more questions regarding how to build a nested interactive list :)

Answer 2

After the responses from Joshua Brown and Alan Thompson, I got to reviewing the API docs in Reagent again to understand what with-let did.

Then I noticed after-render, which was exactly what I needed. To fix the problem in my example, add after-render in the handle-enter-key-down! like this.

(defn handle-enter-key-down!
  "Handle a key-down event for the Enter/Return key. Insert a new headline
  in the tree and focus it, ready for editing."
  [root-ratom span-id]
  (let [id-of-new-child (increment-leaf-index span-id)]
    (graft-topic! root-ratom id-of-new-child empty-test-topic)
    (let [id-of-new-editor (change-tree-id-type id-of-new-child "editor")
          id-of-new-label (change-tree-id-type id-of-new-child "label")]
      (r/after-render
        (fn []
          (swap-display-properties id-of-new-label id-of-new-editor)
          (.focus (get-element-by-id id-of-new-editor)))))))

Since the identifiers for the new label and text input exist after the render, swapping their display properties now works as expected and the newly visible input can be focused.

I believe this also fixes the potential race condition that existed before (but did not manifest) when inserting new headlines at other positions in the vector as well.

Answer 3

I think you have already identified the problem as a race condition between adding the new element in Reagent and it getting created in the DOM (where get-element-by-id is looking for it).

The simplest answer (besides adding 25 ms sleeps everywhere) is to use an event loop library like re-frame to schedule a "set-focus" event which will be processed on the next pass through the event loop.

---

As an aside, I never use concat or subvec. Keep it simple with take & drop, and always wrap the output of a fn with (vec ...) to force it in to a plain vector w/o any sneaky/problematic laziness.