CODEWITHSUNDEEP
jsonrecursionclojuredatomic
adamneilson
adamneilson

Reputation: 758

Serializing recursive refs in Datomic

I have a user entity type in my Datomic database which can follow other user types. My issue comes when one user follows another user who is already following them:

User A follows user B and also User B follows user A

When I try to serialize (using Cheshire) I get a StackOverflowError because of (I'm guessing) infinite recursion on the :user/follows-users attribute.

How would I go about serializing (to json for an API) two Datomic entities that reference each another in such a way?

Here's a basic schema:

; schema
[{:db/id #db/id[:db.part/db]
:db/ident :user/username
:db/valueType :db.type/string
:db/cardinality :db.cardinality/one
:db/unique :db.unique/identity
:db.install/_attribute :db.part/db}

{:db/id #db/id[:db.part/db]
:db/ident :user/follows-users
:db/valueType :db.type/ref
:db/cardinality :db.cardinality/many
:db.install/_attribute :db.part/db}

; create users  
{:db/id #db/id[:db.part/user -100000]
 :user/username "Cheech"} 
{:db/id #db/id[:db.part/user -200000]
 :user/username "Chong"}

; create follow relationships
{:db/id #db/id[:db.part/user -100000]
 :user/follows-users  #db/id[:db.part/user -200000]}
{:db/id #db/id[:db.part/user -200000]
 :user/follows-users  #db/id[:db.part/user -100000]}]

And once the database is set up etc. on repl:

user=> (use '[cheshire.core :refer :all])
nil

user=> (generate-string (d/touch (d/entity (d/db conn) [:user/username "Cheech"]))) 
StackOverflowError   clojure.lang.RestFn.invoke (RestFn.java:433)

Upvotes: 2

Views: 553

Answers (2)

adamneilson
adamneilson

Reputation: 758

I'm a bit of a n00b to Datomic and am certain there must be a more idiomatic way of doing what @arthur-ulfeldt suggests above but in case anyone else is looking for a quick pointer on how to go about serializing Datomic EntityMaps into json where a self-referencing ref exists, here's the code that solves my problem:

(defn should-pack?
  "Returns true if the attribute is type 
  ref with a cardinality of many"
  [attr]
  (->>
    (d/q '[:find ?attr
           :in $ ?attr
           :where
           [?attr :db/valueType ?type]
           [?type :db/ident :db.type/ref]
           [?attr :db/cardinality ?card]
           [?card :db/ident :db.cardinality/many]]
         (d/db CONN) attr)
    first
    empty?
    not))

(defn make-serializable 
  "Stop infinite loops on recursive refs" 
  [entity]
  (def ent (into {} entity))
  (doseq [attr ent]
    (if (should-pack? (first attr))
      (def ent (assoc ent 
                      (first attr) 
                      (map #(get-entity-id %) (first (rest attr)))))))
  ent)

Upvotes: 0

Arthur Ulfeldt
Arthur Ulfeldt

Reputation: 91587

The eager expansion of linked data structures is only safe in any language if they are cycle free. An api that promises to "eagerly expand data only until a cycle is found and then switch to linking (by user id)" may be harder to consume reliably than one that never expanded and always returned enough users to follow all the links in the response. For instance the request above could return the JSON:

[{"id": -100000,
  "username": "Cheech",
  "follows-users": [-200000]}
 {"id": -200000,
  "username": "Chong",
  "follows-users": [-100000]}]

Where the list of selected users is found by reducing walk of the users graph into a set.

Upvotes: 1

Related Questions