CODEWITHSUNDEEP
solrriak
Matt
Matt

Reputation: 1038

Does riak 2.0 allow object updates to be atomic with solr updates?

When I update an object that which is being indexed by solr, I enter an inconsistent state between riak and solr. Is it possible to create a post commit hook to inform me when the update to solr has been completed so that I can guarantee that the cluster is consistent? Or do I have to think of a more Is there a clever way to handle eventual consistency?

Here are reproducible steps that describe the problem:

curl localhost:8098/buckets/bucket/keys/3eaVbY3BWgIN3BFv4riEc6cAqPk -d'{ "key":"3eaVbY3BWgIN3BFv4riEc6cAqPk" }' -H 'Content-Type: application/json'
curl -i -XDELETE localhost:8098/buckets/bucket/keys/3eaVbY3BWgIN3BFv4riEc6cAqPk && curl 'localhost:8098/search/query/index_name?q=key:3eaVbY3BWgIN3BFv4riEc6cAqPk&wt=json'

HTTP/1.1 204 No Content
Vary: Accept-Encoding
Server: MochiWeb/1.1 WebMachine/1.10.5 (jokes are better explained)
Date: Mon, 29 Dec 2014 21:31:58 GMT
Content-Type: application/json
Content-Length: 0

{"responseHeader":{"status":0,"QTime":6,"params":{"q":"key:3eaVbY3BWgIN3BFv4riEc6cAqPk","shards":"127.0.0.1:8093/internal_solr/index_name","127.0.0.1:8093":"(_yz_pn:62 AND (_yz_fpn:62)) OR _yz_pn:61 OR _yz_pn:58 OR _yz_pn:55 OR _yz_pn:52 OR _yz_pn:49 OR _yz_pn:46 OR _yz_pn:43 OR _yz_pn:40 OR _yz_pn:37 OR _yz_pn:34 OR _yz_pn:31 OR _yz_pn:28 OR _yz_pn:25 OR _yz_pn:22 OR _yz_pn:19 OR _yz_pn:16 OR _yz_pn:13 OR _yz_pn:10 OR _yz_pn:7 OR _yz_pn:4 OR _yz_pn:1","wt":"json"}},
    "response":{
       "numFound":1,"start":0,"maxScore":2.9095426,"docs":[              
           {
              "key":"3eaVbY3BWgIN3BFv4riEc6cAqPk",
              "_yz_id":"1*default*index_name*3eaVbY3BWgIN3BFv4riEc6cAqPk*4",
              "_yz_rk":"3eaVbY3BWgIN3BFv4riEc6cAqPk",
              "_yz_rt":"default",
              "_yz_rb":"index_name"
           }
        ]
      }
  }

Upvotes: 0

Views: 134

Answers (1)

Joe
Joe

Reputation: 28356

Using Yokozuna search in Riak 2.0+, the search indexes are updated by the riak_kv_vnode module here. This means that the index function is not called via precommit hook like it was in legacy search, instead is called asynchronously after the put process has passed the object to be stored to the backend, i.e. after the reply has been sent to the client.

The code that acutally performs the indexing is here. I don't see any place in there that would permit a notification event or hook without needing to compile the module from source.

I would expect that index process would finish rather quickly, but it could leave a window perhaps as large a several seconds between the object being updated in kv and the corresponding update appearing in the search index.

Upvotes: 3

Related Questions