CODEWITHSUNDEEP
mapreducecouchbase
shmish111
shmish111

Reputation: 3797

Couchbase lag and views

I think that Couchbase may be able to solve a lot of problems in my current project but there are a few basic things that I need to be sure of first:

1) Say I have a cluster with 10 nodes, I add a document to node A. I then ask node B for that document, will I definitely get the document back? Is there a risk that I ask node B and it thinks the document doesn't exist?

2) Now say I write a view so that I can see all documents for a certain author. I add a document to node A. I know that there is a certain lag time for this document to appear in the view as it is being indexed. What I want to know is will that lag time be very very small as I am only indexing 1 document?

Basically I have a website which manages something in the order of 100 million documents. There are thousands of users at any one time so there are around 10 - 20 servers running. With the current architecture, if I insert a document into node A, when I look for that document I may well be accessing node B to get it. So my overall question is:

The user inserts a document into node A, I then show them a screen of all their documents. If I were using Couchbase, this screen would be the result of querying a view keyed on Author. How likely is it that the user won't be able to see the document they just posted?

Upvotes: 2

Views: 503

Answers (2)

mikewied
mikewied

Reputation: 5343

  1. You will always be able to get the document back no matter what node you ask. If you use one of the Couchbase SDK's then they will always make sure they ask the node that has the document. If you use moxi then it will do the same so you shouldn't need to worry about this.

  2. This depends on how heavy your write workload is. In the current architecture (2.0) a document must hit disk before it is indexed. If you have a heavy write workload then the lag will increase since there will be a longer line of items that need to be written to disk. I suggest doing benchmarking with your specific application requirements in mind for this use case and posting any problems or issues you have on the Couchbase forums.

Also, you will want to take a look at the commands that have durability requirements and doing queries with the stale query parameter set to false. The durability commands will allow you to have you application block until an item hits disk and doing a query with stale set to false will make sure your view is up to date with the latest items put into Couchbase. This process will ensure your users always see the latest results of you

EDIT: Couchbase 3.0 no longer requires items to be persisted to disk before they can be queried.

Upvotes: 2

elwarren
elwarren

Reputation: 326

View updates are a tunable parameter. By default a thread wakes up every 5 seconds, checks for changes and updates the index if it finds them. Either tune these parameters or use the view parameter stale=false for queries.

From the docs under "Automated index updates":

In addition to a configurable update interval, you can also update all indexes automatically in the background. You configure automated update through two parameters, the update time interval in seconds and the number of document changes that occur before the views engine updates an index. These two parameters are updateInterval and updateMinChanges:

  • updateInterval: the time interval in milliseconds, default is 5000 milliseconds. At every updateInterval the views engine checks if the number of document mutations on disk is greater than updateMinChanges. If true, it triggers view update. The documents stored on disk potentially lag documents that are in-memory for tens of seconds.
  • updateMinChanges: the number of document changes that occur before re-indexing occurs, default is 5000 changes.

http://docs.couchbase.com/admin/admin/Views/views-operation.html

Upvotes: 0

Related Questions