CODEWITHSUNDEEP
javacouchbase
Abinash
Abinash

Reputation: 103

Why does bulk load causes indexing on Couchbase Server?

I am reading bulk data from a Couchbase bucket and inserting this bulk data to another bucket. I am using couchbase java sdk 1.4.4 version.Using views for reading the whole data.

I am using the set api to insert the data. After inserting the whole data into Couchbase, I am closing the Couchbase client.

Data gets inserted into Couchbase but as soon as it gets inserted, it starts indexing the data which I can see from couch base web console. As per couch base documentation, it should start indexing when we are reading the data , based on STALE parameter.

The other issue which I am facing is, at my java application console, it shows lots of warnings until indexing is not done.

jvm 1    | WARN  [OperationFuture] Exception thrown wile executing com.couchbase.client.CouchbaseClient$15.operationComplete()
jvm 1    | java.lang.IllegalStateException: Shutting down
jvm 1    |      at net.spy.memcached.MemcachedClient.broadcastOp(MemcachedClient.java:298) ~[spymemcached-2.11.4.jar:2.11.4]
jvm 1    |      at net.spy.memcached.MemcachedClient.broadcastOp(MemcachedClient.java:292) ~[spymemcached-2.11.4.jar:2.11.4]
jvm 1    |      at com.couchbase.client.CouchbaseClient.observe(CouchbaseClient.java:1670) ~[couchbase-client-1.4.4.jar:1.4.4]
jvm 1    |      at com.couchbase.client.CouchbaseClient.observePoll(CouchbaseClient.java:1803) ~[couchbase-client-1.4.4.jar:1.4.4]
jvm 1    |      at com.couchbase.client.CouchbaseClient$15.onComplete(CouchbaseClient.java:1443) ~[couchbase-client-1.4.4.jar:1.4.4]
jvm 1    |      at com.couchbase.client.CouchbaseClient$15.onComplete(CouchbaseClient.java:1412) ~[couchbase-client-1.4.4.jar:1.4.4]
jvm 1    |      at net.spy.memcached.internal.AbstractListenableFuture$1.run(AbstractListenableFuture.java:117) ~[spymemcached-2.11.4.
jvm 1    |      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [na:1.7.0_21]
jvm 1    |      at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [na:1.7.0_21]
jvm 1    |      at java.util.concurrent.FutureTask.run(FutureTask.java:166) [na:1.7.0_21]
jvm 1    |      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_21]
jvm 1    |      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_21]
jvm 1    |      at java.lang.Thread.run(Thread.java:722) [na:1.7.0_21]

Questions:
Why is index creation starting before reading the views?
Why are there warning messages coming on console?.

Please let me know if you need more information.

Upvotes: 1

Views: 392

Answers (1)

Paddy
Paddy

Reputation: 1195

As documented in the Couchbase server manual index are run automatically. Here are the default parameters for the automated index updates.

{
     "updateInterval":5000,
     "updateMinChanges":5000,
     "replicaUpdateMinChanges":5000 
}

With these setting the system will check every 5 seconds to see if there has been 5000 mutations made to the bucket. If that is true it will start indexing. Once the bulk load code does 5000 sets the indexing will began.

You can read more about automated index updates in the manual.

Upvotes: 1

Related Questions