CODEWITHSUNDEEP
jsonwindowscouchbasenosql
Matthew Groves
Matthew Groves

Reputation: 26169

Couchbase cbdocloader not loading documents from zip file

I recently installed Couchbase 4.5 beta on Windows 10. I'm following along with the free training videos and labs at learn.couchbase.com, specifically the CB110 course.

One step in the lab requires me to load up sample data with cbdocloader. I did this before with Couchbase 4.5 developer preview and it worked fine, but now it's not loading any documents.

It creates the bucket, but it doesn't load documents into it. Here's my powershell output:

PS C:\Users\mgroves\Desktop> cbdocloader -u Administrator -p password -b couchmu
sic1 -n 127.0.0.1:8091 -s 100 .\couchmusic1-countries-20151228-win.zip
[2016-05-12 10:23:50,480] - [rest_client] [6240] - INFO - existing buckets : [u'
couchmusic1', u'hello-couchbase', u'travel-sample']
[2016-05-12 10:23:50,496] - [rest_client] [6240] - INFO - found bucket couchmusi
c1
bucket creation is successful
.
bucket: couchmusic1-countries-20151228-win.zip, msgs transferred...
       :                total |       last |    per sec
 byte  :                    0 |          0 |        0.0
done
PS C:\Users\mgroves\Desktop>

I've made one of the zip files available on dropbox if you'd like to try: couchmusic1-countries-20151228.zip

I suspect this is probably user error and not related to the Couchbase release, but I don't know for sure.

UPDATE: I ran with the -v flag (v for "verbose"), and below is the output from that. I'm still not seeing what the issue is:

PS C:\Users\mgroves\Desktop> cbdocloader -u Administrator -p password -b couchmu
sic1 -n 127.0.0.1:8091 -s 100 -v .\couchmusic1-countries-20151228-win.zip
[2016-05-12 10:40:06,549] - [rest_client] [7764] - INFO - existing buckets : [u'
couchmusic1', u'hello-couchbase', u'travel-sample']
[2016-05-12 10:40:06,561] - [rest_client] [7764] - INFO - found bucket couchmusi
c1
bucket creation is successful
2016-05-12 10:40:06,594: mt cbtransfer...
2016-05-12 10:40:06,595: mt  source : json://.\couchmusic1-countries-20151228-wi
n.zip
2016-05-12 10:40:06,605: mt  sink   : http://127.0.0.1:8091
2016-05-12 10:40:06,612: mt  opts   : {'username': '<xxx>', 'destination_vbucket
_state': 'active', 'verbose': 1, 'extra': {'max_retry': 10.0, 'rehash': 0.0, 'dc
p_consumer_queue_length': 1000.0, 'data_only': 0.0, 'uncompress': 0.0, 'nmv_retr
y': 1.0, 'conflict_resolve': 1.0, 'cbb_max_mb': 100000.0, 'report': 5.0, 'mcd_co
mpatible': 1.0, 'try_xwm': 1.0, 'backoff_cap': 10.0, 'batch_max_bytes': 400000.0
, 'report_full': 2000.0, 'flow_control': 1.0, 'batch_max_size': 1000.0, 'seqno':
 0.0, 'design_doc_only': 0.0, 'recv_min_bytes': 4096.0}, 'ssl': False, 'threads'
: 4, 'key': None, 'password': '<xxx>', 'id': None, 'destination_operation': None
, 'source_vbucket_state': 'active', 'silent': False, 'dry_run': False, 'single_n
ode': False, 'bucket_destination': 'couchmusic1', 'vbucket_list': None, 'bucket_
source': None}
2016-05-12 10:40:06,726: mt bucket: couchmusic1-countries-20151228-win.zip
2016-05-12 10:40:06,749: w3   source : json://.\couchmusic1-countries-20151228-w
in.zip(couchmusic1-countries-20151228-win.zip@N/A)
2016-05-12 10:40:06,760: w3   sink   : http://127.0.0.1:8091(couchmusic1-countri
es-20151228-win.zip@N/A)
2016-05-12 10:40:06,767: w3          :                total |       last |    pe
r sec
2016-05-12 10:40:06,772: w3    batch :                    1 |          1 |
 28.6
2016-05-12 10:40:06,776: w3    byte  :                    0 |          0 |
  0.0
2016-05-12 10:40:06,779: w3    msg   :                    0 |          0 |
  0.0
.
bucket: couchmusic1-countries-20151228-win.zip, msgs transferred...
       :                total |       last |    per sec
 batch :                    1 |          1 |        8.0
 byte  :                    0 |          0 |        0.0
 msg   :                    0 |          0 |        0.0
done
PS C:\Users\mgroves\Desktop>

Upvotes: 0

Views: 787

Answers (1)

Matthew Groves
Matthew Groves

Reputation: 26169

Turns out, this is an issue that occurred because of a change between Couchbase 4.5 developer preview and Couchbase 4.5 beta.

Apparently, these couchmusic json files aren't in the correct format. There was a case to allow the import of these invalid json zip files for backwards compatibility (see https://github.com/couchbase/couchbase-cli/commit/3794ffa8fdfcdd5224cb4e332d5ef882aa8140b5). However, another case appears to have broken this (see: https://github.com/couchbase/couchbase-cli/commit/c892c9241d1e6997fa30317af791d6fcde73aeaa).

In any case, there are two problems: 1) The example json files for couchmusic aren't in the correct format 2) Backwards compatibility import is broken

I've spoken to the Couchbase support team, and they are going to try to get this issue reactivated and fixed before the Couchbase 4.5 release (you can view the issue here if you'd like: https://issues.couchbase.com/browse/MB-18905)

Upvotes: 1

Related Questions