CODEWITHSUNDEEP
jsoncsvcouchbase
LocustHorde
LocustHorde

Reputation: 6399

storing csv in couchbase in json format

I've got some strings that I will be concatenating over time to a document on couchbase. I'm doing this with clojure, the code is pretty straight forward 

;create document when new
(cbc/add client key value)

;append when document exists
(c/async-prepend client key (str "," value) (c/get-cas-id client key)

And this all works fine. I can roughly outline the steps for the problem I want to discuss as follows

  1. Adding value: "1"
  2. Getting value: 1
  3. Appending value ",2"
  4. Getting value: 1,2

This works fine, and everything is dandy. But this question isn't really about prepending, but prepending to couchbase in a way that it is still a valid json. Because, 1,2 is not a valid json, the couchbase UI shows either a base64 encoded value or it sometimes says invalid json (even though the correct value is returned from memcache and api's).

From the couchbase forum I learnt that this is the expected behaviour. I'd really like to avoid this and stored the values (and append them) in a valid json format.

I could do something like 

{ "vals" : [1,2] }

and append to vals each time, but now, if I append 3, it would turn out to be [1,2], 3 which, again, is not a valid json.

I don't want to get the value, remove the ending bracket, add the new value, end the bracket and save again because (1) that defeats the purpose of appending and (2) the documents gets bigger and bigger and it's not efficient to read all value to add a new one each time.

Finally, my questions here boil down to two things:

1) Is it okay to leave the data as is - I'm getting the right data from api's anyway, the ui can't show me the data or shows a base64 encoded data so I can't edit it.. that's the only downside (why would I want to edit data from the UI anyway?)

2) IF, there is a way around this, is there a way to store the data in a way that makes it valid json every time, so it shows up in the UI as well as API. - downside to this being now the document is a lot bigger and there is a lot more processing needed to get the required data ({ "value": [1, 2]} as opposed 1,2)

not to mention I still don't know how to add further values to it!

Advise please!

Upvotes: 0

Views: 391

Answers (2)

m03geek
m03geek

Reputation: 2538

I don't know if it would work for you, but you can try to do such trick: Store some valid JSON doc in database with some property that hold an key that points to that csv values. Example:

doc: { "doc.id":"some_id", "doc.type":"some_type", "doc.values":"values:for:doc_some_id" }

then you append/prepend your csv data to that key values:for:doc_some_id, so you get:

"key": "values:for:doc_some_id", "value": "1,2,3..."

And then in emit function you get correct json doc, then get key that holds values, then get your values by that key. See this link for information about collated views. I never used them, but I think to implement what you want try to follow steps from that example.

Upvotes: 0

Don Dickinson
Don Dickinson

Reputation: 6258

Pretty sure there's not a way around this. with append and prepend you're going to break your json. the downside of it is that you will not be able to query the data with views - that functionality requires valid json. So, as long as you're ok with retrieving data via the key values, there is no problem with non-json data in the couchbase server.

Upvotes: 1

Related Questions