CODEWITHSUNDEEP
couchdbreplication
Derek Ekins
Derek Ekins

Reputation: 11391

CouchDB filtered replication - remove a document

I am trying to setup filtered replication between a master and user database. Documents in the master contain a list of user groups that have permission to the document.

{
  _id: 'one',
  groups: ['a', 'b']
}

{
  _id: 'two',
  groups: ['c', 'd']
}

I created a filtered view of the database that only allows user with the group to get a copy of the replicated document (hard coding the group of 'a' in this example)

{
  filters = {
    users = function(doc, req){
      return doc.groups.indexOf(req.query.group) != -1;
    }
  }
}

I then create a replication document in the _replicator database

{
  source: "master",
  target: "user1",
  filter: "replication/user",
  query_params: {group: "a"},
  create_target: true
}

Once this document is created replication begins and the document 'one' is replicated from master to user1. Document 'two' is not replicated - just what I want.

Subsequently the user is moved from group 'a' to group 'c' so I create a new replication document:

{
  source: "master",
  target: "user1",
  filter: "replication/user",
  query_params: {group: "c"},
  create_target: true
}

The behaviour I want is for document 'one' to be removed from the user database and for document 'two' to be replicated. As it happens document 'one' remains and document 'two' is replicated. Obviously the replication filter does not allow for deletions in the target database unless the document is deleted in the source database.

How then should this scenario be handled? Or is there an alternate structure I should be considering?

Upvotes: 6

Views: 2614

Answers (2)

Ewan Makepeace
Ewan Makepeace

Reputation: 5416

This is a tricky subject. The options we have considered are:

  1. When the group changes copy the document and save the new group to the copy, then delete the original. The delete will propagate through the filter and remove from the remote DB. Downsides are loss of revisions and breakage if the ID was referenced from other docs.
  2. Have the user database write a manifest file listing all the docs in the DB, which syncs back to the master (we had this already to track document delivery, and read status). Compare that to the intended (filtered) document list and tell the user database to PURGE (not DELETE) the docs that should no longer be present.

Best reference I have found is here: http://pouchdb.com/2015/04/05/filtered-replication.html

Upvotes: -1

Akshat Jiwan Sharma
Akshat Jiwan Sharma

Reputation: 16060

As far as I know there is no way to modify documents using replication. But there are two approaches that you can take.

First you can create a new database and replicate to it. For instance if your query parameter changes to user c instead of replicating it to user1 create another database some name and replicate to it and then delete the original database (or keep it in case you query param changes again). You can even use descriptive names for your source database like "user1_filter_a". This is the most hassle free way to do it but if the number of documents is large and overlapping (like lots of user b's that belong to both groups a and c) and your replicator filter changes rapidly it can be inefficient.

Another way is to use the view and Bulk document api. First create a view that emits the document based on the group field like so

function map(doc){

     emit(doc.groups,doc._id);

  }

and then query with

startkey=["a"]&endKey=["a",{}]&include_docs=true

to get all the documents that you want to delete. Then iterate over the result set and append doc._deleted=true to each document and make a bulk request to the database. All the documents will be deleted (more explanation here) .The advantage of this method is that you can keep a single database.

In short if you want to keep a single database you will have to delete the docs manually. But if you are open to multiple database based on replicator function you can just create a new database every time your filter changes.

Upvotes: 2

Related Questions