CODEWITHSUNDEEP
viewcloudant
Chris Knowles
Chris Knowles

Reputation: 51

Time-sensitive Cloudant view not always returning correct results

I have a view on a Cloudant database that is designed to show events that are happening in the next 24 hours:

function (doc) {

  // activefrom and activeto are in UTC 
  // set start to local time in UTC
  var m = new Date();
  var start = m.getTime();

  // end is start plus 24 hours of milliseconds
  var end = start + (24*60*60*1000);

  // only want approved disruptions for today that are not changed conditions
  if (doc.properties.status === 'Approved' && doc.properties.category != 'changed' && doc.properties.activefrom && doc.properties.activeto){
     if (doc.properties.activeto > start && doc.properties.activefrom < end)
       emit([doc.properties.category,doc.properties.location], doc.properties.timing);    
     }
  }
}

This works fine for most of the time but every now and then the view does not show the expected results.

If I edit the view, even just adding a comment, the output changes to the expected results. If I re-edit the view and remove the change, the results return to the incorrect results.

Is this because of the time-sensitive nature of the view? Is there a better way to achieve the same result?

Upvotes: 0

Views: 77

Answers (1)

Glynn Bird
Glynn Bird

Reputation: 5637

The date that is indexed by your MapReduce function is the time that the server dealing with the work performs the indexing operation.

Cloudant views are not necessarily generated at the point that data is added to the database. Sometimes, depending on the amount of work the cluster is having to do, the Cloudant indexer is not triggered until later. Documents can even remain unindexed until the view is queried. In that circumstance, the date in your index would not be "the time the document was inserted" but "the time the document was indexed/queried", which is probably not your intention.

Not only that, different shards (copies) of the database may process the view build at different times, giving you inconsistent results depending on which server you asked!

You can solve the problem by indexing something from your source document e.g.

if your document looked like:

{ "timestamp": 1519980078159, "properties": { "category": "books", "location": "Rome, IT" } }

You could generate an index using the timestamp value from your document and the view you create would be consistent across all shards and would be deterministic.

Upvotes: 1

Related Questions