Elasticsearch mongodb and river indexing fail

Question

Dear my (I hope) saviors,

I have a very annoying problem with elasticsearch and mondogdb datas indexing.

The situation is like: I have 2 mongodb collections, records and pois, that I need to index on elasticsearch, using river plugin (deprecated, I know).

(Record has a reference (DBRef) to poi and other collections, called otherref# here.)

Now, when I execute the curl call, it happens that....Sometimes all records document are indexed, sometimes just 200 (of 140k). Sometimes 900 pois documents are indexed, sometimes just 200 (never all, about 70k).

So, it seems that the script doesn't work properly.

I've monitored the /var/log/elasticsearch log , but no error has been logged.

Here the indexing script:

curl -XPUT "localhost:9200/lw_index_poi" -d ' 
{
 "mappings": {
 "lw_mapping_poi" : {
   "properties" : {
   "position" : {
      "type" : "geo_shape"
    },
   "poi_id" : {
      "type" : "string",
      "index" : "not_analyzed" 
    }
  }
 }
}
}'

curl -XPUT "localhost:9200/lw_index_record" -d ' 
{
  "mappings": {
    "lw_mapping_record" : {
    "date_detection": false,
      "properties" : {
        "other_ref1" : {
          "type" : "string",
          "index" : "not_analyzed"
        },
        "other_ref2" : {
          "type" : "string",
          "index" : "not_analyzed"
        },
        "poi_ref" : {
          "type" : "string",
          "index" : "not_analyzed"
        }
      }
    }
  }
}'
curl -XPUT "localhost:9200/_river/lw_index_poi/_meta" -d '
{
  "type": "mongodb",
  "mongodb": { 
    "servers":
    [
      { "host": "mongodb", "port": 27017 }
    ],
    "options": { 
      "secondary_read_preference" : false
    },
    "db": "lifewatch",
    "collection": "poi",
    "script": "if (ctx.document.decimalLatitude && ctx.document.decimalLongitude) { ctx.document.position = {}; ctx.document.position.type=\"Point\"; ctx.document.position.coordinates = [ctx.document.decimalLongitude, ctx.document.decimalLatitude]; } ctx.document.poi_id = ctx.document._id; delete ctx.document.decimalLatitude; delete ctx.document.decimalLongitude;"
  }, 
  "index": { 
    "name": "lw_index_poi",
    "type": "lw_mapping_poi"
  }
}'

curl -XPUT "localhost:9200/_river/lw_index_record/_meta" -d '
{
  "type": "mongodb",
  "mongodb": { 
    "servers":
    [
      { "host": "mongodb", "port": 27017 }
    ],
    "options": { 
      "secondary_read_preference" : false
    },
    "db": "lifewatch",
    "collection": "record",
    "script": "if (ctx.document.ref1) { ctx.document.ref1 = ctx.document.ref1.id; delete ctx.document.ref1;};if (ctx.document.poi) { ctx.document.poi_ref = ctx.document.poi.id; delete ctx.document.poi;};if (ctx.document.ref2) { ctx.document.ref2 = ctx.document.ref2.id; delete ctx.document.ref2;};"
  }, 
  "index": { 
    "name": "lw_index_record",
    "type": "lw_mapping_record"
  }
}'

What's wrong?

Thanks in advance