CODEWITHSUNDEEP
elasticsearchelasticsearch-2.0
jmng
jmng

Reputation: 2568

Elasticseach bulk load strangely missing 1 of 3 documents

I'm running into strange problems following the shingles example at https://www.elastic.co/guide/en/elasticsearch/guide/current/shingles.html

When I try to index the three documents on that tutorial, only two of them get indexed, the document with ID 3 is never indexed.

The request POSTed to http://elastic:9200/myIndex/page/_bulk is:

{ "index": { "_id": 1 }}
{ "text": "Sue ate the alligator" }
{ "index": { "_id": 2 }}
{ "text": "The alligator ate Sue" }
{ "index": { "_id": 3 }}
{ "text": "Sue never goes anywhere without her alligator skin purse" }

But the response is:

{
"took": 18,
"errors": false,
"items": [
    {
        "index": {
            "_index": "myIndex",
            "_type": "page",
            "_id": "1",
            "_version": 1,
            "_shards": {
                "total": 1,
                "successful": 1,
                "failed": 0
            },
            "status": 201
        }
    },
    {
        "index": {
            "_index": "myIndex",
            "_type": "page",
            "_id": "2",
            "_version": 1,
            "_shards": {
                "total": 1,
                "successful": 1,
                "failed": 0
            },
            "status": 201
        }
    }
]}

Index and mappings definition:

{
"settings": {
    "number_of_shards": 1,
    "number_of_replicas": 0,
    "analysis": {
        "filter": {
            "filter_shingle": {
                "type": "shingle",
                "max_shingle_size": 5,
                "min_shingle_size": 2,
                "output_unigrams": "false"
            },
            "filter_stop": {
                "type": "stop"
            }
        },
        "analyzer": {
            "analyzer_shingle": {
                "tokenizer": "standard",
                "filter": ["standard", "lowercase", "filter_stop", "filter_shingle"]
            }
        }
    }
},
"mappings": {
    "page": {
        "properties": {
            "text": {
                "type": "string",
                "index_options": "offsets",
                "analyzer": "standard",
                "fields": {
                    "shingles": {
                        "search_analyzer": "analyzer_shingle",
                        "analyzer": "analyzer_shingle",
                        "type": "string"
                    }
                }
            },
            "title": {
                "type": "string",
                "index_options": "offsets",
                "analyzer": "standard",
                "search_analyzer": "standard"
            }
        }
    }
}}

Upvotes: 0

Views: 619

Answers (1)

Val
Val

Reputation: 217514

When posting documents in bulk, you need to make sure to include a new line character after the last line as explained in the official docs

curl -XPOST http://elastic:9200/myIndex/page/_bulk -d '
{ "index": { "_id": 1 }}
{ "text": "Sue ate the alligator" }
{ "index": { "_id": 2 }}
{ "text": "The alligator ate Sue" }
{ "index": { "_id": 3 }}
{ "text": "Sue never goes anywhere without her alligator skin purse" }
'      <--- new line

Upvotes: 1

Related Questions