CODEWITHSUNDEEP
jdbcelasticsearch
Avión
Avión

Reputation: 8376

Elasticsearch JDBC importer not importing entry correctly

Having the following mapping:

curl -XPUT 'localhost:9200/borrador' -d '{
  "mappings": {
    "item": {
       "dynamic": "strict",
       "properties" : {
            "body" : { "type": "string"  },
            "source_id" : { "type": "integer"  },
}}}}'

I'm trying to import my DB to Elasticsearch using the Elasticsearch-JDBC importer. This is the script I'm using:

#!/bin/sh

bin=/usr/share/elasticsearch/elasticsearch-jdbc-2.1.1.2/bin
lib=/usr/share/elasticsearch/elasticsearch-jdbc-2.1.1.2/lib
echo "Indexando base de datos..."
echo '{
    "type" : "jdbc",
    "jdbc" : {
        "url" : "jdbc:mydbip/mydbname",
        "user" : "username",
        "password" : "pw",
        "sql" : "select source_id, body, id as _id from table_name",
        "index" : "borrador",
        "type" : "item"
    }
}' | java \
       -cp "${lib}/*" \
       -Dlog4j.configurationFile=${bin}/log4j2.xml \
       org.xbib.tools.Runner \
       org.xbib.tools.JDBCImporter

Most of the rows of the table are indexed correctly, but the following row from that DB is giving me an error and it's not indexing correctly: http://i.imgur.com/UPEfRS0.png

This is the error that shows up:

[ERROR][org.xbib.elasticsearch.helper.client.BulkTransportClient][elasticsearch[importer][listener][T#1]] bulk [957] failed with 1 failed items, failure message = failure in bulk execution: [3499]: index [borrador], type [item], id [14327140], message [MapperParsingException[failed to parse [body]]; nested: IllegalArgumentException[unknown property [records]];]

As you can see in this case, this specific row has a json format string ({"format":"MS Excel","price":"750","records":"577","recordType":"records"}<!-- com -->) instead of the normal string that has the other entries that are indexing correctly.

What is happening? I would like to store that as a normal string. It's problem of the mapping as it's reading it as a json or something? Even if I remove the "dynamic": "strict", or the entire mapping, it still gives me the error. Thanks in advance.

Upvotes: 1

Views: 418

Answers (1)

Val
Val

Reputation: 217254

By default the JDBC importer tries to detect JSON strings in your data and will parse them. You need to modify the configuration of your importer with the detect_json setting and set it to false:

{
    "type" : "jdbc",
    "jdbc" : {
        "url" : "jdbc:mydbip/mydbname",
        "user" : "username",
        "password" : "pw",
        "sql" : "select source_id, body, id as _id from table_name",
        "index" : "borrador",
        "type" : "item",
        "detect_json": false            <--- add this
    }
}

Upvotes: 2

Related Questions