CODEWITHSUNDEEP
node.jsmongodbelasticsearchsails.js
hirenhcm
hirenhcm

Reputation: 93

Elasticsearch Mapping lost on sails lift with mongo-connector

I am developing an applications using MongoDB, Sails JS and ElasticSearch.

MongoDB is used to write records that are retrieve for the application. ElasticSearch is used for search text and geo locations distance search etc.

I am using mongo-connector to keep my data in sync from MongoDB to ElasticSearch.

Issue is, i am not able to maintain my mappings for geo_point for the fields that store lat and lon or parent/child or analyzer etc. Every time sails server is lifted i see in elasticsearch logs that all the mappings are removed, created and updated, and i lose my mapping for geo_point that is have created manually via the REST after every thing is up and running or even if i have created mapping at bootstrap time of sails js(as a work around).

I have also tried to create a mapping file and placed it in elasticsearch/config/mappings/index/mymapping.json but i get an error 

Caused by: org.elasticsearch.index.mapper.MapperParsingException: Root type mapping not empty after parsing! Remaining fields: ...

Here i tried all the combinations to make this work but no success eg

{"mappings" : {
        "locations" : {
            "dynamic": "false",
            "properties":{
                "location": {
                    "type": "geo_point"
                }
            }
        }
    }
}

Also tried using a template to create the mapping but after that mongo-connector kick in and overrides the mapping.

As of now i am only able to make this work is to stop mongo-connector, delete the oplog.timestamp file, start the sails server(Here at bootstrap time i delete and recreate the mapping for that document) and then start mongo-connector. But this create accidents if we forgots to do a step.

Am i doing any thing wrong or is there a better way to sync the mongodb to elasticsearch without losing the custom mapping or an alternative mongo-connector.

Upvotes: 1

Views: 405

Answers (1)

Val
Val

Reputation: 217284

According to the documentation, if you install a mapping on the filesystem, the file must be named <your_mapping>.json so in your case it should be named locations.json and be placed either in 

elasticsearch/config/mappings/_default/locations.json

or

elasticsearch/config/mappings/<your_index_name>/locations.json

Moreover, you mapping file shouldn't contain the mappings keyword, it should instead look like this:

    {
        "locations" : {
            "dynamic": "false",
            "properties":{
                "location": {
                    "type": "geo_point"
                }
            }
        }
    }

You should try again after correctly naming your mapping file and folders.

Upvotes: 1

Related Questions