CODEWITHSUNDEEP
elasticsearchelasticsearch-6
No1Lives4Ever
No1Lives4Ever

Reputation: 6903

Elasticsearch 6.0 Removal of mapping types - Alternatives

Background

I migrating my ES index into ES version 6. I currenly stuck because ES6 removed the using on "_type" field.

Old Implementation (ES2)

My software has many users (>100K). Each user has at least one document in ES. So, the hierarchy looks like this:

INDEX  ->  TYPE      -> Document
myindex->  user-123  -> document-1

The key point here is with this structure I can easily remove all the document of specific user.

DELETE /myindex/user-123

(Delete all the document of specific user, with a single command)

The problem

"_type" is no longer supported by ES6.

Possible solution

Instead of using _type, use the index name as USER-ID. So my index will looks like:

"user-123" -> "static-name" -> document

Delete user is done by delete index (instead of delete type in previous implementation).

Questions:

Any other tip will be welcome! Thanks.

Upvotes: 0

Views: 308

Answers (2)

Terry
Terry

Reputation: 317

Having these many indexes is definitely not a good approach. If your only concern to delete multiple documents with a single command. Then you can use Delete by Query API provided by ElasticSearch

You can introduce "subtype" attribute in all your document containing value for each document like "user-" value. So in your case, document would looks like.

{
  "attribute1":"value", 
  "subtype":"user-123"
}

Upvotes: 0

Val
Val

Reputation: 217564

I would not have one index per user, it's a waste of resources, especially if there are only 10 docs per user.

What I would do instead is to use filtered aliases, one per user.

So the index would be named users and the type would be a static name, e.g. doc. For user 123, the documents of that user would all be stored in users/doc/xyz and in each document you need to add the user id, e.g.

PUT users/doc/xyz
{
   ...
   "userId": 123,
   ...
}

Then you can define a filtered alias for all documents of user 123, like this:

POST /_aliases
{
    "actions" : [
        {
            "add" : {
                 "index" : "users",
                 "alias" : "user-123",
                 "filter" : { "term" : { "userId" : "123" } }
            }
        }
    ]
}

If you need to delete all documents of user 123, then you can simply do it like this:

POST user-123/_delete_by_query?q=*

Upvotes: 3

Related Questions