CODEWITHSUNDEEP
elasticsearchindexing
nlyn
nlyn

Reputation: 606

ElasticSearch separate index per user

I'm wondering if having thousands of different indexes is a bad idea?

I'm adding a search page to my web app based on ElasticSearch. The search page lets users search for other users on the site by filtering on a number of different indexed criteria (name, location, gender etc). This is fairly straight forward and will require just one index that contains a document every user of the site.

However, I want to also create a page where users can see a list of all of the other users they follow. I want this page to have the same filtering options that are available on the search page. I'm wondering if a good way to go about this would be to create a separate index for each user containing documents for each user they follow?

Upvotes: 2

Views: 2821

Answers (2)

dadoonet
dadoonet

Reputation: 14512

I would like to add to Igor's answer that creating thousand of indexes on a tiny cluster (one or two nodes) can cause some drawbacks. Each shard of an index is a full Lucene instance. That said, you will have many opened files (probably too many opened files) if you have a single node (or a small cluster - in term of nodes).

That's one of the major reasons why I would not define too many indices...

See also File descriptors on installation guide

Upvotes: 5

imotov
imotov

Reputation: 30163

While you can certainly create thousands of indices in elasticsearch, I don't really see the need for it in your use case. I think you can use one index. Simply create an additional child type followers for the main user record. Every time user A follows user B, create a child record of B with the following content: {"followed_by" : "A"}. To get the list of users that current user is following, you can simply add Has Child Filter to you query.

Upvotes: 5

Related Questions