CODEWITHSUNDEEP
elasticsearch
SexyMF
SexyMF

Reputation: 11155

elasticsearch indexes disappeared unexpectedly

I have a single node el cluster. a few hours ago I have lost 2 indexes.
these are the logs I have from the docker. I don't know of all the logs are there.
the logs are too large for here:

 https://pastebin.com/YiLiQkPC

here is sampled log:

{"log":"{"type": "server", "timestamp": "2020-12-16T05:16:00,561Z", "level": "WARN", "component": "o.e.b.JNANatives", "cluster.name": "docker-cluster", "node.name": "37f65704d9bb", "message": "Unable to lock JVM Memory: error=12, reason=Cannot allocate memory" }\n","stream":"stdout","time":"2020-12-16T05:16:00.564490107Z"}
{"log":"{"type": "server", "timestamp": "2020-12-16T05:16:00,564Z", "level": "WARN", "component": "o.e.b.JNANatives", "cluster.name": "docker-cluster", "node.name": "37f65704d9bb", "message": "This can result in part of the JVM being swapped out." }\n","stream":"stdout","time":"2020-12-16T05:16:00.565085052Z"}
{"log":"{"type": "server", "timestamp": "2020-12-16T05:16:00,565Z", "level": "WARN", "component": "o.e.b.JNANatives", "cluster.name": "docker-cluster", "node.name": "37f65704d9bb", "message": "Increase RLIMIT_MEMLOCK, soft limit: 65536, hard limit: 65536" }\n","stream":"stdout","time":"2020-12-16T05:16:00.565449225Z"}
{"log":"{"type": "server", "timestamp": "2020-12-16T05:16:00,565Z", "level": "WARN", "component": "o.e.b.JNANatives", "cluster.name": "docker-cluster", "node.name": "37f65704d9bb", "message": "These can be adjusted by modifying /etc/security/limits.conf, for example: \\n\\t# allow user 'elasticsearch' mlockall\\n\\telasticsearch soft memlock unlimited\\n\\telasticsearch hard memlock unlimited" }\n","stream":"stdout","time":"2020-12-16T05:16:00.565962142Z"}
{"log":"{"type": "server", "timestamp": "2020-12-16T05:16:00,565Z", "level": "WARN", "component": "o.e.b.JNANatives", "cluster.name": "docker-cluster", "node.name": "37f65704d9bb", "message": "If you are logged in interactively, you will have to re-login for the new limits to take effect." }\n","stream":"stdout","time":"2020-12-16T05:16:00.56620427Z"}
{"log":"{"type": "server", "timestamp": "2020-12-16T05:16:00,785Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "37f65704d9bb", "message": "version[7.9.3], pid[6], build[default/docker/c4138e51121ef06a6404866cddc601906fe5c868/2020-10-16T10:36:16.141335Z], OS[Linux/4.19.0-12-cloud-amd64/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/15/15+36-1562]" }\n","stream":"stdout","time":"2020-12-16T05:16:00.787778299Z"}

thanks

Upvotes: 1

Views: 1547

Answers (1)

Amit
Amit

Reputation: 32376

Below log lines clearly show the deletion of your index, was your index deleted during same time??

{"log":"{"type": "server", "timestamp": "2020-12-16T07:21:34,013Z", "level": "INFO", "component": "o.e.c.m.MetadataDeleteIndexService", "cluster.name": "docker-cluster", "node.name": "37f65704d9bb", "message": "[research-fb-private/4d7NP5IqSNyFNvzMLSn5-A] deleting index", "cluster.uuid": "gzpqD1vtTi-6jAasvOm15g", "node.id": "5zzXP2kCQ9eDI0U6WY4j9Q" }\n","stream":"stdout","time":"2020-12-16T07:21:34.014059142Z"}

Also was your deleted index was having the research-fb-private/4d7NP5IqSNyFNvzMLSn5-A name, if yes, than it confirm the deletion of index, you can also check the name of your deleted index, during the time of their deletion in the same logs.

Upvotes: 1

Related Questions