CODEWITHSUNDEEP
elasticsearch
niko85
niko85

Reputation: 303

Elasticseacr index is deleted all of a sudden

I have Elasticsearch server on AWS. And every now and then my Elasticsearch index is being somehow deleted without me doing anything.

Below are the logs from my Elasticsearch server.

[2021-04-19T01:30:00,001][INFO ][o.e.x.s.SnapshotRetentionTask] [ip-10-0-0-94] starting SLM retention snapshot cleanup task
[2021-04-19T01:30:00,009][INFO ][o.e.x.s.SnapshotRetentionTask] [ip-10-0-0-94] there are no repositories to fetch, SLM retention snapshot cleanup task complete
[2021-04-19T02:27:00,000][INFO ][o.e.x.m.MlDailyMaintenanceService] [ip-10-0-0-94] triggering scheduled [ML] maintenance tasks
[2021-04-19T02:27:00,000][INFO ][o.e.x.m.a.TransportDeleteExpiredDataAction] [ip-10-0-0-94] Deleting expired data
[2021-04-19T02:27:00,002][INFO ][o.e.x.m.j.r.UnusedStatsRemover] [ip-10-0-0-94] Successfully deleted [0] unused stats documents
[2021-04-19T02:27:00,002][INFO ][o.e.x.m.a.TransportDeleteExpiredDataAction] [ip-10-0-0-94] Completed deletion of expired ML data
[2021-04-19T02:27:00,002][INFO ][o.e.x.m.MlDailyMaintenanceService] [ip-10-0-0-94] Successfully completed [ML] maintenance task: triggerDeleteExpiredDataTask
[2021-04-19T02:58:46,088][INFO ][o.e.c.m.MetadataDeleteIndexService] [ip-10-0-0-94] [read__me/xc6mW05hTV-Xn1TTqX_FEA] deleting index
[2021-04-19T02:58:46,641][INFO ][o.e.c.m.MetadataDeleteIndexService] [ip-10-0-0-94] [superhero/PHe6S6kSRoSamCaaiPBSFg] deleting index
[2021-04-19T02:58:47,212][INFO ][o.e.c.m.MetadataCreateIndexService] [ip-10-0-0-94] [read__me] creating index, cause [api], templates [], shards [1]/[1]
[2021-04-19T02:58:47,823][INFO ][o.e.c.m.MetadataMappingService] [ip-10-0-0-94] [read__me/ODzWqR5FTeeiBhOBG-EMzA] create_mapping [_doc]
[2021-04-19T07:17:10,800][INFO ][o.e.c.m.MetadataCreateIndexService] [ip-10-0-0-94] [superhero] creating index, cause [api], templates [], shards [1]/[0]

Upvotes: 2

Views: 786

Answers (2)

Jeriko
Jeriko

Reputation: 89

it seems that daily maintenance tasks are responsible for this behavior. Indeed, you can see in logs this line : [2021-04-19T02:27:00,002][INFO ][o.e.x.m.MlDailyMaintenanceService] [ip-10-0-0-94] Successfully completed [ML] maintenance task: triggerDeleteExpiredDataTask

I am not a pro of ES, but check this link : https://discuss.elastic.co/t/how-to-stop-ml-daily-maintenance-service-in-elasticsearch-slm-retention-snapshot-cleanup-task/255623

Upvotes: 1

Bhavya
Bhavya

Reputation: 16172

It is possible that there is an ILM policy defined in your cluster that is managing your indices and is deleting the data.

Upvotes: 1

Related Questions