CODEWITHSUNDEEP
solrapache-zookeeper
Usama
Usama

Reputation: 480

solrCloud zookeeper logs

I have a solrCloud instance and I have a job which inserts data into a Solr collection and removes it all . Every time I insert some data and remove it the size of the zoo_data directory increase. A few logs are generated into this directory. which causes increase in size of the solr. Is there a way I can stop these logs from generation or remove these logs after generation.

NOTE: my core is not shared nor replicated it is running on a single machine.

Upvotes: 2

Views: 1359

Answers (1)

Amrit Sarkar
Amrit Sarkar

Reputation: 23

ZooKeeper servers use local storage to persist transactions. The transactions are logged to transaction logs, similar to the approach of sequential append-only log files used in database systems.

The servers in the ZooKeeper service also keep on saving point-in-time copies or snapshots of the ZooKeeper tree or the namespace onto the local filesystem.

The ZooKeeper snapshot files and transactional logs enable recovery of data in times of catastrophic failure or user error. Starting Zookeeper 3.4.0 we have additional parameters which we can add to Zookeeper configuration file so that purging the logs and snapshot files can be automated.

More comprehensive article with details can be read over here: https://support.lucidworks.com/hc/en-us/articles/228753687-Zookeeper-Log-Snapshot-Maintenance

Upvotes: 1

Related Questions