SOLR - old Transactions-Logs (tlogs) never deleted

Question

first of all to mention i searched a long time but got n solution, so not i try with my specific problem, trying to keep it short:

• solr-spec 4.0.0.2012.10.06.03.04.33

• one master, three slaves

• around 70.000 documents in index

• master gets triggered to full import / generate complete new index ~ once a day

• command line options for trigger are: ?command=full-import&verbose=false&clean=false&commit=true&optimize=true

• slaves trigger master for new index, if GEN increases (full import + hard commit as mentioned), they pull the new index

• no autoCommit / autoSoftCommit set up

the problem ist, that each hard commit the index (~670MB) gets written to disk, once a day, but the old never get deleted. As far as i read solr keeps enough tlogs to be able to restore the last 100 changes to documents, am i right?

In my setup i am sure at least 100 documents (or data sets within the source database) are changed each day, so i dont understand why solr never deletes old tlogs.

I would be glad if someone can point to the right direction, currently i have no clue what to try next. Also i did not find a setup like this one described having problems like this.

Thx ;)

Answer 1

First you'll probably want to update your Solr-version, as there's been a few transaction log reference leaks fixed since 4.0.

A hard commit should usually remove old transaction logs as the documents are written to disk in the index anyway iirc, which may indicate that you're getting bit by some old references hanging around.

Another option would be to turn off the transaction log completely, since you only generate a completely new index each run anyway and dist that one.