CODEWITHSUNDEEP
performancesqlitesolid-state-drive
Ger Teunis
Ger Teunis

Reputation: 955

sqlite updates slow (15 seconds for 1720 records) on SSD disk

Dear fellow developer, for some reason updates for 1720 records takes around 15 seconds when on SSD disk (especially when having trim enabled).

I have tweaked the sqlite settings using the following document (which works well) http://web.utk.edu/~jplyon/sqlite/SQLite_optimization_FAQ.html

I have the following PRAGMA's set to optimize performance and I DO use transactions around the complete set of updates.

sqlite3_exec(database, "PRAGMA cache_size=500000;", nil, nil, nil);
sqlite3_exec(database, "PRAGMA synchronous=OFF", nil, nil, nil);
sqlite3_exec(database, "PRAGMA count_changes=OFF", nil, nil, nil);
sqlite3_exec(database, "PRAGMA temp_store=MEMORY", nil, nil, nil);

It seems the SSD is doing too much (like deleting blocks and such) which makes it block for 15 seconds for just 1720 simple records update.

Weirdly enough: inserting 2500 records is almost instant. Can you help me and give me some pointers how to fix this?

Upvotes: 4

Views: 1999

Answers (1)

Ger Teunis
Ger Teunis

Reputation: 955

I found the answer by doing some good old testing and trying out. It seems that sqlite will freeze on SSDs with TRIM when doing a lot of single UPDATE statements after each other in a transaction.

I've changed the code now: a) prepare a command and reuse that command for all the to-be updated records. b) COMMIT and BEGIN a new transaction every 1500 records

this seems to have fixed the freeze in sqlite. So in short: reuse the prepared sql statement and make sure the transaction doesn't get too large.

Upvotes: 4

Related Questions