CODEWITHSUNDEEP
sqlite
ytrewq
ytrewq

Reputation: 3930

Sqlite3: Inserting multiple rows is still slow

I need to perform about 250 million inserts using sqlite3 and python.

I've wrapped every 500 inserts using SELECT UNION, and also done CREATE INDEX.

But even so, it later becomes slow, performing less than 1 million inserts per hour (about 300 inserts per second)

Is there any way to make it any significantly faster?

Upvotes: 0

Views: 2339

Answers (2)

Igor Medeiros
Igor Medeiros

Reputation: 4126

This article explains very well the issue of fast insert in sqlite3. It comes with a benchmark of different approaches.

Upvotes: 1

Tony Gibbs
Tony Gibbs

Reputation: 2489

This is one of the better/best optimization guides I have read: How do I improve the performance of sqlite

I recently spent some time optimizing an initial load of a large dataset into sqlite and the tips in there helped greatly. I'd recommend just reading through, no point in me doing a c/p of it.

Upvotes: 0

Related Questions