CODEWITHSUNDEEP
sqliteparallel-processing
sequenceGeek
sequenceGeek

Reputation: 797

Obtaining Read/Write Lock for Sqlite Parallel Updates

I write a lot of parallel scripts in python for research purposes and was wondering if it is possible to obtain a read/write lock manually in sqlite for a specific set of commands. Here is an oversimplified example of why I need it:

Simple Example:
Suppose I have a table (A) and I want to simply count the number of rows in it and store the result in another table (B) in parallel. To do so, I run 10 instances of a program that counts rows for a certain range of A and adds the sum of the rows to the property in B.

The problem is that I need to read the property in B, add it to the instance's count, and save it; all while making sure none of the other instances are doing this process. Normally it is only a write lock that is needed - in this case I need a read lock as well...

I was hoping I could do something like this:

  1. read/write lock
  2. Grab current value in B, add it to the instance count
  3. save new value
  4. unlock

Is there a way to do this?

Thanks.

Upvotes: 2

Views: 1660

Answers (1)

thkala
thkala

Reputation: 86333

You can increment an integer value using a single UPDATE statement:

sqlite> CREATE TABLE B(Id INTEGER, Value INTEGER);
sqlite> INSERT INTO B VALUES(0, 15);
sqlite> UPDATE B SET Value=Value + 23 WHERE Id = 0;
sqlite> SELECT * FROM B;
0|38
sqlite>

Using a single UPDATE statement makes this operation atomic, making any extra locking unnecessary.

If you need more complex processing, you can use SQL transactions to ensure that any complex database operations are performed atomically.

In general, you should avoid any locking external to SQLite or messing with the SQLite locking subsystem - doing so is a very good recipe for deadlocks...

EDIT:

To append to a string, you can use the || concatenation operator:

sqlite> CREATE TABLE C(Id INTEGER, Value TEXT);
sqlite> INSERT INTO C VALUES(0, 'X');
sqlite> UPDATE C SET Value=Value || 'Y' WHERE Id = 0;
sqlite> SELECT * FROM C;
0|XY
sqlite>

Upvotes: 4

Related Questions