CODEWITHSUNDEEP
pythonmultithreadingsqlitethreadpool
Milano
Milano

Reputation: 18735

Can't INSERT into Sqlite3 database

I'm very desperate because I tried everything I could to make it work but no success... I've created a pool which is processing some text lines and then inserting them into the database. The problem is that it does not insert into the table nothing.

I know that it is probably because multiple workers trying to modify the database but I've already tried many things to avoid that.

For example this (this is the method for inserting url into the index table):

def insert_into_index(self, url):
    self.lock.acquire()
    self.cur.execute('INSERT INTO index VALUES(?)', (url,))
    self.conn.commit()
    self.lock.release()

Or this (each new use of the method has it's own cursor):

def insert_into_index(self, url):
    cur = self.conn.cursor()
    cur.execute('INSERT INTO index VALUES(?)', (url,))
    self.conn.commit()

The snippet where I create the Pool:

 """ CREATE A POOL """

    pool = Pool(50)
    for category in categories[1:2]:
        task = pool.apply_async(index, args=(category.strip('\n'),))
        task.get()

    pool.close()
    pool.join()

    """ POOL END """

Index() is a method which creates an instance of class leaf(text), which do some things with the text, obtains list of strings and try to insert them into the database.

class leaf():
    def __init__(self, url):
        self.url = url
        self.results = self.get_all_hrefs(url)
        self.dbm = database_file.manager()

        for res in self.results:
              self.dbm.insert_into_index(res)

But I don't have nothing in table index. I've tried everything I knew about with no success. Could you give me an advice?

EDIT: This is an error which is raised when I append task.get() into the loop. 

line 86, in execute
    task.get()
  File "C:\Python27\lib\multiprocessing\pool.py", line 567, in get
    raise self._value
sqlite3.OperationalError: near "index": syntax error

But I thing that it does not tells nothing because the syntax is totally the same as in another method which is called sequentially and works.

Upvotes: 2

Views: 145

Answers (1)

CL.
CL.

Reputation: 180080

index is a keyword. You must quote it, or use a different name.

Upvotes: 2

Related Questions