CODEWITHSUNDEEP
pythonmysqlsqldatabasesqlite
user248237
user248237

Reputation:

How to insert strings with quotes and newlines into sqlite db with Python?

I'm trying to insert strings read from a file into an sqlite database in Python. The strings have whitespace (newline, tab characters, and spaces) and also have appearances of single or double quotes. Here's how I try to do it:

import sqlite3
conn = sqlite3.connect('example.db')
c = conn.cursor()

# Create table
c.execute('''CREATE TABLE test
             (a text, b text)''')

f = open("foo", "w")
f.write("hello\n\'world\'\n")
f.close()

testfield = open("foo").read()

# Insert a row of data
c.execute("INSERT INTO test VALUES ('%s', 'bar')" %(testfield))

# Save (commit) the changes
conn.commit()

I find that this fails with the error: 

    c.execute("INSERT INTO test VALUES ('%s', 'bar')" %(testfield))
sqlite3.OperationalError: near "world": syntax error

How can I achieve this? Do the strings need to be escaped before insertion in the db, and if so how? thanks.

Upvotes: 10

Views: 16038

Answers (1)

Martijn Pieters
Martijn Pieters

Reputation: 1121584

You use SQL parameters instead of string formatting:

c.execute("INSERT INTO test VALUES (?, 'bar')", (testfield,))

When using SQL parameters you let the database library handle the quoting, and even better, give the database to optimize the query and reuse the optimized query plan for multiple executions of the same basic query (with different parameters).

Last but not least, you are much better defended against SQL injection attacks as the database library knows best how to escape dangerous SQL-like values.

To quote the sqlite3 documentation:

Usually your SQL operations will need to use values from Python variables. You shouldn’t assemble your query using Python’s string operations because doing so is insecure; it makes your program vulnerable to an SQL injection attack (see http://xkcd.com/327/ for humorous example of what can go wrong).

Instead, use the DB-API’s parameter substitution. Put ? as a placeholder wherever you want to use a value, and then provide a tuple of values as the second argument to the cursor’s execute() method.

Upvotes: 29

Related Questions