CODEWITHSUNDEEP
pythoncsvsqlite
HeLlOwOrLd
HeLlOwOrLd

Reputation: 59

How do I get to my values from a CSV file to load into my SQL database with python?

I'm trying to figure out how to load my data from a CSV file into my SQL database.

I currently used Sqlite3, because I couldnt install pymssql yet. That is my code so far:

import csv, sqlite3
con = sqlite3.connect("aerzte.db")
cur = con.cursor()
#cur.execute("""CREATE TABLE liste (id INTEGER PRIMARY KEY, Anrede TEXT,     Titel TEXT, Titel2 TEXT, Vorname TEXT, Name TEXT, Praxis TEXT, Straße TEXT, PLZ TEXT, Ort TEXT);""") 
with open('arztliste.csv', 'r') as f:
    file = csv.reader(f)
    columns = next(file)
    query = 'insert into liste({0}) values ({1})'
    query = query.format(','.join(columns), ','.join('?' * len(columns)))
    for data in file:        
        cur.execute(query, data)
    cur.commit()

con.commit()
con.close()

My CSV file Looks like this:

Anrede;Titel;Titel2;Vorname;Name;Praxis;Straße;PLZ;Ort;
Herr;Dr.;med.;Norbert;Braunisch;CoMedicum Landshuter Allee GmbH; Landshuter Allee 45;80637;München;

The first row is the Header with the column values. After that follows the "real" data that should get inserted into those columns. I also alreasy created the database, the Table and the columns. I think the data cant load in because of the semicolons between the diffrent column values. I already replaced them with ","'s, but then at the end the semicolon s missing to end the line. I hope to get any advices soon. Thank You.

Upvotes: 0

Views: 247

Answers (2)

Serge Ballesta
Serge Ballesta

Reputation: 148975

The Python csv module allows you to declare the delimiter. And as you have an additional semicolon at the end of the line, you will get an additional field in each row that you have to ignore.

It does not make sense to commit a cursor: you only commit at the connection level. You must choose if you want to commit after each line (uncommon), at the end of file (may use memory) of every n-th line (use a counter). So you code should become (using this last option)

...
counter = 20                                            # commit every 20-th row
with open('arztliste.csv', 'r') as f:
    file = csv.reader(f, delimiter=";")                 # declare delimiter
    columns = next(file)[:-1]                           # ignore last (empty) field
    query = 'insert into liste({0}) values ({1})'
    query = query.format(','.join(columns), ','.join('?' * len(columns)))
    for data in file:        
        cur.execute(query, data[:-1])                   # ignore last (empty) field
        counter -= 1
        if counter == 0:
            con.commit()
            counter = 20

con.commit()
con.close()

Upvotes: 2

Mahesh Kumaran
Mahesh Kumaran

Reputation: 907

Using csv.DictReader makes the work simpler compared to reader, And i changed it to commas from semi-colon, incase youre gonna use semicolon , specify the delimitter in the reader object

with open('arztliste.csv', 'r') as f:
    file = csv.Dicteader(f)
    csv_data = []
    for element in file : 
         csv_data.append(element)

csv_data now contains list of dictionaries where keys are the headers of your csv file and values are the "real" data .

Once you get the data correct its simple to dump it into the sqldb ,

query = 'INSERT INTO table_name(Anrede,Titel,Titel2,Vorname,Name,Praxis,Straße,PLZ,Ort) VALUES(%s,%s,%s,%s,%s,%s,%s,%s)'

Looping through the values ,

for data in csv_data:
     cur.execute(query,data['Anrede'],data['Titel'],data['Titel2']...data['Ort'])

Upvotes: 2

Related Questions