CODEWITHSUNDEEP
pythondatabasegenerator
nutship
nutship

Reputation: 4934

Implementing generators to populate database

I can't wrap my head around generators fully, sometimes I happen to use them in a right way and sometimes I do not.

I am populating a db from a .csv file:

name, location
Tom, France
Doug, USA
Michael, UK

Here is my code:

def process_csv(filecsv):
    f = open(filecsv)
    f_csv = csv.reader(f)
    headers = next(f_csv)
    User = namedtuple('User', headers)
    for user in f_csv:
        user = [u.strip() for u in user]
        user = User(*user)
        yield user 

def insert(cur, user): 
    u = list(user)[0]   # it's a one-elem list so take it 
    cur.execute("INSERT INTO users (name, location) VALUES(%s, %s)", 
    (u.name, u.location))


if __name__ == '__main__':
    cur = cxn.cursor()
    user = process_csv(filecsv)
    insert(cur, user)

When run this, only the first row got inserted into the db. Can you please advise how to fix it?

Upvotes: 0

Views: 60

Answers (2)

Philippe T.
Philippe T.

Reputation: 1192

You aren't so far from your goal i think, try this :

def process_csv(filecsv):
    f = open(filecsv)
    f_csv = csv.reader(f)
    headers = next(f_csv)
    User = namedtuple('User', headers)
    for user in f_csv:
        user = [u.strip() for u in user]
        user = User(*user)
        yield user 

def insert(cur, user): 
    cur.execute("INSERT INTO users (name, location) VALUES(%s, %s)", 
    (user.name, user.location))

if __name__ == '__main__':
    cur = cxn.cursor()
    for user in process_csv():
        insert(cur,user)

a generateur is an iterator , so you should use it like a iterator

Upvotes: 2

user2357112
user2357112

Reputation: 281528

process_csv(filecsv) returns an iterator over users. insert turns the iterator into a list and explicitly takes the first item. (The comment is wrong; it's not a 1-element list.) Instead, loop over the users and insert each one:

def insert(cur, users): 
    for user in users:
        cur.execute("INSERT INTO users (name, location) VALUES(%s, %s)", 
        (user.name, user.location))

Note that there are almost certainly better ways to perform this database operation. This answer only addresses the generator issue.

Upvotes: 2

Related Questions