CODEWITHSUNDEEP
pythonpython-3.xpandas
marcelo.delta
marcelo.delta

Reputation: 3082

How can I create a loop with pandas read_csv?

How can I create a loop with pandas read_csv?

I need to create a data loop to list and save to the database.

How can I do this loop with the data from a csv?

thank you all for your attention

produtos = pd.read_csv('tabela.csv', delimiter=';')
        for produto in produtos:
            print(produto['NOME'])

Upvotes: 6

Views: 28132

Answers (4)

mins
mins

Reputation: 7504

You can iterate using a reader, which is obtained by specifying a chunksize in the call to read_csv().

Here I use CSV data in memory and read two rows at once. Replace StringIO(s) by your file, and use a chunsize of 1 if you want to read a single row at once:

import pandas as pd
from numpy import random as npr, round
from io import StringIO

# Some CSV data to read, in a string
s = pd.DataFrame(round(npr.randn(10, 4), 2)).to_csv()

# Use the reader in a with/as structure
with  pd.read_csv(StringIO(s),index_col=0, chunksize=2) as reader:
    for i, chunk in enumerate(reader):
        print(f'\nChunk {i:}')
        print(chunk)

Chunk 0
      0     1     2     3
0 -1.50 -1.78 -0.53  1.09
1 -0.35 -0.79  0.20  1.08

Chunk 1
      0     1     2     3
2 -1.44 -1.21 -0.79  1.09
3  0.23  2.13  0.94 -0.04

Chunk 2
[...]

Upvotes: 0

Anderson B. Magalhães
Anderson B. Magalhães

Reputation: 204

If you can create loop with Pandas by column name:

produtos = pd.read_csv('tabela.csv', delimiter=';')
for i, produto in produtos.iterrows():
    print(produto['NOME'])

But if you want to insert directly on your database use sqlalchemy and function to_sql like this:

from sqlalchemy import create_engine
import pandas as pd
...
engine = create_engine("mysql://user:pwd@localhost/database")
produtos = pd.read_csv('tabela.csv', delimiter=';')
if_exists_do = 'append'
produtos.to_sql('table_name', con=engine, if_exists=if_exists_do)

Then it will be inserted on database. The var 'if_exists_do' can receive value 'replace' if you want this.

Upvotes: 0

Biarys
Biarys

Reputation: 1173

If you have files that you need to save, I recommend this

import os
import pandas as pd
from sqlalchemy import create_engine

engine = create_engine('sqlite://', echo=False)
path = "C:/path/to/directory"

# list all files in the directory, assuming this directory 
# contains only files csv files that you need to save 
for file in os.listdir(path):
    df = pd.read_csv(path+file)
    # some other data cleaning/manipulation

    # write dataframe to database
    df.to_sql("table_name", con=engine)

Alternative, you can create a list with all files locations and iterate through that one instead. More info on to_sql() and check out this answer

Upvotes: 4

Fernandoms
Fernandoms

Reputation: 444

To iterate in the DataFrame resulted by calling pandas read_csv you should use the command iterrows() for iteration, as in the below example:

for produto in produtos.iterrows():
    print(produto['NOME'])

Upvotes: 8

Related Questions