CODEWITHSUNDEEP
pythonmemorypandas
홍형기
홍형기

Reputation: 63

Python, Memory Error in making dataframe

When I use pandas DataFrame, occuring the Memory Error.

data's row is 200000 and column is 30.(type: list) fieldnames1 has columns name.(type:list)

Error occured in: 

df = pd.DataFrame(data,columns=[fieldnames1])

what should I do? (python version 2.7 32bit)

Upvotes: 2

Views: 8831

Answers (2)

Javeria
Javeria

Reputation: 101

You can try this line of code:

data=pd.DataFrame.from_csv("train.csv")

This is an alternate of read.csv but it returns Data frame object without giving any memory error P.S the size of the training data is around 73 mb

Upvotes: -1

Julien Marrec
Julien Marrec

Reputation: 11905

As indicated by Klaus, you're running out of memory. The problem occurs when you try to pull the entire text to memory in one go.

As pointed out in this post by Wes McKinney, "a solution is to read the file in smaller pieces (use iterator=True, chunksize=1000) then concatenate then with pd.concat".

Upvotes: 3

Related Questions