CODEWITHSUNDEEP
pythonperformancepandasmemory
user1566200
user1566200

Reputation: 1838

Memory error when using Pandas built-in divide, but looping works?

I have two DataFrames, each having 100,000 rows. I am trying to do the following:

new = dataframeA['mykey']/dataframeB['mykey']

and I get an 'Out of Memory' error. I get the same error if I try:

new = dataframeA['mykey'].divide(dataframeB['mykey'])

But if I loop through each element, like this, it works:

result = []
for idx in range(0,dataframeA.shape[0]):
    result.append(dataframeA.ix[idx,'mykey']/dataframeB.ix[idx,'mykey'])

What's going on here? I'd think that the built-in Pandas functions would be much more memory efficient.

Upvotes: 3

Views: 934

Answers (1)

user1566200
user1566200

Reputation: 1838

@ayhan got it right off the bat.

My two dataframes were not using the same indices. Resetting them worked.

Upvotes: 2

Related Questions