CODEWITHSUNDEEP
pythonpandasduplicates
Niccola Tartaglia
Niccola Tartaglia

Reputation: 1667

drop duplicates pandas dataframe

I am getting an error message when using drop_duplicates to drop duplicate columns from my dataframe.

ValueError: Buffer has wrong number of dimensions (expected 1, got 2)

Below is a min example (notice that I don't have duplicate column names here, since that column won't be added then, so I var1 would be called var0 in my actual dataframe)

dict1 = [{'var0': 0, 'var1': 0, 'var2': 2},
         {'var0': 0, 'var1': 0, 'var2': 4},
         {'var0': 0, 'var1': 0, 'var2': 8},
         {'var0':0, 'var1': 0, 'var2': 12},]
df = pd.DataFrame(dict1, index=['s1', 's2','s1','s2'])
df.T.drop_duplicates().T

Upvotes: 2

Views: 1118

Answers (1)

Massoud
Massoud

Reputation: 523

The problem is with your indexing, when you transpose your DataFrame you will get duplicate column names which are messing it up. See below

dict1 = [{'var0': 0, 'var1': 0, 'var2': 2},
         {'var0': 0, 'var1': 0, 'var2': 4},
         {'var0': 0, 'var1': 0, 'var2': 8},
         {'var0':0, 'var1': 0, 'var2': 12},]
df = pd.DataFrame(dict1, index=['s1', 's2','s1','s2'])
df.reset_index().T.drop_duplicates().T.set_index('index')

Upvotes: 2

Related Questions