CODEWITHSUNDEEP
pythonpython-2.7csv
Nishant Kumar Thakur
Nishant Kumar Thakur

Reputation: 13

Remove redundant rows from csv where a particular column value is same in python

I have a csv file similar to below representation:

**Number,Timestamp,Value1,value2,Value3,Value4**

7680.0,2015-05-06 13:53:07,4.695,7.929,,

7680.0,2015-05-06 13:53:07,,,4.4118,7.8514

7681.0,2015-05-06 21:25:11,4.259,7.924,,

7681.0,2015-05-06 21:25:11,,,4.477,7.6178

I need to convert this file in below format:

**Number,Timestamp,Value1,value2,Value3,Value4**

7680.0,2015-05-06 13:53:07,4.695,7.929,4.4118,7.8514


7681.0,2015-05-06 21:25:11,4.259,7.924,4.477,7.6178

I am new to python 2.

Upvotes: 0

Views: 75

Answers (3)

UNagaswamy
UNagaswamy

Reputation: 2150

This can be easily handled by pandas

import pandas as pd
df = pd.read_csv("file1.csv", header=0, index_col=["**Number", "Timestamp"])
dfnew = df.groupby(df.index).sum()
dfnew.to_csv("file2.csv")

Upvotes: 0

Kathirmani Sukumar
Kathirmani Sukumar

Reputation: 10970

import pandas as pd
df = pd.read_csv('filename.csv')
df_group = df.groupby(['Number','Timestamp']).sum()

Groupby function will group your dataset by Number and Timestamp. Then sum() will sum all numeric columns. I hope this is what your looking for.

Upvotes: 1

Sakamaki Izayoi
Sakamaki Izayoi

Reputation: 153

Probably not the best solution, but this will get it done:

with open('messed_up.csv', 'r') as r and open('new.csv', 'w') as f:
   simValues = []
   for line in r:
       line = line.replace(',,','')
       line = line.split(',,,','')
       try:
           fOne, fTwo, fThree, fFour, fFive, fSix = line.split(',')
           if fOne not in simValues:
               simValues.append(fOne)
               f.write(line)
           else:
               print "[-] " + line + " was detected as similar"
       except Exception as e:
           print "[-] Error : " + str(e)

Upvotes: 0

Related Questions