CODEWITHSUNDEEP
pythoncsv
James
James

Reputation: 305

Extracting some portion of data from text into csv using Python

I have some data in text file as shown below image(it's just a portion of whole data). What is best way to filter out keys? 

{  u'chan': 5,
   u'cls': 0,
   u'codr': u'4/5',
   u'data': u'ABfxqqqpVVVOAAA='
}

Upvotes: 1

Views: 130

Answers (2)

themistoklik
themistoklik

Reputation: 880

For a series of files in the form of what you posted try

import csv
filenames=[...]
#open your output file in append mode

with open('output.csv','a+') as out:
    outwriter = csv.writer(out,dialect='excel')

#write your headers
outwriter.writerow(['header1','header2','header3'])

#for every file
for fname in filenames:
    with open(fname) as f:

        #list that holds each row
        csvlines=[]

        #labels you wanna keep
        keep=["u'data'","u'rssi'","u'timestamp'"]
        lines=f.readlines()
        print lines

        #split at first : character
        lines = map(lambda x:x.split(':',1),lines)

        for line in lines:
            if line[0].strip() in keep:

                csvlines.append(line[1].strip())
    #clean from unnecessary characters
            csvlines=map(lambda x:x.replace("u'","").replace("'","").replace(",",""),csvlines)
    #write it to csv and then reset it for the next file.
            if(len(csvlines)==3):
               print "writing"
               print csvlines
               outwriter.writerow(csvlines)
               csvlines=[]

Upvotes: 0

dubbbdan
dubbbdan

Reputation: 2740

Assuming the sample data is one record of your data, you can use pandas to subset and export directly to an excel workbook.

import pandas as pd

df = pd.DataFrame({  u'chan': 5,
   u'cls': 0,
   u'codr': u'4/5',
   u'data': u'ABfxqqqpVVVOAAA=',
   u'datr': u'SF10BW125',
   u'freq': u'912.9',
   u'lsnr': u'-8.2',
   u'mhdr': u'8007000002001900',
   u'modu': u'LORA',
   u'opts': u'',
   u'port': 5,
   u'rfch': 1,
   u'rssi': -111,
   u'seqn': 25,
   u'size': 16,
   u'timestamp': u'2016-11-17T09:51:44.406724Z',
   u'tmst': 2477375724L},index=[0])
df = df[['data','chan','timestamp','rssi']]
oName = #Path to desired excel workbook
df.to_excel(oName,'Sheet1')

Upvotes: 1

Related Questions