CODEWITHSUNDEEP
pythonjsoncsvtwitter
ramiroaznar
ramiroaznar

Reputation: 266

How to read twitter-json file and save it as a csv with Python

I have a json-text file containing tweets from a certain hashtag. Now I transform it to a matrix with a row for each tweet and an array of columns such as user, time, latitude, longitude and so on. I have written the following code, but when I get the output file the information is not saved. It has just showed the header row:

#import module
import json
from csv import writer

#input file
tweets = ()
for line in open('file.txt'):
    try: 
        tweets.append(json.loads(line))
    except:
        pass

#variables
ids = [tweet['id_str'] for tweet in tweets]
times = [tweet['created_at'] for tweet in tweets]
users = [tweet['user']['name'] for tweet in tweets]
texts = [tweet['text'] for tweet in tweets]
lats = [(T['geo']['coordinates'][0] if T['geo'] else None) for T in tweets]
lons = [(T['geo']['coordinates'][1] if T['geo'] else None) for T in tweets]
place_names = [(T['place']['full_name'] if T['place'] else None) for T in tweets]
place_types = [(T['place']['place_type'] if T['place'] else None) for T in tweets]

#output file
out = open('tweets_file.csv', 'w')
print >> out, 'id,created,text,user,lat,lon,place name,place type'
rows = zip(ids, times, texts, users, lats, lons, place_names, place_types)
csv = writer(out)
for row in rows:
    values = [(value.encode('utf8') if hasattr(value, 'encode') else value) for value in row]
    csv.writerow(values)
out.close()

Please could you help me to find and clear the bug... Thanks in advance.

R.

Upvotes: 0

Views: 3090

Answers (1)

dani herrera
dani herrera

Reputation: 51715

In your code, tweets is a tuple:

'tuple' object has no attribute 'append'

It seems you have copy-paste code from several sources without understand what is doing.

import json
from csv import writer

with open('file.txt') as data_file:    
    data = json.load(data_file)

tweets = data['statuses']

ids = [tweet['id_str'] for tweet in tweets]
times = [tweet['created_at'] for tweet in tweets]
...

Upvotes: 1

Related Questions