CODEWITHSUNDEEP
pythonpandastweepy
overboard182
overboard182

Reputation: 300

Tweepy. storing tweet text in python pandas dataframe

I am following an online tutorial (http://adilmoujahid.com/posts/2014/07/twitter-analytics/) and I am getting stuck despite writing the python script the same. I am not really proficient in python and am having a hard time understanding documentation on maps (which are used in the tutorial). Right now I am getting "valueError Cannot set a frame with no defined index and a value that cannot be converted to a Series" and cannot figure out a fix. I am under the impression that the dataframe will have 3 columns. One with all the tweets, one with the tweets that mention facebook and one with all the tweets that mention microsoft. I also realize that the tutorial is two years old so maybe there is some syntax that is deprecated? Any help appreciated

import json 
import pandas as pd 
import re 

tweets_data_path = "Desktop/twit_dat/tweet1.txt"
tweets_data = []

tweets_file = open(tweets_data_path, "r")
for line in tweets_file:
    try:
        tweet = json.loads(line)
        tweets_data.append(tweet) 
    except:
        continue


tweets = pd.DataFrame()


tweets['text'] = map(lambda tweet: tweet['text'], tweets_data)
tweets['Facebook'] = tweets['text'].apply(lambda tweet: word_in_text('Facebook', tweet))
tweets['Microsoft'] = tweets['text'].apply(lambda tweet: word_in_text('Microsoft', tweet))



def word_in_text(word,text):
     if text == None:
        return False
     word = word.lower()
     text = text.lower() 
     match = re.search(word,text)
     if match:
        return True
     else:
        return False

Here is a sample of the data I am using: http://charon.kean.edu/~jonathan/exampledata.txt

Upvotes: 0

Views: 1821

Answers (1)

Prayalankar
Prayalankar

Reputation: 198

Maybe your pandas version is lower. I replicate the code and works fine on my compiler. See if this is helpful.
https://github.com/pandas-dev/pandas/issues/5632
--this is more of a comment but i don't have that privilege--.

Upvotes: 1

Related Questions