CODEWITHSUNDEEP
pythontwittertweepy
Kabilesh
Kabilesh

Reputation: 1012

Exclude Retweets and Replies when getting Tweets from user timeline - Tweepy

I use the following code for downloading Tweets from user timeline with Tweepy. However, this returns Tweets including Retweets and Replies by the user also. I want only the Tweets posted in user's own timeline. How can I filter this results?

The reason is I want to collect Tweets posted by cosmetics companies about their products. Tweets in their timeline give me this. However, Replies and Retweets looks likes regular conversations, do not talk about products. I want to filter these out. 

import tweepy
import csv
import time

# Twitter API credentials
consumer_key = "xxxxxxx"
consumer_secret = "xxxxx"
access_key = "xxxxxxx"
access_secret = "xxxx"

def get_all_tweets(screen_name):
    # Twitter only allows access to a users most recent 3240 tweets with this method

    # authorize twitter, initialize tweepy
    auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
    auth.set_access_token(access_key, access_secret)
    api = tweepy.API(auth)

    # initialize a list to hold all the tweepy Tweets
    alltweets = []

    # make initial request for most recent tweets (200 is the maximum allowed count)
    new_tweets = api.user_timeline(screen_name=screen_name, count=200)

    # save most recent tweets
    alltweets.extend(new_tweets)

    # save the id of the oldest tweet less one
    oldest = alltweets[-1].id - 1

    # keep grabbing tweets until there are no tweets left to grab
    while len(new_tweets) > 0:
        print
        "getting tweets before %s" % (oldest)

        # all subsiquent requests use the max_id param to prevent duplicates
        new_tweets = api.user_timeline(screen_name=screen_name, count=200, max_id=oldest, include_entities=True)

        # save most recent tweets
        alltweets.extend(new_tweets)

        # update the id of the oldest tweet less one
        oldest = alltweets[-1].id - 1

        print
        "...%s tweets downloaded so far" % (len(alltweets))

    user = api.get_user(screen_name)
    followers_count = user.followers_count


    # transform the tweepy tweets into a 2D array that will populate the csv
    outtweets = [[tweet.id_str, tweet.created_at, tweet.text.encode("utf-8"), 1 if 'media' in tweet.entities else 0,
                  1 if tweet.entities.get('hashtags') else 0, followers_count, tweet.retweet_count, tweet.favorite_count]
                 for tweet in alltweets]


    # write the csv
    with open('tweets.csv', mode='a', encoding='utf-8') as f:
        writer = csv.writer(f)
        #writer.writerow(["id", "created_at", "text", "hasMedia", "hasHashtag", "followers_count", "retweet_count", "favourite_count"])
        writer.writerows(outtweets)

    pass

def main():
    get_all_tweets("@MACcosmetics")


if __name__ == '__main__':
    main()

Upvotes: 4

Views: 10482

Answers (4)

Amrit
Amrit

Reputation: 2165

Unfortunately tweepy does not have this: But instead you can use python-twitter

this has a method

def GetHomeTimeline(self,
                        count=None,
                        since_id=None,
                        max_id=None,
                        trim_user=False,
                        exclude_replies=False,
                        contributor_details=False,
                        include_entities=True):

and should work well in your case

Upvotes: 5

sindhuja parnam
sindhuja parnam

Reputation: 1

tweepy.Cursor(api.user_timeline, screen_name='anyname',include_rts=False)

Option include_rts=False removed retweets .

Reference link.

Upvotes: 0

Richard Willian
Richard Willian

Reputation: 148

I did it that way and it worked:

tweepy.Cursor(api.user_timeline, 
                        screen_name=usuario, 
                        count=None,
                        since_id=None,
                        max_id=None,
                        trim_user=True,
                        exclude_replies=True,
                        contributor_details=False,
                        include_entities=False
                        ).items(200);

Upvotes: 1

Shino Lex
Shino Lex

Reputation: 525

There are some search parameters you can send to twitter to filter the response.

exclude:retweets exclude:replies

so basically "SearchParams" + exclude:retweets exclude:replies should work

There are few more if you want to check it out https://developer.twitter.com/en/docs/tweets/rules-and-filtering/overview/standard-operators.html

Had the same issue and solved it this way right now, I hope it will help someone in the future

Upvotes: 2

Related Questions