CODEWITHSUNDEEP
twittertweepy
ebt_dev
ebt_dev

Reputation: 159

Tweepy: Find all tweets in a specific language

I'd like to extract all tweets in the Arabic language in all countries.

I modified the code in this tutorial. This is my search query. api.search(q="*", count=tweetsPerQry, lang ['ar'],tweet_mode='extended'). I expect to find a very large number of tweets, but I only collected about 7000 tweets.

I tried to know the reason for finding a limited number of tweets, so I modified the query by replacing the lang parameter by geocode to find tweets in a city. I fetched more than 65,000 Arabic tweets. After that, I used the lang parameter with the geocode and I found a very limited number of tweets.

Upvotes: 2

Views: 4221

Answers (1)

TravisPooley
TravisPooley

Reputation: 429

The free twitter API's are good for small projects, but keep in mind that they don't display all of the tweets. Twitter has paid API's that are much more powerful, though what you are trying to achieve should be possible. I ran the query attached bellow, it seemed to work I was able to find a considerable amount of tweets. This method also seemed to work for @ebt_dev too I think it was just the structure of your request was set out like the stream listener version not the cursor search.

# Search Query change the X of .items(X) to the amount of tweets you are looking for
for tweet in tweepy.Cursor(api.search, q='*',tweet_mode='extended', lang='de').items(9999999):
    # Defining Tweets Creators Name
    tweettext = str( tweet.full_text.lower().encode('ascii',errors='ignore')) #encoding to get rid of characters that may not be able to be displayed
    # Defining Tweets Id
    tweetid = tweet.id

    # printing the text of the tweet
    print('\ntweet text: '+str(tweettext))
    # printing the id of the tweet
    print('tweet id: '+str(tweetid))

Upvotes: 5

Related Questions