CODEWITHSUNDEEP
pythontwittertwython
Jamerson2
Jamerson2

Reputation: 151

How to obtain full text field value from Twitter API with TwythonStreamer

Trying to obtain full tweets via the following code. I understand you want to set the parameter tweet_mode to value 'extended', but since I'm not the standard query here I don't know where it would fit. For the text field I always get partial text cut off by '...' followed by a URL. With this configuration, how would you go about getting full tweets:

from twython import Twython, TwythonStreamer
import json
import pandas as pd
import csv

def process_tweet(tweet):
    d = {}
    d['hashtags'] = [hashtag['text'] for hashtag in tweet['entities']['hashtags']]
    d['text'] = tweet['text']
    d['user'] = tweet['user']['screen_name']
    d['user_loc'] = tweet['user']['location']
    return d
    
    
# Create a class that inherits TwythonStreamer
class MyStreamer(TwythonStreamer):     

    # Received data
    def on_success(self, data):

        # Only collect tweets in English
        if data['lang'] == 'en':
            tweet_data = process_tweet(data)
            self.save_to_csv(tweet_data)

    # Problem with the API
    def on_error(self, status_code, data):
        print(status_code, data)
        self.disconnect()
        
    # Save each tweet to csv file
    def save_to_csv(self, tweet):
        with open(r'tweets_about_california.csv', 'a') as file:
            writer = csv.writer(file)
            writer.writerow(list(tweet.values()))

# Instantiate from our streaming class
stream = MyStreamer(creds['CONSUMER_KEY'], creds['CONSUMER_SECRET'], 
                    creds['ACCESS_TOKEN'], creds['ACCESS_SECRET'])
# Start the stream
stream.statuses.filter(track='california', tweet_mode='extended')

Upvotes: 0

Views: 1297

Answers (1)

anon
anon

Reputation:

The tweet_mode=extended parameter has no effect on the v1.1 streaming API, as all Tweets are delivered in both extended and default (140) format.

If the Tweet object has the value truncated: true then there will be an additional element in the payload - extended_tweet. This is where the full_text value will be stored.

Note that this answer only applies to the v1.1 Twitter API, in v2 all Tweet text is returned by default in the streaming API (Twython does not currently support v2).

Upvotes: 2

Related Questions