CODEWITHSUNDEEP
pythonweb-scrapingtwittertwint
Cype
Cype

Reputation: 23

Twint output list is returning 'NoneType' Error

I'm writing a simple program to save a given twitter user's tweets word-by-word into a .csv file, as well as using nltk to tag them with parts of speech.

When attempting to iterate through twint.output.tweets_list, I receive the following error:

twint.get:User:'NoneType' object is not subscriptable

I know for a fact that there are tweets to be returned, so it's not simply missing tweets.

My code is as follows:

import twint
import csv
import nltk

# Configure Twint object

c = twint.Config()
c.Username = "POTUS"
c.Limit = 100

# Run Twint

twint.run.Search(c)

# Open a CSV file and write the tweets and their parts of speech to it

with open('tweets_with_POS.csv', 'w', newline='', encoding='utf-8') as file:
    writer = csv.writer(file)
    writer.writerow(["word", "part_of_speech"])
    for tweet in twint.output.tweets_list:
        words = nltk.word_tokenize(tweet.tweet)
        pos_tags = nltk.pos_tag(words)
        for word, pos in pos_tags:
            writer.writerow([word, pos])

I've tried running the code from a variety of networks, thinking it may be an IP block, but it doesn't seem to be. Any help is appreciated.

You will need to include the following code if you want to reproduce this nltk.download('punkt') nltk.download('averaged_perceptron_tagger')

Upvotes: 0

Views: 699

Answers (1)

Cype
Cype

Reputation: 23

Turns out the issue was with a screwy compatibility issue with Twint.

I ran pip install --upgrade -e git+https://github.com/twintproject/twint.git@origin/master#egg=twint to upgrade twint, which had automatically downloaded one version behind the main branch for some reason.

I then encountered AttributeError: module 'typing' has no attribute '_ClassVar' which I resolved by running pip uninstall dataclasses -y

Upvotes: 0

Related Questions