CODEWITHSUNDEEP
pythonlistnlpnltkstop-words
nia4life
nia4life

Reputation: 363

Remove Stop Words in Python List Using List Comprehension

Python noob so sorry for simple question but I can't find the exact solution for my situation.

I've got a python list, I want to remove stop words from a list. My code isn't removing the stopword if it's paired with another token.

    from nltk.corpus import stopwords
    rawData = ['for', 'the', 'game', 'the movie']
    text = [each_string.lower() for each_string in rawData]
    newText = [word for word in text if word not in stopwords.words('english')]
    print(newText)

current output: ['game', 'the movie']

desired output ['game', 'movie']

I'd prefer to use list comprehension for this.

Upvotes: 0

Views: 2109

Answers (1)

The Pilot Dude
The Pilot Dude

Reputation: 2227

It took me a while to do this because list comprehensions are not my thing. Anyways, this is how I did it:

import functools

stopwords = ["for", "the"]

rawData = ['for', 'the', 'game', 'the movie']
lst = functools.reduce(lambda x,y: x+y, [i.split() for i in rawData])
newText = [word for word in lst if word not in stopwords]
print(newText)

Basically, line 4 splits the list values to make a nested list AND turns the nested list one dimensional.

Upvotes: 1

Related Questions