CODEWITHSUNDEEP
pythontext-mining
Frits Verstraten
Frits Verstraten

Reputation: 2189

Create a stemmer to reduce words to a base form

I am dealing with a case now for which I would like to create my own stemming algorithm. I know that there are some excellent libraries for this but this does not work for this use case.

In essence I would like to import a dictionary so I can loop through words in a sentence and if a word is present in a list, reduce it to its base form.

So in case, fe reduce 'banker' to bank. Im have produced this but this is not scalable.

list_bank = ('banking', 'banker' )
sentence = ("There's a banker")
banker_tags = []

for word in sentence.split():
 print(word)

So in case, fe reduce 'banker' to bank if word in list_bank: #replace word

Any suggestion on how I can get this working?

Upvotes: 0

Views: 757

Answers (1)

user325117
user325117

Reputation:

Put the words and their stems in a dictionary and then use that to look up the stemmed form:

dictionary = { 'banker' : 'bank', 'banking': 'bank' } # Add the rest of your words and stems
sentence = "There's a banker"
for word in sentence.split():
    if word in dictionary:
        word = dictionary[word]
    print(word)

There's
a
bank

Upvotes: 2

Related Questions