CODEWITHSUNDEEP
rtext-miningstop-wordscorpustm
Brian
Brian

Reputation: 7316

Adding custom stopwords in R tm

I have a Corpus in R using the tm package. I am applying the removeWords function to remove stopwords

tm_map(abs, removeWords, stopwords("english"))

Is there a way to add my own custom stop words to this list?

Upvotes: 17

Views: 42178

Answers (6)

Confusion Matrix
Confusion Matrix

Reputation: 118

I am using the stopwords library instead of the tm library. I just decided to put my solution here in case anyone will need it.

# Create a list of custom stopwords that should be added
word <- c("quick", "recovery")
lexicon <-  rep("custom", times=length(word))

# Create a dataframe from the two vectors above
mystopwords <- data.frame(word, lexicon)
names(mystopwords) <- c("word", "lexicon")

# Add the dataframe to stop_words df that exists in the library stopwords
stop_words <-  dplyr::bind_rows(stop_words, mystopwords)
View(stop_words)

Upvotes: 1

Henryk Borzymowski
Henryk Borzymowski

Reputation: 1078

You could also use the textProcessor package. It works quite well:

textProcessor(documents, 
  removestopwords = TRUE, customstopwords = NULL)

Upvotes: 2

Reza Rahimi
Reza Rahimi

Reputation: 643

Save your custom stop words in a csv file (ex: word.csv).

library(tm)
stopwords <- read.csv("word.csv", header = FALSE)
stopwords <- as.character(stopwords$V1)
stopwords <- c(stopwords, stopwords())

Then you can apply custom words to your text file.

text <- VectorSource(text)
text <- VCorpus(text)
text <- tm_map(text, content_transformer(tolower))
text <- tm_map(text, removeWords, stopwords)
text <- tm_map(text, stripWhitespace)

text[[1]]$content

Upvotes: 4

BMALURU
BMALURU

Reputation: 11

It is possible to add your own stopwords to the default list of stopwords that came along with tm install. The "tm" package comes with many data files including stopwords, and note that stopwords files come for many languages. You can add, delete, or update the english.dat file under stopwords directory.
The easiest way to find the stopwords directory is to search for "stopwords" directory in your system through your file browser. And you should find english.dat along with many other language files. Open the english.dat file from RStudio which should enable to edit the file - you can add your own words or drop existing words as needed. It is the same process if you want to edit stopwords in any other language.

Upvotes: 1

Jeff J.
Jeff J.

Reputation: 75

You can create a vector of your custom stopwords & use the statement like this:

tm_map(abs, removeWords, c(stopwords("english"), myStopWords))

Upvotes: 2

James
James

Reputation: 66814

stopwords just provides you with a vector of words, just combine your own ones to this.

tm_map(abs, removeWords, c(stopwords("english"),"my","custom","words"))

Upvotes: 41

Related Questions