CODEWITHSUNDEEP
rstringtext-mining
Kavipriya
Kavipriya

Reputation: 441

Maximum occurrence of any set of words in text in R

Given a set of lines, I have to find maximum occurrence of words(need not be single word, can be set of words also.)

say, I have a text like,

string <- "He is john beck. john beck is working as an chemical engineer. Most of the chemical engineers are john beck's friend"

I want output to be,

john beck - 3
chemical engineer - 2

Is there any function or package which does this?

Upvotes: 1

Views: 125

Answers (2)

Ken Benoit
Ken Benoit

Reputation: 14902

Could also try this, using the quanteda package:

require(quanteda)
mydfm <- dfm(string, ngrams = 1:2, concatenator = "_", stem = TRUE, verbose = FALSE)
topfeatures(mydfm)
##           beck           john      john_beck         chemic chemical_engin          engin             is 
##              3              3              3              2              2              2              2 
##             an        an_chem            are 
##              1              1              1

You lose the stems, but this counts "john beck" three times instead of just two (since without stemming, "john beck's" will be a separate type).

It's simpler though!

Upvotes: 1

lukeA
lukeA

Reputation: 54237

Try this:

string <- "He is john beck. john beck is working as an chemical engineer. Most of the chemical engineers are john beck's friend"
library(tau)
library(tm)
tokens <- MC_tokenizer(string) 
tokens <- tokens[tokens != ""]
string_ <- paste(stemCompletion(stemDocument(tokens), tokens), collapse = " ")

## if you want only bi-grams: 
tab <- sort(textcnt(string_, method = "string", n = 2), decreasing = TRUE)
data.frame(Freq = tab[tab > 1])
#                   Freq
# john beck            3
# chemical engineer    2

## if you want uni-, bi- and tri-grams: 
nmin <- 1; nmax <- 3
tab <- sort(do.call(c, lapply(nmin:nmax, function(x) textcnt(string_, method = "string", n = x) )), decreasing = TRUE)
data.frame(Freq = tab[tab > 1])
#                   Freq
# beck                 3
# john                 3
# john beck            3
# chemical             2
# engineer             2
# is                   2
# chemical engineer    2

Upvotes: 3

Related Questions