CODEWITHSUNDEEP
rspecial-characterstmterm-document-matrix
jz_
jz_

Reputation: 358

I am trying to create a DocumentTermMatrix while keeping all special characters

Trying to do some text mining with R without removing any special characters. For example in the following "LKC" and "LKC_" should be different words. Instead it is dropping the _ and making it the same word. How can I accomplish this?

library(tm)
special = c("OLAC_ LA LAC LAC_ LAC_E AC AC_ AC_E AC_ET",
                      ")LK )LKC )LKC- LK LKC LKC-",
                      "LAC_ LAC_E LKC LKC-")

bagOfWords <- Corpus(VectorSource(special))


mydocsDTM <- DocumentTermMatrix(bagOfWords, control = list(removePunctuation = FALSE,                     
preserve_intra_word_contractions = FALSE,         
preserve_intra_word_dashes = FALSE,
removeNumbers = FALSE,
stopwords = FALSE,
stemming = FALSE
))

  inspect(mydocsDTM)

Upvotes: 1

Views: 473

Answers (1)

Ken Benoit
Ken Benoit

Reputation: 14902

Easily done using the quanteda package, after which you can convert to a DocumentTermMatrix, or just keep using quanteda.

library("quanteda")
qdfm <- dfm(special, tolower = FALSE, what = "fasterword")
qdfm
# Document-feature matrix of: 3 documents, 15 features (57.8% sparse).
# 3 x 15 sparse Matrix of class "dfm"
#        features
# docs    OLAC_ LA LAC LAC_ LAC_E AC AC_ AC_E AC_ET )LK )LKC )LKC- LK LKC LKC-
#   text1     1  1   1    1     1  1   1    1     1   0    0     0  0   0    0
#   text2     0  0   0    0     0  0   0    0     0   1    1     1  1   1    1
#   text3     0  0   0    1     1  0   0    0     0   0    0     0  0   1    1

convert(qdfm, to = "tm")
# <<DocumentTermMatrix (documents: 3, terms: 15)>>
# Non-/sparse entries: 19/26
# Sparsity           : 58%
# Maximal term length: 5
# Weighting          : term frequency (tf)

Upvotes: 0

Related Questions