CODEWITHSUNDEEP
rtmterm-document-matrix
Akira Murakami
Akira Murakami

Reputation: 453

Including all tokens in the term-document matrix in the R tm package

I'm trying to make a term-document matrix with the TermDocumentMatrix function of the tm package in R and found that some words are not included.

> library(tm)
> tdm <- TermDocumentMatrix(Corpus(VectorSource("The book is of great importance.")))
> rownames(tdm)
[1] "book"        "great"       "importance." "the"

Here, the words is and of have been excluded from the matrix. If the corpus only includes the deleted words, it gives the following message.

> tdm <- TermDocumentMatrix(Corpus(VectorSource("of is of is")))
Warning message:
In is.na(x) : is.na() applied to non-(list or vector) of type 'NULL'
> rownames(tdm)
NULL

The message signals that is and of are deleted before the matrix is built, but I have not been able to figure out why it occurs and how I can include all the tokens in the corpus.

Any help is appreciated.

Upvotes: 0

Views: 1742

Answers (1)

user1600826
user1600826

Reputation:

Use the control argument of TermDocumentMatrix

require(tm)
tdm <- TermDocumentMatrix(Corpus(VectorSource("of is of is")), control =  list(stopwords=FALSE, wordLengths=c(0, Inf)))
rownames(tdm)

Upvotes: 3

Related Questions