CODEWITHSUNDEEP
rtwitter
Partha Roy
Partha Roy

Reputation: 1621

Issue while mining tweets in R

In this program I am trying to read n number of tweets by a specific user and show his tweet data after processing , but the problem is when I specify Number of tweets as 10 then it runs well..

Code snipet

#Tweet processing
library("twitteR")
library("tm")

tweets_process<-function(){




tweets<-userTimeline("roypartha97",n=100)

tweets.df<-twListToDF(tweets)


mycorpus<-Corpus(VectorSource(tweets.df$text))
mycorpus<-tm_map(mycorpus,content_transformer(tolower))
mycorpus<-tm_map(mycorpus,removePunctuation)
mycorpus<-tm_map(mycorpus,removeNumbers)

removeUrl<-function(x) gsub("http[:alnum:]*","",x)

mycorpus<-tm_map(mycorpus,removeUrl)
mycorpus<-tm_map(mycorpus,removeWords,stopwords("english"))

mycorpusCopy<-mycorpus
mycorpus<-tm_map(mycorpus,stemDocument,language="english",lazy=TRUE)

for(i in 1:5)
{
    cat(paste("[",i,"]",sep=""))
    writeLines(mycorpus[[i]])
}

#mycorpus<-tm_map(mycorpus,stemCompletion,dictionary=mycorpusCopy,lazy=TRUE)

#tdm<-TermDocumentMatrix(mycorpus,control=list(wordLengths=c(1,Inf)))
#print(tdm)


}

But When I am changing number of tweets to 100 from 10 , there come these problems - 

  [1]Error in UseMethod("stemDocument", x) : 
  no applicable method for 'stemDocument' applied to an object of class "try-error"
In addition: Warning messages:
1: In mclapply(content(x), FUN, ...) :
  scheduled core 1 encountered error in user code, all values of the job will be affected
2: In mclapply(content(x), FUN, ...) :
  scheduled core 1 encountered error in user code, all values of the job will be affected
3: In mclapply(content(x), FUN, ...) :
  scheduled core 1 encountered error in user code, all values of the job will be affected
4: In mclapply(content(x), FUN, ...) :
  scheduled core 1 encountered error in user code, all values of the job will be affected
>

Upvotes: 1

Views: 88

Answers (1)

Partha Roy
Partha Roy

Reputation: 1621

After a lot of trying what I did is in the DocumentTermMatrix creation step , I specified the cleaning process and its working fine.

this is what i used - 

tdm = TermDocumentMatrix(mycorpus,control=list(removepunctuation=TRUE,stopwords=c(stopwords("english"),customstopwords),removeNumbers=TRUE,tolower=TRUE))

Upvotes: 1

Related Questions