CODEWITHSUNDEEP
rapplytext-mininggrepl
lmcshane
lmcshane

Reputation: 1114

R: extract and paste keyword matches

I am new to R and have been struggling with this one. I want to create a new column, that checks if a set of any of words ("foo", "x", "y") exist in column 'text', then write that value in new column.

I have a data frame that looks like this: a->

 id     text        time   username
 1     "hello x"     10     "me"
 2     "foo and y"   5      "you"
 3     "nothing"     15     "everyone"
 4     "x,y,foo"     0      "know"

The correct output should be:

a2 ->

id     text        time   username        keywordtag  
 1     "hello x"     10     "me"          x
 2     "foo and y"   5      "you"         foo,y
 3     "nothing"     15     "everyone"    0 
 4     "x,y,foo"     0      "know"        x,y,foo

I have this:

df1 <- data.frame(text = c("hello x", "foo and y", "nothing", "x,y,foo"))
terms <- c('foo', 'x', 'y')
df1$keywordtag <- apply(sapply(terms, grepl, df1$text), 1, function(x) paste(terms[x], collapse=','))

Which works, but crashes R when my needleList contains 12k words and my text has 155k rows. Is there a way to do this that won't crash R?

Upvotes: 1

Views: 1189

Answers (1)

steveb
steveb

Reputation: 5532

This is a variation on what you have done, and what was suggested in the comments. This uses dplyr and stringr. There may be a more efficient way but this may not crash your R session.

library(dplyr)
library(stringr)

terms      <- c('foo', 'x', 'y')
term_regex <- paste0('(', paste(terms, collapse = '|'), ')')

### Solution: this uses dplyr::mutate and stringr::str_extract_all
df1 %>%
    mutate(keywordtag = sapply(str_extract_all(text, term_regex), function(x) paste(x, collapse=',')))
#       text keywordtag
#1   hello x          x
#2 foo and y      foo,y
#3   nothing           
#4   x,y,foo    x,y,foo

Upvotes: 2

Related Questions