CODEWITHSUNDEEP
rnlpaccess-tokentext-miningquanteda
Lazel102
Lazel102

Reputation: 13

How to view tokens in quanteda after applying a dictionary

This is my first time asking a question here, so pleace excuse, if I am not handling it appropriately. I used the R package quanteda to analyse text documents.

My problem is now that I would like to see the text after I applied the dictionary that I developed. In order to apply the dictionary I tokenized the corpus, but then I couldn't find a function or method which allows me to see the tokenized text. I looked at the quanteda website and the cheat sheet but couldn't find any solution. This is basically the important part of my code: 

tokens_text_dict <- tokens_text %>% 
  tokens_lookup(dict_Info_priv, exclusive = FALSE)

EDIT: Moved code from comment to question:

text1 <- "a b c"
corpus1 <- corpus(text1)
tokens <- tokens(corpus1)
dict1 <- dictionary(list(A = "a")
tokens1_dict <- tokens_text %>% 
   tokens_lookup(dict1, exclusive = FALSE)

I am looking for the command which would give the "A b c", which should now be in tokens1_dict.

I would appreciate some help a lot!

Best wishes

Yannick

Upvotes: 1

Views: 538

Answers (1)

Ken Benoit
Ken Benoit

Reputation: 14902

There are two easy ways to view your tokens. In quanteda v2, there are options for printing the tokens object to the console. (See ?`print-quanteda`)

> print(tokens1_dict, max_ndoc = -1, max_ntok = -1)
Tokens consisting of 1 document.
text1 :
[1] "A" "b" "c"

Or, can use the View() function, which calls the display method for inspecting a list (of which a tokens object is a special type). This action is also triggered in RStudio by clicking on the object's name from the Environment pane.

> View(tokens1_dict)

enter image description here

Upvotes: 3

Related Questions