CODEWITHSUNDEEP
ramazon-s3
Josh
Josh

Reputation: 1309

Using aws.s3 R package to download json file

I'm trying to use the aws.s3 R package to download a JSON file that I know exists and contains twitter stream data.

To find all of my keys, I use this code:

library(aws.s3)
Sys.setenv("AWS_ACCESS_KEY_ID" = "my_key",
           "AWS_SECRET_ACCESS_KEY" = "my_secret_access_key",
           "AWS_DEFAULT_REGION" = "us-west-2")

bucket <- get_bucket('my_bucket',max = Inf)

keys <- c()
for(i in seq(1:length(bucket))){
  keys[i] <- bucket[[i]]$Key
}

Then I'm trying to actually get the json file from one of the keys using:

get_object(keys[n], bucket = 'my_bucket')

Instead of returning an error OR my json file, I'm getting a list of 182,858 seemingly random two-digit characters like this: 

 [1] 3c 3f 78 6d 6c 20 76 65 72 73 69 6f 6e 3d 22 31 2e 30 22 20 65 6e 63 6f 64 69 6e 67 3d 22 55 54 46 2d 38 22 3f
 [38] 3e 0a 3c 45 72 72 6f 72 3e 3c 43 6f 64 65 3e 4e 6f 53 75 63 68 4b 65 79 3c 2f 43 6f 64 65 3e 3c 4d 65 73 73 61
 [75] 67 65 3e 54 68 65 20 73 70 65 63 69 66 69 65 64 20 6b 65 79 20 64 6f 65 73 20 6e 6f 74 20 65 78 69 73 74 2e 3c
...

Am I using this package incorrectly? Any tips on how to do this better?

Upvotes: 2

Views: 1683

Answers (1)

Josh
Josh

Reputation: 1309

Thomas Leeper answered this on Twitter, but an case anyone comes looking here:

The response object comes back as a "raw" vector so it needs to be wrapped in rawToChar() or passed to fromJSON() after it's retrieved.

Upvotes: 3

Related Questions