twitteR Proxy Authentication with ROAuth

Question

I am trying to use twitteR package from RStudio.
However i am getting an error:

'Proxy Authentication required' and some times - 'cannot find the host'.

There are many threads with the same problem.
I have tried everything (--internet2,R by setting "~/Rgui.exe" http_proxy=http:/999.99.99.99:8080/ http_proxy_user=ask). But it didn't work.

Find my below my code (I am running from my office desktop, RStudio), which requires authentication: proxyuserpwd="**domain//username:pwd**".

I am not sure about this line. I tried various combination but none worked.

rm(list=ls())
library(twitteR)
library(ROAuth)
library(RCurl)

options(RCurlOptions = list(
  verbose = TRUE,
  proxy ="http://proxy1.domain.com:8080",
  proxyuserpwd="domain//username:pwd",
  proxyauth="ntlm"))

reqURL <- "http://api.twitter.com/oauth/request_token"
accessURL <- "http://api.twitter.com/oauth/access_token"
authURL <- "http://api.twitter.com/oauth/authorize"
consumerKey <- '.............'
consumerSecret <- '..........'
twitCred <- OAuthFactory$new(consumerKey=consumerKey,
                          consumerSecret=consumerSecret,
                         requestURL=reqURL,
                         accessURL=accessURL,
                         authURL=authURL)

download.file(url="http://curl.haxx.se/ca/cacert.pem",destfile="cacert.pem")
twitCred$handshake(cainfo="cacert.pem")



< Proxy-Authenticate: BASIC realm="Access to this server requires AD Authentication.    Prefix your user ID with domain."
Host: api.twitter.com
Accept: */*
Proxy-Connection: Keep-Alive
Content-Length: 221
Content-Type: application/x-www-form-urlencoded

Error: Proxy Authentication Required
> registerTwitterOAuth(twitCred)
< HTTP/1.1 407 Proxy Authentication Required
* Authentication problem. Ignoring this.
< Proxy-Authenticate: NTLM
< Cache-Control: no-cache
< Pragma: no-cache
< Content-Type: text/html; charset=utf-8
* HTTP/1.1 proxy connection set close!
< Proxy-Connection: close
< Set-Cookie: BCSI-CS-3bf0ea03406bba28=2; Path=/
< Connection: close
< Content-Length: 899
< 
* Closing connection #0
Error in registerTwitterOAuth(twitCred) : 
oauth has not completed its handshake

However when I tried with the below code.

library("RCurl")
opts <- list(
proxy         = "proxy1.domain.com", 
proxyusername = "domain", 
proxypassword = "pwd",
proxyport     = 8080
)
getURL("http://stackoverflow.com", .opts = opts)

...I am successful.

>sessionInfo()  
R version 2.15.1 (2012-06-22)
Platform: i386-pc-mingw32/i386 (32-bit)

locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252    LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C                           LC_TIME=English_United States.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] twitteR_1.1.7  rjson_0.2.13   ROAuth_0.9.3   digest_0.6.3   RCurl_1.95-4.1    bitops_1.0-5  

loaded via a namespace (and not attached):
[1] tools_2.15.1

I tried Multiple Threads solutions from here and their solution none worked. I am not sure where i am making the mistake.(is it the domain that i need to put along with username troubling me) Can someone through some light on this issue.

UPDATE:

When I ran the same code in RGui (setting "~/Rgui.exe" http_proxy=http:/999.99.99.99:8080/http_proxy_user=ask), it has prompted for username and password to download cacert.pem.
But when I try to handshake it throws the same error:

Error: Proxy Authentication Required.

UPDATE:

I am getting the following Error:
Error in function (type, msg, asError = TRUE) : Received HTTP code 407 from proxy after CONNECT

UPDATE As Thomas suggested - tried

reqURL <- "https://api.twitter.com/oauth/request_token"
accessURL <- "https://api.twitter.com/oauth/access_token"
authURL <- "http://api.twitter.com/oauth/authorize"
consumerKey <- '-----'
consumerSecret <- '-------'
twitCred <- OAuthFactory$new(consumerKey=consumerKey,
                         consumerSecret=consumerSecret,
                         requestURL=reqURL,
                         accessURL=accessURL,
                         authURL=authURL)

h <- getCurlHandle(
proxy         = "proxy1.domain.com", 
proxyusername = "username",  #but i have to always prefix my domain with username(Domain\username or domain\\username or domain//username)
proxypassword = "password",
proxyport     = 8080,
cainfo = "cacert.pem")
twitCred$handshake(curl=h)

I got the following Error
Error in strsplit(response, "&") : non-character argument
i am updating the link with trace back output
5: strsplit(response, "&")
4: lapply(X = X, FUN = FUN, ...)
3: sapply(strsplit(response, "&")[[1]], strsplit, "=")
2: parseResponse(resp)
1: twitCred$handshake(curl = h)

Answer 1

I am also working behind a proxy server and was able to access twitter using following code.

consumer_key <- '<put your consumer key here>'
consumer_secret <- '<put your consumer secret here>'
access_token <- "<put your access token here>"
access_secret <- "<put your access token secret here>"
set_config(use_proxy(url='proxy- address',port-number, username, password))
setup_twitter_oauth(consumer_key,consumer_secret, access_token , access_secret)

And was able to fetch tweets. I have also written a blog with step by step procedure to solve this problem. You can access it from http://analyticsinall.blogspot.in/search/label/Analytics%20for%20Business

Answer 2

Thanks thomas, for Your Help - I solved the Problem.

rm(list=ls())
library(twitteR)
library(ROAuth)
library(RCurl)

reqURL <- "http://api.twitter.com/oauth/request_token"
accessURL <- "http://api.twitter.com/oauth/access_token"
authURL <- "http://api.twitter.com/oauth/authorize"
consumerKey <- '-----'
consumerSecret <- '------'
twitCred <- OAuthFactory$new(consumerKey=consumerKey,
                          consumerSecret=consumerSecret,
                         requestURL=reqURL,
                         accessURL=accessURL,
                         authURL=authURL)

h <- getCurlHandle(
proxy         = "proxy1.domain.com", 
 proxyusername = "username", 
proxypassword = "pwd",
proxyport     = 8080,     cainfo = "cacert.pem")

twitCred$handshake(curl=h)

To enable the connection, please direct your web browser to:
http://api.twitter.com/oauth/authorize?oauth_token=6VJ4mdc71BxVtgbI6YT0eNHqFCGe41f28MjiHl2KSvs When complete, record the PIN given to you and provide it here:

I have replaced https with http

Answer 3

I really don't know anything about proxy servers, but why don't you try this for your handshake line (explicitly specifying the curl handle):

h <- getCurlHandle(
      proxy         = "proxy1.domain.com", 
      proxyusername = "domain", 
      proxypassword = "pwd",
      proxyport     = 8080,
      cainfo = "cacert.pem")
twitCred$handshake(curl=h)