try demo，get this：

Error: could not find function "oauth2.0_token"

Error: could not find function "oauth2.0"

e.g. don't add to parsers list. Instead use type = "text" and parse yourself.

e.g.

library(httr)
search <- "httr package daterange:2456294-2456659"
r <- GET("https://www.google.com/search", query = list(q = search))
stop_for_status(r)


c <- content(r)
xpathSApply(c, "//a/@href")

from http://stackoverflow.com/questions/20888625/scrape-url-from-google-search-using-r-httr

Possible in principle, but currently make it very easy to crash R.

library(httr)
down_progress <- function() {
  bar <- NULL

  function(down, up) {
    total <- down[1]
    if (total == 0) {
      cat(".")
      return()
    }

    current <- down[2]
    if (is.null(bar)) {
      bar <<- txtProgressBar(max = total)
    }
    setTxtProgressBar(bar, current)
  }
}
prg <- config(noprogress = 0L, progressfunction = down_progress())
x <- GET("http://courses.had.co.nz/12-oscon/slides.zip", prg)
x <- GET("https://dl.dropbox.com/u/41902/ggplot2-course.zip", prg)

How can the authentication token be refreshed using the refresh_token instead of the user needing to grant access again? Is it possible for httr to automatically refresh using the refresh_token when the access_token ised but has expired?

Problem described as follows:

With the OAuth 2.0 client secret for my app held in an environment variable named "GANALYTICS_CONSUMER_SECRET"

Note: The client_id below has been replaced with a dummy value.

Using Windows 7 64-bit and R version 3.0.0, the below code in the R GUI will successfully request access from the user to Google Analytics, then pass the resulting authentication token back to R via a redirect to the localhost. (Note this does not currently work in RStudio.)

library(httr)
authorize.urlPath <- "auth"
  access.urlPath <- "token"
  base_url <- "https://accounts.google.com/o/oauth2"
  appname <- "GANALYTICS"
  client_id <- "123456789012.apps.googleusercontent.com"
  scope_url <- "https://www.googleapis.com/auth/analytics.readonly"
  endpoint <- oauth_endpoint(
    request = NULL,
    authorize = authorize.urlPath,
    access = access.urlPath,
    base_url = base_url
  )
  app <- oauth_app(
    appname = appname,
    key = client_id
  )
  access_token <- oauth2.0_token(
    endpoint = endpoint,
    app = app,
    scope = scope_url
  )

Passing access_token to sign_oauth2.0 results in the a malformed configuration query if passed to GET, i.e. the list object is flattened to a string starting as follows "access_token=list%28access%5Ftoken%20%3D%20%22 and ending %22%29"

However, passing access_token$access_token to sign_oauth2.0 works.

Therefore the following is executed to create the configure and send the request to Google Analytics (note the Google Analytics profile ID has been replaced with a dummy value):

request.config <- sign_oauth2.0(access_token = access_token$access_token)
query.url <- "https://www.googleapis.com/analytics/v3/data/ga?ids=ga%3A12345678&dimensions=ga%3Adate&metrics=ga%3Avisits&start-date=2013-04-21&end-date=2013-05-05&max-results=50"`
response <- GET(
  url = query.url,
  request.config
)

The above successfully retrieves the requested data from Google Analytics.

However, the token expires after 3600 seconds and needs to be refreshed using the refresh_token item from the access_token list object:

> print(access_token)
$access_token
[1] "########"

$token_type
[1] "Bearer"

$expires_in
[1] 3600

$refresh_token
[1] "########"

Note: the $access_token and $refresh_token have been masked from view.

Once the token has expired, if used as above, it results in Google Analytics returning a 401 HTTP status code with the following authentication error message "Login Required"

How can the authentication token be refreshed using the refresh_token instead of the user needing to grant access again? Is it possible for httr to automatically refresh using the refresh_token when the access_token ised but has expired?

Thank you.

E.g. http://httpbin.org/gzip.

Need to retrieve binary data by default, and then convert to text based on content type.

> library(httr)
> url <- "ftp://ftp.ensembl.org/pub/release-69/gtf/ailuropoda_melanoleuca/Ailuropoda_melanoleuca.ailMel1.69.gtf.gz"
> HEAD(url)
Error in from:to : result would be too long a vector
In addition: Warning messages:
1: In max(i) : no non-missing arguments to max; returning -Inf
2: In max(i) : no non-missing arguments to max; returning -Inf

Although

curl -I ftp://ftp.ensembl.org/pub/release-69/gtf/ailuropoda_melanoleuca/Ailuropoda_melanoleuca.ailMel1.69.gtf.gz

Works fine.

Probably same basic model as OAuth, but need to check.

Provide implementation for google as reference for implementors.

Currently, oauth_listener uses the built in httpd server which has the limitation that if help has already been started you can not change the port. Switching to httpuv would remedy this (and the blocking nature of httpuv might actually be an advantage). It should be a simple switch given that httpuv supports Rook.

Running the Google Demo with a custom app from the API Console and a devtools github httr install,
i'm getting an error for step3 "Get OAuth credentials":

  cred <- oauth2.0(google, myapp, 
             scope = "https://www.googleapis.com/auth/userinfo.profile")

Fehler: konnte Funktion "oauth2.0" nicht finden

de-en:

Error: could not find function "oauth2.0"

There may be useful logic in RCurl::isBinaryContent and RCurl::dynCurlReader

Download from http://curl.haxx.se/ca/cacert.pem and set default config to use it.

> parse_url("http://[email protected]:80/?a=1&b=2")$password
[1] NA

Should be NULL.

At least for google:

• create new google account
• store encrypted key and secret in travis
• store cached key

Tests

• tests old token invalidated when cloned token refreshed
• sensible error message if permission revoked when refreshing

e.g.

parsers$`text/csv` <- function(x, ...) read.csv(text = x, stringsAsFactors = FALSE, ...)

parsed_content for XML content returns in class XMLDocument. It would be much more useful to have an object of class XMLInternalDocument, as returned by XML::xmlParse, which can be used with getNodeSet, xpathSApply etc.

Maybe I'm just a dense user in this case though. How do I go from XMLDocument to an XMLInternalDocument object anyhow? (Normally I've used getURLContent followed by xmlParse, which gives this format).

Thanks!

See https://groups.google.com/d/msg/rdevtools/3bfSKMri7zo/7CgsLoI8YhEJ

Here's my RScript:

library('httr')
myapi <- oauth_endpoint(NULL, "authorize", "security/authorize", base_url= "https://myapi.com/v2")
myapp <- oauth_app("myapi", "my_key", "my_secret")
cred <- oauth2.0_token(myapi, myapp)

Here's the result:

Loading required package: Rook
Loading required package: tools
Loading required package: brew
Waiting for authentication in browser...
Authentication complete.
Error in file(con, "r") : cannot open the connection

Here's what the browser says after I login:

HTTP GET: http://localhost:43580/custom/OAuth/cred?code=TUZ2SlJkZDBDUzFReFNFalFaUU9keHk1bllCSTU0cC9TaXZTY1ZITXptSmV4K2NCS3h5bytEckVFbU1ESXFSUnZVZ3NoRWZUcHNoN1FtcTU3cVNPUkFUdk5mNHB4QkpKVjFmTENwc0N4ZkRqYWRyZW9weXUrMGtjZGpvak5lTGNPYUlsWk1PUHJMS2JzOHlST0VEZjB0YVFWTXhIZTZueTRoNHJYLzI3SEhDR0pOT3NTTlJHcG9QNW1vRGdteDM4dHF4Z1E5L0RFRTZuayt4UFYxMmRFWEZMYStsZFdDZmc1bmw5Q2NWdTI1ekdxQUtaYnAvbmQxMFlhUWxvUTIyTFdXeG90UHpJbGJjUUVvLzRPeGdGTUQzRmlDWms4eWJ6Q0VMR3NtbGV5RDlLNEFDNUNadm5wN2w1UnhxdVlNWm9WQi9Sck1HWHlHaFIvMTZMRFZ1M2pnPT0=

Authentication complete - you can now close this page and return to R.

If I execute the script again, it hangs at "Waiting for authentication in browser..." and the browser says this after I login:

HTTP GET: http://localhost:1410/custom/OAuth/cred?code=UTJCSzZRbGk0TnUwU2EwQ1hFUFc0ODZnQ25RWUlOdUxzMVcwSmRTS25maEw4dnVsYjI2dGxOemFnMTErU09EYlVpYk5FaG85MmNjV0taTHFsV2MzdlE5Z244bWN4NExLbVVCSWpGMExVcjZZU0pkSEtFeG94MmVaQkdCR2JuZ01oM0hYZDIvVUt3cnZlZFJxU3ppYWFTM3R0SDIyUFRSamVZdjNMYThRbjhkRkZKYitZY3N6MlRJamtrMkl2eHJEQk44Y1JyZVlBOTV5QlJacVVzQXlkd0E2c2N0SFNSN05LTjhxSzA4YTdCUkVSK08vYkJvSUxqVTFNbytJcStVSjJmdHdtNldZTFFMVncrek1hODdoaXd4ZFVGR1JMajlvYURoMVRvZUdkcFZrQzhYaHpvWHhHZWwzK25jZFlvbk5ObXlqNDdSUUdvVlpkKzlkbFRYbnNnPT0=

Oops! Google Chrome could not connect to localhost:1410

Probably using the ROauth package

A script I've written using httr works fine interactively, but dies with the following error when run via Rscript:

Error in getCurlHandle(cookiefile = cookie_path, .defaults = list()) :
  could not find function "getClass"
Calls: POST ... handle_url -> find_handle -> handle -> getCurlHandle -> .Call

The workaround is to add library(methods) to the user code. Presumably httr should do that...

I'm getting this curl call to work just fine

curl --form "[email protected]" http://taxosaurus.org/submit

Where myfile.txt is

"Dichorisandra diederichsanae"
"Echeveria longiflora"
"Gottschea cuspidiloba"
"Virola loretensis"
"Pterobryella praenitens"
"Ochthocosmus floribundus"
"Geissois superba"
"Ginkgo biloba"
"Escallonia angustifolia"
"Pterigynandrum julaceum"

But I can't quite get this to work using POST or postForm. I think I'm missing something obvious, but I don't know what it might be.

curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, FALSE);

I haven't had this error prior to this with other APIs, but the Springer Journals API for some reason gives something in the header that makes parsed_content think that it is html or perhaps xml that is returned when json is actually returned. I want JSON output, but I get error when wrapping the below call in parsed_content()

For example

tt <- GET(http://api.springer.com/metadata/json?api_key=2pn639bshyrju39zfewygkgd&q=dna)

Then element 4 in the list gives
tt[4]

content-type in tt[4] is given as "text/html; charset=utf-8", which is perhaps where the problem is?

Thanks! Scott

oauth_callback with no args, how to change a callback url like http://127.0.0.1:1410/

For http status messages, to make it easier for package users to react to specific types of error. stop_for_status should generate error of correct type

I'm probably doing something wrong here, but it's not clear how I specify content requests in PUT.
I'm trying to access the Figshare API with httr.

Authentication with httr and oauth 1 seems to work fine:

figshare_auth <-  function(cKey = getOption("FigshareKey", stop("Missing Figshare consumer key")),
       cSecret = getOption("FigsharePrivateKey", stop("Missing Figshare app secret")),
       token = getOption("FigshareToken", stop("Missing Figshare token")),
       token_secret = getOption("FigsharePrivateToken", stop("Missing Figshare Secret Token"))){
  require(httr)
  myapp <- oauth_app("rfigshare", key = cKey, secret=cSecret)
  sig <-  httr:::sign_ouath1.0(myapp, token = token, token_secret = token_secret) 
  } 

session <- figshare_auth()

And most of the basic GET commands work. But I try to a post method and get:

figshare_create <- 
function(title="test", description="description of test", type="dataset", session){
  require(RJSONIO)
  base <- "http://api.figshare.com/v1"
  method <- paste("my_data/articles")
  request <- paste(base, method, sep="/")
  meta <- toJSON(list("title"=title, "description"=description,
                      "type"=type, "Content-Type" = "application/json"))
  POST(request, config=session, body=meta)
}

and get the error

figshare_create(session=session)

Response [http://api.figshare.com/v1/my_data/articles]
  Status: 401
{"error": "Incorect content_type. Expected application/json, received:"} 

The GET -based methods work fine (those calls also require the 'session' variable, but I guess this could still be an authentication issue since they shouldn't actually need to authenticate since those should be public methods)?

Any ideas? (I'm building out API in rfigshare package here: https://github.com/ropensci/rfigshare)

Is there (or could you consider adding) a method to escape HTML characters for URLs, e.g. replacing : with %3A, etc?

I'm trying to make a GET request to the greycite API like so:

http://greycite.knowledgeblog.org/json?uri=http%3A%2F%2Fwww.carlboettiger.info

But would rather have the user pass the query url as such "http://carlboettiger.info". (I thought I recalled you had a method for this but couldn't find it just now).

Thanks!

Can you bump the version number on the github copy to indicate that this version isn't identical to the 0.1.1 version on CRAN? (README also says the package isn't on CRAN at all yet). Thanks!

regarding this SO post on httr and this SO post on RCurl - it's currently impossible to load a large file if that file is bigger than available RAM.

i have 64GB of RAM, so these two lines work for me in R x64 and do not work in R 32-bit (both on windows) --

require(httr)
x <- GET( "http://www2.census.gov/acs2011_5yr/pums/csv_pus.zip" )

This is probably my failure to implement correctly rather than a bug, but I cannot get the file upload step to work on the FigShare API.

• Here's my upload function
• Here's the Ruby version of the function from the official documentation.

Perhaps I am not handling the content-type correctly, or perhaps I am using the call to "upload_file" in the wrong place. Here's the error from the verbose debug message (secure keys wiped):

> fs_upload(myid, "rfigshare.png", session=session)
* Connection #0 seems to be dead!
* Closing connection #0
* About to connect() to api.figshare.com port 80 (#0)
*   Trying 107.20.170.85... * connected
> PUT /v1//my_data/articles/95796/files HTTP/1.1
Host: api.figshare.com
Accept: */*
Accept-Encoding: gzip
Cookie: figsh.09=339b6670bd7b6947765a00000000000000063580
Content-Type: multipart/form-data
Authorization: OAuth oauth_consumer_key="qMXXXXXXXXXXXXXXX", oauth_nonce="AD2U40JuVE", oauth_signature="ZXrwXXXXXXXXXXVHwQ%3D", oauth_signature_method="HMAC-SHA1", oauth_timestamp="1347300651", oauth_token="SrXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX", oauth_version="1.0"
Content-Length: 72
Expect: 100-continue

< HTTP/1.1 100 Continue
* We are completely uploaded and fine
< HTTP/1.1 401 Unauthorized
< Server: nginx/0.8.54
< Date: Mon, 10 Sep 2012 18:10:53 GMT
< Content-Type: application/json; charset=UTF-8
< Connection: keep-alive
< Content-Length: 383
< 
* Connection #0 to host api.figshare.com left intact
> 

i.e. if output is json (or xml?) should be able to get back as an R list.

I am not sure if this is an issue with httr or RStudio, but I was not able to complete the browser authentication if I run the github demo from within RStudio. I suspect it is because RStudio opens a different port by default. Here is part of a message which might help.

In server$start(port = 1410, quiet = TRUE) :
  RStudio has already started the web server on port 32387

I receive a 404 error when trying to authenticate on a client machine using RStudio Server. I believe the 404 derives from the fact that the endpoint is localhost rather than the IP address of the machine hosting the Rstudio server session.

Wondering if there's a workaround for this.

Reset config options after each call. May be possible to do in make_request by setting all defined config options to have an empty string.

to e.g. retry after pause, or after some other action (like updating auths), or returning a default value, or retry with verbose, or browser, ...

Maybe call it something like on_failure ?

See email to Duncan

it looks like there is a problem connecting to the fitbit api using the oauth1.0 method: after tokens are retrieved from the api, all calls are rejected with an invalid signature error.

It seems that the issue is with the oauth_signature function using curlEscape, while the OAuth 1.0 specification requires curlPercenEncode (see http://oauth.net/core/1.0/#encoding_parameters for details).

Changing the code solves the issue for the fitbit site, although I could not test whether this breaks anything for other api .

library(httr)
url <- "http://www.kegg.jp/kegg-bin/show_pathway?eco00260/eco%3ab0002%09%23ff0000,%23ffff00/eco%3ac00263%09%2300ff00,yellow"
GET(url)

results in:

Response [http://www.kegg.jp/kegg-bin/show_pathway?eco00260/eco%3ab0002%09%23ff0000,%23ffff00/eco%3ac00263%09%2300ff00,yellow=NA]
Status: 200
Content-type: text/html
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>Usage</title>
<link rel="stylesheet" href="/css/kegg2.css" type="text/css">
</head>
<body>
<h3>cannot create 0 (no link ko_kid)type </h3>
</body>
</html>

Whereas if I paste that URL into a browser, I get different (much longer) results.
Am I doing something wrong or is it a bug?
Dan

sessionInfo()
R Under development (unstable) (2012-12-15 r61341)
Platform: x86_64-apple-darwin9.8.0/x86_64 (64-bit)

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] httr_0.2

loaded via a namespace (and not attached):
[1] digest_0.6.0 RCurl_1.95-3 stringr_0.6.2 tools_3.0.0

The set of mime types used by parse_auto is a fixed property of any given version of httr. It is generated manually by the package using the function cache_media, which pulls the set of mime types from a hopefully authoritative source.

However, it's missing such gems as gz: application/x-gzip.

It can't be replaced (as far as I know) during use of the package because I can't replace data in a namespace (only functions) using techniques such as unlockBinding.

request: allow package user to specify a mime type source to replace or extend mime types during a session, or add a simple mime type source.

addMimeTypes(nameValueList)
replaceMimeTypes(nameValueList)
addMimeType("gz","application/x-gzip")

Related to: simpler mechanism to replace mime type handlers.

Simple acceptance tests that omit/include those components which are optional should suffice. Perhaps in conjunction with build_url.

Hi,

Thanks for this great package, incredibly useful and easy to use :-) One issue I have encountered is with setting a "Referer" header if the Referer URL is https...

> test <- GET('https://google.ca', c(add_headers('Referer'='https://www.google.com'), verbose()))
* Adding handle: conn: 0x4f02c30
* Adding handle: send: 0
* Adding handle: recv: 0
* Curl_addHandleToPipeline: length: 1
* - Conn 1 (0x4f02c30) send_pipe: 1, recv_pipe: 0
* About to connect() to google.ca port 443 (#1)
*   Trying 74.125.228.120...
* Connected to google.ca (74.125.228.120) port 443 (#1)
* successfully set certificate verify locations:
*   CAfile: /bla/R/x86_64-pc-linux-gnu-library/3.0/RCurl/CurlSSL/cacert.pem
  CApath: /etc/ssl/certs
* SSL connection using ECDHE-RSA-RC4-SHA
* Server certificate:
*    subject: C=US; ST=California; L=Mountain View; O=Google Inc; CN=*.google.ca
*    start date: 2013-11-20 14:48:09 GMT
*    expire date: 2014-03-20 00:00:00 GMT
*    subjectAltName: google.ca matched
*    issuer: C=US; O=Google Inc; CN=Google Internet Authority G2
*    SSL certificate verify ok.
> GET / HTTP/1.1
Host: google.ca
Accept: */*
Accept-Encoding: gzip
https://www.google.com

< HTTP/1.1 301 Moved Permanently
< Location: https://www.google.ca/
< Content-Type: text/html; charset=UTF-8
[...]

Note that the Referer header value is present in the request but the key is MIA...

Your assistance would be greatly appreciated.

Thanks!

google, facebook, github, ...

Trying to manipulate your Google API demo for use with the Google Docs API. I can't seem to get past the point of defining cred.

Everything "seems" to work fine:

library(httr)
google <- oauth_endpoint(
  NULL, 
  "auth", 
  "token",
  base_url = "https://accounts.google.com/o/oauth2"
  )

myapp <- oauth_app(
  "google", 
  "100...apps.googleusercontent.com", 
  secret="MY_SECRET"
  )

cred <- oauth2.0_token(
  google, 
  myapp,
  scope = "https://www.googleapis.com/auth/drive.readonly"
  )

A browser window asking for authorization pops up, and then an otherwise blank page loads saying "Authentication complete - you can now close this page and return to R.".

However, looks at cred, I see:

error 
"redirect_uri_mismatch"

I'm not at all familiar with OAuth, but would be happy to debug upon instruction.

I was trying to extend the github demo to create a repo on github. I did the oauth dance successfully and was able to execute the GET requests correctly. But when I tried to create a repo on github, I failed.

Github documentation says that I need to pass the name of the repo as the scope, but I was unable to figure out where I should pass this. The POST request I sent was

POST('https://api.github.com/user/repos', github_sig)

PS. For some reason oauth2.0_token is not exported. Is that intentional?

There seems to be a problem with the current version on CRAN: after loading the library, calling the sign_ouath1.0 function gives the following error message

> sign_ouath1.0
Error: object 'sign_ouath1.0' not found

although the corresponding help vignette is there.

• Generate random key: HTTR_SECRET=$(cat /dev/urandom | head -c 10000 | openssl sha1)
• Encrypt secret with travis: travis encrypt HTTR_SECRET=$HTTR_SECRET
• Locally, encrypt file using password: openssl aes-256-cbc -k "$HTTR_SECRET" -in keys.rds.enc -out keys.rds
• In before_script, decrypt file with openssl aes-256-cbc -k "$HTTR_SECRET" -in keys.rds.enc -d -a -out keys.rds

Set cookielist or cookiejar to tempfile, and then parse info$cookielist, splitting into tabs and selecting elements 6 and 7.