Comments (9)
@lawinslow, @jiwalker-usgs, or @jread-usgs
I think this is the default using httr, but I'm still on RCurl. Anyone see an easy way to do in in RCurl?
Here's a comparison:
library(httr)
r <- GET("http://nwis.waterservices.usgs.gov/nwis/iv/?site=05114000&format=waterml,1.1&ParameterCd=00060&startDT=2014-10-10&endDT=2014-10-10")
headers(r)
#vs:
library(RCurl)
h <- basicHeaderGatherer()
r2 <- getURI("http://nwis.waterservices.usgs.gov/nwis/iv/?site=05114000&format=waterml,1.1&ParameterCd=00060&startDT=2014-10-10&endDT=2014-10-10", headerfunction = h$update)
h$value()
The headers(r) has"
$content-encoding
[1] "gzip"
The h$value() from r2 doesn't say anything about gzip.
from dataretrieval.
Hmmm. Do you have to specify that you accept gzip in the getURI call?
getURI("http://nwis.waterservices.usgs.gov/nwis/iv/?site=05114000&format=waterml,1.1&ParameterCd=00060&startDT=2014-10-10&endDT=2014-10-10", httpheader=c('Accept-Encoding'='gzip'), headerfunction = h$update)
my initial guess
from dataretrieval.
h$value()
Content-Encoding
"gzip"
from dataretrieval.
You don't get an error with that getURI request?
Error in curlPerform(curl = curl, .opts = opts, .encoding = .encoding) :
embedded nul in string: '\037<U+008B>\b'
from dataretrieval.
Well WTF, worked for me a second ago
from dataretrieval.
getURI("http://nwis.waterservices.usgs.gov/nwis/iv/?site=05114000&format=waterml,1.1&ParameterCd=00060&startDT=2014-11-11&endDT=2014-11-11", httpheader=c('Accept-Encoding'='gzip'), headerfunction = h$update, encoding='gzip')
from dataretrieval.
@ldecicco-USGS Does this work for you now? (also, slack test)
from dataretrieval.
Boom. Works. Now I'll see if it helps or not.
from dataretrieval.
It seems like you don't need the httpheader=c('Accept-Encoding'='gzip')
, just:
getURI(url,
headerfunction = h$update,
encoding='gzip')
And it is a tad faster. Nice.
from dataretrieval.
Related Issues (20)
- Inclusive Date Ranges HOT 1
- WaterML1.1 Depreciated? HOT 2
- SSL certificate problem HOT 6
- How to pull all site data from an entire state from readNWISsite HOT 4
- readNWISsite HOT 2
- Problem with the function readNWISdata HOT 2
- Need to escape WQP site IDs with spaces in them or NLDI queries.
- readNWISdata returning more parameter codes than requested HOT 3
- Enable Caching of data HOT 12
- broken web services link in whatNWISdata documentation
- Timeout error when attempting to retrieve data HOT 5
- POST in WQP not working? HOT 2
- Add citation guidance to tutorial HOT 2
- Site services w/qw output
- Consider being smarter about providers in WQP HOT 1
- How do I know what flow_cd column means and it's values? HOT 2
- allow data frame as input to importWQP HOT 1
- Drainage area retrieval from sites HOT 1
- Unexpected warning when providing duplicate pcodes to `pcode_to_name`
- retrieve water year summary statistics? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dataretrieval.