Giter VIP home page Giter VIP logo

hurley's Introduction

Hurley

Hurley is a ruby gem with no runtime dependencies that provides a common interface for working with different HTTP adapters. It is an evolution of Faraday, with rethought internals.

Hurley revolves around three main classes: Client, Request, and Response. A Client sets the default properties for all HTTP requests, including the base url, headers, and options.

require "hurley"

# If you prefer Addressable::URI, require this too:
# This is required automatically if `Addressable::URI` is defined when Hurley
# is being loaded.
require "hurley/addressable"

client = Hurley::Client.new "https://api.github.com"
client.header[:accept] = "application/vnd.github+json"
client.query["a"] = "?a is set on every request too"

client.scheme # => "https"
client.host   # => "api.github.com"
client.port   # => 443

# See Hurley::RequestOptions in lib/hurley/options.rb
client.request_options.timeout = 3

# See Hurley::SslOptions in lib/hurley/options.rb
client.ssl_options.ca_file = "path/to/cert.crt"

# Verbs head, get, put, post, patch, delete, and options are supported.
response = client.get("users/tater") do |req|
  # These properties can be changed on a per-request basis.
  req.header[:accept] = "application/vnd.github.preview+json"
  req.query["a"] = "override!"

  req.options.timeout = 1
  req.ssl_options.ca_file = "path/to/cert.crt"

  req.verb   # => :get
  req.scheme # => "https"
  req.host   # => "api.github.com"
  req.port   # => 443
end

# You can also use Hurley class level shortcuts, which use Hurley.default_client.
response = Hurley.get("https://api.github.com/users/tater")

response.header[:content_type] # => "application/json"
response.status_code           # => 200
response.body                  # => {"id": 1, ...}
response.request               # => same as `request`

# Is this a 2xx response?
response.success?

# Is this a 3xx redirect?
response.redirection?

# Is this is a 4xx response?
response.client_error?

# Is this a 5xx response?
response.server_error?

# What kind of response is this?
response.status_type # => One of :success, :redirection, :client_error, :server_error, or :other

# Timing of the response, in ms
response.ms

# Responses automatically follow 5 redirections by default.

response.via      # Array of Request objects that redirected.
response.location # => New Request built from Location header URL.

# You can tune the number of redirections, or disable them per Client or Request.

# This client follows up to 10 redirects
client.request_options.redirection_limit = 10
client.get "/foo" do |req|
  # this specific request never follows any redirects.
  req.options.redirection_limit = 0
end

Connections

By default, a Hurley::Client uses a Hurley::Connection instance to make requests with net/http. You can swap the connection with any object that responds to #call with a Request, and returns a Response. This will not interrupt other client properties or callbacks.

client = Hurley::Client.new "https://api.github.com"
client.connection = lambda do |req|
  # return a Hurley::Response!
end

URLs

Hurley joins a Client endpoint with a given request URL to produce the final URL that is requested.

client = Hurley::Client.new "https://a:[email protected]/v1?a=1"

client.get "user" do |req|
  req.url.user     # => "a"
  req.url.password # => "b"
  req.url          # https://api.com/v1/user?a=1
end

# Absolute paths remove any path prefix
client.get "/v2/user" do |req|
  req.url.user     # => "a"
  req.url.password # => "b"
  req.url          # https://api.com/v2/user?a=1
end

client.get "user?a=2" do |req|
  req.url.user     # => "a"
  req.url.password # => "b"
  req.url          # https://api.com/v1/user?a=2
end

# Basic auth can be overridden
client.get "https://c:[email protected]/v1/user" do |req|
  req.url.user     # => "c"
  req.url.password # => "d"
  req.url          # https://api.com/v1/user?a=1
end

client.get "https://staging.api.com/v1/user" do |req|
  req.url.user     # => nil, since the host changed
  req.url.password # => nil
  req.url          # https://staging.api.com/v1/user
end

Hurley uses Hurley::Query::Nested for all query encoding and decoding by default. This can be changed globally, per client, or per request. Typically you won't create Hurley::Query instances manually, and will use Hurley::Query.parse for parsing.

# Nested queries

q = Hurley::Query::Nested.new(:a => [1,2], :h => {:a => 1})
q.to_query_string # => "a[]=1&a[]=2&h[a]=1"

Hurley::Query::Nested.parse(q.to_query_string)
# => #<Hurley::Query::Nested {"a"=>["1", "2"], "h"=>{"a"=>"1"}}>

# Flat queries

q = Hurley::Query::Flat.new(:a => [1,2])
q.to_query_string # => "a=1&a=2"

Hurley::Query::Flat.parse(q.to_query_string)
# => #<Hurley::Query::Nested {"a"=>["1", "2"]}>

# Change it globally.
Hurley.default = Hurley::Query::Flat

# Change it for just this client.
client = Hurley::Client.new
client.request_options.query_class = Hurley::Query::Flat

# Change it for just this request.
client.get "/foo" do |req|
  req.options.query_class = Hurley::Query::Flat
end

Headers

A Client's Header is passed down to each request. Header keys can be overridden by the request. Headers are stored internally in canonical form, which is capitalized with dashes: "Content-Type", for example.

See Hurley::Header for all of the common header keys that have symbolized shortcuts.

client = Hurley::Client.new "https://api.com"
client.header[:content_type] = "application/json"

# Same as:
client.header["content-type"] = "application/json"
client.header["Content-Type"] = "application/json"

client.get "/something.atom" do |req|
  # Default user agent
  req.header[:user_agent] # => "Hurley v#{Hurley::VERSION}"

  # Change a header
  req.header[:content_type] = "application/atom"
end

Posting Forms

Hurley will encode form bodies for you, while setting default Content-Type and Content-Length values as necessary. Multipart forms are supported too, using Hurley::UploadIO objects.

# Works with HTTP verbs: post, put, and patch

# Send a=1 with Content-Type: application/x-www-form-urlencoded
client.post("/form", :a => 1)

# Send a=1 with Content-Type: text/plain
client.post("/form", {:a => 1}, "text/plain")

# Send file with Content-Type: multipart/form-data
client.post("/multipart", :file => Hurley::UploadIO.new("filename.txt", "text/plain"))

The default query parser (Hurley::Query::Nested) is used by default. You can change it globally, per client, or per request. See the "URLs" section.

Client Callbacks

Clients can define "before callbacks" that yield a Request, or "after callbacks" that yield a Response. Multiple callbacks of the same type are added in order.

client.before_call do |req|
  # modify request before it's called
end

client.before_callbacks # => ["#<Proc:...>"]

client.after_call do |res|
  # modify response after it's called
end

# You can set a name to identify the callback
client.before_call :upcase do |req|
  req.body.upcase!
end

client.before_callbacks # => [:upcase]

# You can also pass an object that responds to #call and #name.
class Upcaser
  def name
    :upcaser
  end

  def call(req)
    req.body.upcase!
  end
end

client.before_call(Upcaser.new)
client.before_callbacks # => [:upcaser]

Streaming the Response Body

A Request object can take a callback that receives the response body in chunks as they are read from the socket. Hurley connections that don't support streaming will yield the entire response body once.

client.get "big-file" do |req|
  req.on_body do |res, chunk|
    puts "#{res.status_code}: #{chunk}"
  end

  # This streams the body for 200 or 201 responses only:
  req.on_body(200, 201) do |res, chunk|
    puts "#{res.status_code}: #{chunk}"
  end
end

Testing

Hurley includes a Test connection object for testing. This lets you make requests without hitting a real endpoint.

require "hurley"
require "hurley/test"

client = Hurley::Client.new "https://api.github.com"

client.connection = Hurley::Test.new do |test|
  # Verbs head, get, put, post, patch, delete, and options are supported.
  test.get "/user" do |req|
    # req is a Hurley::Request
    # Return a Rack-compatible response.
    [200, {"Content-Type" => "application/json"}, %({"id": 1})]
  end
end

client.get("/user").body # => {"id": 1}

TODO

  • Backport Faraday adapters as gems
    • Excon
    • Typhoeus
  • Integrate into Faraday reliant gems:
  • Tomdoc all the things
  • Fix allll the bugs
  • Release v1.0

hurley's People

Contributors

ben-m avatar kkirsche avatar o-i avatar pengwynn avatar sferik avatar sqrrrl avatar steved avatar technoweenie avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

hurley's Issues

on_body listeners lost during redirects

Hurley::Response.location doesn't copy over the body_receiver to the new request. Bodies won't be received if the initial response contains a redirect.

Will send PR shortly.

Shutting this project down

This project is incomplete, and isn't seeing any new work. I don't have any time to work on it, and unfortunately need to shut it down :( I'm not going to yank any gem versions, but I won't be replying to issues or pull requests from here on out.

You're probably better off with Faraday if you can't decide which of ruby's many http clients you want to use.

simple query building unclear

query = { time: Time.now, fred: "hello")
In httpclient i used to be able to do httpclient.get(uri,query). This worked nicely.

In Hurley i tried the following:
c=Hurley::Cleint.new url="https://datadrop.wolframcloud.com/api/v1.0/Add?"
c.get(Hurley::Query::Flat.new(query))

this comes close to working but strips the ? from the url.

Eventually i had to do the following:

q="https://datadrop.wolframcloud.com/api/v1.0/Add?"+ Hurley::Query::Flat.new(qdata).to_s
res=Hurley.get(q)
this does not seem to in keeping with the style of Hurley... Any suggestions for improving the handling of my hash structured query or keeping the question mark in the url?

Best approach for testing adapter gems

Do you have any thoughts about the best way for Hurley adapter gems to run integration tests?

It's really nice that Hurley::Test::Integration has moved into the lib directory (which saves nastiness like this):

faraday_gem_location = Gem.loaded_specs['faraday'].gem_dir
integration_location = File.join(faraday_gem_location, 'test', 'adapters', 'integration')
require integration_location

However, what's the best way for the gems to spin up the server and trigger the tests? This probably works but is ugly:

adapter-gem/script/test:

#!/usr/bin/env bash
hurley_path=`bundle show hurley_path`
$hurley_path/script/test $@

adapter-gem/script/server:

#!/usr/bin/env bash
hurley_path_path=`bundle show hurley_path`
$hurley_path_path/script/server $@

0.3 release

There are a few pending PRs that fix streaming responses (#28, #34) - please cut a new gem. Since I'm using this as a dependency of another gem, I can't just pull from git. And as much as I don't want to monkey patch it, I'll likely have to if these aren't fixed soon :(

Hurley.default not being respected

The README says that one can:

# Change it globally.
Hurley.default = Hurley::Query::Flat

Although this is not implemented (or has been removed). In order to globally change it, I had to monkey patch:

module Hurley
  class Query
    def self.default
      @default ||= Flat
    end
  end
end

Url.parse does not handle repeated query parameters correctly

Found an issue where flat parameters are discarded when passing in a URL to client.get. Specifically, I'm calling client.get(url) with an absolute URL that has a repeated parameter. Even though the client instance is configured to use the flat encoder, I'm seeing only the last value.

Looks like client.request is calling URL.join() to merge the URL with the base URL (in this case the base is null.) This is where it breaks:

Example:
Hurley::Url.join(nil, 'https://www.foo.com?a=b&a=b2')
#<Hurley::Url https://www.foo.com?a=b2>

Notice only the last value of a is preserved. Expected result is that both pairs would be retained.

Issue Getting Basic Auth to Work

Hey guys,

So I'm trying to use Hurley and I have to work with one API endpoint which uses HTTP Basic Auth. It keeps giving me access denied via Hurley but if I copy / paste the URL I generate with the https://username:[email protected]/api/end/point.json

response = @client.get("https://#{username}:#{password}@rubygems.org/api/v1/api_key.#{format}")

puts response.status_code
puts response.body

and I'm getting:

401
HTTP Basic: Access denied.

Pasting the result of "https://#{username}:#{password}@rubygems.org/api/v1/api_key.#{format}" works though.

on_body filtering broken

Streaming response bodies in response to certain status codes is broken, particularly with get requests. This is due to the fact that the body is read in perform_request, before the response status and headers are available.

The status code is always null and callbacks that filter on a given status code are never called. Working on a PR that fixes this.

Issue getting from http://github.com on JRuby

Hey there! Ran the following steps to see whether hurley would allow me past a redirection limitation I was hitting with open-uri, but instead ran into this error:

[1] pry(main)> require 'hurley'
=> true
[2] pry(main)> Hurley.get("http://github.com")
TypeError: can't convert Net::ReadAdapter into String
from org/jruby/ext/stringio/StringIO.java:143:in `initialize'

Works fine if I run on MRI 2.1.5, but unfortunately my target app is JRuby-based.

JRuby version:

jruby 1.7.16.1 (1.9.3p392) 2014-10-28 4e93f31 on Java HotSpot(TM) 64-Bit Server VM 1.8.0_20-b26 [darwin-x86_64]

Mutex in hurley.rb

Why in hurley.rb this code needed?

MUTEX = Mutex.new
  def self.default_client
    @default_client ||= mutex { Client.new }
  end

It looks, that in hurley repository there is no code with threads, which share same data, which must be locked.

What about the middleware

I've got an extensive middleware stack used in my production Faraday code, including

  • HTTP caching
  • retry logic
  • instrumentation (ActiveSupport::Notifications)
  • circuit breaker logic
  • automatic Avro decoding
  • error mapping

Having a unified interface for this in Faraday was nice. Are you planning on offering the same kind of functionality in Hurley?

Hurley Typhoeus

Hey,

I was wondering what the interest would be in having a Hurley Typhoeus connection made and how best to submit that to Lost Island for review / use.

If you could share any thoughts or preferences on that, I'd love to work on that as I really enjoy working with Hurley over other clients

Crash when query string contains &&

Line 45 in query.rb causes a crash if pair is nil, which can happen in the (possibly invalid but seen in the wild) case where the query string contains &&.

def parse_query(raw_query)
    raw_query.to_s.split(AMP).each do |pair|
        escaped_key, escaped_value = pair.split(EQ, 2)
        key = CGI.unescape(escaped_key)
        value = escaped_value ? CGI.unescape(escaped_value) : nil
        send(:decode_pair, key, value)
    end
end

Could easily fix by simply wrapping the block in if pair != nil, but maybe there's a nicer ruby-like to do it (still pretty new to it)?

For future reference, would an immediate pull-request be preferred to raising an issue like this?

Test stubs aren't consumed by the connection.

I don't know if this is a desired behavior of the Test connection, but it is one that made a lot of sense when working with Faraday. When defining multiple stubs for the same URI, only the first one gets used by the connection and the subsequent responses aren't used, even when the client perform multiple requests

connection.get('/test') { [200, {}, 'foo'] }
connection.get('/test') { [200, {}, 'bar'] }

client.get('/test').body # => 'foo'
client.get('/test').body # => Expected to be 'bar', but it's 'foo'.

Is this the expected behavior of some oversight when porting the API from Faraday?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.