typhoeus / ethon Goto Github PK

Very simple libcurl wrapper.

License: MIT License

Ruby 100.00%

ethon's Issues

Option to enforce multipart post

setopt for off_t potentially broken on 32-bit systems

The off_t options (:max_send_speed_large and :max_recv_speed_large) consistently get set as :int (using value_for), this then gets passed on to set_option, which calls (I assume) easy_setopt_fixnum (which takes a long). I don't have a 32-bit system to test this on, but I think this is wrong, because an off_t is larger than a long on those systems.

Also, if you try to specify an off_t of 1<<62 or bigger (still a valid long!), set_option will try to call easy_setopt_bignum...

Different params encoding wit libcurl/7.19.7 & libcurl/7.24.0

I got a perculiar issue while using Ethon (via Typhoeus) and I was wondering if you can help me out on this one

First let me set the stage:

Using typhoeus (0.6.5)
Using vcr (2.5.0)
libcurl/7.24.0 (on dev machine)
libcurl/7.19.7 (on CI server)

We run rspecs that make extensive use of https://github.com/vcr/vcr to record various Typhoeus HTTP requests and replay them.

We record the cassettes on our local developer machines and replay them on our CI server.

Now we have a HTTP request that look like this: http://example.com/path?locale=en&auth_token=123

What happens when we record a cassette is the request/response gets saved under that URI.

However, when we run the tests on the CI servers VCR tries to find them with the following URI: http://example.com/path?locale=en&auth%5Ftoken=123

So there is something fishy going on. I think I've confined the problem to the different libcurl versions. Here's an example with Ethon.easy:

dev machine

>> require 'typhoeus'
false
>> easy = Ethon::Easy.new
#<Ethon::Easy:0x007fa09bb21770 @body_write_callback=#<Proc:0x007fa09bb21680@/Users/sascha/.rvm/gems/ruby-1.9.3-p392-turbo/gems/ethon-0.6.1/lib/ethon/easy/callbacks.rb:37>, @handle=#<FFI::AutoPointer address=0x007fa09e038600>, @header_write_callback=#<Proc:0x007fa09bb211a8@/Users/sascha/.rvm/gems/ruby-1.9.3-p392-turbo/gems/ethon-0.6.1/lib/ethon/easy/callbacks.rb:50>, @debug_callback=#<Proc:0x007fa09bb20ed8@/Users/sascha/.rvm/gems/ruby-1.9.3-p392-turbo/gems/ethon-0.6.1/lib/ethon/easy/callbacks.rb:64>, @response_body="", @response_headers="", @debug_info=#<Ethon::Easy::DebugInfo:0x007fa09bb20b40 @messages=[]>>
>> easy.http_request("www.example.com/path", :get, { params: {locale: 'en', auth_token: 123}, verbose: true })
nil
>> easy.perform
About to connect() to www.example.com port 80 (#0)
  Trying 93.184.216.119...
connected
Connected to www.example.com (93.184.216.119) port 80 (#0)
GET /path?locale=en&auth_token=123 HTTP/1.1
Host: www.example.com
Accept: */*

HTTP/1.1 404 Not Found
Accept-Ranges: bytes
Cache-Control: max-age=604800
Content-Type: text/html
Date: Thu, 12 Sep 2013 08:26:33 GMT
Etag: "3012602696"
Expires: Thu, 19 Sep 2013 08:26:33 GMT
Last-Modified: Fri, 09 Aug 2013 23:54:35 GMT
Server: ECS (lax/286D)
X-Cache: HIT
x-ec-custom-error: 1
Content-Length: 1270

Connection #0 to host www.example.com left intact
:ok

CI server

>> require 'typhoeus'
 => true 
>> easy = Ethon::Easy.new
 => #<Ethon::Easy:0x0000000259ede8 @body_write_callback=#<Proc:0x0000000259e690@/usr/local/rvm/gems/ruby-1.9.3-p374@location-app-master/gems/ethon-0.6.1/lib/ethon/easy/callbacks.rb:37>, @handle=#<FFI::AutoPointer address=0x00000002ed42c0>, @header_write_callback=#<Proc:0x000000025a7060@/usr/local/rvm/gems/ruby-1.9.3-p374@location-app-master/gems/ethon-0.6.1/lib/ethon/easy/callbacks.rb:50>, @debug_callback=#<Proc:0x000000025a6778@/usr/local/rvm/gems/ruby-1.9.3-p374@location-app-master/gems/ethon-0.6.1/lib/ethon/easy/callbacks.rb:64>, @response_body="", @response_headers="", @debug_info=#<Ethon::Easy::DebugInfo:0x000000025a5c38 @messages=[]>> 
>> easy.http_request("www.example.com/path", :get, { params: {locale: 'en', auth_token: 123}, verbose: true })
 => nil 
>> easy.perform
About to connect() to www.example.com port 80 (#0)
  Trying 93.184.216.119... connected
Connected to www.example.com (93.184.216.119) port 80 (#0)
GET /path?locale=en&auth%5Ftoken=123 HTTP/1.1
Host: www.example.com
Accept: */*

HTTP/1.1 404 Not Found
Accept-Ranges: bytes
Cache-Control: max-age=604800
Content-Type: text/html
Date: Thu, 12 Sep 2013 08:28:41 GMT
Etag: "3012602696"
Expires: Thu, 19 Sep 2013 08:28:41 GMT
Last-Modified: Fri, 09 Aug 2013 23:54:35 GMT
Server: ECS (lax/286D)
X-Cache: HIT
x-ec-custom-error: 1
Content-Length: 1270

Connection #0 to host www.example.com left intact
 => :ok

You see the difference clearly here:

GET /path?locale=en&auth_token=123 HTTP/1.1
vs:
GET /path?locale=en&auth%5Ftoken=123 HTTP/1.1

I'm sure I'm doing something wrong. Can anyone help me out on this one? Thanks!

accept options for multi

http://curl.haxx.se/libcurl/c/curl_multi_setopt.html

Ethon::Easy should either fix url or throw error if is not in desired format

Example:
easy = Ethon::Easy.new
easy.http_request(
'www.example.com/my_path?query_param=1',
:get,
{ :params => {:option_param => 2}}
)

easy.url == "www.example.com/my_path?query_param=1?option_param=2"

Consider using Addressable::URI instead URI

https://github.com/sporkmonger/addressable

Provide access to raw request data

It would be nice if we could access the raw HTTP request data for later inspection/auditing/debugging.

0.5.12: undefined method `each' for nil:NilClass - response_callbacks.rb:43

A NoMethodError occurred in search#index:

undefined method `each' for nil:NilClass
ethon (0.5.12) lib/ethon/easy/response_callbacks.rb:43:in `complete'

-------------------------------
Request:
-------------------------------

  * URL       : https://level.travel/search/Moscow-RU-to-Any-EG-departure-11.05.2013-for-13-nights-2-adults-0-kids-1..5-stars
  * IP address: 82.97.205.46
  * Parameters: {"controller"=>"search", "action"=>"index", "query"=>"Moscow-RU-to-Any-EG-departure-11.05.2013-for-13-nights-2-adults-0-kids-1..5-stars"}
  * Rails root: /home/deployer/apps/leveltravel/current
  * Timestamp : 2013-04-23 16:24:41 +0400

-------------------------------
Backtrace:
-------------------------------

  ethon (0.5.12) lib/ethon/easy/response_callbacks.rb:43:in `complete'
  ethon (0.5.12) lib/ethon/easy/operations.rb:24:in `perform'
  typhoeus (0.5.4) lib/typhoeus/request/operations.rb:26:in `run'
  typhoeus (0.5.4) lib/typhoeus/request/block_connection.rb:31:in `run'
  typhoeus (0.5.4) lib/typhoeus/request/stubbable.rb:23:in `run'
  typhoeus (0.5.4) lib/typhoeus/request/before.rb:26:in `run'
  typhoeus (0.5.4) lib/typhoeus/request/actions.rb:22:in `get'
  app/apis/weather_bug.rb:27:in `live_request'
  app/apis/weather_bug.rb:19:in `live'
  app/controllers/search_controller.rb:45:in `index'
  actionpack (3.2.11) lib/action_controller/metal/implicit_render.rb:4:in `send_action'
  actionpack (3.2.11) lib/abstract_controller/base.rb:167:in `process_action'
  actionpack (3.2.11) lib/action_controller/metal/rendering.rb:10:in `process_action'
  actionpack (3.2.11) lib/abstract_controller/callbacks.rb:18:in `block in process_action'
  activesupport (3.2.11) lib/active_support/callbacks.rb:513:in `_run__906001855511687055__process_action__4226426739626338382__callbacks'
  activesupport (3.2.11) lib/active_support/callbacks.rb:405:in `__run_callback'
  activesupport (3.2.11) lib/active_support/callbacks.rb:385:in `_run_process_action_callbacks'
  activesupport (3.2.11) lib/active_support/callbacks.rb:81:in `run_callbacks'
  actionpack (3.2.11) lib/abstract_controller/callbacks.rb:17:in `process_action'
  actionpack (3.2.11) lib/action_controller/metal/rescue.rb:29:in `process_action'
  actionpack (3.2.11) lib/action_controller/metal/instrumentation.rb:30:in `block in process_action'
  activesupport (3.2.11) lib/active_support/notifications.rb:123:in `block in instrument'
  activesupport (3.2.11) lib/active_support/notifications/instrumenter.rb:20:in `instrument'
  activesupport (3.2.11) lib/active_support/notifications.rb:123:in `instrument'
  actionpack (3.2.11) lib/action_controller/metal/instrumentation.rb:29:in `process_action'
  actionpack (3.2.11) lib/action_controller/metal/params_wrapper.rb:207:in `process_action'
  activerecord (3.2.11) lib/active_record/railties/controller_runtime.rb:18:in `process_action'
  actionpack (3.2.11) lib/abstract_controller/base.rb:121:in `process'
  actionpack (3.2.11) lib/abstract_controller/rendering.rb:45:in `process'
  actionpack (3.2.11) lib/action_controller/metal.rb:203:in `dispatch'
  actionpack (3.2.11) lib/action_controller/metal/rack_delegation.rb:14:in `dispatch'
  actionpack (3.2.11) lib/action_controller/metal.rb:246:in `block in action'
  actionpack (3.2.11) lib/action_dispatch/routing/route_set.rb:73:in `call'
  actionpack (3.2.11) lib/action_dispatch/routing/route_set.rb:73:in `dispatch'
  actionpack (3.2.11) lib/action_dispatch/routing/route_set.rb:36:in `call'
  journey (1.0.4) lib/journey/router.rb:68:in `block in call'
  journey (1.0.4) lib/journey/router.rb:56:in `each'
  journey (1.0.4) lib/journey/router.rb:56:in `call'
  actionpack (3.2.11) lib/action_dispatch/routing/route_set.rb:601:in `call'
  exception_notification (3.0.1) lib/exception_notifier.rb:41:in `call'
  warden (1.2.1) lib/warden/manager.rb:35:in `block in call'
  warden (1.2.1) lib/warden/manager.rb:34:in `catch'
  warden (1.2.1) lib/warden/manager.rb:34:in `call'
  actionpack (3.2.11) lib/action_dispatch/middleware/best_standards_support.rb:17:in `call'
  rack (1.4.5) lib/rack/etag.rb:23:in `call'
  rack (1.4.5) lib/rack/conditionalget.rb:25:in `call'
  actionpack (3.2.11) lib/action_dispatch/middleware/head.rb:14:in `call'
  actionpack (3.2.11) lib/action_dispatch/middleware/params_parser.rb:21:in `call'
  actionpack (3.2.11) lib/action_dispatch/middleware/flash.rb:242:in `call'
  rack (1.4.5) lib/rack/session/abstract/id.rb:210:in `context'
  rack (1.4.5) lib/rack/session/abstract/id.rb:205:in `call'
  actionpack (3.2.11) lib/action_dispatch/middleware/cookies.rb:341:in `call'
  activerecord (3.2.11) lib/active_record/query_cache.rb:64:in `call'
  activerecord (3.2.11) lib/active_record/connection_adapters/abstract/connection_pool.rb:479:in `call'
  actionpack (3.2.11) lib/action_dispatch/middleware/callbacks.rb:28:in `block in call'
  activesupport (3.2.11) lib/active_support/callbacks.rb:405:in `_run__4493510109563642895__call__1195701628514063054__callbacks'
  activesupport (3.2.11) lib/active_support/callbacks.rb:405:in `__run_callback'
  activesupport (3.2.11) lib/active_support/callbacks.rb:385:in `_run_call_callbacks'
  activesupport (3.2.11) lib/active_support/callbacks.rb:81:in `run_callbacks'
  actionpack (3.2.11) lib/action_dispatch/middleware/callbacks.rb:27:in `call'
  rack (1.4.5) lib/rack/sendfile.rb:102:in `call'
  actionpack (3.2.11) lib/action_dispatch/middleware/remote_ip.rb:31:in `call'
  actionpack (3.2.11) lib/action_dispatch/middleware/debug_exceptions.rb:16:in `call'
  actionpack (3.2.11) lib/action_dispatch/middleware/show_exceptions.rb:56:in `call'
  railties (3.2.11) lib/rails/rack/logger.rb:32:in `call_app'
  railties (3.2.11) lib/rails/rack/logger.rb:18:in `call'
  actionpack (3.2.11) lib/action_dispatch/middleware/request_id.rb:22:in `call'
  rack (1.4.5) lib/rack/methodoverride.rb:21:in `call'
  rack (1.4.5) lib/rack/runtime.rb:17:in `call'
  rack (1.4.5) lib/rack/lock.rb:15:in `call'
  rack-cache (1.2) lib/rack/cache/context.rb:136:in `forward'
  rack-cache (1.2) lib/rack/cache/context.rb:245:in `fetch'
  rack-cache (1.2) lib/rack/cache/context.rb:185:in `lookup'
  rack-cache (1.2) lib/rack/cache/context.rb:66:in `call!'
  rack-cache (1.2) lib/rack/cache/context.rb:51:in `call'
  railties (3.2.11) lib/rails/engine.rb:479:in `call'
  railties (3.2.11) lib/rails/application.rb:223:in `call'
  railties (3.2.11) lib/rails/railtie/configurable.rb:30:in `method_missing'
  unicorn (4.6.2) lib/unicorn/http_server.rb:552:in `process_client'
  unicorn (4.6.2) lib/unicorn/http_server.rb:632:in `worker_loop'
  unicorn (4.6.2) lib/unicorn/http_server.rb:500:in `spawn_missing_workers'
  unicorn (4.6.2) lib/unicorn/http_server.rb:142:in `start'
  unicorn (4.6.2) bin/unicorn:126:in `'
  /home/deployer/apps/leveltravel/shared/bundle/ruby/1.9.1/bin/unicorn:23:in `load'
  /home/deployer/apps/leveltravel/shared/bundle/ruby/1.9.1/bin/unicorn:23:in `'

Ethon::Errors::Select: An error occured on select: 4

I'm encountering this error while using Typhoeus.

I've logged it as a typhoeus issue: typhoeus/typhoeus#252

However, it might be better classed as a ethon issue?

Problem uploading a file

Hello,

I've been using ethon in order to upload an XML file to Amazon's MWS, and I haven't been able to achieve a success yet.

When I use Ethon::Easy's readdata option, I can't get it to work, neither with a string containing the file content, or with a file.

So, is it possible to successfully set the value for CURLOPT_INFILE/CURLOPT_READDATA with this gem?

Thanks!

Timeout ignored

I set both timeout and connectiontimeout
Typhoeus::Request.new line, timeout: 30, followlocation: true, connecttimeout: 10
But at this place
https://github.com/typhoeus/ethon/blob/master/lib/ethon/multi/operations.rb#L126
with long-loading url the script waits infinite amount of time, ignoring both timeout values.

Bump FFI gem version to latest version in gemspec

Current ffi gem version is 1.3.1 and ethon requires v. ~> 1.2.0 This is causing issues in my project when other gems depend on the latest version of ffi

add delete.

Socks support

There seems to be an issues with socks support:

typhoeus/typhoeus#81
wpscanteam/wpscan#7

/cc @antel @ethicalhack3r @erwanlr

Core dump

I'm getting a core dump with a really simple request using ruby 1.9.3p194 (2012-04-20 revision 35410) [x86_64-linux]. I'm sure this is not a typhoeus (edit: sorry, I meant Ethon) problem, but you may have some ideas? It started as soon as I added the httppost.

$: << './lib'
require 'ethon'

e = Ethon::Easy.new
e.http_request(
  'http://<bucket-name>.s3.amazonaws.com',
  :post,
  { 
    :body => 'hi there',
    :httppost => 'multipart/form-data',
    :params => {

    } 
  } 
)
e.perform
puts e.response_body

check multi socket implementation

http://curl.haxx.se/libcurl/c/curl_multi_socket_action.html

Find a better gem name.

url encoding for arrays

Hi,

recently I had some problems with your encoding of arrays in the query params.
While adding an index in the brackets is a nice idea to make sure the array will be sorted correctly. But most it seems to break with the W3C suggestion (http://www.w3.org/TR/url/#collect-the-url-parameters) and implementation in some frameworks.

An option for setting the array encoding would be nice. So you could enable or disable this feature.

Best regards
Steffen

P.S.: Thanks for building Typhoeus.

Memory leak?

Hey there.

I noticed some of my Rails processes were constantly growing, so I set out to try to figure out why. Long story short, I got bleak_house (https://github.com/evan/bleak_house) running my app and profiled several code paths. The only path which seemed to grow objects constantly was that using typhoeus/ethon.

Here is a bleak snippet of the object_count/line/object_type after 4000 requests:

4002 ~/.rvm/gems/ruby-1.8.7-p371/gems/ethon-0.5.7/lib/ethon/easy/http/actionable.rb:62:Hash
4002 ~/.rvm/gems/ruby-1.8.7-p371/gems/ethon-0.5.7/lib/ethon/easy/http/actionable.rb:62:Ethon::Easy::Form
4002 ~/.rvm/gems/ruby-1.8.7-p371/gems/ethon-0.5.7/lib/ethon/easy/form.rb:37:Proc
4002 ~/.rvm/gems/ruby-1.8.7-p371/gems/ethon-0.5.7/lib/ethon/easy/form.rb:27:__unknown__
4002 ~/.rvm/gems/ruby-1.8.7-p371/gems/ethon-0.5.7/lib/ethon/easy/form.rb:27:__scope__
4002 ~/.rvm/gems/ruby-1.8.7-p371/gems/ethon-0.5.7/lib/ethon/easy/form.rb:27:Array

I believe I successfully forced GC by doing about 10MB of string concatenation and running GC.start, and watched the total object count drop.

For reference the code path using typhoeus/ethon looks like:

response = Typhoeus::Request.get(url, :timeout_ms => 5000)
raise "Timeout!" if response.timed_out?
data = JSON.parse(response.body)

Any ideas?

#copypostfields= should not escape NUL bytes

Currently, #set_form copies the body using #copypostfields= if body is given as a string.

However, for example from Faraday which builds all the body string including multipart requests, the body can contain NUL bytes and it is escaped as "\0". The problem goes even worse because we call #postfieldsize= with the original string.

~~We could fix https://github.com/typhoeus/ethon/blob/master/lib/ethon/easy/options.rb#L174 like this:~~

def copypostfields=(value)
  Curl.set_option(:copypostfields, value, handle) # no escaping here :)
end

Thanks!

Make put work (like post)

Code documentation

Deal with file uploads.

multipart post is slow.

Every request method should have a body

SystemStackError

@vegetas4 encountered this ethon error when running Wpscan (https://github.com/wpscanteam/wpscan/)

Opening this bug for him to see if it is an error in ethon.

Wpscan Bug:
wpscanteam/wpscan#257

Quote from our bug:

When I go to perform a brute force attack this error occurs
/home/user/.rvm/gems/ruby-2.0.0-p247/gems/ethon-0.5.12/lib/ethon/easy/util.rb:20: stack level too deep (SystemStackError)

Ethon's using curl_easy_escape but not free'ing the string

This is another cause of the ethon memory leaks. In queryable:

  def to_s
    query_pairs.map{ |pair|
      return pair if pair.is_a?(String)

      pair.map{ |e|
        escape && @easy ? @easy.escape(e.to_s) : e
      }.join("=")
    }.join('&')
  end

Easy escape:

def escape(value)
  Curl.easy_escape(handle, value, 0)
end

That returns a string, but the underlying string isn't ever escape. From http://curl.haxx.se/libcurl/c/curl_easy_escape.html

This function converts the given input string to an URL encoded string and returns that as a new allocated string.....
...
You must curl_free(3) the returned string when you're done with it.

I'm not 100% sure what the value of using curl's escape function is versus using ruby's CGI. But to stay with the curl one, I believe the FFI signature will need to be changed to return a pointer, which gets manually converted to a ruby string, and then have the original string freed (or something along those lines).

Segmentation fault with rspec

I have the following output at the end of rspec with https://github.com/wpscanteam/wpscan :

/home/erwan/.rvm/gems/ruby-1.9.3-p362/gems/ethon-0.5.10/lib/ethon/multi.rb:26: [BUG] Segmentation fault
ruby 1.9.3p362 (2012-12-25 revision 38607) [x86_64-linux]

-- Control frame information -----------------------------------------------
c:0005 p:---- s:0012 b:0012 l:000011 d:000011 CFUNC  :multi_cleanup
c:0004 p:0024 s:0008 b:0008 l:0001c0 d:000007 BLOCK  /home/erwan/.rvm/gems/ruby-1.9.3-p362/gems/ethon-0.5.10/lib/ethon/multi.rb:26
c:0003 p:---- s:0006 b:0006 l:000005 d:000005 FINISH
c:0002 p:---- s:0004 b:0004 l:000003 d:000003 CFUNC  :call
c:0001 p:0000 s:0002 b:0002 l:001698 d:001698 TOP   

-- Ruby level backtrace information ----------------------------------------
/home/erwan/.rvm/gems/ruby-1.9.3-p362/bin/rspec:0:in `call'
/home/erwan/.rvm/gems/ruby-1.9.3-p362/gems/ethon-0.5.10/lib/ethon/multi.rb:26:in `block in finalizer'
/home/erwan/.rvm/gems/ruby-1.9.3-p362/gems/ethon-0.5.10/lib/ethon/multi.rb:26:in `multi_cleanup'

-- C level backtrace information -------------------------------------------
Segmentation fault

$ curl --version
curl 7.29.0 (x86_64-unknown-linux-gnu) libcurl/7.29.0 OpenSSL/0.9.8o zlib/1.2.3.4 libidn/1.15 libssh2/1.2.6
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smtp smtps telnet tftp 
Features: IDN IPv6 Largefile NTLM NTLM_WB SSL libz

Frozen after "ETHON: Libcurl initialized" from Typheous

My server freezes right after that line is output. I'm doing a simple request of the form

Typhoeus.get(request_url,
    headers: headers.merge({'Content-Type' => 'application/json'})
  )

This happens every time. I'm using Ethon 0.5.3 and curl "7.30.0 (x86_64-apple-darwin13.0) libcurl/7.30.0 SecureTransport zlib/1.2.5".

Commenting out those lines solves my freezing issue. I even tried putting a timeout around those lines of code, but that did nothing to help.

Any help would be greatly appreciated. I know that 7.31 is out now, so I'm going to try updating to that.

Linux FDSet too big

FD_SETSIZE = ::Ethon::Libc.getdtablesize
layout :fds_bits, [:long, FD_SETSIZE / ::FFI::Type::LONG.size]

getdtablesize returns the maximum number of file descriptors. An fd_set has a bit for each of these. FFI::Type::LONG.size returns the number of bytes (not bits) for a long, so the resulting array is 8 times too big.

On a related note: according to man getdtablesize, it just does Process.getrlimit(Process::RLIMIT_NOFILE)[0] (so no need to import getdtablesize). Since rlimits can change, this function is not guaranteed to give you a struct definition that is big enough. You might be able to use the hard limit from getrlimit, but even that is no guarantee. See also http://etutorials.org/Programming/secure+programming/Chapter+3.+Input+Validation/3.13+Preventing+File+Descriptor+Overflows+When+Using+select/

Ethon::Easy::Utils#escape_zero_byte corrupts strings

Ethon::Easy::Utils#escape_zero_byte just corrupts the payloads, why not %-encode them instead?

Bump FFI version in gemspec

ffi is at version 1.1.5 but the current ethon depends on ffi ~> 1.0.11. This is causing issues for me when updating typhoeus to 0.5.1.

Can you bump the ffi dependency version?

Support HTTP PURGE verb (and maybe other custom ones)

I'm not sure if this belongs in Typhoeus or Ethon, but Typhoeus used to support HTTP PURGE which is used by HTTP caches like Varnish to signal that entries should be removed.

The error I get now is: uninitialized constant Ethon::Easy::Http::Purge.

If I add this, the request works (it just copies Ethon::Easy::Http::Get):

module Ethon
  class Easy
    module Http

      # This class knows everything about making GET requests.
      class Purge
        include Ethon::Easy::Http::Actionable
        include Ethon::Easy::Http::Postable

        # Setup easy to make a GET request.
        #
        # @example Setup.
        #   get.set_params(easy)
        #
        # @param [ Easy ] easy The easy to setup.
        def setup(easy)
          super
          easy.customrequest = "GET" unless form.empty?
        end
      end
    end
  end
end

But maybe there's a way to allow for custom HTTP verbs?

Figure out when to escape params/form and when not

Fix 1.8 builds.

Can't start applications

Ethon 0.4.4 prevents application from starting up. 0.4.2 works fine.

Error message:

/Users/myusr/.rvm/gems/ruby-1.9.2-p290/gems/typhoeus-    0.5.0.alpha/lib/typhoeus/responses/informations.rb:9:in `<module:Informations>': uninitialized     constant Ethon::Easies (NameError)

Full trace: https://gist.github.com/3722049

include spec directory in gem itself

please include the spec file in the gem itself like most gems do. Currently we have to get it from github tag. It would be better to get it directly from the gem itself.

FFI Problem

Hi Guys,

I get the following error when I add 'typhoeus' to my Gemfile.

/Users/govinda/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/ffi-1.9.2/lib/ffi.rb:14:in `require': dlopen(/Users/govinda/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/ffi-1.9.2/lib/ffi_c.bundle, 9): Library not loaded: /Users/headius/.rvm/rubies/ruby-head/lib/libruby.2.1.0.dylib (LoadError)
  Referenced from: /Users/govinda/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/ffi-1.9.2/lib/ffi_c.bundle
  Reason: image not found - /Users/govinda/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/ffi-1.9.2/lib/ffi_c.bundle
    from /Users/govinda/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/ffi-1.9.2/lib/ffi.rb:14:in `rescue in <top (required)>'
    from /Users/govinda/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/ffi-1.9.2/lib/ffi.rb:3:in `<top (required)>'
    from /Users/govinda/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/ethon-0.6.1/lib/ethon.rb:2:in `require'
    from /Users/govinda/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/ethon-0.6.1/lib/ethon.rb:2:in `<top (required)>'
    from /Users/govinda/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/typhoeus-0.6.5/lib/typhoeus.rb:2:in `require'
    from /Users/govinda/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/typhoeus-0.6.5/lib/typhoeus.rb:2:in `<top (required)>'
    from /Users/govinda/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/bundler-1.3.5/lib/bundler/runtime.rb:72:in `require'
    from /Users/govinda/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/bundler-1.3.5/lib/bundler/runtime.rb:72:in `block (2 levels) in require'
    from /Users/govinda/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/bundler-1.3.5/lib/bundler/runtime.rb:70:in `each'
    from /Users/govinda/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/bundler-1.3.5/lib/bundler/runtime.rb:70:in `block in require'
    from /Users/govinda/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/bundler-1.3.5/lib/bundler/runtime.rb:59:in `each'
    from /Users/govinda/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/bundler-1.3.5/lib/bundler/runtime.rb:59:in `require'
    from /Users/govinda/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/bundler-1.3.5/lib/bundler.rb:132:in `require'
    from /Users/govinda/workspace/test/mytest/config/application.rb:12:in `<top (required)>'
    from /Users/govinda/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/railties-4.0.0/lib/rails/commands.rb:76:in `require'
    from /Users/govinda/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/railties-4.0.0/lib/rails/commands.rb:76:in `block in <top (required)>'
    from /Users/govinda/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/railties-4.0.0/lib/rails/commands.rb:73:in `tap'
    from /Users/govinda/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/railties-4.0.0/lib/rails/commands.rb:73:in `<top (required)>'
    from bin/rails:4:in `require'
    from bin/rails:4:in `<main>'

Without the gem everything works just fine. Is my setup missing anything vital?

I have the following environment with Ruby 2.0.0p247, Rubygems 2.0.3 ...

My Gemfile.lock looks like this:

GEM
  remote: https://rubygems.org/
  specs:
    actionmailer (4.0.0)
      actionpack (= 4.0.0)
      mail (~> 2.5.3)
    actionpack (4.0.0)
      activesupport (= 4.0.0)
      builder (~> 3.1.0)
      erubis (~> 2.7.0)
      rack (~> 1.5.2)
      rack-test (~> 0.6.2)
    activemodel (4.0.0)
      activesupport (= 4.0.0)
      builder (~> 3.1.0)
    activerecord (4.0.0)
      activemodel (= 4.0.0)
      activerecord-deprecated_finders (~> 1.0.2)
      activesupport (= 4.0.0)
      arel (~> 4.0.0)
    activerecord-deprecated_finders (1.0.3)
    activesupport (4.0.0)
      i18n (~> 0.6, >= 0.6.4)
      minitest (~> 4.2)
      multi_json (~> 1.3)
      thread_safe (~> 0.1)
      tzinfo (~> 0.3.37)
    arel (4.0.1)
    atomic (1.1.14)
    builder (3.1.4)
    coffee-rails (4.0.1)
      coffee-script (>= 2.2.0)
      railties (>= 4.0.0, < 5.0)
    coffee-script (2.2.0)
      coffee-script-source
      execjs
    coffee-script-source (1.6.3)
    erubis (2.7.0)
    ethon (0.6.1)
      ffi (>= 1.3.0)
      mime-types (~> 1.18)
    execjs (2.0.2)
    ffi (1.9.2)
    hike (1.2.3)
    i18n (0.6.5)
    jbuilder (1.5.2)
      activesupport (>= 3.0.0)
      multi_json (>= 1.2.0)
    jquery-rails (3.0.4)
      railties (>= 3.0, < 5.0)
      thor (>= 0.14, < 2.0)
    json (1.8.1)
    mail (2.5.4)
      mime-types (~> 1.16)
      treetop (~> 1.4.8)
    mime-types (1.25)
    minitest (4.7.5)
    multi_json (1.8.2)
    mysql2 (0.3.13)
    polyglot (0.3.3)
    rack (1.5.2)
    rack-test (0.6.2)
      rack (>= 1.0)
    rails (4.0.0)
      actionmailer (= 4.0.0)
      actionpack (= 4.0.0)
      activerecord (= 4.0.0)
      activesupport (= 4.0.0)
      bundler (>= 1.3.0, < 2.0)
      railties (= 4.0.0)
      sprockets-rails (~> 2.0.0)
    railties (4.0.0)
      actionpack (= 4.0.0)
      activesupport (= 4.0.0)
      rake (>= 0.8.7)
      thor (>= 0.18.1, < 2.0)
    rake (10.1.0)
    rdoc (3.12.2)
      json (~> 1.4)
    sass (3.2.12)
    sass-rails (4.0.1)
      railties (>= 4.0.0, < 5.0)
      sass (>= 3.1.10)
      sprockets-rails (~> 2.0.0)
    sdoc (0.3.20)
      json (>= 1.1.3)
      rdoc (~> 3.10)
    sprockets (2.10.0)
      hike (~> 1.2)
      multi_json (~> 1.0)
      rack (~> 1.0)
      tilt (~> 1.1, != 1.3.0)
    sprockets-rails (2.0.1)
      actionpack (>= 3.0)
      activesupport (>= 3.0)
      sprockets (~> 2.8)
    thor (0.18.1)
    thread_safe (0.1.3)
      atomic
    tilt (1.4.1)
    treetop (1.4.15)
      polyglot
      polyglot (>= 0.3.1)
    turbolinks (1.3.0)
      coffee-rails
    typhoeus (0.6.5)
      ethon (~> 0.6.1)
    tzinfo (0.3.38)
    uglifier (2.3.0)
      execjs (>= 0.3.0)
      json (>= 1.8.0)

PLATFORMS
  ruby

DEPENDENCIES
  coffee-rails (~> 4.0.0)
  jbuilder (~> 1.2)
  jquery-rails
  mysql2
  rails (= 4.0.0)
  sass-rails (~> 4.0.0)
  sdoc
  turbolinks
  typhoeus
  uglifier (>= 1.3.0)

Difference with `curb` gem

curb has been there for a while now. What's the difference between both libs ?

implicit Easy#prepare

I want to suggest making Easy#prepare implicit, make it a private method and allow it to run right before the perform. If several runs required, it's possible to check if it was prepared already.

Don't let unsopported options slip through.

See https://github.com/typhoeus/ethon/blob/master/lib/ethon/easy.rb#L136.

start server before spec run.

getdtablesize not found Windows 7 x64

Summary

Just requiring typhoeus on windows causes it to break. The issue is due to: 477bee8

Perhaps only defining the constant for non-windows makes sense?

Details

Windows 7 x64

ruby 1.9.3p125 (2012-02-16) [i386-mingw32]

curl 7.21.1 (i686-pc-mingw32) libcurl/7.21.1 OpenSSL/0.9.8r zlib/1.2.3
Protocols: dict file ftp ftps http https imap imaps ldap ldaps pop3 pop3s rtsp smtp smtps telnet tftp
Features: Largefile NTLM SSL SSPI libz

Error

$ irb --noreadline
irb(main):001:0> require 'typhoeus'
FFI::NotFoundError: Function 'getdtablesize' not found in [msvcrt.dll]
        from c:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/ffi-1.0.11/lib/ffi/library.rb:249:in `attach_function'
        from c:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/ethon-0.5.3/lib/ethon/libc.rb:5:in `<module:Libc>'
        from c:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/ethon-0.5.3/lib/ethon/libc.rb:2:in `<module:Ethon>'
        from c:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/ethon-0.5.3/lib/ethon/libc.rb:1:in `<top (required)>'
        from c:/RailsInstaller/Ruby1.9.3/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require'
        from c:/RailsInstaller/Ruby1.9.3/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require'
        from c:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/ethon-0.5.3/lib/ethon.rb:7:in `<top (required)>'
        from c:/RailsInstaller/Ruby1.9.3/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require'
        from c:/RailsInstaller/Ruby1.9.3/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require'
        from c:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/typhoeus-0.5.1/lib/typhoeus.rb:2:in `<top (required)>'
        from c:/RailsInstaller/Ruby1.9.3/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:59:in `require'
        from c:/RailsInstaller/Ruby1.9.3/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:59:in `rescue in require'
        from c:/RailsInstaller/Ruby1.9.3/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:35:in `require'
        from (irb):1
        from c:/RailsInstaller/Ruby1.9.3/bin/irb:12:in `<main>'

Ethon's form freeing uses a method that doesn't exist

I'm currently working on fixing the memory leaks in Ethon, and I stumbled across what seems to be a bug in form.

If look look at the form finalizer: https://github.com/richievos/ethon/blob/master/lib/ethon/easy/form.rb#L37

it is using Curl.formfree:

proc { Curl.formfree(form.first) if form.multipart? }

But that method isn't defined on the Curl object. I'm pretty sure functions.rb needs to have:

base.attach_function :formfree,               :curl_formfree,            [:pointer],                     :void

rspec Ethon::Easy::Options when requesting when connecttimeout when cannot connect sometimes fails

Instead of :couldnt_connect you sometimes get :operation_timedout
https://travis-ci.org/typhoeus/ethon/builds/7900086
https://travis-ci.org/typhoeus/ethon/builds/7914090

Options :proxyauth and :proxytype use wrong setopt interface

They are strings, but should be longs. This will probably get fixed with #57, but reporting separately just in case.

Add Rubinius support

I have a proposal. I know it's a big goal but I think we as a community reached a point when we should now take other rubies as first class citizens in our ecosystem and start supporting them globally for us soon to live in a world where we could just switch rubies beneath and our applications will still work without a single change in our code.

I'm getting some FFI related errors trying to run Ethon's tests under current rbx-head. I'm not particularly good at this FFI and C extension stuff, I know that Rubinius has it's own implementation of FFI, but I'm not sure how it plays with ffi gem, the API is little bit different. I would really love to see if we could come closer to supporting Rubinius in Ethon (therefore in Typhoeus too), maybe just track down those bugs and inconsistencies and report them to more appropriate places (I can't tell if it is an Ethon's issue, FFI's or even on Rubinius side really). I think it would be nice if it is currently even possible.

Thanks for any update on this. I'm really curious.

Add patch and options.

NoMethodError: undefined method `easy_setopt_array' for Ethon::Curl:Module

Arose in the context of a Typhoeus issue

License missing from gemspec

RubyGems.org doesn't report a license for your gem. This is because it is not specified in the gemspec of your last release.

via e.g.

spec.license = 'MIT'
# or
spec.licenses = ['MIT', 'GPL-2']

Including a license in your gemspec is an easy way for rubygems.org and other tools to check how your gem is licensed. As you can imagine, scanning your repository for a LICENSE file or parsing the README, and then attempting to identify the license or licenses is much more difficult and more error prone. So, even for projects that already specify a license, including a license in your gemspec is a good practice. See, for example, how rubygems.org uses the gemspec to display the rails gem license.

There is even a License Finder gem to help companies/individuals ensure all gems they use meet their licensing needs. This tool depends on license information being available in the gemspec. This is an important enough issue that even Bundler now generates gems with a default 'MIT' license.

I hope you'll consider specifying a license in your gemspec. If not, please just close the issue with a nice message. In either case, I'll follow up. Thanks for your time!

Appendix:

If you need help choosing a license (sorry, I haven't checked your readme or looked for a license file), GitHub has created a license picker tool. Code without a license specified defaults to 'All rights reserved'-- denying others all rights to use of the code.
Here's a list of the license names I've found and their frequencies

p.s. In case you're wondering how I found you and why I made this issue, it's because I'm collecting stats on gems (I was originally looking for download data) and decided to collect license metadata,too, and make issues for gemspecs not specifying a license as a public service :). See the previous link or my blog post about this project for more information.

typhoeus / ethon Goto Github PK

ethon's Issues

Summary

Details

Error

Recommend Projects

Recommend Topics

Recommend Org