Comments (13)
I believe there's an individual socket pool for each Easy handle. If you use Multi, it will keep a shared connection pool. Or reutilize the Easy handle by setting the url again and calling #perform.
from curb.
Switching this example to use Curb::Multi I see the same behavior (multiple sockets created).
The man page for curl_easy_perform() says:
You can do any amount of calls to curl_easy_perform(3) while using the
same handle. If you intend to transfer more than one file, you are even
encouraged to do so. libcurl will then attempt to re-use the same con-
nection for the following transfers, thus making the operations faster,
less CPU intense and using less network resources. Just note that you
will have to use curl_easy_setopt(3) between the invokes to set options
for the following curl_easy_perform.
Which leads me to believe that Curl::Easy#perform should behave as I expect.
from curb.
this should fix the issue. http://github.com/taf2/curb/commit/73e11030c9debfc6c51f32e6aef4f597281cf6db
It keeps the original multi handle around between invocations.
I added two benchmarks in bench/curb_easy.rb and bench/nethttp_test.rb
here's the results for me:
time ruby bench/curb_easy.rb
Duration 0.059045 seconds
real 0m0.083s
user 0m0.019s
sys 0m0.014s
time ruby bench/nethttp_test.rb
Duration 0.063115 seconds
real 0m0.731s
user 0m0.406s
sys 0m0.070s
from curb.
Just to point it out, libcurl already uses a multi handle to perform a easy handle even if you don't manually add the handle.
from curb.
I think the issue with curb here is different... Unless libcurl provides away to get at that multi handle? For each easy request curb was creating a new multi handle, effectively throwing away any open connections. Now instead, when calling the curb easy perform it will reuse it's internal multi handle... Hope that makes sense?
from curb.
I'm wondering why create a multi for each easy handle if libcurl does that by itself inside easy_perform?
from curb.
curl_easy_perform blocks the whole ruby interpreter. To avoid the blocking and have libcurl play nice with ruby, all IO is run through rb_thread_select. In order to expose that kind of IO a multi interface was necessary... Unless there is an interface to expose what select method libcurl uses or access the easy handle's internal multi interface?
from curb.
Did some testing in 1.8.7 and 1.9.1 and it does some evil blocking indeed. Since we don't have access to libcurl's select or internal multi, a suggestion would be to fit rb_thread_schedule() in a callback and get rid of multi and the fds hassle.
from curb.
you mean for example have a libcurl callback on the easy handle such as on_progress or another that calls maybe rb_thread_schedule or even yields ?
from curb.
Yup, but according to libcurl docs, on_progress callback does not seem to be a good choice since it keeps being called every second even if there is no transfer running. I've did a quick test by placing rb_thread_schedule inside CURLOPT_WRITEFUNCTION callback and works . I didn't run any benchmark tho.
from curb.
if there's not transfer running than it's blocked... write function would only be called when data is received?
from curb.
I really think select is the right thing here, you want the OS to tell you when to idle and when to read/write otherwise let ruby run.
from curb.
Alright, it's using persistent connections now
from curb.
Related Issues (20)
- on_failure receive incorrect argument HOT 1
- README: Change URLs to be to curl.se HOT 1
- New version request HOT 4
- Is CURLOPT_REQUEST_TARGET needed for Request-URI different than Host header? HOT 2
- Library not loaded: @rpath/libgssapi_krb5.2.2.dylib HOT 1
- Hello, I am new into coding, so i was trying to set up environment for ESG-BERT: https://github.com/mukut03/ESG-BERT using pytorch, but i am unable to solve this error on windows using anaconda, the server is running but unhealthy. Error(ESG-BERT) C:\Users\Administrator\anaconda3\envs\ESG-BERT>curl http://localhost:8080/ping { "status": "Unhealthy" }
- [BUG] Segmentation fault with Ruby 2.3 & curb 1.0.1 HOT 1
- LICENSE file contents does not match MIT license in the gemspec file HOT 1
- Gem install fails in windows 10 wsl v1 HOT 1
- Build against curl-impersonate HOT 1
- Investigate Ruby T_DATA warning HOT 1
- Segfault in Multi after upgrading to v1.0.3 HOT 5
- CURLOPT_NOPROXY not supported
- Curl::Err::HostResolutionError on WSL
- how do I `download_byte_limit` with curb
- `test_connection_keepalive(TestCurbCurlMulti)` skipped with Curl 8+ HOT 1
- Cannot edit cookies.
- Error Installing on Ubuntu 20.04.6 LTS
- error running the curb test suite with curl 8.7.1 HOT 5
- Not working with libcurl 8.8 HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from curb.