Giter VIP home page Giter VIP logo

katipo's Issues

Problem running coveralls in Travis CI 

===> Running coveralls...
===> Uncaught error in rebar_core. Run with DEBUG=1 to see stacktrace
===> When submitting a bug report, please include the output of `rebar3 report "your command"`
make: *** [coveralls] Error 1
The command "PATH=.:$PATH make update test dialyzer coveralls" exited with 2.
cache.2
store build cache

Something broken after #112

Repro: run this a few times and the port process should crash

katipo:req(test,
           #{body => ["i", "am", [], <<"an">>, "iolisttttttttttttttttttttttttttttttttttttttttttt thats really long"],
             url => <<"http://127.0.0.1:1080/">>}).

Support for async interface

Hello,

Right now the API provides only methods that blocks the current process until the response is received. Can you add API's that sends the response when is ready to a specified PID ?

Silviu

better cowlib dependency?

Hey, I notice its locked at 2.4.0 but it makes it difficult to use this lib with {:plug_cowboy, "~> 2.0"}

How to disable HTTP/2 for GET queries

Hi @puzza007

I'm trying to send a GET request without HTTP2.
Even if I specify {pipelining, nothing}, this query seems to be sent using HTTP2.

{ok, _} = application:ensure_all_started(katipo).
Pool = api_server.
{ok, _} = katipo_pool:start(Pool, 2, [{pipelining, nothing}]).
ReqHeaders = [{<<"User-Agent">>, <<"katipo">>}].
Req = #{url => <<"https://www.vancleefarpels.com/">>,
        method => get,
        headers => ReqHeaders,
        connecttimeout_ms => 5000,
        ssl_verifyhost => false,
        ssl_verifypeer => false}.
{ok, #{status := 200,
       headers := RespHeaders,
       cookiejar := CookieJar,
       body := RespBody}} = katipo:req(Pool, Req).
       
 ** exception error: no match of right hand side value 
   {error,#{code => unknown_error,
                 message => <<"HTTP/2 stream 1 was not closed cleanly: INTERNAL_ERROR (err 2)">>}}

Is there a way to tell katipo to not use HTTP/2?
Many thanks

Getting bad options when trying to fetch URL using proxy

Hi @puzza007 ,

I'm getting an error when trying to fetch a webpage via a GET request, and using a proxy:

> {ok, _} = application:ensure_all_started(katipo).
> {ok, _} = katipo_pool:start(pool, 1, [ {pipelining, nothing} ]),
> katipo:get(pool, "https://www.google.com/",  #{proxy => "http://localhost:3128", 
                                                 ssl_verifyhost => false,
                                                 ssl_verifypeer => false,
                                                 connecttimeout_ms => 5000}).
                                                 
 {error,#{code => bad_opts,
         message =>
             <<"[{url,\"https://www.google.com/\"},{proxy,\"http://localhost:3128\"}]">>}}

My proxy is working perfectly, and I'm able to test it using cURL.

$ curl -x "http://localhost:3128" "https://www.google.com/" > google.html 
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 19649    0 19649    0     0   214k      0 --:--:-- --:--:-- --:--:--  215k
$ ls -la google.html
.rw-r--r-- 20k zab 11 Sep 01:22 google.html

It fails with a wrong/closed port 9999, which proves my proxy is working as expected:

$ curl -x "http://localhost:9999" "https://www.google.com/"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (7) Failed to connect to localhost port 9999 after 0 ms: Couldn't connect to server

What am i missing?

My setup:
. Erlang: 26.0.2 | 25.3.2.4 | 24.2.1
. katipo from master
. macOS: Ventura 13.5.1

Help appreciated
Z.

Consider adding cacert (aside capath)

From a short research I just did, capath (or in libcurl's terms, CURLOPT_CAPATH) will be rarely the thing that the library user actually wants. The usual use case of using such option is in case of a self-signed certificate.

The problem is, CURLOPT_CAPATH is expected to be a directory containing CA certificate files along with their hashes. Typically, the user will want to provide just a single CA certificate file. I suggest adding another option, cacert, which will translate to CURLOPT_CAINFO, which is an option that supports just that scenario.

I can prepare a PR for that if that addition will get settled.

Erlang/OTP 20.0 compatibility

Are there any plans to update this project to make it compatible with Erlang 20.0? Right now, it fails to build on my machine, and it seems due to changes introduced in Erlang versions more recent than 19.3. I played around with travis-ci, it builds fine with 19.3, but shows errors for 20.0: https://travis-ci.org/nroi/katipo

compilation fails on Elixir 1.11.2 / OTP 23 / rebar 3.13.1

I have the following compilation error :

===> Rebar3 detected a lock file from a newer version. It will be loaded in compatibility mode, but important information may be missing or lost. It is recommended to upgrade Rebar3.
===> Compiling katipo
make: Entering directory '/home/pascal/boruta/boruta_umbrella/deps/katipo/c_src'
cc /home/pascal/boruta/boruta_umbrella/deps/katipo/c_src/katipo.o  -L /usr/lib/erlang/lib/erl_interface-4.0.1/lib -lerl_interface -lei -lcurl -lssl -lcrypto -levent -o /home/pascal/boruta/boruta_umbrella/deps/katipo/c_src/../priv/katipo
/usr/bin/ld: cannot find -lerl_interface
collect2: error: ld returned 1 exit status
make: *** [Makefile:52: /home/pascal/boruta/boruta_umbrella/deps/katipo/c_src/../priv/katipo] Error 1
make: Leaving directory '/home/pascal/boruta/boruta_umbrella/deps/katipo/c_src'
===> Hook for compile failed!

** (Mix) Could not compile dependency :katipo, "/home/pascal/.mix/rebar3 bare compile --paths /home/pascal/boruta/boruta_umbrella/_build/test/lib/*/ebin" command failed. You can recompile this dependency with "mix deps.compile katipo", update it with "mix deps.update katipo" or clean it with "mix deps.clean katipo"

As I don't know much about Erlang, I don't even know where to start.
any clues ?

Publish new version to hex.pm

Hi @puzza007 ,

I was wondering if you could publish a new release to hex.pm repository. We are looking forward to using OTP 26 and we need the fix introduced in #144

Thank you very much

fatal error: 'event.h' file not found on MacOS Ventura M1 Pro

Hi,

I'm having an issue when compiling katipo on MacOS Ventura 13.5 M1 as you can see in the logs below:

$ rebar3 compile
===> Verifying dependencies...
===> Fetching katipo v1.0.4
===> Fetching rebar3_hex v7.0.7
===> Fetching hex_core v0.8.4
===> Fetching verl v1.1.1
===> Analyzing applications...
===> Compiling verl
===> Compiling hex_core
===> Compiling rebar3_hex
===> Fetching rebar3_proper v0.12.1
===> Analyzing applications...
===> Compiling rebar3_proper
===> Fetching coveralls v1.4.0
===> Analyzing applications...
===> Compiling coveralls
===> Fetching metrics v2.5.0
===> Fetching worker_pool v6.0.0
cc -O3 -std=c99 -arch x86_64 -Wall -Wmissing-prototypes -fPIC -I /Users/giorgio.torres/.asdf/installs/erlang/24.3.4.13/erts-12.3.2.13/include/ -I /Users/giorgio.torres/.asdf/installs/erlang/24.3.4.13/lib/erl_interface-5.2.2/include  -c -o /Users/giorgio.torres/projects/test_katipo/_build/default/lib/katipo/c_src/katipo.o /Users/giorgio.torres/projects/test_katipo/_build/default/lib/katipo/c_src/katipo.c
/Users/giorgio.torres/projects/test_katipo/_build/default/lib/katipo/c_src/katipo.c:2:10: fatal error: 'event.h' file not found
#include <event.h>
         ^~~~~~~~~
1 error generated.
make: *** [/Users/giorgio.torres/projects/test_katipo/_build/default/lib/katipo/c_src/katipo.o] Error 1
===> Hook for compile failed!

Steps to reproduce

  1. Clone https://github.com/GPrimola/test_katipo
  2. Run rebar3 compile

Workaround

I found this answer on StackExchange that helped pass through the fatal error: 'event.h' file not found.

But then I get the following error:

$ export LDFLAGS="-L/opt/homebrew/lib"
$ export CPPFLAGS="-I/opt/homebrew/include"
$
$ rebar3 compile                      
===> Verifying dependencies...
cc -O3 -std=c99 -arch x86_64 -Wall -Wmissing-prototypes -fPIC -I /Users/giorgio.torres/.asdf/installs/erlang/24.3.4.13/erts-12.3.2.13/include/ -I /Users/giorgio.torres/.asdf/installs/erlang/24.3.4.13/lib/erl_interface-5.2.2/include -I/opt/homebrew/include -c -o /Users/giorgio.torres/projects/test_katipo/_build/default/lib/katipo/c_src/katipo.o /Users/giorgio.torres/projects/test_katipo/_build/default/lib/katipo/c_src/katipo.c
cc /Users/giorgio.torres/projects/test_katipo/_build/default/lib/katipo/c_src/katipo.o -L/opt/homebrew/lib -L /Users/giorgio.torres/.asdf/installs/erlang/24.3.4.13/lib/erl_interface-5.2.2/lib -lei -lcurl -lssl -lcrypto -levent -o /Users/giorgio.torres/projects/test_katipo/_build/default/lib/katipo/c_src/../priv/katipo
ld: warning: ignoring file /Users/giorgio.torres/projects/test_katipo/_build/default/lib/katipo/c_src/katipo.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64
Undefined symbols for architecture arm64:
  "_main", referenced from:
     implicit entry/start for main executable
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [/Users/giorgio.torres/projects/test_katipo/_build/default/lib/katipo/c_src/../priv/katipo] Error 1
===> Hook for compile failed!

which I solved by running the same command shown just after the ===> Verifying dependencies... but with the flag -arch arm64:

cc -O3 -std=c99 -arch arm64 -Wall -Wmissing-prototypes -fPIC -I /Users/giorgio.torres/.asdf/installs/erlang/24.3.4.13/erts-12.3.2.13/include/ -I /Users/giorgio.torres/.asdf/installs/erlang/24.3.4.13/lib/erl_interface-5.2.2/include -I/opt/homebrew/include -c -o /Users/giorgio.torres/projects/test_katipo/_build/default/lib/katipo/c_src/katipo.o /Users/giorgio.torres/projects/test_katipo/_build/default/lib/katipo/c_src/katipo.c

and only after that I can compile katipo:

$ rebar3 compile
===> Verifying dependencies...
cc /Users/giorgio.torres/projects/test_katipo/_build/default/lib/katipo/c_src/katipo.o -L/opt/homebrew/lib -L /Users/giorgio.torres/.asdf/installs/erlang/24.3.4.13/lib/erl_interface-5.2.2/lib -lei -lcurl -lssl -lcrypto -levent -o /Users/giorgio.torres/projects/test_katipo/_build/default/lib/katipo/c_src/../priv/katipo
===> Analyzing applications...
===> Compiling worker_pool
===> Compiling metrics
===> Compiling katipo
===> Analyzing applications...
===> Compiling test_katipo

I suspect there's a kind of misconfiguration with my Homebrew so the include and libs for libevent aren't being found, but also I don't know why when compiling c_src/katipo.c the architecture is not set correctly:

$ arch
arm64

Is anybody facing a similar issue?

EDIT

Versions

$ brew -v
Homebrew 4.1.3
Homebrew/homebrew-core (git revision 7a144369d9e; last commit 2023-07-12)
Homebrew/homebrew-cask (git revision cb0391d439; last commit 2023-07-12)

$ gcc -v
Apple clang version 14.0.3 (clang-1403.0.22.14.1)
Target: arm64-apple-darwin22.6.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

$ cc -v
Apple clang version 14.0.3 (clang-1403.0.22.14.1)
Target: arm64-apple-darwin22.6.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

$ rebar3 -v
rebar 3.22.0 on Erlang/OTP 24 Erts 12.3.2.13

Limit number of connections

Hi,
Is there a way to limit the number of TCP connections opened? It seems that pool size only sets the number of "workers" not connections.

Thanks,
LP

NTLM support?

Hi,
sorry to ask this kind of question, but I did not find in your readme how to map between curl command and Erlang code, in particular is the support for NTLM authentication? My command is:

curl -H "<Header>" --ntlm --negotiate -u "<UserName>:<Password>" <URL> -XPOST --data-binary <BinaryContent>

How would I translate it into Katipo language?

Thanks a lot!

Enhance Why in README

Can you please describe which problems you found with existing http clients, and why you cannot solve them so you wrote a new solution based on libcurl ?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.