vlet / p5-protocol-http2 Goto Github PK

View Code? Open in Web Editor NEW

27.0 4.0 13.0 274 KB

HTTP/2 protocol implementation (RFC 7540)

License: Other

Perl 100.00%

p5-protocol-http2's Introduction

NAME

Protocol::HTTP2 - HTTP/2 protocol implementation (RFC 7540)

SYNOPSIS

use Protocol::HTTP2;

# get protocol identification string for secure connections
print Protocol::HTTP2::ident_tls;           # h2

# get protocol identification string for non-secure connections
print Protocol::HTTP2::ident_plain;         # h2c

DESCRIPTION

Protocol::HTTP2 is HTTP/2 protocol implementation (RFC 7540) with stateful decoders/encoders of HTTP/2 frames. You may use this module to implement your own HTTP/2 client/server/intermediate on top of your favorite event loop over plain or tls socket (see examples).

STATUS

Current status - beta. Structures, module names and methods seems like stable. I've started this project to understand internals of HTTP/2 and may be it will never become production, but at least it works.

| Spec                    |      status     |
| ----------------------- | --------------- |
| Negotiation             |   ALPN, NPN,    |
|                         | Upgrade, direct |
| Preface                 |        +        |
| Headers (de)compression |        +        |
| Stream states           |        +        |
| Flow control            |        ±        |
| Stream priority         |        ±        |
| Server push             |        +        |
| Connect method          |        -        |


| Frame           | encoder | decoder |
| --------------- |:-------:|:-------:|
| DATA            |    ±    |    +    |
| HEADERS         |    +    |    +    |
| PRIORITY        |    +    |    +    |
| RST_STREAM      |    +    |    +    |
| SETTINGS        |    +    |    +    |
| PUSH_PROMISE    |    +    |    +    |
| PING            |    +    |    +    |
| GOAWAY          |    +    |    +    |
| WINDOW_UPDATE   |    +    |    +    |
| CONTINUATION    |    ±    |    +    |

- -- not implemented
± -- incomplete
- -- implemented (may even work)

MODULES

Protocol::HTTP2::Client

Client protocol decoder/encoder with constructor of requests

Protocol::HTTP2::Server

Server protocol decoder/encoder with constructor of responses/pushes

Protocol::HTTP2::Connection

Main low level module for protocol logic and state processing. Connection object is a mixin of Protocol::HTTP2::Frame (frame encoding/decoding), Protocol::HTTP2::Stream (stream operations) and Protocol::HTTP2::Upgrade (HTTP/1.1 Upgrade support)

Protocol::HTTP2::HeaderCompression

Module implements HPACK - Header Compression for HTTP/2 (RFC 7541).

Protocol::HTTP2::Constants

Module contains all defined in HTTP/2 protocol constants and default values

Protocol::HTTP2::Trace

Module for debugging. You can setup HTTP2_DEBUG environment variable to change verbosity of the module (output to STDOUT). Default level is error.

$ export HTTP2_DEBUG=debug
$ perl ./http2_program

LICENSE

This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

AUTHOR

Vladimir Lettiev [email protected]

p5-protocol-http2's People

Contributors

Stargazers

Watchers

Forkers

meteoritt mashthekeys mjonss bondarev-aws pkg-perl-tools iflagri manwar fgasper junhochoi pelov ottohollmann ogayot yoshikazusawa

p5-protocol-http2's Issues

accessing reconstructed streamed objects

I'm trying to use this module as an http2 server. I've got a client which opens a connection to the server and streams objects. The tcp connection forms and I see the client streaming objects. I'm using the example code provided here (http://search.cpan.org/~crux/Protocol-HTTP2-0.15/lib/Protocol/HTTP2/Server.pm). In the $handle->on_read, I can dump out the $handle->{rbuf} info I see the raw data coming from the client. in the code, this information is then past to $server->feed{$handle->{rbuf}} which I think takes the streaming data and decodes.

What I'm hoping to find is a way to access the reconstructed objects-- how would I get access to an entire reconstructured object? It's not clear how I gain access to these object in the server code.

Thanks

Unclear how to make the connection idle

How is one suppose to use this to add new requests sequentially?

Since it will finish the connection (by sending an GOAWAY) as soon as there are no active_sessions.

I want to be able to create a new request and then wait for the reply and then create another request etc. like:

while (my $msg = get_new_msg()) {
    $client->request(... $msg ..., on_done => ... );
    while (my $frame = $client->next_frame) {
        syswrite $socket, $frame;
    }
    my $reply;
    sysread $socket, $reply, 4096;
    $client->feed($reply);
    # Now there will be no active streams!
}

Currently it will finish the connection in Client::active_streams
223 $self->{con}->finish unless $self->{active_streams} > 0;
which prohibits subsequent use of the http2 connection.

As a workaround I can do $client->active_streams(+1) when I have created $client.

I propose $client->allow_idle(0|1) to control this instead and perhaps a defined way to shutdown it down like $client->shutdown.

Unable to install under Centos 8

Centos 8 - tests are failed
Centos 7, Fedora 30, 31 - OK

[root@d3a191917302 /]# cpanm Protocol::HTTP2 -v
cpanm (App::cpanminus) 1.7044 on perl 5.026003 built for x86_64-linux-thread-multi
Work directory is /root/.cpanm/work/1570966752.2652
You have make /usr/bin/make
You have /usr/bin/curl
You have /usr/bin/tar: tar (GNU tar) 1.30
Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by John Gilmore and Jay Fenlason.
You have /usr/bin/unzip
Searching Protocol::HTTP2 () on cpanmetadb ...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   989  100   989    0     0   8106      0 --:--:-- --:--:-- --:--:--  8173
--> Working on Protocol::HTTP2
Fetching http://www.cpan.org/authors/id/C/CR/CRUX/Protocol-HTTP2-1.09.tar.gz ... ##O=#   #                                                       ########################################################################################################################################## 100.0%
OK
Unpacking Protocol-HTTP2-1.09.tar.gz
Protocol-HTTP2-1.09/Build.PL
Protocol-HTTP2-1.09/Changes
Protocol-HTTP2-1.09/LICENSE
Protocol-HTTP2-1.09/META.json
Protocol-HTTP2-1.09/README.md
Protocol-HTTP2-1.09/cpanfile
Protocol-HTTP2-1.09/examples/client-anyevent.pl
Protocol-HTTP2-1.09/examples/client-io-socket-ssl.pl
Protocol-HTTP2-1.09/examples/client-tls-anyevent.pl
Protocol-HTTP2-1.09/examples/extract_huff_codes.pl
Protocol-HTTP2-1.09/examples/extract_static_table.pl
Protocol-HTTP2-1.09/examples/server-anyevent.pl
Protocol-HTTP2-1.09/examples/server-io-socket-ssl.pl
Protocol-HTTP2-1.09/examples/server-tls-anyevent.pl
Protocol-HTTP2-1.09/examples/test.crt
Protocol-HTTP2-1.09/examples/test.key
Protocol-HTTP2-1.09/lib/Protocol/HTTP2.pm
Protocol-HTTP2-1.09/lib/Protocol/HTTP2/Client.pm
Protocol-HTTP2-1.09/lib/Protocol/HTTP2/Connection.pm
Protocol-HTTP2-1.09/lib/Protocol/HTTP2/Constants.pm
Protocol-HTTP2-1.09/lib/Protocol/HTTP2/Frame.pm
Protocol-HTTP2-1.09/lib/Protocol/HTTP2/Frame/Continuation.pm
Protocol-HTTP2-1.09/lib/Protocol/HTTP2/Frame/Data.pm
Protocol-HTTP2-1.09/lib/Protocol/HTTP2/Frame/Goaway.pm
Protocol-HTTP2-1.09/lib/Protocol/HTTP2/Frame/Headers.pm
Protocol-HTTP2-1.09/lib/Protocol/HTTP2/Frame/Ping.pm
Protocol-HTTP2-1.09/lib/Protocol/HTTP2/Frame/Priority.pm
Protocol-HTTP2-1.09/lib/Protocol/HTTP2/Frame/Push_promise.pm
Protocol-HTTP2-1.09/lib/Protocol/HTTP2/Frame/Rst_stream.pm
Protocol-HTTP2-1.09/lib/Protocol/HTTP2/Frame/Settings.pm
Protocol-HTTP2-1.09/lib/Protocol/HTTP2/Frame/Window_update.pm
Protocol-HTTP2-1.09/lib/Protocol/HTTP2/HeaderCompression.pm
Protocol-HTTP2-1.09/lib/Protocol/HTTP2/Huffman.pm
Protocol-HTTP2-1.09/lib/Protocol/HTTP2/HuffmanCodes.pm
Protocol-HTTP2-1.09/lib/Protocol/HTTP2/Server.pm
Protocol-HTTP2-1.09/lib/Protocol/HTTP2/StaticTable.pm
Protocol-HTTP2-1.09/lib/Protocol/HTTP2/Stream.pm
Protocol-HTTP2-1.09/lib/Protocol/HTTP2/Trace.pm
Protocol-HTTP2-1.09/lib/Protocol/HTTP2/Upgrade.pm
Protocol-HTTP2-1.09/minil.toml
Protocol-HTTP2-1.09/t/00_compile.t
Protocol-HTTP2-1.09/t/01_HeaderCompression.t
Protocol-HTTP2-1.09/t/02_Huffman.t
Protocol-HTTP2-1.09/t/03_connection.t
Protocol-HTTP2-1.09/t/04_continuation.t
Protocol-HTTP2-1.09/t/05_trace.t
Protocol-HTTP2-1.09/t/06_upgrade.t
Protocol-HTTP2-1.09/t/07_ping.t
Protocol-HTTP2-1.09/t/08_priority.t
Protocol-HTTP2-1.09/t/09_client_server_tcp.t
Protocol-HTTP2-1.09/t/10_settings.t
Protocol-HTTP2-1.09/t/11_server_stream.t
Protocol-HTTP2-1.09/t/12_leaks.t
Protocol-HTTP2-1.09/t/13_request_with_body.t
Protocol-HTTP2-1.09/t/14_keepalive.t
Protocol-HTTP2-1.09/t/continuation.request.data
Protocol-HTTP2-1.09/t/lib/PH2ClientServerTest.pm
Protocol-HTTP2-1.09/t/lib/PH2Test.pm
Protocol-HTTP2-1.09/META.yml
Protocol-HTTP2-1.09/MANIFEST
Entering Protocol-HTTP2-1.09
Checking configure dependencies from META.json
Checking if you have Module::Build::Tiny 0.035 ... Yes (0.039)
Running Build.PL
Configuring Protocol-HTTP2-1.09 ... Creating new 'Build' script for 'Protocol-HTTP2' version '1.09'
OK
Checking dependencies from MYMETA.json ...
Checking if you have Test::LeakTrace 0 ... Yes (0.16)
Checking if you have Test::More 0.98 ... Yes (1.302135)
Checking if you have Net::SSLeay > 1.45 ... Yes (1.88)
Checking if you have Test::TCP 0 ... Yes (2.22)
Checking if you have MIME::Base64 3.11 ... Yes (3.15)
Checking if you have Scalar::Util 0 ... Yes (1.49)
Checking if you have AnyEvent 0 ... Yes (7.17)
Building and testing Protocol-HTTP2-1.09 ... cp lib/Protocol/HTTP2/HeaderCompression.pm blib/lib/Protocol/HTTP2/HeaderCompression.pm
cp lib/Protocol/HTTP2/Trace.pm blib/lib/Protocol/HTTP2/Trace.pm
cp lib/Protocol/HTTP2/Constants.pm blib/lib/Protocol/HTTP2/Constants.pm
cp lib/Protocol/HTTP2/HuffmanCodes.pm blib/lib/Protocol/HTTP2/HuffmanCodes.pm
cp lib/Protocol/HTTP2/Frame/Push_promise.pm blib/lib/Protocol/HTTP2/Frame/Push_promise.pm
cp lib/Protocol/HTTP2/Frame/Data.pm blib/lib/Protocol/HTTP2/Frame/Data.pm
cp lib/Protocol/HTTP2/Frame/Continuation.pm blib/lib/Protocol/HTTP2/Frame/Continuation.pm
cp lib/Protocol/HTTP2/StaticTable.pm blib/lib/Protocol/HTTP2/StaticTable.pm
cp lib/Protocol/HTTP2/Frame/Window_update.pm blib/lib/Protocol/HTTP2/Frame/Window_update.pm
cp lib/Protocol/HTTP2/Frame/Priority.pm blib/lib/Protocol/HTTP2/Frame/Priority.pm
cp lib/Protocol/HTTP2/Upgrade.pm blib/lib/Protocol/HTTP2/Upgrade.pm
cp lib/Protocol/HTTP2/Frame/Goaway.pm blib/lib/Protocol/HTTP2/Frame/Goaway.pm
cp lib/Protocol/HTTP2/Server.pm blib/lib/Protocol/HTTP2/Server.pm
cp lib/Protocol/HTTP2/Frame/Headers.pm blib/lib/Protocol/HTTP2/Frame/Headers.pm
cp lib/Protocol/HTTP2/Connection.pm blib/lib/Protocol/HTTP2/Connection.pm
cp lib/Protocol/HTTP2/Client.pm blib/lib/Protocol/HTTP2/Client.pm
cp lib/Protocol/HTTP2/Frame/Ping.pm blib/lib/Protocol/HTTP2/Frame/Ping.pm
cp lib/Protocol/HTTP2.pm blib/lib/Protocol/HTTP2.pm
cp lib/Protocol/HTTP2/Huffman.pm blib/lib/Protocol/HTTP2/Huffman.pm
cp lib/Protocol/HTTP2/Frame/Rst_stream.pm blib/lib/Protocol/HTTP2/Frame/Rst_stream.pm
cp lib/Protocol/HTTP2/Frame.pm blib/lib/Protocol/HTTP2/Frame.pm
cp lib/Protocol/HTTP2/Frame/Settings.pm blib/lib/Protocol/HTTP2/Frame/Settings.pm
cp lib/Protocol/HTTP2/Stream.pm blib/lib/Protocol/HTTP2/Stream.pm
t/00_compile.t ............ ok
t/01_HeaderCompression.t .. ok
t/02_Huffman.t ............ ok
t/03_connection.t ......... ok
t/04_continuation.t ....... ok
t/05_trace.t .............. ok
t/06_upgrade.t ............ ok
t/07_ping.t ............... ok
t/08_priority.t ........... ok
t/09_client_server_tcp.t ..
    #   Failed test 'no errors'
    #   at t/09_client_server_tcp.t line 115.
    #          got: 'Some problem with SSL CTX: examples/test.crt: failed to use local certificate chain (cert_file or cert) at t/lib/PH2ClientServerTest.pm line 45.
    #
    #  at /usr/local/lib64/perl5/AnyEvent/Socket.pm line 1266.
    # '
    #     expected: ''

    #   Failed test 'no errors'
    #   at t/09_client_server_tcp.t line 115.

    #   Failed test 'no errors'
    #   at t/09_client_server_tcp.t line 115.
    #          got: 'error occured
    #  at t/09_client_server_tcp.t line 77.
    # '
    #     expected: ''
    #          got: 'Some problem with SSL CTX: examples/test.crt: failed to use local certificate chain (cert_file or cert) at t/lib/PH2ClientServerTest.pm line 45.
    #
    #  at /usr/local/lib64/perl5/AnyEvent/Socket.pm line 1266.
    # '
    #     expected: ''
    # Looks like you planned 0 tests but ran 4.

    #   Failed test 'no errors'
    #   at t/09_client_server_tcp.t line 115.
    #          got: 'Some problem with SSL CTX: examples/test.crt: failed to use local certificate chain (cert_file or cert) at t/lib/PH2ClientServerTest.pm line 45.
    #
    #  at /usr/local/lib64/perl5/AnyEvent/Socket.pm line 1266.
    # '
    #     expected: ''
    # Looks like you planned 4 tests but ran 7.
    # Looks like you failed 3 tests of 7 run.

    #   Failed test 'no errors'
    #   at t/09_client_server_tcp.t line 115.
    #          got: 'error occured
    #  at t/09_client_server_tcp.t line 77.
    # '
    #     expected: ''
    # Looks like you planned 4 tests but ran 9.
    # Looks like you failed 5 tests of 9 run.
t/09_client_server_tcp.t .. 1/?
#   Failed test 'client/server'
#   at t/09_client_server_tcp.t line 117.

#   Failed test 'client/server'
#   at t/09_client_server_tcp.t line 117.

#   Failed test 'client/server'
#   at t/09_client_server_tcp.t line 117.
# Looks like you failed 3 tests of 3.
t/09_client_server_tcp.t .. skipped: (no reason given)
t/10_settings.t ........... ok
t/11_server_stream.t ...... ok
t/12_leaks.t .............. ok
t/13_request_with_body.t .. ok
t/14_keepalive.t .......... ok

Test Summary Report
-------------------
t/09_client_server_tcp.t (Wstat: 768 Tests: 3 Failed: 3)
  Failed tests:  1-3
  Non-zero exit status: 3
  Parse errors: Plan (1..0) must be at the beginning or end of the TAP output
                More than one plan found in TAP output
                More than one plan found in TAP output
                Bad plan.  You planned 0 tests but ran 3.
Files=15, Tests=37,  1 wallclock secs ( 0.03 usr  0.05 sys +  0.77 cusr  0.29 csys =  1.14 CPU)
Result: FAIL
FAIL

on_error callback seems to disappear, causing undefined coderef errors

A long-running stream can generate a condition the on_cancel callback disappears.

I don’t have a repro setup now because I fixed it locally. If you’d like I can try to put something together, but the fix seems pretty obvious.

Add on_goaway callback

Add a on_goaway callback to Procotol::HTTP2::Client->new. It should be called when the server sends a GOAWAY frame. It should receive last_stream_id, error_code and the additional data (which is usually a text).

Right now, the only way to see these frames is to turn on debugging. However, this frame is important to a client so that it can stop opening new streams and prepare to shutdown the connection. Also the error code and the additional data can be very useful for diagnosing problems.

I don't mind trying to make this change myself - just want to know if I have a green light for that.

t/09_client_server_tcp.t fails on Ubuntu

The current release 1.10 fails test t/09_client_server_tcp.t with the following error:

    #   Failed test 'no errors'
    #   at t/09_client_server_tcp.t line 115.
    #          got: 'Can't call method "push_write" on an undefined value at t/lib/PH2ClientServerTest.pm line 182.
    # '
    #     expected: ''

    #   Failed test 'no errors'
    #   at t/09_client_server_tcp.t line 115.
    #          got: 'Can't call method "push_write" on an undefined value at t/lib/PH2ClientServerTest.pm line 182.
    # '
    #     expected: ''
    # Looks like you failed 2 tests of 6.

#   Failed test 'client/server'
#   at t/09_client_server_tcp.t line 117.
# Looks like you failed 1 test of 1.
t/09_client_server_tcp.t .. 
# Subtest: client/server
    # test: without tls
    ok 1 - get response headers
    ok 2 - get body
    ok 3 - no errors
    # test: without tls, upgrade
    ok 4 - no errors
    # test: tls/npn
connection error
    not ok 5 - no errors
    # test: tls/alpn
connection error
    not ok 6 - no errors
    1..6
not ok 1 - client/server
1..1
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/1 subtests

Ubuntu Bug: https://bugs.launchpad.net/ubuntu/+source/libprotocol-http2-perl/+bug/2023586

Note: The same package builds on Debian.

Protocol::HTTP2::Client on_done gives undef header/content when tried with multiple streams

Hi,
When I use Protocol::HTTP2::Client with request sent with multiple parallel requests(streams), the on_done() cb is returning undef values for header, content for all streams exept the last one(for which it returns valid data.).
my code snippet:
$self->client->request(
%$req,
on_done => sub {
my ($header, $content) = @;
$msg = "_make_next_client_request::on_done!!!!!!!!!!!!!!!!!!!!!!!!!!";
warn Dumper($msg);
warn Dumper($header);
warn Dumper($content);

Note that I send more than one(4) requests using above code and then wait for responses using below code:
while ( $self->{'resp_revcd'} == 0 ) {
$self->{_io}->can_read;
while ( sysread $self->_socket, my $data, 4096 ) {
$self->_client->feed($data);
my $msg = "Recvd Response!!!!!!!!!!!!!!!!!!!!!!!";
warn Dumper($msg);
warn Dumper($data);

I could see that for all streams created(using request()), the response frames are received nd on_done() is called. Also the streams is correctly getting closed.

ouput for all streams(except first):
$VAR1 = 'Stream 7 changed state from HALF_CLOSED to CLOSED';
$VAR1 = '_make_next_client_request::on_done!!!!!!!!!!!!!!!!!!!!!!!!!!';
$VAR1 = undef;
$VAR1 = undef;

output for first stream:
$VAR1 = 'Stream 1 changed state from HALF_CLOSED to CLOSED';
$VAR1 = '_make_next_client_request::on_done!!!!!!!!!!!!!!!!!!!!!!!!!!';
$VAR1 = [
':status',
'410',
'xxxxx-xxx',
'dddddddddddddddddddddddddddd'
];
$VAR1 = '{"rrrrrrrr":"aaaaaaaaaaaaaaaa","timestamp":111111111111111111111111}';

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.