Replay process can not tee some request with error "Can not parse request" about goreplay HOT 10 CLOSED

I released new version, pls check it https://github.com/buger/gor/releases/tag/0.7.5

from goreplay.

The problem is still there, but the error is different.

from goreplay.

For some request, I remove "Expect: 100-continue" from http header, and it works. So I guess perhaps Go's http library does not support "Expect: 100-continue". Is that a bug?

from goreplay.

I believe what you are seeing here is the tail end of a post that has already had most of the request sent. Basically the listener thinks the message is complete and sends it to the replay, then the listener gets more packets from the message which has already been shipped and then forwards that message to the replay.

from goreplay.

There were some changes in recent v0.8.4 which may help

from goreplay.

This is my first time posting here and I just want to congratulate you on this excellent piece of software. Its been working great for us. This is the first issue we've experienced with it so far.

I've still observed this issue in the 0.8.4 release. I run gor as follows :

sudo gor --input-raw :8080 --output-http xxx:8080 --output-http-method POST

The replay breaks on the following conversation.

POST /some/path HTTP/1.1
Content-Type: application/xml
Content-Length: xxx
Host: xxx:8080
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.0.3 (java 1.5)
Expect: 100-Continue

HTTP/1.1 100 Continue

<?xml version="1.0" encoding="UTF-8"?>
<X></X>

HTTP/1.1 200 OK
Content-Length: 0
Date: Wed, 10 Sep 2014 09:29:44 GMT
Server: Apache

The following errors are outputted by gor.

2014/09/10 09:00:46 Request error: Post http://xxx:8080/some/path: unexpected EOF
2014/09/10 09:00:46 Cannot parse request <?xml version="1.0" encoding="UTF-8"?>
<X></X> malformed HTTP version "encoding=\"UTF-8\"?>"

The 'Expect: 100-Continue' header is used by clients to give the server a chance to reject the request before the client goes to the trouble of sending the request body. It's normally used when the request body is large. The client sends the request headers including 'Expect: 100-Continue'. If it gets a 'HTTP/1.1 100 Continue' back it knows it's then good to send on the request body. http://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html#sec8.2.3

It looks as though gor's request parsing does not understand this header. For the first part of the conversation it expects to find a request body. It doesn't find anything so outputs the error 'unexpected EOF'. For the second part of the conversation it expects a HTTP version header, but instead finds a request body.

from goreplay.

It looks like output_http just delegates the request parsing to http://golang.org/pkg/net/http/#ReadRequest which does not have any special handling of this header.

from goreplay.

Yeah, its pretty simple parser right now...

from goreplay.

OK. Thanks. I see there is another issue for this #77 where you mention that handling this isn't so straightforward. I didn't see this earlier.

from goreplay.

@stephencurran @TigerMee well, it took a while, but now it should fixed by #158

from goreplay.