sockjs / sockjs-protocol Goto Github PK

These all are valid per the spec (http://tools.ietf.org/html/rfc6265#section-5.2.4)

Set-Cookie: JSESSIONID=abcdef; Path=/
Set-Cookie: JSESSIONID=abcdef; PaTh=/
Set-Cookie: JSESSIONID=abcdef; PATH=/

The test suite only accepts lower-case attribute-name for the Path attribute, i.e.:
Set-Cookie: JSESSIONID=abcdef; path=/

To prevent clickjacking new browsers introduced X-Frame-Options header. Although SockJS does not have any clickable elements, it would be good to send this header for hygiene. Especially for pages that are intended to be framed (htmlfile, iframe.html):

https://developer.mozilla.org/en/The_X-FRAME-OPTIONS_response_header
http://blogs.msdn.com/b/ieinternals/archive/2010/03/30/combating-clickjacking-with-x-frame-options.aspx
https://www.owasp.org/index.php/Clickjacking#Defending_with_response_headers

The protocol spec needs better structure. The simplest way to start fixing structure is to have a table of contents.

@mrjoes in https://groups.google.com/group/sockjs/msg/b57a83de3b32db87 suggested:

4. ETag in JSONP transport. Again, Tornado makes assumption that ETag is 
always required and will generate one if there's none set by the 
application. Is there reason to check for ETag even if Cache-Control is set 
to 'no-store, no-cache, must-revalidate, max-age=0'? 

Currently /echo service is required to have JSESSIONID option set. This is wrong and makes the deployment of QUnit tests tedious.

We should leave the JSESSIONID requirements alone from all the service used by QUnit tests and introduce another /cookie_needed_echo service especially to check cookies. This will be in similar fashion to /disabled_websocket_echo service.

We have few hacks for the unicode, we should test them

If client goes away, the streaming request will be abrupty closed.

There is no way to figure out what data was actually delivered to the browser and what was lost.

In such case, we should just drop the whole session, SockJS on the client side should report the same error as for timeout - something within the lines of timeout/network error.

There are some issues when running the tests on mac. Investigate and fix!

Right now we JSON-encode all the traffic. This is rather suboptimal.

It looks that we're using at least three flavours of http request abstractions in the tests. That's rather wrong.

We should use the RawHttpClient for everything.

From the README:

To run the tests against your server (**the source assumes your server is at http://localhost:8080**).

In my opinion this should be configurable (probably from command-line).

test_abort_* tests are checking for a rough optimization - they expect the "o" frame.

http://sockjs.github.com/sockjs-protocol/sockjs-protocol-0.2.html#section-134

In fact they should expect "o" frame or a close frame with a network error described.

Hi!

Can't finalize HtmlFile, EventSource, XhrStreaming tests, due to way reading from <VERB>_async() is performed.
Example for HtmlFile:

test_no_callback (__main__.HtmlFile) ... ok
test_transport (__main__.HtmlFile) ...  AND-HERE-IT-BLOCKS

What should testing server do for test to go on?

--Vladimir

Hello,
the test suite for SockJS 0.3 FAILs on test_xhr_server_encodes for mine server implementation with the following assertion:

AssertionError: '"\xe2\x80\x8c\xe2\x80\x8d\xe2\x80\x8e\xe2\x80\x8f\xe2\x80\xa8\xe2\x80\xa9\xe2\x80\xaa\xe2\x80\xab\xe2\x80\xac\xe2\x80\xad\xe2\x80\xae\xe2\x80\xaf\xe2\x81\xa0\xe2\x81\xa1\xe2\x81\xa2\xe2\x81\xa3\xe2\x81\xa4\xe2\x81\xa5\xe2\x81\xa6\xe2\x81\xa7\xe2\x81\xa8\xe2\x81\xa9\xe2\x81\xaa\xe2\x81\xab\xe2\x81\xac\xe2\x81\xad\xe2\x81\xae\xe2\x81\xaf\xef\xbf\xb0\xef\xbf\xb1\xef\xbf\xb2\xef\xbf\xb3\xef\xbf\xb4\xef\xbf\xb5\xef\xbf\xb6\xef\xbf\xb7\xef\xbf\xb8\xef\xbf\xb9\xef\xbf\xba\xef\xbf\xbb\xef\xbf\xbc\xef\xbf\xbd\xef\xbf\xbe\xef\xbf\xbf"' != '"\\u200c\\u200d\\u200e\\u200f\\u2028\\u2029\\u202a\\u202b\\u202c\\u202d\\u202e\\u202f\\u2060\\u2061\\u2062\\u2063\\u2064\\u2065\\u2066\\u2067\\u2068\\u2069\\u206a\\u206b\\u206c\\u206d\\u206e\\u206f\\ufff0\\ufff1\\ufff2\\ufff3\\ufff4\\ufff5\\ufff6\\ufff7\\ufff8\\ufff9\\ufffa\\ufffb\\ufffc\\ufffd\\ufffe\\uffff"'

I have hand checked the first few characters and they seems to be the same, except the first string is UTF8 encoded and the second one is UTF16 encoded (plus the '' are for some reason escaped, but that might be python's print feature).

For example "0xE2 0x80 0x8C" is the same as "0x200C" (http://www.fileformat.info/info/unicode/char/200c/index.htm).

I think that both should be correct and one should be able to determine the encoding based on the first two octets, which are at the moment being ignored by the tests, see the RFC's third section:

3.  Encoding

   JSON text SHALL be encoded in Unicode.  The default encoding is
   UTF-8.

   Since the first two characters of a JSON text will always be ASCII
   characters [RFC0020], it is possible to determine whether an octet
   stream is UTF-8, UTF-16 (BE or LE), or UTF-32 (BE or LE) by looking
   at the pattern of nulls in the first four octets.

           00 00 00 xx  UTF-32BE
           00 xx 00 xx  UTF-16BE
           xx 00 00 00  UTF-32LE
           xx 00 xx 00  UTF-16LE
           xx xx xx xx  UTF-8

Moreover, the encoding header is set to UTF8, so it might make more sense to expect UTF8 rather than UTF16.

But mayhaps I've just made some erroneous assumptions, in that case, please forgive me for wasting your time and let me know.

Cheers,
Petr.

Doing r.read() == None check is quite confusing. Instead, introduce is_closed function.

That should make tests much easier to debug.

As discussed:
https://groups.google.com/group/sockjs/msg/89fc223c8c966dca

2. Support command line clients for SockJS. 
Currently, SockJS has a small bit of framing on top of websockets 
protocol. That makes creating a standalone command-line 
SockJS client difficult - it must understand SockJS framing. 
As opposed to current websockets url: 
  /prefix/server_id/session_id/websocket 
i'm thinking of creating a new websockets url, that will expose 
websockets without any additional framing, for example: 
   /prefix/websocket 
That would make it easier to connect to sockjs from other 
environments than browsers. 

People seem to be confused by:

• transfer-encoding
• content-length and
• connection headers

We should write few basic tests to check if the servers adhere to http specs sanely.

Hi,

I got a weird problem. The test suite hangs on any transport that tests for prelude. Example for Eventsource

File "sockjs-protocol-0.2.py", line 929, in test_response_limit
  self.assertTrue(r.read()) # prelude
File "<my_path>/sockjs-protocol/utils.py", line 65, in read
  data =  self.res.read(10240)
File "httplib.py", line 561, in read
  s = self.fp.read(amt)
File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/socket.py", line 351, in read
  data = self._sock.recv(left)

I tried testing Node.js server and there wasn't such problem.

What I telnet the server, I get the response right away

GET /echo/000/642c3ab9-c850-418b-87bf-1ec381186467/eventsource HTTP/1.1
Host: localhost:8081
Accept-Encoding: identity

HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: http://localhost
Connection: keep-alive
Set-Cookie: JSESSIONID=dummy;path=/
Content-Type: text/event-stream; charset=UTF-8
Cache-Control: no-store, no-cache, must-revalidate, max-age=0


data: o

data: a[]

What's there I can not see?

Hi!

Please, clarify how /close route should work for different transports?

E.g., should WebSocket first establish new session answering o and then close it with c[3000, "Go away!"] as prescribed in https://github.com/sockjs/sockjs-protocol/blob/master/sockjs-protocol-0.0.4-incomplete.py#L543-544 ? This contradicts to statement "close - server immediately closes the session" done in requirements section.

Also, should we consider /close route as purely transient, for testing purpose only?

TIA,
--Vladimir

Twice in the tests we use POST_async with load set to False. This is racy.

We should change the protocol to always send headers immediately, also on XHR-polling.

This requires a protocol change.

Is there any particular reason why the specification strictly disallows response bodies on 4xx-level responses?
The RFC2616 #10.4 suggests that the server SHOULD respond with an entity describing situation
to the client:

Except when responding to a HEAD request, the server SHOULD include an entity containing
an explanation of the error situation, and whether it is a temporary or permanent condition.

For heartbeats, the client should be allowed to send an empty frame, something like []. We should write tests for that.

As described here:

• https://secure.wikimedia.org/wikipedia/en/wiki/HTTP_cookie#HttpOnly_cookie

We may also use path= thing.

@mrjoes suggests that for broken_json (and others?) we should think about using native websockets closing frames: http://tools.ietf.org/html/draft-ietf-hybi-thewebsocketprotocol-10#section-1.4

We should definetely use native close frames for raw_websocket url.

As described in sockjs/sockjs-node#88 , getting iOS6 Safari to behave better is quite trivial - we just need to decorate POST responses with no-cache cache-control header.

@mrjoes reports that this is not tested:

https://github.com/sockjs/sockjs-node/blob/master/src/trans-xhr.coffee#L56-58

Hi!

https://github.com/sockjs/sockjs-protocol/blob/master/sockjs-protocol-0.0.4-incomplete.py#L445-449 causes:

======================================================================
ERROR: test_simpleSession (__main__.Protocol)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "sockjs-protocol-0.0.4-incomplete.py", line 446, in test_simpleSession
    r2 = POST(trans_url + '/xhr')
  File "/home/dvv/LUA/sockjs-protocol/utils.py", line 79, in POST
    return HttpResponse('POST', url, **kwargs)
  File "/home/dvv/LUA/sockjs-protocol/utils.py", line 36, in __init__
    self._load()
  File "/home/dvv/LUA/sockjs-protocol/utils.py", line 52, in _load
    self.body = self.res.read()
  File "/home/dvv/LUA/sockjs-protocol/httplib.py", line 546, in read
    s = self.fp.read()
  File "/usr/lib/python2.7/socket.py", line 351, in read
    data = self._sock.recv(rbufsize)
timeout: timed out

on both sockjs-luvit and latest sockjs-node. Solution is to always read response headers -- append at L39 of utils.py:

        else:
            self.res = self.conn.getresponse()

Please, confirm or invalidate.

TIA,
--Vladimir

Heartbeats are not tested now. They should be.

@mrjoes says

http://groups.google.com/group/sockjs/browse_thread/thread/d4cf931f28e65858

that bad things can happen to websockets behind avast. We should understand the problem and make sure that SockJS behaves correctly.

Reference:  https://github.com/LearnBoost/socket.io/wiki/Socket.IO-and-firewall-software

We should review JSONP protocol encodings and make it similar to Htmlfile transport.

It's redundant, the same functionality could be achieved by just running a streaming transport and waiting for a timeout.

http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.7

says that 405 requires Allow header. But this isn't a good header for OPTIONS request:

http://www.w3.org/TR/cors/#access-control-allow-methods-response-he

We should make sure we adhere to specs.

.

We should have a simple but efficient smoke test.

Of course, it's impossible to test all the quirks and details, but it would be nice if we could easily run 1K client connections and look at latency, throughput, cpu and memory usage.

Awesome project and I assume you will be doing this eventually. But it would be useful to know that we could run 0.0.4 tests with, say, this version of sockjs-erlang.

Not a big deal now, when there are few implementations. But when things explode like in the socket.io world, there will be a many-to-many relationship between implementations and specs and it will get crazy to keep track.

Plus, even now, it would still be useful to always know at a glance what version of the spec an implementation is working towards.

And thanks for sockjs!

$ ./venv/bin/python sockjs-protocol-0.1.py
[...snip...]
ImportError: No module named ws4py.client.threadedclient

Fixed with pip --environment=venv install ws4py, but I'm not sure how to integrate it into this project. Should it just be part of the readme?

We can avoid custom framing on websockets connections. This can make further transition out of using SockJS even simpler.

Some websocket implementations can't receive/send binary data over websockets yet:
https://bugzilla.mozilla.org/show_bug.cgi?id=666349

We also need to make that working over fallback protocols.

We have a use case where we have a cluster of sock.js-capable servers behind load balancer and each of nodes need to have 2 endpoints:

1. for websockets transport - without stickyness - handled by our own code
2. for rest of the fallback transports - with stickyness - handled by one of sock.js server implementations in the same node

Is this use case already supported?
I understand, that sock.js javascript client library need to be modified to choose to which endpoint to connect?

cc: @oribrost , @nivertech

Using native websocket connection, on server side, we can receive cookies. That is useful for authorization. We should be able to pass cookies using other transports.

That raises a greater quesiton - how does ws:// behave in cross domain situation.

But still, passing cookies (or something similar) may be usefull.

@mrjoes on the mailing list http://groups.google.com/group/sockjs/browse_thread/thread/709ae905e315e1a6 noticed that once in the sockjs-protocol we do:

    ws.send(u'["a"]')
    self.assertEqual(ws.recv(), u'a["a"]')

and later:

    ws1.send(u'"a"')
    self.assertEqual(ws1.recv(), u'a["a"]')

Xhr-polling uses /xhr url, this is not consistent with other transports.

Hi!

From description:

"It is undefined what happens when two clients share the same session_id" -- a security leak happens.

"It is a client responsibility to choose identifier with enough entropy." -- it opens a way to enter alien session.

Though I must admit browser security is a die-hard, I believe a rethink is needed here to shift session management to server.

Cheers,
--Vladimir

sometimes we use text/plain, sometimes text/plain; charset=utf-8. We should chose one.

Hi,

I've ported sockjs-erlang to cowboy master (https://github.com/spawngrid/sockjs-erlang/tree/cowboy-master). Everything seems to be fine, but I comparing to the master sockjs (that uses much older cowboy) there is one failure:

$ ./venv/bin/python sockjs-protocol-0.3.3.py
............E.......................................................
======================================================================
ERROR: test_synchronous (__main__.Http10)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "sockjs-protocol-0.3.3.py", line 1449, in test_synchronous
    print c.closed()
  File "/Users/yrashk/tmp/sockjs-protocol/utils_03.py", line 245, in closed
    r = self.s.recv(1) == ''
timeout: timed out

I investigated this issue and I can't think of anything but to blame the test suite :). Here's why:

It fails here https://github.com/sockjs/sockjs-protocol/blob/master/sockjs-protocol-0.3.3.py#L1449, I figured out that what happens here is that connection variable is an empty string (I was printing out values)

However, when I run curl I get this:

* Connected to localhost (127.0.0.1) port 8081 (#0)
> GET /echo HTTP/1.1
> User-Agent: curl/7.21.4 (universal-apple-darwin11.0) libcurl/7.21.4 OpenSSL/0.9.8r zlib/1.2.5
> Host: localhost:8081
> Accept: */*
>
< HTTP/1.1 200 OK
< connection: keep-alive
< server: Cowboy
< date: Thu, 25 Oct 2012 21:10:09 GMT
< content-length: 19
< Content-Type: text/plain; charset=UTF-8
<
Welcome to SockJS!

As you can see, it has connection: keep-alive header. When I modify the test-suite to force it in believing it got connection: keep-alive the test passes without a problem.

Any ideas why this might be happening?

There's a significant difference between behaviour of primitive transports such as polling and more advanced transport such as websockets in case of closing connection in user app (see http://pastie.org/pastes/3011573). So whereas websocket tranport do deliver all the messages, primitive transports buffer messages and if user call #close, buffer won't be ever send, hence these messages shall not be delivered.

This behaviour should be tested.

Botanicus in 0c3d4bb suggested that we shouldn't define body for 404. It should be implementation specific.

On the other hand some internal workings do rely on a sane (ie: small and cacheable) 404 response:

• https://github.com/sockjs/sockjs-protocol/blob/master/sockjs-protocol-0.0.4-incomplete.py#L484

Also, #3 is related.