guyzmo / event-source-library Goto Github PK

Currently events are matched with tokens e. g. in buffer_event() we expect target to be set. Is there a way to define 'wildcard' events? Such an event should be dispatched to every connected client regardless of a token.

I have difficulty in understanding the terms 'token' and 'action' from your documentation. Are these used in the same context in listener.py and client.py?

Existing docs only briefly touch the question of integrating with EventSource without use of external post handler. EventSourceHandler.buffer_event() is advised but more elaborate examples are certainly required: how can we construct EventSourceHandler, how can we pass it to tornado's application and to external code which will call buffer_event().

To quote @socketpair from ticket #6 :

also, what is purpose of buffer_event? push() is not blocking anyway. tornado's self.write(), self.flush() are not blocking functions. So, you may just push() events at any time without blocking.

I think it is worth a try to get to a better design.

Hi,

I'm interested in mostly the EventSourceClient part of this library. Why does the constructor build its own url from ssl, url, action, and target parameters, instead of a single url parameter? An SSE server can respond at any location, and it isn't clear what that URL should be.

I guess I'm looking for a generic python EventSource client implementation, and this client is geared towards working with the included corresponding server library which abstracts the concept of 'action' and 'target', right? The CLI interface also won't work with a generic SSE endpoint, since it doesn't take a full URL.

E.g., our EventSource URL endpoints will (eventually) be like: https://stream.wikimedia.org/v2/stream/recentchanges

Would you mind if I forked your EventSourceClient code into a new repository (with credit and link back here), and updated it to be more generic and python 3 compatible?

Thanks!
-Andrew

I get the following error when trying to execute the client side code.

Fatal Python error: pycurl: libcurl link-time version is older than compile-time version
Abort trap: 6

• install event-source-library via pip install
• run server on non-defaul port: eventsource-server -P 8888 -i -k 50000
• try to tun client on default port: eventsource-client 42:42:42:42:42:42 -r 5000 - it will terminate with following error:

Traceback (most recent call last):
  File "/home/msuraev/.local/lib/python2.7/site-packages/eventsource/client.py", line 139, in handle_stream
    (field, value) = line.split(":", 1)
ValueError: need more than 1 value to unpack
ERROR:eventsource.client:HTTP 599: Failed writing body (0 != 877)

Hmm, looking further into this, seems like it's not the port to be blamed - I've got the same error if I run eventsource-server -i -k 50000 instead.

Note, that running both client and the server with -P 8888 fixes it. What's the default port used if -P not specified? In general, I think every option with default value should explicitly mention that value in '--help' output to make troubleshooting easier.

Incidentally, trying to connect with some "empty" port leads to expected error:

eventsource-client 42:42:42:42:42:42 -r 5000 -P 1111 

ERROR:eventsource.client:HTTP 599: Failed to connect to 127.0.0.1 port 1111: Connection refused

So it seems like eventsource-client got confused by smth else running on the default port but I don't understand how it's possible - if the port is already occupied than eventsource-server would have complained about it?

So, write_error() does not work

When running examples outlined in a readme I've got the same output twice from client:

eventsource-client test -r 5000 -P 8888
INFO:eventsource.client:received Event<0,ping,{"B": "XXX"}>
INFO:eventsource.client:received Event<0,ping,{"B": "XXX"}>

Does this mean that there are 2 events have arrived or it's just artefact of debugging?

How many concurrent connected clients were you able to have on one instance?

There are few things which seem dated in the existing documentation:

• it refers to eventsource_listener which it's actually located in ./eventsource/listener.py
• KEEPALIVE setting does not mention unit (seconds? minutes?)
• incompatible examples: server uses -P 8888 setting while client doesn't so trying to follow it will lead to cryptic error:

Traceback (most recent call last):
File "client.py", line 139, in handle_stream
(field, value) = line.split(":", 1)
ValueError: need more than 1 value to unpack
ERROR:eventsource.client:HTTP 599: Failed writing body (0 != 877)

The right entry would be "python client.py 42:42:42:42:42:42 -r 5000 -P 8888"

• both client and requestor use 'token' but it's not clear why it is mandatory
• the r"/(.)/(.)" argument to tornado.web.Application looks mysterious - some reference to what it is would be helpful
• likely a typo: class OneLineIdEvent(OneLineEvent,IdEvent) -> class OneLineIdEvent(OneLineEvent,EventId)

The following error gets thrown when launching eventsource with last version of tornado (3.0.1):

DEBUG:eventsource.listener:get(XXX,poll)
DEBUG:eventsource.listener:set_connected(XXX)
ERROR:tornado.application:Uncaught exception GET /poll/XXX (::1)
HTTPRequest(protocol='http', host='localhost:8000', method='GET', uri='/poll/XXX', version='HTTP/1.1', remote_ip='::1', body='', headers={'Host': 'localhost:8000', 'Content-Type': 'text/event-stream', 'Accept-Encoding': 'gzip,deflate', 'Accept': '*/*', 'User-Agent': 'Mozilla/5.0 (compatible; pycurl)'})
Traceback (most recent call last):
  File ".../event-source-library/eggs/tornado-3.0.1-py2.7.egg/tornado/web.py", line 1055, in _stack_context_handle_exception
    raise_exc_info((type, value, traceback))
  File ".../event-source-library/eggs/tornado-3.0.1-py2.7.egg/tornado/web.py", line 1178, in wrapper
    result = method(self, *args, **kwargs)
  File ".../event-source-library/eventsource/listener.py", line 346, in get
    self._lock[target].get(self._event_loop)
  File ".../event-source-library/eggs/toro-0.5-py2.7.egg/toro/__init__.py", line 186, in get
    future = _TimeoutFuture(deadline, self.io_loop)
  File ".../event-source-library/eggs/toro-0.5-py2.7.egg/toro/__init__.py", line 64, in __init__
    self._timeout_handle = io_loop.add_timeout(deadline, callback)
  File ".../event-source-library/eggs/tornado-3.0.1-py2.7.egg/tornado/ioloop.py", line 681, in add_timeout
    timeout = _Timeout(deadline, stack_context.wrap(callback), self)
  File ".../event-source-library/eggs/tornado-3.0.1-py2.7.egg/tornado/ioloop.py", line 740, in __init__
    raise TypeError("Unsupported deadline %r" % deadline)
TypeError: Unsupported deadline <bound method EventSourceHandler._event_loop of <eventsource.listener.EventSourceHandler object at 0x1f63150>>
ERROR:tornado.access:500 GET /poll/XXX (::1) 2.51ms
DEBUG:eventsource.listener:push_keepalive()
ERROR:tornado.application:Uncaught exception GET /poll/XXX (::1)
HTTPRequest(protocol='http', host='localhost:8000', method='GET', uri='/poll/XXX', version='HTTP/1.1', remote_ip='::1', body='', headers={'Host': 'localhost:8000', 'Content-Type': 'text/event-stream', 'Accept-Encoding': 'gzip,deflate', 'Accept': '*/*', 'User-Agent': 'Mozilla/5.0 (compatible; pycurl)'})
Traceback (most recent call last):
  File ".../event-source-library/eggs/tornado-3.0.1-py2.7.egg/tornado/web.py", line 1055, in _stack_context_handle_exception
    raise_exc_info((type, value, traceback))
  File ".../event-source-library/eggs/tornado-3.0.1-py2.7.egg/tornado/web.py", line 1178, in wrapper
    result = method(self, *args, **kwargs)
  File ".../event-source-library/eventsource/listener.py", line 178, in push_keepalive
    self.write(": keepalive %s\r\n\r\n" % (unicode(time.time())))
  File ".../event-source-library/eggs/tornado-3.0.1-py2.7.egg/tornado/web.py", line 513, in write
    raise RuntimeError("Cannot write() after finish().  May be caused "
RuntimeError: Cannot write() after finish().  May be caused by using async operations without the @asynchronous decorator.
ERROR:tornado.general:Cannot send error response after headers written
DEBUG:eventsource.listener:get(XXX,poll)

Until this issue is solved, I'm setting up a branch that works with up to date tornado, and make
the master branch stuck with tornado 2.2.1, which is the last version of tornado ESL has been tested with.

problems arise at listener.py:346 and at listener.py:178, there's an error on how failure is being dealt.

It will be called on each Tornado's envet loop interation. On heavy-loaded web-servers it eat CPU. This is the problem.

1. Please use collections.deque instead of self._events array (!)
2. require callback from tornado only when queue changes from empty to non-empty
3. stop triggering callback when queue become empty

for ev in self._events[target]:
self._events[target].remove(ev)
yield ev

iterating over array and modifying it - very bad idea. Also, remove() complexity is o(n). very bad.