benoitc / restkit Goto Github PK
View Code? Open in Web Editor NEWan HTTP resource kit for Python
Home Page: http://restkit.readthedocs.org
License: Other
an HTTP resource kit for Python
Home Page: http://restkit.readthedocs.org
License: Other
This isn't working out:
pool = restkit.SimplePool()
self.elastic = restkit.Resource("http://localhost:9200/", pool_instance=pool)
for ...
self.elastic.post("project/project", payload=jsonlib.write(d),
headers={'Content-Type': 'application/json'})
After a varying number of posts (seems to be more than 1000), I get this trace.
File "build/bdist.linux-x86_64/egg/restkit/resource.py", line 164, in post
File "build/bdist.linux-x86_64/egg/restkit/resource.py", line 205, in request
File "build/bdist.linux-x86_64/egg/restkit/client.py", line 357, in request
File "build/bdist.linux-x86_64/egg/restkit/client.py", line 387, in do_send
File "build/bdist.linux-x86_64/egg/restkit/client.py", line 202, in make_connection
File "build/bdist.linux-x86_64/egg/restkit/util/sock.py", line 53, in connect
<class 'socket.error'>: (99, 'Cannot assign requested address')
Meanwhile, strace shows that restkit was not reusing the connection, but instead was making new ones, building up about 30 connections before closing them (and then repeating):
...
socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 30
connect(30, {sa_family=AF_INET, sin_port=htons(9200), sin_addr=inet_addr("10.65.8.237")}, 16) = 0
sendto(30, "POST /project/project HTTP/1.1\r\nHost: localhost:9200\r\nUser-Agent: restkit/2.1.0\r\nAccept-Encoding: identity\r\nContent-Length: 225\r\nContent-Type: application/json\r\n\r\n", 159, 0, NULL, 0) = 159
sendto(30, "{...}", 225, 0, NULL, 0) = 225
recvfrom(30, "HTTP/1.1 200 OK\r\nAccess-Control-Allow-Origin: *\r\nContent-Type: application/json; charset=UTF-8\r\nContent-Length: 93\r\n\r\n{\"ok\":true,\"_index\":\"project\",\"_type\":\"project\",\"_id\":\"618cfa96-1054-4aed-bfb1-870852ce81ce\"}", 8192, 0, NULL, NULL) = 211
select(0, NULL, NULL, NULL, {0, 100000}) = 0 (Timeout)
socket(PF_FILE, SOCK_STREAM, 0) = 31
...
Generally, am I supposed to catch socket.timeout
? Would be nice if this was caught in restkit and raised as a restkit.errors.Timeout.
I found this in client.py:323
elif name == "Accept-Encoding":
accept_encoding = 'identity'
I don't know where or why, but in couchdbkit 0.4.10 with restkit 2.1.0 any attempt to do a GET on an auth enabled db will fail because 2 BasicAuth objects are being added on certain requests. I can't trace the logic to find out where, but I've worked around it by modifying lines 202 and 203 from:
# make request
http = HttpConnection(**self.client_opts)
to:
# make request
myfilters = self.client_opts['filters']
if len(myfilters)>1:
print '\terror, more than one filter!!!!'
print 'try to remove one:'
self.client_opts['filters'] = [myfilters[0]]
http = HttpConnection(**self.client_opts)
This must be a showstopper for everyone using 0.4.10 and 2.1.0. I'm surprised unit tests passed.
The except statement at line 424 in client.py catches any exception. That includes KeyboardInterrupt, which means that, if you're using restkit from a command line script, you cannot abort it using Ctrl+C while a request is running.
I would change that line to:
except Exception, e:
if isinstance(e, KeyboardInterrupt): raise
Line 17 in basicauth.py has a 'print tries' line that shouldn't be there.
In def update_uri of the Resource class line 237 should probably be
self.original['uri'] = util.make_uri(self.original['uri'], path,
self.initial['uri'] = util.make_uri(self.initial['uri'], path,
When using restkit through couchdbkit, connecting to a server that isn't actually running couchdb produces a very confusing error message. For instance, this code
import couchdbkit
s = couchdbkit.Server('http://192.168.0.10:5984')
d = s['bar']
print list(d.all_docs())
gives this exception
Traceback (most recent call last):
File "foo.py", line 5, in <module>
print list(d.all_docs())
File "/home/cap/tmp/ve/lib/python2.6/site-packages/couchdbkit/client.py", line 956, in __len__
return self.count()
File "/home/cap/tmp/ve/lib/python2.6/site-packages/couchdbkit/client.py", line 890, in count
self._fetch_if_needed()
File "/home/cap/tmp/ve/lib/python2.6/site-packages/couchdbkit/client.py", line 921, in _fetch_if_needed
self.fetch()
File "/home/cap/tmp/ve/lib/python2.6/site-packages/couchdbkit/client.py", line 903, in fetch
self._result_cache = self.view._exec(**self.params).json_body
File "/home/cap/tmp/ve/lib/python2.6/site-packages/couchdbkit/client.py", line 991, in _exec
return self._db.res.get(self.view_path, **params)
File "/home/cap/tmp/ve/lib/python2.6/site-packages/restkit/resource.py", line 113, in get
params_dict=params_dict, **params)
File "/home/cap/tmp/ve/lib/python2.6/site-packages/couchdbkit/resource.py", line 108, in request
payload=payload, headers=headers, **params)
File "/home/cap/tmp/ve/lib/python2.6/site-packages/restkit/resource.py", line 189, in request
headers=self.make_headers(headers))
File "/home/cap/tmp/ve/lib/python2.6/site-packages/restkit/client.py", line 448, in request
return self.perform(request)
File "/home/cap/tmp/ve/lib/python2.6/site-packages/restkit/client.py", line 406, in perform
raise RequestError("socker.error: %s" % str(e))
restkit.errors.RequestError: socker.error: getaddrinfo returns an empty list
even though getaddrinfo returns correctly and the real problem is that the connection is refused.
We have had to downgrade to 0.9.4 - on that version we get responses from couchdb in 0.02s and with 1.2.x we get 0.3-0.5s. Quite the speed difference.
Raising this ticket as a note for anyone else suffering a similar issue.
This is on Python 2.5 and not using SSL.
The pool.html of the restkit site has a wrong line which is
from restkit.poll.rgevent import GeventPool
It must say pool instead of poll
Is there any roadmap for python 3 support? I have some scripts that I might like to convert to py3k soon but are using restkit. I may be able to help somewhat with the translation, but I'm not sure if I can properly dedicate the time. Since it's pure python (as far as I can tell), it shouldn't take much effort to make it compatible.
if the first item in the list is the one to replace, headers will become invalid by having two identical headers.
l. 152 ; reskit/util/init.py
should be: >= 0 and not > 0
Hi,
I have troubles with restkit, on debian unstable using pip.
# pip install couchapp
# hash -r
# couchapp
...
from restkit.util import oauth2 as oauth
ImportError: cannot import name oauth2
# pip install couchdbkit
# cd $DJANGO_PROJECT
# python manage.py runserver
...
from restkit.util.misc import deprecated_property
ImportError: No module named misc
Looking at the site-package the oauth2 and misc are there. No comprendo...
I think issue #29 is only partially resolved. Forgot to also modify modify the part util.make_uri(self.original['uri'], path
should also be
util.make_uri(self.initial['uri'], path
self.initial['uri'] = util.make_uri(self.original['uri'], path,
self.initial['uri'] = util.make_uri(self.initial['uri'], path,
replace current parser by http-parser (http parsing in C + fallback in python.
(I had to hack resource.py:231 to not raise a new exception, since that was losing the real traceback here.)
File "eggs/restkit-1.3.1-py2.5.egg/restkit/resource.py", line 135, in get
return self.request("GET", path=path, headers=headers, **params)
File "eggs/restkit-1.3.1-py2.5.egg/restkit/resource.py", line 227, in request
headers=headers)
File "eggs/restkit-1.3.1-py2.5.egg/restkit/resource.py", line 175, in do_request
headers=headers)
File "eggs/restkit-1.3.1-py2.5.egg/restkit/client.py", line 324, in request
return self.do_send()
File "eggs/restkit-1.3.1-py2.5.egg/restkit/client.py", line 381, in do_send
return self.start_response()
File "eggs/restkit-1.3.1-py2.5.egg/restkit/client.py", line 466, in start_response
buf2.seek(0)
exceptions.AttributeError: 'bool' object has no attribute 'seek'
That shouldn't be allowed to happen. It seems like a problem with filter_headers sometimes returning a StringIO and sometimes a False. Its docstring is completely wrong about its return types, BTW.
Then I guess client.py:438 happens because the input data ends, but buf2 never got set to some good headers. This is happening when my URL is very long, though I'm still not sure if it's a problem with restkit or the server I'm talking to.
It seems that the library fails to ssl.wrap_socket the connection is no extra ssl_args are provided to Resource.init.
The problem is in restkit.util.sock.connect wich expects nonempty ssl_args to wrap the socket. The ssl_args dict is usually empty and evaluates to false. This would result in making a http request on a ssl connection without making the proper ssl handshake when a resource is initiated with no ssl-specific kwargs.
Moreover this has been hard to identify due to very poor exception handling in the library (discarded exceptions or tracebacks and complete lack of exception logging), see: client.py:413, sock.py:49, resource.py:180
This may be intended, or it may have gone out of sync with time. restkit.request
doesn't take the following options that can be passed on to HttpConnection
: timeout
, force_follow_redirect
, max_follow_redirect
and response_class
.
Perhaps the last three justify having a distinction as more advanced usages that warrant the initialization of a HttpConnection
or Resource
object, but maybe timeout
would be a justified part of this call.
It took a long time to discover that this code is silently ignoring filters who raise an AttributeError:
{{{
for f in self.filters:
try:
func = getattr(f, kind)
func(client)
except AttributeError:
continue
}}}
I don't know what that handler is supposed to be screening for, but it ought to be a lot more specific so it doesn't catch unrelated problems.
It seems like the HttpResponse.closed attribute is supposed to help you raise AlreadyRead to protect against certain programming errors. However, nothing in HttpResponse ever sets self.closed=True, and it's easy to get an empty string back:
In [33]: r = request(url); r.body_string(); r.body_string()
Out[33]: 'html>\nhead>\ntitle>Welcome to nginx!/title>\n/head>\nbody bgcolor="white" text="black">\ncenter>h1>Welcome to nginx!/h1>/center>\n/body>\n/html>\n'
Out[33]: '' # < should be AlreadyRead error?
In [38]: r=Resource(url).get(); r.body; r.body
Out[38]: '..good result..'
Out[38]: ''
Delete
Store a session and cache results on the fs (default backend).
If I lose my network connection by closing my laptop and opening my laptop, after calling get(), and before body_string() returns, the python process will hang.
I create a subclass of resource with a timeout:
Resource.init(self,
url,
filters=[auth],
pool_instance=pool,
client_opts={'timeout':30}
)
I call get:
r = self.get(path, headers, **kwargs)
I close my laptop right around here.
return json.loads(r.body_string())
The process will hang for at least 10 minutes, maybe longer. I added an interrupt handler that starts pdb when you press ^C. Here are the results:
I haven't checked this out in the code or even the http spec, but when I upgraded from pre-1.0 to 1.3.1, some fetches broke because the sender was writing "Content-type: image/jpeg\n\n". Now I have to write "Content-type: image/jpeg\r\n\r\n" for the fetch to succeed.
The error on the client is below. This might be related to my other recent bug. I tried to fetch the latest git version, but my git clone always breaks.
return restkit.Resource(url, timeout=8).get().body
File "/usr/local/lib/python2.6/dist-packages/restkit-1.3.1-py2.6.egg/restkit/resource.py", line 135, in get
return self.request("GET", path=path, headers=headers, **params)
File "/usr/local/lib/python2.6/dist-packages/restkit-1.3.1-py2.6.egg/restkit/resource.py", line 231, in request
raise RequestError(e)
RequestError: 'bool' object has no attribute 'seek'
and in case anyone cares about the git problem:
% git clone http://github.com/benoitc/restkit.git
...
Getting pack 7a4a7ab2bc26d5f09d360b543a2983ca21ba0e04
which contains 0f4d515dbe14e426d52b4456007a3c796071c7fd
error: Unable to get pack file http://github.com/benoitc/restkit.git/objects/pack/pack-7a4a7ab2bc26d5f09d360b543a2983ca21ba0e04.pack
transfer closed with 1804158 bytes remaining to read
error: Unable to find 0f4d515dbe14e426d52b4456007a3c796071c7fd under http://github.com/benoitc/restkit.git
Cannot obtain needed object 0f4d515dbe14e426d52b4456007a3c796071c7fd
while processing commit 46f166e06f3b14672bfe448e19160e920cf4e93e.
fatal: Fetch failed.
lip6 implementation
I've been running using restkit 2.1.1 for a week or so and everything has ticked over nicely, but when I upgrade to restkit 2.1.5 I begin to get this type of error (on the second GET request, never the first, in a session):
There a typo there :
restkit/conn/gevent_manager.py", line 54, in release
if self.connections.qsize() >= self.nb_connecions:
It should be self.nb_connections.
needs s/OAuthfilter/OAuthFilter/
I have an authentication module (for google appengine) that uses urllib2 (by installing an opener).
The project that this replaced (py-restclient) claimed to have urllib2 support in its online documentation, but I couldn't find any released version of the library that actually had the classes from that documentation.
Is it possible to use restkit with urllib2?
If your request have a Range header the whole body don't need to be read and WSGI server should .close() the output stream
TeeInput must have a close() method and stop streaming the response if needed
Assuming I'm not doing something wrong ... when I use oauth2 with query parameters (e.g. ?foo=bar&foo2=bar2) each of the parameters are duplicated in the final URL, which then fails authorization.
The reason for this appears to be that the params already exist in req.url but are then duplicated in the params argument passed to Request.from_consumer_and_token in OAuthFilter.on_request.
A fix that works for me (but may not be the best) is to modify OAuthFilter.on_request to strip the query params from req.uri as follows:
req_url_noargs = urlparse.urlunparse((req.uri[0], req.uri[1], req.uri[2], req.uri[3], '', ''))
oauth_req = Request.from_consumer_and_token(self.consumer,
token=self.token, http_method=req.method,
http_url=req_url_noargs, parameters=params)
The preceding fix strips both the query (and fragment) components from the ParseResult object and reforms the URL using urlparse.urlunparse. The resulting output works and authorizes properly. There may, of course, be a better solution.
====================================================================== ERROR: tests.010-test-proxies.test_001 ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib64/python2.7/site-packages/nose/case.py", line 187, in runTest self.test(*self.arg) File "/var/tmp/paludis/dev-python-restkit-2.3.3/work/restkit-2.3.3/tests/010-test-proxies.py", line 15, in wrapper return func(req) File "/var/tmp/paludis/dev-python-restkit-2.3.3/work/restkit-2.3.3/tests/010-test-proxies.py", line 23, in test_001 resp = req.get_response(proxy) File "/usr/lib64/python2.7/site-packages/webob/request.py", line 935, in get_response application, catch_exc_info=False) File "/usr/lib64/python2.7/site-packages/webob/request.py", line 904, in call_application app_iter = application(self.environ, start_response) File "/var/tmp/paludis/dev-python-restkit-2.3.3/work/restkit-2.3.3/restkit/contrib/wsgi_proxy.py", line 86, in __call__ pool_instance=self.pool) File "/var/tmp/paludis/dev-python-restkit-2.3.3/work/restkit-2.3.3/restkit/__init__.py", line 94, in request headers=headers) File "/var/tmp/paludis/dev-python-restkit-2.3.3/work/restkit-2.3.3/restkit/client/request.py", line 220, in request self.set_body(body, found_headers, chunked=chunked) File "/var/tmp/paludis/dev-python-restkit-2.3.3/work/restkit-2.3.3/restkit/client/request.py", line 154, in set_body content_length = str(os.fstat(body.fileno())[6]) UnsupportedOperation: fileno ====================================================================== ERROR: tests.010-test-proxies.test_004 ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib64/python2.7/site-packages/nose/case.py", line 187, in runTest self.test(*self.arg) File "/var/tmp/paludis/dev-python-restkit-2.3.3/work/restkit-2.3.3/tests/010-test-proxies.py", line 15, in wrapper return func(req) File "/var/tmp/paludis/dev-python-restkit-2.3.3/work/restkit-2.3.3/tests/010-test-proxies.py", line 62, in test_004 resp = req.get_response(proxy) File "/usr/lib64/python2.7/site-packages/webob/request.py", line 935, in get_response application, catch_exc_info=False) File "/usr/lib64/python2.7/site-packages/webob/request.py", line 904, in call_application app_iter = application(self.environ, start_response) File "/var/tmp/paludis/dev-python-restkit-2.3.3/work/restkit-2.3.3/restkit/contrib/wsgi_proxy.py", line 86, in __call__ pool_instance=self.pool) File "/var/tmp/paludis/dev-python-restkit-2.3.3/work/restkit-2.3.3/restkit/__init__.py", line 94, in request headers=headers) File "/var/tmp/paludis/dev-python-restkit-2.3.3/work/restkit-2.3.3/restkit/client/request.py", line 220, in request self.set_body(body, found_headers, chunked=chunked) File "/var/tmp/paludis/dev-python-restkit-2.3.3/work/restkit-2.3.3/restkit/client/request.py", line 154, in set_body content_length = str(os.fstat(body.fileno())[6]) UnsupportedOperation: fileno ====================================================================== ERROR: tests.010-test-proxies.test_005 ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib64/python2.7/site-packages/nose/case.py", line 187, in runTest self.test(*self.arg) File "/var/tmp/paludis/dev-python-restkit-2.3.3/work/restkit-2.3.3/tests/010-test-proxies.py", line 15, in wrapper return func(req) File "/var/tmp/paludis/dev-python-restkit-2.3.3/work/restkit-2.3.3/tests/010-test-proxies.py", line 71, in test_005 resp = req.get_response(proxy) File "/usr/lib64/python2.7/site-packages/webob/request.py", line 935, in get_response application, catch_exc_info=False) File "/usr/lib64/python2.7/site-packages/webob/request.py", line 904, in call_application app_iter = application(self.environ, start_response) File "/var/tmp/paludis/dev-python-restkit-2.3.3/work/restkit-2.3.3/restkit/contrib/wsgi_proxy.py", line 86, in __call__ pool_instance=self.pool) File "/var/tmp/paludis/dev-python-restkit-2.3.3/work/restkit-2.3.3/restkit/__init__.py", line 94, in request headers=headers) File "/var/tmp/paludis/dev-python-restkit-2.3.3/work/restkit-2.3.3/restkit/client/request.py", line 220, in request self.set_body(body, found_headers, chunked=chunked) File "/var/tmp/paludis/dev-python-restkit-2.3.3/work/restkit-2.3.3/restkit/client/request.py", line 154, in set_body content_length = str(os.fstat(body.fileno())[6]) UnsupportedOperation: fileno ====================================================================== ERROR: tests.010-test-proxies.test_006 ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib64/python2.7/site-packages/nose/case.py", line 187, in runTest self.test(*self.arg) File "/var/tmp/paludis/dev-python-restkit-2.3.3/work/restkit-2.3.3/tests/010-test-proxies.py", line 15, in wrapper return func(req) File "/var/tmp/paludis/dev-python-restkit-2.3.3/work/restkit-2.3.3/tests/010-test-proxies.py", line 84, in test_006 resp = req.get_response(proxy) File "/usr/lib64/python2.7/site-packages/webob/request.py", line 935, in get_response application, catch_exc_info=False) File "/usr/lib64/python2.7/site-packages/webob/request.py", line 904, in call_application app_iter = application(self.environ, start_response) File "/var/tmp/paludis/dev-python-restkit-2.3.3/work/restkit-2.3.3/restkit/contrib/wsgi_proxy.py", line 86, in __call__ pool_instance=self.pool) File "/var/tmp/paludis/dev-python-restkit-2.3.3/work/restkit-2.3.3/restkit/__init__.py", line 94, in request headers=headers) File "/var/tmp/paludis/dev-python-restkit-2.3.3/work/restkit-2.3.3/restkit/client/request.py", line 220, in request self.set_body(body, found_headers, chunked=chunked) File "/var/tmp/paludis/dev-python-restkit-2.3.3/work/restkit-2.3.3/restkit/client/request.py", line 154, in set_body content_length = str(os.fstat(body.fileno())[6]) UnsupportedOperation: fileno ====================================================================== ERROR: tests.010-test-proxies.test_007 ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib64/python2.7/site-packages/nose/case.py", line 187, in runTest self.test(*self.arg) File "/var/tmp/paludis/dev-python-restkit-2.3.3/work/restkit-2.3.3/tests/010-test-proxies.py", line 15, in wrapper return func(req) File "/var/tmp/paludis/dev-python-restkit-2.3.3/work/restkit-2.3.3/tests/010-test-proxies.py", line 93, in test_007 resp = req.get_response(proxy) File "/usr/lib64/python2.7/site-packages/webob/request.py", line 935, in get_response application, catch_exc_info=False) File "/usr/lib64/python2.7/site-packages/webob/request.py", line 904, in call_application app_iter = application(self.environ, start_response) File "/var/tmp/paludis/dev-python-restkit-2.3.3/work/restkit-2.3.3/restkit/contrib/wsgi_proxy.py", line 86, in __call__ pool_instance=self.pool) File "/var/tmp/paludis/dev-python-restkit-2.3.3/work/restkit-2.3.3/restkit/__init__.py", line 94, in request headers=headers) File "/var/tmp/paludis/dev-python-restkit-2.3.3/work/restkit-2.3.3/restkit/client/request.py", line 220, in request self.set_body(body, found_headers, chunked=chunked) File "/var/tmp/paludis/dev-python-restkit-2.3.3/work/restkit-2.3.3/restkit/client/request.py", line 154, in set_body content_length = str(os.fstat(body.fileno())[6]) UnsupportedOperation: fileno ====================================================================== ERROR: tests.010-test-proxies.test_008 ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib64/python2.7/site-packages/nose/case.py", line 187, in runTest self.test(*self.arg) File "/var/tmp/paludis/dev-python-restkit-2.3.3/work/restkit-2.3.3/tests/010-test-proxies.py", line 15, in wrapper return func(req) File "/var/tmp/paludis/dev-python-restkit-2.3.3/work/restkit-2.3.3/tests/010-test-proxies.py", line 103, in test_008 resp = req.get_response(proxy) File "/usr/lib64/python2.7/site-packages/webob/request.py", line 935, in get_response application, catch_exc_info=False) File "/usr/lib64/python2.7/site-packages/webob/request.py", line 904, in call_application app_iter = application(self.environ, start_response) File "/var/tmp/paludis/dev-python-restkit-2.3.3/work/restkit-2.3.3/restkit/contrib/wsgi_proxy.py", line 86, in __call__ pool_instance=self.pool) File "/var/tmp/paludis/dev-python-restkit-2.3.3/work/restkit-2.3.3/restkit/__init__.py", line 94, in request headers=headers) File "/var/tmp/paludis/dev-python-restkit-2.3.3/work/restkit-2.3.3/restkit/client/request.py", line 220, in request self.set_body(body, found_headers, chunked=chunked) File "/var/tmp/paludis/dev-python-restkit-2.3.3/work/restkit-2.3.3/restkit/client/request.py", line 154, in set_body content_length = str(os.fstat(body.fileno())[6]) UnsupportedOperation: fileno ---------------------------------------------------------------------- Ran 96 tests in 2.964s FAILED (errors=6)
on Gentoo Linux, with python-2.7 and WebOb-1.0.1.
It seems that the availability of a fileno method does not guarantee that the underlying object really supports it.
I ran into this when trying to access the Flickr API via restkit. One of the two required query parameters to Flickr's API is 'method'. Of course, the Resource.request() method's first parameter is named 'method' so that's a no-go. Python spits out a big error:
Traceback (most recent call last):
File "./images.py", line 32, in
res = f.search('obama', 1)
File "./images.py", line 22, in search
return self.get(*_params)
File "/home/sadpengu/sandbox/pylons/lib/python2.6/site-packages/restkit-2.2.2-py2.6.egg/restkit/resource.py", line 135, in get
return self.request("GET", path=path, headers=headers, *_params)
File "./images.py", line 25, in request
res = Resource.request(self, _method, payload, headers, **params).body_string()
TypeError: request() got multiple values for keyword argument 'method'
The easy fix is to rename 'method' to '_method' on lines 181 and 202 of resource.py. I suppose you'd want to go ahead and rename 'path', 'payload', and 'headers' while you're at it. I've done so in my local copy just to get it to work.
However, this is more of a hack than a long-term fix, as this wouldn't help if you happened to run across a site that requires '_method' as one of the query parameters. What would really be needed in this case is a way to pass a plain-old dict of query parameters instead of passing them as arguments, as convenient as that is.
This is the ca_certs parameter in ssl.SSLSocket:
The ca_certs file contains a set of concatenated “certification authority” certificates, which are used to validate certificates passed from the other end of the connection.
This also requires adding this to the provided < 2.6 SSLSocket in sock.py for backwards compatibility.
Hello,
I noticed that some basic auth base64 encoding don't work well using : base64.encodestring
this is a very rare case but sometime the encoded string has a linebreak :
http://github.com/benoitc/restkit/blob/master/restkit/filters.py#L46
Using the following code fixed the issue:
encode = base64.b64encode("%s:%s" % self.credentials)
Regards,
Xav
HTTPS support appears broken. Tested using 1.1.2 and against master:
>>> from restkit import request
>>> resp = request('https://api.twilio.com')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "restkit/__init__.py", line 91, in request
headers=headers)
File "restkit/client.py", line 348, in request
return self.do_send(req_headers, self.body, chunked)
File "restkit/client.py", line 354, in do_send
s = self.make_connection()
File "restkit/client.py", line 190, in make_connection
self.key_file, self.cert_file)
AttributeError: 'HttpConnection' object has no attribute 'key_file'
Hi,
in the actual version 3.0.2 the http proxy support is missing in SimpleProxy. It is a bug or what i'm missing?
regards,
Chris
This happens often, but random when trying to fetch existing documents from couchdb. I can't see any pattern, sometimes it works and sometimes it throws this error.
In couchdb, there is nothing unusual logged. Just the normal get requests.
I'm using the latest restkit/couchdbkit on couchdb 0.10.1
URL: http://localhost:5000/a860bb8f9175a79de4593f64663178d2/company/chart_of_accounts
File '/home/loxs/pylons/lib/python2.6/site-packages/WebError-0.10.2-py2.6.egg/weberror/evalexception.py', line 431 in respond
app_iter = self.application(environ, detect_start_response)
File '/home/loxs/pylons/lib/python2.6/site-packages/Beaker-1.5.1-py2.6.egg/beaker/middleware.py', line 73 in call
return self.app(environ, start_response)
File '/home/loxs/pylons/lib/python2.6/site-packages/Beaker-1.5.1-py2.6.egg/beaker/middleware.py', line 152 in call
return self.wrap_app(environ, session_start_response)
File '/home/loxs/pylons/lib/python2.6/site-packages/Routes-1.11-py2.6.egg/routes/middleware.py', line 130 in call
response = self.app(environ, start_response)
File '/home/loxs/pylons/lib/python2.6/site-packages/Pylons-1.0b1-py2.6.egg/pylons/wsgiapp.py', line 107 in call
response = self.dispatch(controller, environ, start_response)
File '/home/loxs/pylons/lib/python2.6/site-packages/Pylons-1.0b1-py2.6.egg/pylons/wsgiapp.py', line 304 in dispatch
return controller(environ, start_response)
File '/home/loxs/kontado/kontado/trunk/kontado/kontado/lib/base.py', line 34 in call
return WSGIController.call(self, environ, start_response)
File '/home/loxs/pylons/lib/python2.6/site-packages/Pylons-1.0b1-py2.6.egg/pylons/controllers/core.py', line 207 in call
response = self._inspect_call(self.before)
File '/home/loxs/pylons/lib/python2.6/site-packages/Pylons-1.0b1-py2.6.egg/pylons/controllers/core.py', line 105 in _inspect_call
result = self._perform_call(func, args)
File '/home/loxs/pylons/lib/python2.6/site-packages/Pylons-1.0b1-py2.6.egg/pylons/controllers/core.py', line 57 in _perform_call
return func(**args)
File '/home/loxs/kontado/kontado/trunk/kontado/kontado/lib/base.py', line 23 in before
c.company = Company.get(company_id)
File '/home/loxs/pylons/lib/python2.6/site-packages/couchdbkit-0.4.4-py2.6.egg/couchdbkit/schema/base.py', line 476 in get
return cls._db.get(docid, rev=rev, wrapper=cls.wrap)
File '/home/loxs/pylons/lib/python2.6/site-packages/couchdbkit-0.4.4-py2.6.egg/couchdbkit/client.py', line 351 in get
doc = maybe_raw(self.res.get(docid), raw=_raw_json)
File 'build/bdist.linux-i686/egg/restkit/resource.py', line 132 in get
File '/home/loxs/pylons/lib/python2.6/site-packages/couchdbkit-0.4.4-py2.6.egg/couchdbkit/resource.py', line 138 in request
payload=body, headers=headers, **params)
File 'build/bdist.linux-i686/egg/restkit/resource.py', line 231 in request
AttributeError: 'NoneType' object has no attribute 'status_int'
Python’s urlparse
splits a URL path into 4 parts: path
, params
, query
, and fragment
. Restkit silently drops the second one, params
, when it reconstructs the URL to send to the server:
req_path = urlparse.urlunparse(('','', path, '',
self.uri.query, self.uri.fragment))
The above (line 265 in client/request.py), should be:
req_path = urlparse.urlunparse(('','', path, self.uri.params,
self.uri.query, self.uri.fragment))
Python 2.5.2 (335e875cb0fe, Dec 28 2010, 20:31:56)
[PyPy 1.4.1] on win32
Type "help", "copyright", "credits" or "license" for more information.
And now for something completely different: ``it's duct tape all the way down
but not the way up''
>>>> from restkit import *
>>>> c = Client(use_proxy=True)
>>>> c.request("http://www.goggle.com.gt")
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "restkit\client.py", line 448, in request
return self.perform(request)
File "restkit\client.py", line 323, in perform
connection = self.get_connection(request)
File "restkit\client.py", line 213, in get_connection
sck, addr, extra_headers = self.proxy_connection(request, addr, ssl)
NameError: global name 'ssl' is not defined
This fixed it up for me:
diff --git a/restkit/client.py b/restkit/client.py
index 7a4bb42..66011b9 100644
--- a/restkit/client.py
+++ b/restkit/client.py
@@ -210,7 +210,7 @@ class Client(object):
extra_headers = []
sck = None
if self.use_proxy:
- sck, addr, extra_headers = self.proxy_connection(request, addr, ssl)
+ sck, addr, extra_headers = self.proxy_connection(request, addr, is_ssl)
if not sck:
sck = self._manager.find_socket(addr, is_ssl)
if sck is None:
UI allowing an admin to browse the sessions. Session can be find by user/date or session id.
In client.py
def do_send(self):
tries = 2
while True:
try:
# apply on request filters
for bf in self.request_filters:
bf.on_request(self)
# get socket
self._sock = self.make_connection()
Notice that make_connection is called after filters are applied. This is needed for SimpleProxy since it redirects the request host. Not sure if this breaks other filters or any other functionality.
I am paging through a view where all of the keys and values are null. Here's the python code that generates the view.
def map(doc):
if doc['doc_type']=="Tweet" and doc.get('plc'):
yield None, None
This view finds around .5 million out of 30 million tweets, so I need to paginate the results. The problem is that CouchDB ignores startkey_docid if you omit the startkey parameter. The following urls give different results:
http://localhost:5984/hou/_design/tweet/_view/plc?startkey_docid=T12&limit=3
returns the first three documents in the view.
http://localhost:5984/hou/_design/tweet/_view/plc?startkey=null&startkey_docid=T12&limit=3
returns the first three documents starting from T12
However, there isn't a way to set startkey to null with couchdbkit. encode_params in resource.py removes startkey if it is None and runs it through json.dumps if it is not.
db.view()
If my url is 1000 chars long (lots of query params), this line in client.py assembles a 1000+ char new string and passes it to log.info:
log.info('Start request: %s %s' % (self.method, self.url))
If I turn off the INFO logging, that line is a waste. Instead, you can use this form:
log.info('Start request: %s %s', self.method, self.url)
That way, the big strings are simply passed (no copies), and the logging library decides if it needs to interpolate the string for a log or not.
Similarly, log.debug("Request headers: [%s]" % str(req_headers)) can become log.debug("Request headers: [%s]", req_headers), since the %s formatter will perform a str() operation anyway.
Allows someone to stop, restart or kill a session. (based on ACL and secure cookie)
Here is a snippet of my code:
log.debug('======================== before proxy =======================')
log.debug(request.params)
req = Request(request.environ)
req.environ['PATH_INFO'] = '/products/studio/storeOrderInfo'
req.environ['CONTENT_LENGTH'] = str(len(req.body))
req.environ['QUERY_STRING'] = 'number=' + str(request.params.get('studioId'))
req.content_length = str(len(req.body))
resp = req.get_response(glob.ibProductsProxy)
studioDoc = json.loads(resp.body)
log.debug('========================== after proxy =====================')
log.debug(request.params)
And here is what it outputs to the log:
10:37:34,641 DEBUG [invoices.controllers.studio] ======================== before proxy =======================
10:37:34,641 DEBUG [invoices.controllers.studio] UnicodeMultiDict([('studioId', u'1103125'), ('docLang', u'bg'), ('docId', u'0aa9706ac565df58037b17b229189386')])
10:37:34,642 INFO [restkit.client] Start request: POST http://work.office.n7/products/studio/storeOrderInfo?number=1103125
10:37:34,668 DEBUG [invoices.controllers.studio] ========================== after proxy =====================
10:37:34,669 DEBUG [invoices.controllers.studio] UnicodeMultiDict([('number', u'1103125')])
As you see, my pylons request object gets modified by the proxy.
restkit need a new http connection handler'
If you wonder what is HostProxy, here it is:
http://github.com/benoitc/restkit/blob/master/restkit/contrib/wsgi_proxy.py#L141
Here is my code:
http://friendpaste.com/YP3SIiapVkLisdzNBbjhw
And here is the output to the log file:
http://friendpaste.com/BKKosZp5Iq3eBghA09AJw
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.