beanstalkd / pybeanstalk Goto Github PK

As suggested by @mahmoudimus; see https://pypi.python.org/pypi/tox.

• Python 3.5.1
• Windows 7 & 10 (but I think this is reproductible on any Windows version)

When importing the pybeanstalk package in any possible manner, the following exception is raised:

Traceback (most recent call last):
  File "local_server.py", line 1, in <module>
    from war import app
  File "C:\war\war.py", line 3, in <module>
    from beanstalk import job
  File "C:\{SomeFolders}\Python35\lib\site-packages\beanstalk\__init__.py", line 1, in <module>
    import serverconn
ImportError: No module named 'serverconn'

By "any possible manner", I'm mean I tried the following import statements in my C:\war\war.py file:

import beanstalk
from beanstalk import job, serverconn
from beanstalk import serverconn

I don't understand what is the issue. Maybe this is related to relative imports in Python 3?

Hi,

I get an error when trying to run the example simple_clients.py using beanstalkd 1.4.4:

kangaroo:examples cbare$ python simple_clients.py producer 127.0.0.1
handling args
setting up connection
No handlers could be found for logger "beanstalk.serverconn"
starting producer loop
This is data to be consumed (0)!
usage: example.py TYPE server [port]
TYPE is one of: [producer|consumer]
Traceback (most recent call last):
File "simple_clients.py", line 53, in
main()
File "simple_clients.py", line 41, in main
producer_main(connection)
File "simple_clients.py", line 15, in producer_main
data.Queue()
File "/Library/Python/2.6/site-packages/beanstalk/job.py", line 107, in Queue
self.Server.put(self._serialize(), self.pri, self.delay, self.ttr)
File "/Library/Python/2.6/site-packages/beanstalk/serverconn.py", line 51, in caller
return self._do_interaction(_res(_args, **kw))
File "/Library/Python/2.6/site-packages/beanstalk/serverconn.py", line 99, in _do_interaction
return self._get_response(handler)
File "/Library/Python/2.6/site-packages/beanstalk/serverconn.py", line 90, in _get_response
res = handler(recv)
File "/Library/Python/2.6/site-packages/beanstalk/protohandler.py", line 133, in call
return self.h(val)
File "/Library/Python/2.6/site-packages/beanstalk/protohandler.py", line 153, in handler
checkError(response)
File "/Library/Python/2.6/site-packages/beanstalk/errors.py", line 37, in checkError
raise err
beanstalk.errors.ExpectedCrlf: Server returned: EXPECTED_CRLF
Exception beanstalk.errors.UnexpectedResponse: UnexpectedResponse('Response was: WN_COMMAND ',) in <bound method Job.__del of <beanstalk.job.Job object at 0x101163890>> ignored

These are supported in beanstalkd since a long time ago:

beanstalkd/beanstalkd#24

The test test_MultiServerConn.py produces the following output a number several times (on Debian stable, newest Python 2.x.x with beanstalkd from debian pkg):

Exception ServerInUse: ServerInUse('AsyncServerConn(127.0.0.1:11303#[Open][Waiting]) is currently in use!', <0x139986747569216 AsyncServerConn(127.0.0.1:11303#[Open][Waiting])>) in method __del__ of <beanstalk.job.Job object at 0x0000000001ea3298> ignored

It's always the connection to 11303, with a different Job object each time.

There's also one error and one failure:

ERROR: test_MultiServerConn.test_tube_operations
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/pythonapps/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/media/psf/Code/thirdparty/pybeanstalk/tests/test_MultiServerConn.py", line 296, in test_tube_operations
    "Was expecting %s, got %s" % (expecting,
NameError: global name 'expecting' is not defined

======================================================================
FAIL: test_MultiServerConn.test_reserve_timeout_works
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/pythonapps/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/media/psf/Code/thirdparty/pybeanstalk/tests/test_MultiServerConn.py", line 323, in test_reserve_timeout_works
    assert conn.stats()['data']['current-jobs-ready'] == 0, "The server is not empty "\
AssertionError: The server is not empty of jobs so test behaviour cannot be guaranteed.  Bailing out.

How about https://travis-ci.org ? @kr, to set it up for pybeanstalk, Travis CI needs permission to access beanstalk organization repositories; hope you're ok with that.

Currently, the Twisted client tests cannot be run using nose. Help would be appreciated; it probably just needs some debugging by someone who knows Twisted and nose better than I.

Another alternative would be switching to pytest for all tests, under which I know I can make the Twisted client tests run.

Currently, none exist.

Looks like this repository is not that much maintained, and there are many forks with some improvements.

It would be good if we could roll the pybeanstalk improvements into one "canonical" repository and publish the package on PyPI.

@sophacles , would it be ok for you to move the repository to https://github.com/beanstalkd and encourage forkers maybe join the organization as well? I could ask Keith Rarick @kr (author of beanstalkd) if it would be ok for him.

Twisted 15.2.0 introduced a new improved logging framework. Make the pybeanstalk Twisted client use it.

In addition to people already here, there are forks by @dustin, @pragnesh, @AlekSi, @rubenk, @gerry, @jasiek @labjack, @kaulie, @wuguobao and @ZoomQuiet.

This issue is an attempt to reach them via GitHub mentions and let them know that pybeanstalk has been transferred to the beanstalkd GitHub organization, and, that issues and pull requests are most welcome!

Please, if your GitHub username is on the above list and you see this, comment on this issue and let us know what you'd like to see happen to pybeanstalk - and whether you'd perhaps even like to join the GitHub organization to help improve pybeanstalk.

In _get_response, error msg refers to server.ip and server.port, which do not exist. They should instead refer to self.server and self.port.

Special method del is called by Python when the last reference of the item is being destroyed. See https://docs.python.org/2/reference/datamodel.html#object.__del__.

Given that, I don't understand why there's del implementation at https://github.com/beanstalkd/pybeanstalk/blob/master/beanstalk/job.py#L65-L66 that calls self.Finish, in other words the Job is trying to delete itself from beanstalk server???

I can understand someone might somewhat mistakenly try to support deleting a job by del job, but this is dangerous: an application may have more references hanging around when del is called, and on the other hand, there are cases where the reference ends up destroyed inadvertently.

Apparently this has been noticed before as well...

I am not aware of pybeanstalk using the __del__ implementation internally. I will remove the __del__ implementation unless someone objects for a good reason.

The BeanStalk client class uses __getattr__ to call protocol implementations. This however obfuscates/breaks access to Python __nonzero__and zope.interface __conform__ special methods of the BeanStalk instance passed as the conn parameter to the job.Job constructor. This results in an error when trying to use job.Job with Twisted.

It would be convenient if the tests could be run without having to descend to tests/ . Right now it's not possible because of how config.py looks for the tests.cfg file.

@sophacles , @jamescasbon,

Could you please help in documenting how the tests are to be run? I just tried to run the tests by going to the tests directory and running nosetests. It looks like that is the way to go, but I got some errors.

I am not really familiar with nose, but taking a look at some of the files, I understand some of the tests require a live beanstalkd server (or more than one server). That caused at least some of the errors, perhaps even all of them.

Perhaps we could split the tests into three groups (at least in the documentation):

• those that require no live server
• those that need one server
• those that need more than one

So we could ask pull request submitters to provide and run tests without forcing them to set up multiple beanstalk servers just because some tests require them.

This should be easy by using nose to run just some of the tests, either manually by module or test name, or perhaps using some nose-specific criteria for grouping?

update package structure, switch to setuptools, etc.