Comments (7)
ankush
commented on June 2, 2024
Test dependencies are specified in tox, maybe the test can be skipped if sentry_sdk is not installed?
Lines 4 to 11 in 3c05d28
from rq.
kloczek
commented on June 2, 2024
Looks like it would be better IMO to move those dependencies to pryproject.toml 😋
OK looks like sentry is missing in my case. Thx.
I have now build queue clogged so result of add that module to build dependencies will have in 2-3 hours.
from rq.
selwin
commented on June 2, 2024
Test dependencies are already defined in pyproject.toml.
Should I just delete tox.ini? I'm not familiar with tox.
from rq.
kloczek
commented on June 2, 2024
Looks like on opening this ticker I forgot to copy list of installed modules in build env
List of installed modules in build env:
async-timeout 4.0.3
build 1.2.1
certifi 2023.7.22
click 8.1.7
editables 0.5
exceptiongroup 1.1.3
hatchling 1.21.1
importlib_metadata 7.1.0
iniconfig 2.0.0
installer 0.7.0
packaging 24.0
pathspec 0.12.1
pluggy 1.4.0
psutil 5.9.8
pyproject_hooks 1.0.0
pytest 8.1.1
python-dateutil 2.9.0.post0
redis 5.0.3
sentry-sdk 1.44.1
tokenize_rt 5.2.0
tomli 2.0.1
trove-classifiers 2024.4.10
urllib3 2.2.1
wheel 0.43.0
zipp 3.18.1At the moment I have other issue after add missing sentry.
Looks like redis server which I'm starting before star pytest probably has something missing.
I'm saruing it with redis-server --bind 127.0.0.1 --port 6379 --daemonize yes
and almost all units are failing with
> raise response
E redis.exceptions.ResponseError: MISCONF Redis is configured to save RDB snapshots, but it's currently unable to persist to disk. Commands that may modify the data set are disabled, because this instance is configured to report errors during writes if RDB snapshotting fails (stop-writes-on-bgsave-error option). Please check the Redis logs for details about the RDB error.
/usr/lib/python3.10/site-packages/redis/connection.py:536: ResponseErrorBTW I think that you should borrow starting and stopping redis server like it has aioredis https://github.com/aio-libs/aioredis-py/
from rq.
kloczek
commented on June 2, 2024
OK I've removed binding to localhost and all started working.
IMO definitely you should try to start redis like aioredis handles that and such redis should be listening only on localhost. without use system configuration because by default redis complains in my case that it cannot write to system configuration file (/etc/redis/redis.conf).
Also if pytest would be not able to locate redis server binary it would be able gently suggest to install in build env reds server,
In below pytest output you may find some pytest warnings as well.
Here is pytest output:
+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-rq-1.16.1-2.fc37.x86_64/usr/lib64/python3.10/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-rq-1.16.1-2.fc37.x86_64/usr/lib/python3.10/site-packages
+ /usr/bin/pytest -ra -m 'not network' --deselect tests/test_dependencies.py::TestDependencies::test_dependencies_are_met_at_execution_time --deselect tests/test_sentry.py::TestSentry::test_failure_capture
557583:C 13 Apr 2024 12:50:44.063 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
557583:C 13 Apr 2024 12:50:44.063 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
557583:C 13 Apr 2024 12:50:44.063 * Redis version=7.2.4, bits=64, commit=00000000, modified=0, pid=557583, just started
557583:C 13 Apr 2024 12:50:44.063 # Warning: no config file specified, using the default config. In order to specify a config file use /usr/bin/redis-server /path/to/redis.conf
557583:M 13 Apr 2024 12:50:44.064 * Increased maximum number of open files to 10032 (it was originally set to 1024).
557583:M 13 Apr 2024 12:50:44.064 * monotonic clock: POSIX clock_gettime
557583:M 13 Apr 2024 12:50:44.065 * Running mode=standalone, port=6379.
557583:M 13 Apr 2024 12:50:44.066 * Server initialized
557583:M 13 Apr 2024 12:50:44.066 * Ready to accept connections tcp
============================= test session starts ==============================
platform linux -- Python 3.10.14, pytest-8.1.1, pluggy-1.4.0
rootdir: /home/tkloczko/rpmbuild/BUILD/rq-1.16.1
configfile: pyproject.toml
collected 402 items / 2 deselected / 400 selected
tests/test_callbacks.py ............. [ 3%]
tests/test_cli.py ..................................... [ 12%]
tests/test_commands.py ... [ 13%]
tests/test_connection.py .... [ 14%]
tests/test_decorator.py ............... [ 18%]
tests/test_dependencies.py ........ [ 20%]
tests/test_fixtures.py .. [ 20%]
tests/test_helpers.py . [ 20%]
tests/test_job.py ...................................................... [ 34%]
......................... [ 40%]
tests/test_maintenance.py . [ 40%]
tests/test_queue.py .................................................... [ 53%]
[ 53%]
tests/test_registry.py ........................... [ 60%]
tests/test_results.py ............. [ 63%]
tests/test_retry.py ...... [ 65%]
tests/test_scheduler.py ................557583:M 13 Apr 2024 12:51:45.055 * 10000 changes in 60 seconds. Saving...
557583:M 13 Apr 2024 12:51:45.056 * Background saving started by pid 557748
557748:C 13 Apr 2024 12:51:45.061 * DB saved on disk
557748:C 13 Apr 2024 12:51:45.062 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB
557583:M 13 Apr 2024 12:51:45.158 * Background saving terminated with success
.s...... [ 71%]
tests/test_sentry.py .. [ 71%]
tests/test_serializers.py . [ 72%]
tests/test_timeouts.py . [ 72%]
tests/test_utils.py ................. [ 76%]
tests/test_worker.py ....................ss.........s...........s.s..... [ 89%]
................sssss...s..ss... [ 97%]
tests/test_worker_pool.py ....... [ 99%]
tests/test_worker_registration.py .... [100%]
=============================== warnings summary ===============================
rq/connections.py:35: 1 warning
tests/test_cli.py: 31 warnings
tests/test_connection.py: 3 warnings
/home/tkloczko/rpmbuild/BUILD/rq-1.16.1/rq/connections.py:35: DeprecationWarning: The Connection context manager is deprecated. Use the `connection` parameter instead.
warnings.warn(
rq/connections.py:58: 1 warning
tests/test_callbacks.py: 12 warnings
tests/test_cli.py: 33 warnings
tests/test_commands.py: 1 warning
tests/test_connection.py: 4 warnings
tests/test_decorator.py: 1 warning
tests/test_dependencies.py: 18 warnings
tests/test_fixtures.py: 1 warning
tests/test_helpers.py: 1 warning
tests/test_job.py: 1 warning
tests/test_maintenance.py: 1 warning
tests/test_queue.py: 2 warnings
tests/test_registry.py: 8 warnings
tests/test_results.py: 1 warning
tests/test_retry.py: 1 warning
tests/test_scheduler.py: 4 warnings
tests/test_sentry.py: 1 warning
tests/test_timeouts.py: 4 warnings
tests/test_utils.py: 1 warning
tests/test_worker.py: 13 warnings
tests/test_worker_registration.py: 1 warning
/home/tkloczko/rpmbuild/BUILD/rq-1.16.1/rq/connections.py:58: DeprecationWarning: The `push_connection` function is deprecated. Pass the `connection` explicitly instead.
warnings.warn(
rq/connections.py:72: 1 warning
tests/test_callbacks.py: 12 warnings
tests/test_cli.py: 33 warnings
tests/test_commands.py: 1 warning
tests/test_connection.py: 4 warnings
tests/test_decorator.py: 1 warning
tests/test_dependencies.py: 18 warnings
tests/test_fixtures.py: 1 warning
tests/test_helpers.py: 1 warning
tests/test_job.py: 1 warning
tests/test_maintenance.py: 1 warning
tests/test_queue.py: 2 warnings
tests/test_registry.py: 8 warnings
tests/test_results.py: 1 warning
tests/test_retry.py: 1 warning
tests/test_scheduler.py: 4 warnings
tests/test_sentry.py: 1 warning
tests/test_timeouts.py: 4 warnings
tests/test_utils.py: 1 warning
tests/test_worker.py: 13 warnings
tests/test_worker_registration.py: 1 warning
/home/tkloczko/rpmbuild/BUILD/rq-1.16.1/rq/connections.py:72: DeprecationWarning: The `pop_connection` function is deprecated. Pass the `connection` explicitly instead.
warnings.warn(
tests/__init__.py:34
/home/tkloczko/rpmbuild/BUILD/rq-1.16.1/tests/__init__.py:34: PytestUnknownMarkWarning: Unknown pytest.mark.ssl_test - is this a typo? You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
f = pytest.mark.ssl_test(f)
tests/__init__.py:29
/home/tkloczko/rpmbuild/BUILD/rq-1.16.1/tests/__init__.py:29: PytestUnknownMarkWarning: Unknown pytest.mark.slow - is this a typo? You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
f = pytest.mark.slow(f)
tests/test_callbacks.py: 12 warnings
tests/test_decorator.py: 1 warning
/home/tkloczko/rpmbuild/BUILD/rq-1.16.1/rq/job.py:250: DeprecationWarning: Passing a string or function for `on_success` is deprecated, pass `Callback` instead
warnings.warn(
tests/test_callbacks.py: 7 warnings
tests/test_decorator.py: 3 warnings
tests/test_job.py: 1 warning
/home/tkloczko/rpmbuild/BUILD/rq-1.16.1/rq/job.py:260: DeprecationWarning: Passing a string or function for `on_failure` is deprecated, pass `Callback` instead
warnings.warn(
tests/test_callbacks.py::QueueCallbackTestCase::test_enqueue_many_callback
tests/test_callbacks.py::QueueCallbackTestCase::test_enqueue_with_stopped_callback
tests/test_callbacks.py::QueueCallbackTestCase::test_enqueue_with_stopped_callback
tests/test_callbacks.py::QueueCallbackTestCase::test_enqueue_with_stopped_callback
tests/test_callbacks.py::SyncJobCallback::test_stopped_callback
tests/test_callbacks.py::JobCallbackTestCase::test_job_creation_with_stopped_callback
tests/test_job.py::TestJob::test_persistence_of_callbacks
/home/tkloczko/rpmbuild/BUILD/rq-1.16.1/rq/job.py:270: DeprecationWarning: Passing a string or function for `on_stopped` is deprecated, pass `Callback` instead
warnings.warn(
tests/test_callbacks.py: 11 warnings
tests/test_cli.py: 82 warnings
tests/test_connection.py: 4 warnings
tests/test_decorator.py: 25 warnings
tests/test_dependencies.py: 5 warnings
tests/test_job.py: 108 warnings
tests/test_queue.py: 96 warnings
tests/test_registry.py: 2 warnings
tests/test_retry.py: 2 warnings
tests/test_worker.py: 152 warnings
tests/test_worker_registration.py: 7 warnings
/home/tkloczko/rpmbuild/BUILD/rq-1.16.1/rq/connections.py:108: DeprecationWarning: The `resolve_connection` function is deprecated. Pass the `connection` explicitly instead.
warnings.warn(
tests/test_callbacks.py: 14 warnings
tests/test_cli.py: 93 warnings
tests/test_connection.py: 4 warnings
tests/test_decorator.py: 25 warnings
tests/test_dependencies.py: 7 warnings
tests/test_fixtures.py: 1 warning
tests/test_job.py: 120 warnings
tests/test_queue.py: 97 warnings
tests/test_registry.py: 8 warnings
tests/test_results.py: 4 warnings
tests/test_retry.py: 3 warnings
tests/test_worker.py: 241 warnings
tests/test_worker_registration.py: 53 warnings
/home/tkloczko/rpmbuild/BUILD/rq-1.16.1/rq/connections.py:87: DeprecationWarning: The `get_current_connection` function is deprecated. Pass the `connection` explicitly instead.
warnings.warn(
tests/test_callbacks.py: 2 warnings
tests/test_cli.py: 3 warnings
tests/test_job.py: 6 warnings
tests/test_results.py: 2 warnings
tests/test_worker.py: 13 warnings
/home/tkloczko/rpmbuild/BUILD/rq-1.16.1/rq/job.py:860: DeprecationWarning: job.result is deprecated, use job.return_value instead.
warnings.warn("job.result is deprecated, use job.return_value instead.", DeprecationWarning)
tests/test_callbacks.py: 2 warnings
tests/test_job.py: 3 warnings
tests/test_registry.py: 1 warning
tests/test_results.py: 2 warnings
tests/test_timeouts.py: 1 warning
tests/test_worker.py: 4 warnings
/home/tkloczko/rpmbuild/BUILD/rq-1.16.1/rq/job.py:796: DeprecationWarning: job.exc_info is deprecated, use job.latest_result() instead.
warnings.warn("job.exc_info is deprecated, use job.latest_result() instead.", DeprecationWarning)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
SKIPPED [1] tests/test_scheduler.py:381: SSL tests disabled
SKIPPED [1] tests/test_worker.py:333: Slow tests disabled
SKIPPED [1] tests/test_worker.py:311: Slow tests disabled
SKIPPED [1] tests/test_worker.py:683: Slow tests disabled
SKIPPED [1] tests/test_worker.py:964: Slow tests disabled
SKIPPED [1] tests/test_worker.py:708: Slow tests disabled
SKIPPED [1] tests/test_worker.py:1318: Slow tests disabled
SKIPPED [1] tests/test_worker.py:1378: Slow tests disabled
SKIPPED [1] tests/test_worker.py:1402: Slow tests disabled
SKIPPED [1] tests/test_worker.py:1353: Slow tests disabled
SKIPPED [1] tests/test_worker.py:1331: Slow tests disabled
SKIPPED [1] tests/test_worker.py:1503: Slow tests disabled
SKIPPED [1] tests/test_worker.py:1489: Slow tests disabled
SKIPPED [1] tests/test_worker.py:1519: Slow tests disabled
=== 386 passed, 14 skipped, 2 deselected, 1491 warnings in 106.70s (0:01:46) ===
+ /usr/libexec/redis-shutdown
awk: fatal: cannot open file `/etc/redis/redis.conf' for reading: Permission denied
awk: fatal: cannot open file `/etc/redis/redis.conf' for reading: Permission denied
awk: fatal: cannot open file `/etc/redis/redis.conf' for reading: Permission denied
awk: fatal: cannot open file `/etc/redis/redis.conf' for reading: Permission deniedFeel free to close this ticked as main issue of pytest fails has been resolved.
from rq.
Related Issues (20)
- ValueError: Invalid attribute name for job callback HOT 1
- Job.get_status() does not always return JobStatus Enum
- database connection issues when using rq with flask-sqlalchemy HOT 1
- Work-horse termination HOT 1
- DeprecationWarning: datetime.datetime.utcnow() HOT 1
- Rq Worker.all cannot find worker
- Are two connections per worker needed? HOT 1
- keys of command in MULTI calls must be in same slot HOT 3
- How to configure worker-pool via configuration file ? HOT 1
- multiple __init__ , version 1.16 HOT 1
- RQ Tasks Failed Upon Importing Native MacOS modules HOT 3
- Jobs wrongly moved to failed registry (stuck in intermediate queue) HOT 9
- Replace Redis with open source alternative? HOT 5
- How do I log with INFO level from a job?
- no way to catch the redis.exceptions.ConnectionError on pubsub.get_message HOT 4
- Integrating OpenCV with FastAPI and Redis RQ Causes Task/Job Failures HOT 4
- Installing setproctitle >= 1.3.0 breaks RQ on MacOS HOT 1
- Job B depends on job A. A fails. Is there a parameter set that would result in B marked as failed?
- Using `@job` decorator and `func.delay()` causes subclass classmethods to not work correctly HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from rq.