Comments (16)
jape-dev
commented on May 29, 2024
1
Just wanted to post this in case anyone finds this thread looking for a solution to the same issue.
From celery docs
Beat needs to store the last run times of the tasks in a local database file (named celerybeat-schedule by default), so it needs access to write in the current directory, or alternatively you can specify a custom location for this
For some reason my application cannot read the celerybeat-schedule file. But there is a work around to store the active schedule in Redis instead. There a few packages that allow you to do this. I tried RedBeat and it worked 🎉
from build-a-saas-app-with-flask.
nickjj
commented on May 29, 2024
Hi,
Did you make any modifications to CELERYBEAT_SCHEDULE , and did this originally work using the course's source code where the beat server was integrated with the worker using -B?
Also in the future, can you use the course platform's discussion board or email me directly for support on the course? It's just easier to keep track of things, and others will be able to check out the answer to help avoid duplicates. Not too many folks use github issues for reporting this course's issues since the app in this repo isn't the same in the course.
from build-a-saas-app-with-flask.
jape-dev
commented on May 29, 2024
Hi @nickjj ,
Yes, I've left the CELERYBEAT_SCHEDULE config as is. Yes, I originally tried using the integrated approach with the -B arg. I get a similar traceback:
File "/usr/local/lib/python3.7/site-packages/kombu/utils/objects.py", line 42, in __get__
celery_1 | return obj.__dict__[self.__name__]
celery_1 | KeyError: 'scheduler'
celery_1 |
celery_1 | During handling of the above exception, another exception occurred:
celery_1 |
celery_1 | Traceback (most recent call last):
celery_1 | File "/usr/local/lib/python3.7/site-packages/celery/beat.py", line 485, in setup_schedule
celery_1 | self._store = self._open_schedule()
celery_1 | File "/usr/local/lib/python3.7/site-packages/celery/beat.py", line 475, in _open_schedule
celery_1 | return self.persistence.open(self.schedule_filename, writeback=True)
celery_1 | File "/usr/local/lib/python3.7/shelve.py", line 243, in open
celery_1 | return DbfilenameShelf(filename, flag, protocol, writeback)
celery_1 | File "/usr/local/lib/python3.7/shelve.py", line 227, in __init__
celery_1 | Shelf.__init__(self, dbm.open(filename, flag), protocol, writeback)
celery_1 | File "/usr/local/lib/python3.7/dbm/__init__.py", line 94, in open
celery_1 | return mod.open(file, flag, mode)
celery_1 | _gdbm.error: [Errno 22] Invalid argument
celery_1 | [2020-05-24 20:40:38,998: ERROR/Beat] Process Beat
celery_1 | Traceback (most recent call last):
celery_1 | File "/usr/local/lib/python3.7/site-packages/kombu/utils/objects.py", line 42, in __get__
celery_1 | return obj.__dict__[self.__name__]
celery_1 | KeyError: 'scheduler'
celery_1 |
celery_1 | During handling of the above exception, another exception occurred:
celery_1 |
celery_1 | Traceback (most recent call last):
celery_1 | File "/usr/local/lib/python3.7/site-packages/celery/beat.py", line 485, in setup_schedule
celery_1 | self._store = self._open_schedule()
celery_1 | File "/usr/local/lib/python3.7/site-packages/celery/beat.py", line 475, in _open_schedule
celery_1 | return self.persistence.open(self.schedule_filename, writeback=True)
celery_1 | File "/usr/local/lib/python3.7/shelve.py", line 243, in open
celery_1 | return DbfilenameShelf(filename, flag, protocol, writeback)
celery_1 | File "/usr/local/lib/python3.7/shelve.py", line 227, in __init__
celery_1 | Shelf.__init__(self, dbm.open(filename, flag), protocol, writeback)
celery_1 | File "/usr/local/lib/python3.7/dbm/__init__.py", line 94, in open
celery_1 | return mod.open(file, flag, mode)
celery_1 | _gdbm.error: [Errno 22] Invalid argument
celery_1 |
celery_1 | During handling of the above exception, another exception occurred:
celery_1 |
celery_1 | Traceback (most recent call last):
celery_1 | File "/usr/local/lib/python3.7/site-packages/billiard/process.py", line 327, in _bootstrap
celery_1 | self.run()
celery_1 | File "/usr/local/lib/python3.7/site-packages/celery/beat.py", line 673, in run
celery_1 | self.service.start(embedded_process=True)
celery_1 | File "/usr/local/lib/python3.7/site-packages/celery/beat.py", line 588, in start
celery_1 | humanize_seconds(self.scheduler.max_interval))
celery_1 | File "/usr/local/lib/python3.7/site-packages/kombu/utils/objects.py", line 44, in __get__
celery_1 | value = obj.__dict__[self.__name__] = self.__get(obj)
celery_1 | File "/usr/local/lib/python3.7/site-packages/celery/beat.py", line 632, in scheduler
celery_1 | return self.get_scheduler()
celery_1 | File "/usr/local/lib/python3.7/site-packages/celery/beat.py", line 627, in get_scheduler
celery_1 | lazy=lazy,
celery_1 | File "/usr/local/lib/python3.7/site-packages/celery/beat.py", line 467, in __init__
celery_1 | Scheduler.__init__(self, *args, **kwargs)
celery_1 | File "/usr/local/lib/python3.7/site-packages/celery/beat.py", line 226, in __init__
celery_1 | self.setup_schedule()
celery_1 | File "/usr/local/lib/python3.7/site-packages/celery/beat.py", line 493, in setup_schedule
celery_1 | self._store = self._destroy_open_corrupted_schedule(exc)
celery_1 | File "/usr/local/lib/python3.7/site-packages/celery/beat.py", line 481, in _destroy_open_corrupted_schedule
celery_1 | return self._open_schedule()
celery_1 | File "/usr/local/lib/python3.7/site-packages/celery/beat.py", line 475, in _open_schedule
celery_1 | return self.persistence.open(self.schedule_filename, writeback=True)
celery_1 | File "/usr/local/lib/python3.7/shelve.py", line 243, in open
celery_1 | return DbfilenameShelf(filename, flag, protocol, writeback)
celery_1 | File "/usr/local/lib/python3.7/shelve.py", line 227, in __init__
celery_1 | Shelf.__init__(self, dbm.open(filename, flag), protocol, writeback)
celery_1 | File "/usr/local/lib/python3.7/dbm/__init__.py", line 94, in open
celery_1 | return mod.open(file, flag, mode)
celery_1 | _gdbm.error: [Errno 22] Invalid argument
celery_1 | [2020-05-24 20:40:39,002: WARNING/Beat] Process Beat:
celery_1 | [2020-05-24 20:40:39,008: WARNING/Beat] Traceback (most recent call last):
celery_1 | [2020-05-24 20:40:39,009: WARNING/Beat] File "/usr/local/lib/python3.7/site-packages/kombu/utils/objects.py", line 42, in __get__
celery_1 | return obj.__dict__[self.__name__]
celery_1 | [2020-05-24 20:40:39,010: WARNING/Beat] KeyError: 'scheduler'
celery_1 | [2020-05-24 20:40:39,010: WARNING/Beat] During handling of the above exception, another exception occurred:
celery_1 | [2020-05-24 20:40:39,011: WARNING/Beat] Traceback (most recent call last):
celery_1 | [2020-05-24 20:40:39,013: WARNING/Beat] File "/usr/local/lib/python3.7/site-packages/celery/beat.py", line 485, in setup_schedule
celery_1 | self._store = self._open_schedule()
celery_1 | [2020-05-24 20:40:39,014: WARNING/Beat] File "/usr/local/lib/python3.7/site-packages/celery/beat.py", line 475, in _open_schedule
celery_1 | return self.persistence.open(self.schedule_filename, writeback=True)
celery_1 | [2020-05-24 20:40:39,016: WARNING/Beat] File "/usr/local/lib/python3.7/shelve.py", line 243, in open
celery_1 | return DbfilenameShelf(filename, flag, protocol, writeback)
celery_1 | [2020-05-24 20:40:39,017: WARNING/Beat] File "/usr/local/lib/python3.7/shelve.py", line 227, in __init__
celery_1 | Shelf.__init__(self, dbm.open(filename, flag), protocol, writeback)
celery_1 | [2020-05-24 20:40:39,017: WARNING/Beat] File "/usr/local/lib/python3.7/dbm/__init__.py", line 94, in open
celery_1 | return mod.open(file, flag, mode)
celery_1 | [2020-05-24 20:40:39,018: WARNING/Beat] _gdbm.error: [Errno 22] Invalid argument
celery_1 | [2020-05-24 20:40:39,019: WARNING/Beat] During handling of the above exception, another exception occurred:
celery_1 | [2020-05-24 20:40:39,020: WARNING/Beat] Traceback (most recent call last):
celery_1 | [2020-05-24 20:40:39,021: WARNING/Beat] File "/usr/local/lib/python3.7/site-packages/billiard/process.py", line 327, in _bootstrap
celery_1 | self.run()
celery_1 | [2020-05-24 20:40:39,024: WARNING/Beat] File "/usr/local/lib/python3.7/site-packages/celery/beat.py", line 673, in run
celery_1 | self.service.start(embedded_process=True)
celery_1 | [2020-05-24 20:40:39,025: WARNING/Beat] File "/usr/local/lib/python3.7/site-packages/celery/beat.py", line 588, in start
celery_1 | humanize_seconds(self.scheduler.max_interval))
celery_1 | [2020-05-24 20:40:39,026: WARNING/Beat] File "/usr/local/lib/python3.7/site-packages/kombu/utils/objects.py", line 44, in __get__
celery_1 | value = obj.__dict__[self.__name__] = self.__get(obj)
celery_1 | [2020-05-24 20:40:39,027: WARNING/Beat] File "/usr/local/lib/python3.7/site-packages/celery/beat.py", line 632, in scheduler
celery_1 | return self.get_scheduler()
celery_1 | [2020-05-24 20:40:39,028: WARNING/Beat] File "/usr/local/lib/python3.7/site-packages/celery/beat.py", line 627, in get_scheduler
celery_1 | lazy=lazy,
celery_1 | [2020-05-24 20:40:39,029: WARNING/Beat] File "/usr/local/lib/python3.7/site-packages/celery/beat.py", line 467, in __init__
celery_1 | Scheduler.__init__(self, *args, **kwargs)
celery_1 | [2020-05-24 20:40:39,030: WARNING/Beat] File "/usr/local/lib/python3.7/site-packages/celery/beat.py", line 226, in __init__
celery_1 | self.setup_schedule()
celery_1 | [2020-05-24 20:40:39,031: WARNING/Beat] File "/usr/local/lib/python3.7/site-packages/celery/beat.py", line 493, in setup_schedule
celery_1 | self._store = self._destroy_open_corrupted_schedule(exc)
celery_1 | [2020-05-24 20:40:39,032: WARNING/Beat] File "/usr/local/lib/python3.7/site-packages/celery/beat.py", line 481, in _destroy_open_corrupted_schedule
celery_1 | return self._open_schedule()
celery_1 | [2020-05-24 20:40:39,032: WARNING/Beat] File "/usr/local/lib/python3.7/site-packages/celery/beat.py", line 475, in _open_schedule
celery_1 | return self.persistence.open(self.schedule_filename, writeback=True)
celery_1 | [2020-05-24 20:40:39,033: WARNING/Beat] File "/usr/local/lib/python3.7/shelve.py", line 243, in open
celery_1 | return DbfilenameShelf(filename, flag, protocol, writeback)
celery_1 | [2020-05-24 20:40:39,034: WARNING/Beat] File "/usr/local/lib/python3.7/shelve.py", line 227, in __init__
celery_1 | Shelf.__init__(self, dbm.open(filename, flag), protocol, writeback)
celery_1 | [2020-05-24 20:40:39,035: WARNING/Beat] File "/usr/local/lib/python3.7/dbm/__init__.py", line 94, in open
celery_1 | return mod.open(file, flag, mode)
Apologies - I'll be sure to use the more appropriate forums for these kind of questions next time round.
from build-a-saas-app-with-flask.
nickjj
commented on May 29, 2024
Which section of the course are you working on?
Also have you tried to do docker-compose down, delete the old celerybeat-schedule file and then docker-compose up?
from build-a-saas-app-with-flask.
jape-dev
commented on May 29, 2024
I'm working on section 20-internationalizaton-i18n. Yes I have tried to delete the celerybeat-schedule file as I've seen that recommended in other related posts. However this doesn't seem to have worked as upon running docker-compose up again a new celerybeat-schedule file gets created which seems to throw an error when it is loaded by shelve as per the traceback.
It's worth mentioning that I didn't get this error when working on section 16 where the periodic tasks were first introduced.
from build-a-saas-app-with-flask.
jape-dev
commented on May 29, 2024
I've gone back and rebuilt the application through the different course sections. It seems that I first start getting this issue in section 17-building-the-snake-eyes-game.
from build-a-saas-app-with-flask.
nickjj
commented on May 29, 2024
Did you do the docker-compose down step?
from build-a-saas-app-with-flask.
jape-dev
commented on May 29, 2024
Yep I did: docker-compose down --> deleted celerybeat-schedule --> docker-compose up and get the same error.
Have pulled down a fresh version of the course folder and gone into section 17 and getting the same error.
from build-a-saas-app-with-flask.
nickjj
commented on May 29, 2024
I'm not able to reproduce that here.
Here's what I did to test:
- Download the zip file that you get early on in the course
- Extract it locally
- Move into section 17's code base
docker-compose build --no-cachedocker-compose down -vdocker-compose up
At this point, everything is up and running including Celery.
Step 4 ensures everything is built from scratch and step 5's -v removes all named volumes (all PostgreSQL and Redis data will be purged).
If you perform the same workflow, what happens?
from build-a-saas-app-with-flask.
jape-dev
commented on May 29, 2024
I've followed your workflow exactly and unfortunately still getting the same error. Even getting the error when repeating the same steps for section 16 now too.
I'm starting to think it could be something to do with my Docker Toolbox setup. I'm using the latest release:
Client:
Version: 19.03.1
API version: 1.40
Go version: go1.12.7
Git commit: 74b1e89e8a
Built: Wed Jul 31 15:18:18 2019
OS/Arch: windows/amd64
Experimental: false
Server: Docker Engine - Community
Engine:
Version: 19.03.5
API version: 1.40 (minimum version 1.12)
Go version: go1.12.12
Git commit: 633a0ea838
Built: Wed Nov 13 07:28:45 2019
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: v1.2.10
GitCommit: b34a5c8af56e510852c35414db4c1f4fa6172339
runc:
Version: 1.0.0-rc8+dev
GitCommit: 3e425f80a8c931f88e6d94a8c831b9d5aa481657
docker-init:
Version: 0.18.0
GitCommit: fec3683
https://github.com/docker/toolbox/releases
Are you aware of any complications that may arise from this version? Or is this part of the build only compatible with docker desktop?
from build-a-saas-app-with-flask.
nickjj
commented on May 29, 2024
It'll work with Docker Toolbox too. Lots of folks are using it in the course.
When was the last time you downloaded the zip file from the course?
As a hail mary move you could always try deleting the VM that Docker Toolbox created and letting Docker Toolbox create a new one for you the next time you launch the Docker Quickstart Terminal.
from build-a-saas-app-with-flask.
jape-dev
commented on May 29, 2024
I just downloaded a fresh version of the course zip file today while I was trying out your workflow. Hence why I'm so confused and starting to think it's something to do with my Docker Toolbox 😅
I've tried docker-machine restart but I'll give deleting the VM a go using docker-machine kill
from build-a-saas-app-with-flask.
jape-dev
commented on May 29, 2024
Removing the docker machine with docker-machine rm and letting docker toolbox create a new one still didn't work 😢 I've been doing some more digging and found a thread that suggested adding a celery-beat.conf file may work. I'll give this a shot https://docs.celeryproject.org/en/stable/userguide/daemonizing.html#init-script-celerybeat
from build-a-saas-app-with-flask.
nickjj
commented on May 29, 2024
If you email me from the account you purchased the course on I'm open to doing a quick Google Hangouts or Zoom screenshare session. It might be faster than a bunch of back / forths here.
So far you're the only person who has reported this and thousands of people have completed the course based on the stats I see.
from build-a-saas-app-with-flask.
jape-dev
commented on May 29, 2024
That would be great - just sent you an email. Thanks
from build-a-saas-app-with-flask.
nickjj
commented on May 29, 2024
Closing this here, as the issue isn't related to the code in this repo.
from build-a-saas-app-with-flask.
Related Issues (20)
- Illegal instruction: 4 HOT 8
- Several questions for solution evaluation HOT 6
- Revisiting OS X Postgres permissions issue HOT 15
- Internal sever error and db reset issue on Mac HOT 3
- Error importing module add: HOT 12
- Error importing module db: HOT 5
- Getting blank responses from cat watch HOT 44
- Cannot access admin page HOT 2
- CSRF problem HOT 17
- Adding Custom CSS and JS HOT 9
- SMTPAuthenticationError 534 HOT 4
- Getting a ImportError when running with docker compose HOT 4
- redis need a version bump HOT 3
- gunicorn does not run on windows HOT 2
- ERROR when building HOT 4
- how can i configure postgresql over here can you help me out HOT 1
- ImportError: No module named abc HOT 2
- FLASK_ENV error in build HOT 10
- why not django? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from build-a-saas-app-with-flask.