Comments (4)
It looks like the issue you're experiencing with Celery jobs in Superset being scheduled 12 hours ahead could be due to a timezone misalignment, particularly in how timezones are handled and converted within Superset's scheduling logic. This misalignment might stem from the timezone setting in the Superset UI not being correctly recognized or applied when calculating the next execution times for jobs.
To address this, ensure that the timezone you set in the Superset UI is correctly formatted and recognized by the pytz
library, as any incorrect or unrecognized timezone will default to UTC. Additionally, you might need to review and possibly customize the scheduling logic in Superset, especially if the default behavior doesn't align with your expectations. This could involve adjusting how datetime values are localized to the specified timezone and ensuring accurate conversion back to UTC if necessary.
There's also a related issue and suggested solution that involves replacing datetime.utcnow()
with datetime.now(timezone.utc)
in the scheduler code, which has been discussed in the Superset GitHub repository. This change aims to correct the timezone handling and might be a good starting point for resolving the scheduling discrepancies you're facing.
If after these steps you're still encountering issues, expanding logging to include key variables like triggered_at
, start_at
, stop_at
, and the final scheduled times can help identify where the mismatches occur, providing further insight into the problem and aiding in troubleshooting.
Details
Sources
- Timezone Misalignment in Apache Superset 3.0.1 Alert&Reports
- superset/tasks/scheduler.py
- superset/tasks/cron_util.py
About Dosu
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.
Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.
To continue the conversation, mention @dosu.
from superset.
hi @nathan-gilbert you found some solutión about this problem? i have the same issue, thanks for you comment
from superset.
@tridoxx No, I have found no solution yet.
from superset.
hi @nathan-gilbert i did this correctión, first you need to modify the file "superset/task/schedule.py" in my case, i am using docker so, for some strange reason, i need to modify this file inside the "superset_worker" container, not the "superset_app" container. so i edit the to this.
`import pytz
utc_now = datetime.utcnow()
and change the line
#async_options = {"eta": schedule}
to
async_options = {"eta": utc_now} `
something like this.
and reboot the machine. for me this work, the superset wil take the real utc time, and subtract the correct UTC time from the timezone defined directly by the alert generator on the superset app.
for me is working now without problem, check that is correct using the crontab on superset, to execute the alert every minute, and use the next comand to check logs "docker logs superset_worker --since 1h" if you are using the docker "superset_worker"
from superset.
Related Issues (20)
- Drill uses aliases in group by statements generated by SQLA resulting in no data when aggregating columns HOT 1
- Worker instance keep restarting after upgrade to 4.0.0-dev HOT 6
- 4.0 - Imported charts shows error on embedding HOT 6
- sql lab hive 、Spark SQL get tablename list error, Source code error found HOT 1
- exp claim on JWT token is not respected and token is still valid after expiration time HOT 3
- Sql Lab: Getting incorrect error line number in Superset while executing queries HOT 2
- The tooltip of Country Map is not readable HOT 4
- [SIP-132] Proposal for SQL Lab addon plugins HOT 4
- [SIP-133] Update "Time Range" filter in dashboard HOT 7
- HTTP Headers Configuration on config.py not overriding. HOT 3
- "embed dashboard" button does not show up HOT 1
- Missing CSV file in email of report / alert HOT 1
- Running superset db upgrade results is about a SQLite syntax error. HOT 1
- How to customize the row number in the exported excel file? HOT 2
- There is no filter box in apache superset version 3.1.3 HOT 2
- Published Dashboard without charts can only be reached by owners HOT 1
- SPAM
- Angular run time error of superset-embedded-sdk HOT 3
- DISPLAY_MAX_ROW 10.000 / 100.000 HOT 2
- Tables with long numbers are truncated when exported to excel HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from superset.