Comments (10)
@alexcjohnson great! thank you for the info, I was just curious. I'm working on another PR so I may learn enough through that to take a swing at this. No pressure implied though; "OSS maintainers owe you nothing" and all that.
from dash.
@jborman-stonex Thanks! I'll check that out
from dash.
@alexcjohnson Those params make a lot of sense to me. I'd also like to be able to enable retry_jitter
and retry_exponential_backoff
to avoid the situation of "Everything is retrying all at once and thus there's a windfall of requests to work through".
from dash.
Without being an expert in the inner-workings of Dash, here's my alternate suggestion in pseudocode inside the existing implementation, noted with JKUNSTLE NOTE
.
def _make_job_fn(fn, celery_app, progress, key):
cache = celery_app.backend
# JKUNSTLE NOTE: added bind=True, add `self` param to function
@celery_app.task(name=f"long_callback_{key}", bind=True)
def job_fn(self, result_key, progress_key, user_callback_args, context=None):
def _set_progress(progress_value):
if not isinstance(progress_value, (list, tuple)):
progress_value = [progress_value]
cache.set(progress_key, json.dumps(progress_value, cls=PlotlyJSONEncoder))
maybe_progress = [_set_progress] if progress else []
ctx = copy_context()
def run():
c = AttributeDict(**context)
c.ignore_register_page = False
context_value.set(c)
try:
if isinstance(user_callback_args, dict):
user_callback_output = fn(*maybe_progress, **user_callback_args)
elif isinstance(user_callback_args, (list, tuple)):
user_callback_output = fn(*maybe_progress, *user_callback_args)
else:
user_callback_output = fn(*maybe_progress, user_callback_args)
except PreventUpdate:
# Put NoUpdate dict directly to avoid circular imports.
cache.set(
result_key,
json.dumps(
{"_dash_no_update": "_dash_no_update"}, cls=PlotlyJSONEncoder
),
)
# JKUNSTLE NOTE: catches user-raised retry exception so
# Dash can control parameterization api.
except RetryBGTask as e:
# JKUNSTLE NOTE: retry in 1 second.
# should optionally be configured by e.args parameters, passed up from user function.
self.retry(countdown=1)
except Exception as err: # pylint: disable=broad-except
cache.set(
result_key,
json.dumps(
{
"long_callback_error": {
"msg": str(err),
"tb": traceback.format_exc(),
}
},
),
)
else:
cache.set(
result_key, json.dumps(user_callback_output, cls=PlotlyJSONEncoder)
)
ctx.run(run)
return job_fn
from dash.
@T4rk1n You've helped me with a feature request before- hoping to bump for some help!
from dash.
I think retrying callbacks (normal callbacks too) could be a good feature addition.
The API, I think it could have the arguments be in callback arguments and I also like the args in the error so something like.
from dash import RetryCallback
class CustomError(Exception):
pass
@callback(
...,
retry=True, # Global retry all errors.
retry_after=360, # Global retry after argument
retry_on_error=[CustomError, RetryCallback], # Retry only on those errors.
)
def cb(*args):
raise RetryCallback(after=1) # Custom retry after argument.
For the implementation I think we could catch the exceptions in dispatch and use the renderer to resend the same callback in a timeout. Can catch directly for normal callbacks and bg response has key in the response dict "long_callback_error"
that is currently re-raised as LongCallbackError
, would need to add more info to reconstitute the error for the retry_on_error
.
cc @alexcjohnson @Coding-with-Adam
from dash.
I like it! I might change retry_after
and after
to retry_delay
and delay
to be a little more specific ("after" could be an event instead of a time, or could even be taken as "retry after these errors")
Two other pieces come to mind you might want with this:
max_tries
- might default to something like 5 so this doesn't kick off an infinite loop if the error isn't really transient, but we could supportNone
or some such if you want to allow an infinite loop.- Some way to report on retry status. Maybe
retry_status=Output('my_div', 'children')
, and by default we construct a string likeattempt #1 failed with CustomError, trying again
but you can provide your own message likeraise RetryCallback(status=f"Oops, the database was locked on attempt #{ctx.retry_num}, we'll try again in 2 minutes")
orstatus=dcc.Markdown(...)
or whatever.
from dash.
@alexcjohnson @T4rk1n What's the typical timeline on a feature request like this for ya'll? (Not meant in a demanding tone, just curious so I can set objectives based on your timeline)
from dash.
@JamesKunstle it's a great idea, but hasn't made it onto our roadmap yet so I can't say when we'd get to it. Best bet if you want it quick is to contribute a PR, or if your org has resources to commit we could talk about sponsoring development.
from dash.
Just passing by, but the tenacity
package might be of interest here as it provides a flexible decorator interface.
Off the top of my head, it should be stacked like so:
@app.callback(...)
@tenacity.retry(...)
def my_callback(...):
...
from dash.
Related Issues (20)
- Pattern Matching Callback Renders All Components When Adding New One HOT 2
- Can't build custom components with react-docgen 7
- CSS styling to dash debug button resets on refresh HOT 1
- [Feature Request] Add `outputs` and `outputs_list` to `window.dash_clientside.callback_context` HOT 1
- [BUG] `id` passed through `dcc.Loading` not visible in DOM HOT 4
- Switching back and forth between dcc.Tabs doesn't seem to release memory in the browser
- add dash player to dash core components
- [BUG] Error generating typescript components HOT 1
- [BUG] No module named _plotly_utils HOT 1
- clarify instructions for contributing to Dash on Windows HOT 8
- No Layout Exception when using string elements when the layout is a list
- [Feature Request] tabIndex of Div should also accept number type HOT 2
- Remove LogoutButton HOT 7
- Markdown does not update code component properly after callback update. HOT 3
- [BUG] Callback running keyword doesn't work when component is not in layout
- [BUG] Environment variables from .env file ignored in Dash.run
- [BUG] `dcc.Location` doesn't refresh current page - callback does nothing when returns same value as its current state HOT 4
- Layout as list of components does not work when layout is a function HOT 2
- bool(DashTable) is always False HOT 4
- Differences in dcc.Store storage_type performance with dash 2.17.1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dash.