cert-polska / karton-dashboard Goto Github PK

While testing, sometimes is nessesary to flush graph to trace consumers affected by certain task

ERROR:karton.dashboard.app:Exception on /api/queue/karton.archive-extractor [GET]
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 2446, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1951, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1820, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1949, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1935, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/local/lib/python3.7/site-packages/karton/dashboard/app.py", line 239, in get_queue_api
    return jsonify(QueueView(queue).to_dict())
  File "/usr/local/lib/python3.7/site-packages/karton/dashboard/app.py", line 101, in to_dict
    "crashed": sorted(crashed, key=lambda t: t.last_update, reverse=True),
  File "/usr/local/lib/python3.7/site-packages/karton/dashboard/app.py", line 101, in <lambda>
    "crashed": sorted(crashed, key=lambda t: t.last_update, reverse=True),
AttributeError: 'str' object has no attribute 'last_update'

see also #26

Right now, waving the cursor around the screen causes the screen to flash a bit.

Hello,

I met a problem with karton-dashboard when I try to collect /vars using prometheus every 5 secs with more than 10k tasks.

Karton-dashboard exceeds 5s limit (context deadline exceeded prometheus error) and tries to collect data again - and karton-dashboard prepares two reports at the same time, using two cpu cores, but the second request exceeds 5s limit too, the third request will be born - and it looks like deadlock of requests.

CPU exhaust ia a bad news, but it is bearable. Every such request consumes a lot of ram, so soon the server will have no ram, this way I have no possibility even to kill karton-dashboard)

This issue wrote just for acknolegement about the situation with a lot of tasks - I think it is a architecure issue and probably it is fixable if we will use less resouce-consuming stats obtaining.

Since the update to set_task_status removed the consumer argument, the dashboard can no longer cancel tasks.

This line here fails.

Would you mind updating this to support karton 4.4.0?

Since we probably don't want to expose our internal services hostnames

Task is marked as finished, but reference is kept in consumer queue karton.queue.<priority>:<identity>

Ability to set telegram bot token and choose the situations of notification:

• Service changed count of workers
• Service has no workers
• Service workers 0 -> 1
• Chosen service or any of them generated Crashed task

what do you think about a helper for this? I.e. some kind of get_queue(state) that either returns a queue or :abort:s with 404

Originally posted by @msm-code in #32 (comment)

Currently, the crash queue shows a RED number of crashes in the dashboard. Sometimes there are crashes that are fixed or known, and the user wants to resolve it, but they have to live with this scary red number for a couple of days until the crahsed task would be removed from the queue.

I suggest adding an X button that will remove individual tasks. I also suggest adding a button to clean the crashed queue (see #30, it's similar)

Hi all,
I run the karton dashboard with docker .
I would like to add metric to grafana and I'm not sure how to config the data source in Grafana.
I see the logs under /varz.

what is the URL and port that I should fill when I config the data source in grafana?
thanks

Due to how mworks operates, /docz endpoint (available as button on navbar) is broken, as no README.md is available in docker container.
Same goes for nested routes (/docs/doc/*) as they expect markdown files in /docs directory.
There is also path traversal in /docs/doc endpoint (it requires file to end with .md though).

Looks like this endpoint doesn't work anymore:

ERROR:karton.dashboard.app:Exception on /api/queues [GET]
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 2446, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1951, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1820, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1949, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1935, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/local/lib/python3.7/site-packages/karton/dashboard/app.py", line 196, in get_queues_api
    for identity, queue in state.queues.items()
  File "/usr/local/lib/python3.7/site-packages/karton/dashboard/app.py", line 196, in <dictcomp>
    for identity, queue in state.queues.items()
  File "/usr/local/lib/python3.7/site-packages/karton/dashboard/app.py", line 100, in to_dict
    "tasks": sorted(tasks, key=lambda t: t.last_update, reverse=True),
  File "/usr/local/lib/python3.7/site-packages/karton/dashboard/app.py", line 100, in <lambda>
    "tasks": sorted(tasks, key=lambda t: t.last_update, reverse=True),
AttributeError: 'str' object has no attribute 'last_update'

It went undetected pretty long, so I assume it's not used by anyone. We can probably safely remove it instead of fixing it (but I have no hard stance on this).

I wish we could e2e test this, but this looks like a pain to setup test environment for.

Add a button for "Retry All" at the top of the table

karton.inspect classes doesn't implement to_dict() methods