ternaris / marv-robotics-nodes Goto Github PK

View Code? Open in Web Editor NEW

2.0 2.0 0.0 0 B

marv-robotics-nodes's People

Contributors

Stargazers

Watchers

marv-robotics-nodes's Issues

Warnings in `marv fileset scan`

With marv fileset scan I get the following warnings. Not sure if they are normal, but wanted to let you know.

$ marv fileset scan
2016-10-02 14:09:27,729 INFO rospy.topics topicmanager initialized
libdc1394 error: Failed to initialize libdc1394
/marv/venv/local/lib/python2.7/site-packages/matplotlib/font_manager.py:279: UserWarning: Matplotlib is building the font cache using fc-list. This may take a moment.
  warnings.warn('Matplotlib is building the font cache using fc-list. This may take a moment.')
2016-10-02 14:09:28,659 INFO marv._node Registered nodes in order: ['fileset', u'md5', u'bagset_name', u'bagmeta', u'messages', u'camera_frames', u'diag_count', u'fulltext', u'gnss_plots', u'navsatfix', u'geo_json_trajectory']
2016-10-02 14:09:28,685 WARNING marv._scan Ignoring changed mtime of files for now
2016-10-02 14:09:28,688 INFO rospy.core signal_shutdown [atexit]

PS.: I am referring to the libdc1394 error, the matplotlib UserWarning and the warning about mtime being ignored.

Document devserver on other IPs than localhost

Since I am testing marv-robotics inside a docker container, I wanted to access the devserver from the host and got the following response:

$ curl http://localhost:5000
curl: (56) Recv failure: Connection reset by peer

Only while I am writing this I noticed that the devserver is only listening on 127.0.0.1:

marv@c98eb34848e5:/marv/site$ marv devserver
2016-10-02 14:13:09,896 INFO marv.app.listing Listing is up-to-date
2016-10-02 14:13:09,909 INFO werkzeug  * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
2016-10-02 14:13:09,926 INFO werkzeug  * Restarting with inotify reloader
2016-10-02 14:13:10,312 INFO marv.app.listing Listing is up-to-date
2016-10-02 14:13:10,323 WARNING werkzeug  * Debugger is active!
2016-10-02 14:13:10,323 INFO werkzeug  * Debugger pin code: 145-388-684

Fortunately the marv devserver --help and a good guess leads to marv devserver --public, which gets around the issue.

You might want to mention this on the "getting started" documentation, otherwise it might lead to frustration for users wanting to try out marv-robotics for the first time.

Usability: Tagging

Just a few remarks on my first experience with tagging. These are only very minor issues.

It was not directly obvious to me why typing a tag in the box and clicking "save tags" did not add a tag. Now that I know you have to click "create tag ..." or press enter and have a chance to adding/removing multiple tags at once before saving the UI does make sense to me.
clicking "tag" and then hitting "enter" adds an "empty string" tag. Not sure if you want to allow that. I is not visible in the list of tags (probably a grey box with 0px width).

Document simple authentication with uwsgi

I just wanted to leave this here for other's reference. It should probably make it at some point into the setup documentation.

With bagbunker I configured apache with basic_auth for the whole site, to prevent anyone from viewing the datasets. Note that since all of this is in the Bosch intranet, I'm not too worried about malicious attackers and thus not using SSL is fine for now.

Something similar can be done with uwsgi even easier. Simply adding the following line to uwsgi.conf will get you started with the most basic form of adding http basic auth to the whole marv robotics site:

route = .* basicauth:Marv Robotics,username:password

OSM widget satellite images levels

It seems for the OSM widget the 4 closest levels for sattellite images don't work. For the map view the full range of zoom levels seems available.

Feedback for setup with indigo on ubuntu 14.04 docker image

I setup a dockerfile to install marv robotics on an image based on the ros:indigo dockerimage. The setup is somewhat similar to the bagbunker docker image, but arguably a bit simpler, and it supports changing the user-id when starting the container such that there is no permission problem with the mounted volumes, for example:

docker run -v /var/lib/marv-robotics/site:/marv/site -v /vol/vps-storage-local/15_zeno_bagbunker:/mnt/bags:ro --name marv -p 5000 -p 5001:8000 -it -e "MARV_USERID=$(id -u res_bagbunker)" -e "MARV_GROUPID=$(id -g res_bagbunker)" --detach  si0vmc1062.de.bosch.com/zenoway/zeno-marv-robotics

The current Dockerfile can be found here for your reference: https://gist.github.com/NikolausDemmel/7281ae200f6c251b5695e96563ab60a9

In the following I want to discuss a few minor issues that I noticed while setting it up, which should probably make it into the setup documentation at some point.

For system dependencies I needed to make the follwoing adaptations for Indigo:
- ros-kinetic-ros-base --> ros-indigo-ros-base
- python-cv-bridge --> ros-indigo-cv-bridge
During pip install i get the following warning. Not sure if it needs to be addressed or not:

/marv/venv/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
 InsecurePlatformWarning

During pip install I get the following error:

/marv/venv/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#snimissingwarning.
 SNIMissingWarning

I solved this by running pip install pyOpenSSL ndg-httpsclient pyasn1. In order for that to be successful, I needed to install the following additional system dependencies: apt-get install python-dev libffi-dev libssl-dev.
4. I needed to upgrade pip before to make sure the install -c command line parameter was available: pip install --upgrade pip.

That should be all that is needed on top of the install instructions in the README and all of this is in the Dockerfile linked above. The rest in the Dockerfile is really just a bit of user and entrypoint management, and some docker and proxy specific boilerplate.

Only a blank page in Bosch Windows browser

From my Linux machine my test-site of marv robotics works fine, but from my Bosch Windows machine I only see a blank page (in both Firefox and IE). When opening the JS Console in Firefox, I see the following:

TypeError: arguments is not iterable
G()
 main-built.js:7614
create()
 main-built.js:7617
<anonym>
 main-built.js:6763
require()
 main-built.js:6763
<anonym>
 main-built.js:7627
 main-built.js:7614:928

Error parsing time data in aggregate json

I get the following error when rerunning

After running all nodes once, if I attempt to execute marv node run again, I get the following error. It looks like marv chokes on our handcrafted "automated testing" bagfile with a very regular date sometime in 2030.

$ marv node run --all-nodes --all-filesets
2016-10-04 09:55:18,893 INFO rospy.topics topicmanager initialized
libdc1394 error: Failed to initialize libdc1394
/marv/venv/local/lib/python2.7/site-packages/matplotlib/font_manager.py:279: UserWarning: Matplotlib is building the font cache using fc-list. This may take a moment.
  warnings.warn('Matplotlib is building the font cache using fc-list. This may take a moment.')
2016-10-04 09:55:22,938 INFO marv._node Registered nodes in order: ['fileset', u'md5', u'bagset_name', u'bagmeta', u'messages', u'camera_frames', u'diag_count', u'fulltext', u'gnss_plots', u'navsatfix', u'geo_json_trajectory']
2016-10-04 09:56:04,473 WARNING marv._aggregate Exception in aggregate context
Traceback (most recent call last):
  File "/marv/venv/local/lib/python2.7/site-packages/marv/_site.py", line 149, in node_run
    need_run = self.nodes.need_run(agg, rerun=rerun, **kw)
  File "/marv/venv/local/lib/python2.7/site-packages/marv/_node.py", line 178, in need_run
    elif node.name not in agg:
  File "/marv/venv/lib/python2.7/_abcoll.py", line 369, in __contains__
    self[key]
  File "/marv/venv/local/lib/python2.7/site-packages/marv/_aggregate.py", line 185, in __getitem__
    return json_load_agg(f)
  File "/marv/venv/local/lib/python2.7/site-packages/marv/_aggregate.py", line 75, in json_load_agg
    return json.load(*args, object_pairs_hook=object_pairs_hook, **kw)
  File "/usr/lib/python2.7/json/__init__.py", line 290, in load
    **kw)
  File "/usr/lib/python2.7/json/__init__.py", line 351, in loads
    return cls(encoding=encoding, **kw).decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode
    obj, end = self.scan_once(s, idx)
  File "/marv/venv/local/lib/python2.7/site-packages/marv/_aggregate.py", line 64, in object_pairs_hook
    return datetime.strptime(value, '%Y-%m-%dT%H:%M:%S.%f')
  File "/usr/lib/python2.7/_strptime.py", line 325, in _strptime
    (data_string, format))
ValueError: time data '2030-03-25T00:00:00' does not match format '%Y-%m-%dT%H:%M:%S.%f'
2016-10-04 09:56:16,381 INFO rospy.core signal_shutdown [atexit]

Are jobs run in parallel?

If I run multiple nodes (e.g. all nodes on all bag files), are those executed in parallel or sequentially? How many parallel threads are being executed?

Bag download link does not work

When I try to download a bag file, I get the following page:

Not Found

The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.

The server log shows:

[pid: 1|app: 0|req: 15/17] 10.44.53.229 (zenoway) {44 vars in 931 bytes} [Tue Oct  4 13:02:45 2016] GET /marv/api/2/fileset-details/858b00cf-0439-4acf-85bd-cb45b5730c30/download/0 => generated 233 bytes in 38 msecs (HTTP/1.1 404) 2 headers in 72 bytes (1 switches on coe 0)

DateFormatter error in `gnss_plots`

I'm not sure if the bags that cause this error have some strange data in it. Please provide instructions on what info you would need to further debug this.

2016-10-02 13:46:40,352 WARNING marv.node.gnss_plots f2b14c7f-c885-4825-bd33-d0f07649717d No orientations found
2016-10-02 13:46:40,708 ERROR marv.node.gnss_plots f2b14c7f-c885-4825-bd33-d0f07649717d error
Traceback (most recent call last):
  File "/marv/venv/local/lib/python2.7/site-packages/marv/_aggregate.py", line 278, in update
    value = node(self)
  File "/marv/venv/local/lib/python2.7/site-packages/marv/_node.py", line 103, in __call__
    rv = self.node_func(**inputs)
  File "/marv/venv/local/lib/python2.7/site-packages/marv/_node.py", line 57, in __call__
    return self.func(*args, **kw)
  File "/marv/venv/local/lib/python2.7/site-packages/marv_robotics/nodes/gnss.py", line 189, in gnss_plots
    positions[ptopic], orientations[otopic])
  File "/marv/venv/local/lib/python2.7/site-packages/marv_robotics/nodes/gnss.py", line 112, in render
    fig.savefig(plt_path)
  File "/marv/venv/local/lib/python2.7/site-packages/matplotlib/figure.py", line 1557, in savefig
    self.canvas.print_figure(*args, **kwargs)
  File "/marv/venv/local/lib/python2.7/site-packages/matplotlib/backend_bases.py", line 2237, in print_figure
    **kwargs)
  File "/marv/venv/local/lib/python2.7/site-packages/matplotlib/backends/backend_agg.py", line 593, in print_jpg
    buf, size = self.print_to_buffer()
  File "/marv/venv/local/lib/python2.7/site-packages/matplotlib/backends/backend_agg.py", line 562, in print_to_buffer
    FigureCanvasAgg.draw(self)
  File "/marv/venv/local/lib/python2.7/site-packages/matplotlib/backends/backend_agg.py", line 463, in draw
    self.figure.draw(self.renderer)
  File "/marv/venv/local/lib/python2.7/site-packages/matplotlib/artist.py", line 63, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "/marv/venv/local/lib/python2.7/site-packages/matplotlib/figure.py", line 1129, in draw
    renderer, self, dsu, self.suppressComposite)
  File "/marv/venv/local/lib/python2.7/site-packages/matplotlib/image.py", line 139, in _draw_list_compositing_images
    a.draw(renderer)
  File "/marv/venv/local/lib/python2.7/site-packages/matplotlib/artist.py", line 63, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "/marv/venv/local/lib/python2.7/site-packages/matplotlib/axes/_base.py", line 2384, in draw
    mimage._draw_list_compositing_images(renderer, self, dsu)
  File "/marv/venv/local/lib/python2.7/site-packages/matplotlib/image.py", line 139, in _draw_list_compositing_images
    a.draw(renderer)
  File "/marv/venv/local/lib/python2.7/site-packages/matplotlib/artist.py", line 63, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "/marv/venv/local/lib/python2.7/site-packages/matplotlib/axis.py", line 1126, in draw
    ticks_to_draw = self._update_ticks(renderer)
  File "/marv/venv/local/lib/python2.7/site-packages/matplotlib/axis.py", line 959, in _update_ticks
    tick_tups = [t for t in self.iter_ticks()]
  File "/marv/venv/local/lib/python2.7/site-packages/matplotlib/axis.py", line 906, in iter_ticks
    for i, val in enumerate(majorLocs)]
  File "/marv/venv/local/lib/python2.7/site-packages/matplotlib/dates.py", line 464, in __call__
    raise ValueError('DateFormatter found a value of x=0, which is '
ValueError: DateFormatter found a value of x=0, which is an illegal date.  This usually occurs because you have not informed the axis that it is plotting dates, e.g., with ax.xaxis_date()

Feature request: exclude specific bag files in the scanroot

There should be a way to exclude specific bag files in the bagroot from being added to marv robotics.

We have talked about this before and there are many ways, how this could be configured, of which two examples are: 1) a ignore file similar to .gitignore or .dockerignore and 2) a marker file for while folders / sub paths similar to CATKIN_IGNORE.

I'm just mentioning this again here as an issue, because while testing marv robotics with our share of bagfiles, I noticed that marv robotics adds all our duplicate bag files that we keep in an .original folder (the originally recorded data even if the bag file hast undergone message migration or similar alterations. For bagbunker it a matter of just not creating a .md5 file for the bags in the .original folder to ensure that they we not included.

ternaris / marv-robotics-nodes Goto Github PK

marv-robotics-nodes's People

Contributors

Stargazers

Watchers

marv-robotics-nodes's Issues

Warnings in `marv fileset scan`

Document devserver on other IPs than localhost

Usability: Tagging

Document simple authentication with uwsgi

OSM widget satellite images levels

Feedback for setup with indigo on ubuntu 14.04 docker image

Only a blank page in Bosch Windows browser

Error parsing time data in aggregate json

Are jobs run in parallel?

Bag download link does not work

DateFormatter error in `gnss_plots`

Feature request: exclude specific bag files in the scanroot

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent