nguy / artview Goto Github PK
View Code? Open in Web Editor NEWARM Radar Toolkit Viewer
Home Page: http://nguy.github.io/artview
License: BSD 3-Clause "New" or "Revised" License
ARM Radar Toolkit Viewer
Home Page: http://nguy.github.io/artview
License: BSD 3-Clause "New" or "Revised" License
Permanent issue on documentation.
At the AMS radar conference I tried making a conda package for ARTView which would make installing easier for users. My testing was promising but the package did not work. I think this was due to a mismatch between the ARTView tag I choose (v1.0.1) and the version of Py-ART used (1.5.0).
Is there an interest in a conda package for ARTView? If available users could install a fixed version of ARTView using conda install -c channel artview
.
I currently make conda package for every Py-ART release and conda is the primary means by which users install Py-ART. We highly encourage users to choose this over other install methods (setup.py, pip, etc) unless they need to use features that have not made it into a release and would need to install from source.
In ARM-DOE/pyart#390 I have a task of creating conda packages for other radar tools. If there is interest I'll add ARTView to this list. To accomplish this it would be helpful if "good" version of ARTView were tagged with a version number.
So the good news is that we have a small, but growing user community. The bad news is that is community is silent. This is absolutely normal, but it would be really important to create a conversation. Ideas?
I was thinking: to make artview more useful for pyart users we should add some convenience functions that allow a fast and simple visualization, with no need of mounting the application; that is, you are working with a radar, making corrections and stuff, and now you want to visualize, just run something like view(radar)
. For that I would like to mount function like this
artview.view.view(radar)
artview.view.view(radar, "reflectivity")
artview.view.view(grid)
also to allow applications with more than one display:
artview.view.start() # alias to app = QtGui.QApplication([ ])
artview.view.addRadar(radar)
artview.view.addGrid(grid)
artview.view.execute() # alias to app.exec_()
This is easy to be done, I would just like to hear some opinions. I also plan to have this before the September presentation, as it can really come to hand to work/present together with pyart (@scollis @jjhelmus).
With conda 3.16.0 on OX 10.10.5, installed latest
artview : de80c70
and pyart: a07b186fd7b9603d5e718daef1ad088bc2510567
I opened a CFRadial file and got the following crash:
Opening file /scr/eldora2/rsfdata/front/cfradial/spol/qpe/20150404/cfrad.20150404_224032.000_to_20150404_224032.000_SPOL_v664_s00_el85.00_SUR.nc
Traceback (most recent call last):
File "/usr/local/anaconda/bin/artview", line 5, in
artview.main.main(sys.argv)
File "/usr/local/anaconda/lib/python2.7/site-packages/artview/main.py", line 18, in main
artview.run(DirIn, filename, field)
File "/usr/local/anaconda/lib/python2.7/site-packages/artview/scripts/standard.py", line 50, in run
parent=MainMenu)
File "/usr/local/anaconda/lib/python2.7/site-packages/artview/components/plot_radar.py", line 137, in init
self.LaunchGUI()
File "/usr/local/anaconda/lib/python2.7/site-packages/artview/components/plot_radar.py", line 166, in LaunchGUI
self._set_figure_canvas()
File "/usr/local/anaconda/lib/python2.7/site-packages/artview/components/plot_radar.py", line 672, in _set_figure_canvas
self.layout.addWidget(self.canvas, 1, 0, 7, 6)
TypeError: arguments did not match any overloaded call:
QGridLayout.addWidget(QWidget): argument 1 has unexpected type 'FigureCanvasQTAgg'
QGridLayout.addWidget(QWidget, int, int, Qt.Alignment alignment=0): argument 1 has unexpected type 'FigureCanvasQTAgg'
QGridLayout.addWidget(QWidget, int, int, int, int, Qt.Alignment alignment=0): argument 1 has unexpected type 'FigureCanvasQTAgg'
I'm planning on a large refactor and reorganization of the Grid class in Py-ART and want to check on how much ARTView depends on the organization of the class. If needed it should be possible to keep a compatibility layer which allow access to the old Grid attributes but if this is not needed I would like to avoid it.
Allow user to set the style sheet for those that like dark background. Would need to institute something like the following in the plot_*.py files:
with plt.style.context(('dark_background')):
d.plot_ppi('reflectivity',0)
With the tools, I've noticed that selecting one tool after another can cause some behavior that could effect the use of ARTView. For example in Zoom/Pan is selected and then ROI is selected the user can no longer select curved lines. Instead the image pans. This can be alleviated by resetting the default.
Maybe a reset of the tools could be added or disconnecting mouse events before connecting for each tool.
Would like to implement hand editing data in Solo-like fashion. The functionality desired is to uses the polygon selected by SelectRegion instance and apply mask to that region if desired.
Has anybody had any issues installing ArtView? I was able to install Pyart without any issues but when I try to install Artview, I receive an error message.
The line I'm using to install the package into Anaconda is:
conda install -c https://conda.anaconda.org/jjhelmus artview
With the error being:
Fetching package metadata: ......
Solving package specifications: .
Error: Unsatisfiable package specifications.
Generating hint:
[ COMPLETE ]|###################################################| 100%
Hint: the following packages conflict with each other:
Use 'conda info artview' etc. to see the dependencies for each package.
Any help would be much appreciated! Thanks!
I open this to discuss if and when class should initialize V variables. I usually don't like this, as it is giving class to much power, and therefore multiple classes can conflict with each other over this.
This is happening in two places now: Menu
initialize Vradar
and Display
initialize Vlims
. The first one, I did, because the main function of Menu
is to govern over Vradar
, so it is not that bad that it open it. Nevertheless I believe this should be done some what differently, not sure exactly how.
During the reorganization process the ability to display RHI and/or Airborne data has been lost.
Need to track down why this isn't working. The program tries to pull something up and opens a window, which leads me to believe that it may be something we are doing with the Variables, etc?
An idea that came up at the AMS conference was potentially caching a limited (say 20?) Vradar instances as they are loaded if someone is navigating a directory. This would allow quicker loading of display if the graphics, etc. did not need to be rendered each time (imagine flipping back and forth).
We could make the filelist a dictionary and with None placeholders and replace with Vradar if the filename was called.
Thoughts?
I am trying to run artview on some BAMEX data that has been converted to cfradial using RadxConvert, and I get the following output. I have downloaded the most recent build to incorporate any of the bug fixes from yesterday, and that made no difference. Am I missing something or is this actually a bug of some sort?
pc111:artview-master stechma2$ python artview.py /Volumes/Data/BAMEX/development/output/20030608/
Opening file /Volumes/Data/BAMEX/development/output/20030608/cfrad.20030608_172000.631_to_20030608_172003.407_TA-E_v-999_s996_az-16.53_AIR.nc
Traceback (most recent call last):
File "artview.py", line 660, in
Browse(pathDir=fDirIn, airborne=airborne, rhi=rhi)
File "artview.py", line 133, in init
self._initial_openfile()
File "artview.py", line 395, in _initial_openfile
self._openfile()
File "artview.py", line 432, in _openfile
self.SetTiltRadioButtons()
File "artview.py", line 287, in SetTiltRadioButtons
tiltbutton[ntilt].pack(expand=1, side=Tk.TOP, anchor="w")
IndexError: list index out of range
Thanks Nick!
This I idea just came to me: add a plugin for manual velocity unfolding, that is, the user select a region (with the SelectRegion
plug-in or with some cluster identification tool) that he knows is folded and we unfold it! Using a SelectRegion
it is pretty easy, just need to link Vradar and Vpoints and perform the operation.
@jjhelmus, this is probably a pure ARTview functionality, but I just wanted to let you know, may be you have some idea that we can use/contribute to pyart.
I have two problems here.
DisplayLimits
is a example of that, unless this is to grow in the future, a function would work just as well, with the advanced that Display
is less dependent of how this is organized, as long as it keep returning valid values. I also wouldn't like to see this grow larger that it is today, this kind of tasks of selecting appropriated values, although important, tend to grow into big ugly parts.Ui_LimsDialog
uses a Qdialog, that is, it blocks the program until it is closed. I don't know if this was the intention, but if so, there is not reason to make it that complicated, with V variables etc. A simple function could work just as fine, just as selecting new title/unit does, and let Display
update Vlims. If that was not the intention, then I will wait you finish with the implementation before making further comments.Instead of buttons for Data min and max, X- and Y-axis min and max, use a popup dialog to enter new values.
I attempted to run artview on a cfradial PPI file, selected a file, and then received the following:
[snesbitt@pc105:~/python/artview]$ python artview.py ~/data/gpm/lpvex/KUM/cfdisag/20100921/
Opening file /Users/snesbitt/data/gpm/lpvex/KUM/cfdisag/20100921/cfrad.20100921_083356.000_to_20100921_083404.000_Kump_v1_s00_el0.30_PPI.nc
Opening file /Users/snesbitt/data/gpm/lpvex/KUM/cfdisag/20100921/cfrad.20100921_083356.000_to_20100921_083404.000_Kump_v1_s00_el0.30_PPI.nc
Plotting reflectivity field, Tilt 1
Traceback (most recent call last):
File "artview.py", line 636, in
Browse(pathDir=fDirIn, airborne=airborne, rhi=rhi)
File "artview.py", line 133, in init
self._initial_openfile()
File "artview.py", line 396, in _initial_openfile
self._openfile()
File "artview.py", line 440, in _openfile
self._update_plot()
File "artview.py", line 521, in _update_plot
self.cbar.set_label(self.radar.fields[self.field]['units'])
KeyError: 'reflectivity'
My file has the following fields, I'm guessing this is the issue:
short ZT(time, range) ;
ZT:units = "" ;
ZT:sampling_ratio = 1.f ;
ZT:_FillValue = -32768s ;
ZT:scale_factor = 0.01f ;
ZT:add_offset = 0.f ;
ZT:grid_mapping = "grid_mapping" ;
ZT:coordinates = "time range" ;
short DZ(time, range) ;
DZ:long_name = "reflectivity" ;
DZ:units = "dBZ" ;
DZ:sampling_ratio = 1.f ;
DZ:_FillValue = -32768s ;
DZ:scale_factor = 0.01f ;
DZ:add_offset = 0.f ;
DZ:grid_mapping = "grid_mapping" ;
DZ:coordinates = "time range" ;
short VR(time, range) ;
VR:long_name = "radial velocity" ;
VR:units = "m/s" ;
VR:sampling_ratio = 1.f ;
VR:_FillValue = -32768s ;
VR:scale_factor = 0.01f ;
VR:add_offset = 0.f ;
VR:grid_mapping = "grid_mapping" ;
VR:coordinates = "time range" ;
short SW(time, range) ;
SW:long_name = "spectrum width" ;
SW:units = "m/s" ;
SW:sampling_ratio = 1.f ;
SW:_FillValue = -32768s ;
SW:scale_factor = 0.01f ;
SW:add_offset = 0.f ;
SW:grid_mapping = "grid_mapping" ;
SW:coordinates = "time range" ;
short DR(time, range) ;
DR:long_name = "differential reflectivity" ;
DR:units = "dB" ;
DR:sampling_ratio = 1.f ;
DR:_FillValue = -32768s ;
DR:scale_factor = 0.01f ;
DR:add_offset = 0.f ;
DR:grid_mapping = "grid_mapping" ;
DR:coordinates = "time range" ;
short KD(time, range) ;
KD:long_name = "differential propagation of phase" ;
KD:units = "deg/km" ;
KD:sampling_ratio = 1.f ;
KD:_FillValue = -32768s ;
KD:scale_factor = 0.01f ;
KD:add_offset = 0.f ;
KD:grid_mapping = "grid_mapping" ;
KD:coordinates = "time range" ;
short RH(time, range) ;
RH:long_name = "correlation H-to-V" ;
RH:units = "" ;
RH:sampling_ratio = 1.f ;
RH:_FillValue = -32768s ;
RH:scale_factor = 0.001f ;
RH:add_offset = 0.f ;
RH:grid_mapping = "grid_mapping" ;
RH:coordinates = "time range" ;
short SQ(time, range) ;
SQ:units = "" ;
SQ:sampling_ratio = 1.f ;
SQ:_FillValue = -32768s ;
SQ:scale_factor = 0.001f ;
SQ:add_offset = 0.f ;
SQ:grid_mapping = "grid_mapping" ;
SQ:coordinates = "time range" ;
short PH(time, range) ;
PH:long_name = "differential phase" ;
PH:units = "deg" ;
PH:sampling_ratio = 1.f ;
PH:_FillValue = -32768s ;
PH:scale_factor = 0.01f ;
PH:add_offset = 0.f ;
PH:grid_mapping = "grid_mapping" ;
PH:coordinates = "time range" ;
Thanks Nick!
On smaller screens the display sizes and window sizes can be skewed in aspect ratio and go off the screen. We may want to implement a dynamic resizing of windows depending on system.
Will have to look how to get these numbers from the OS.
How to make artview installable/importable and how to structure it?
The file 'version' is not found if one is starting artview.py outside of the source directory.
It was suggested to include of having the drop-down tilt menu changing local tilt and the TiltWindow changing the global tilt. I open this issue to discuss this.
As for me, this is not possible, not should it be possible. I will explain why is that bellow.
This has been in my mind for some time now, it is probably something to add to pyart, but I think it is better we to develop it for internal artview usage and them suggest to pyart
For now pyart has two data containers, as I call them, Radar
for polar data and Grid
for Cartesian data. I would like to add a new one (not sure the name) for unstructured data, that is just as list of points (like RoiData). It would be similar to grid in the point that it has a fields
and a axes
dict. The axes could be of several kinds (all optional) polar coords, cartesian coods or index referring to the original object (RayIndex etc.). I thought about adding a reference to the original data container, but I'm inclined not to: if the user wants to use the index axes he must know to what radar that is referring.
@nguy I just opened some new issues to lead us in a new phase of the development. Now, however, I'm here as an user, I were in need of the following two functionalities: a display for grid data and execute pyart.map.grid_from_radars. The intent is to calibrate the interpolations option to get the best equilibrium between quality and execution time.
If we were able to develop this in the next month or so would be great. I know you have other projects and so do I, so handle things as you got time.
With the new upgrades in PR #42 I find artview, at least from the functionalities point of view, is starting to get mature and I can already see in a near future making a release changing from 0.x to 1.0. But we are not quite there, documentation for instance is still an issue.
So I would like to start making a list of things we must solve before taking this step (github milestones?). For me is also important hearing @jjhelmus some criticism on this.
I believe we could use this opportunity for making a little bit of publicity, may be making a small video to show how it work (for me it is really difficult to understand a graphical software with just pictures and text).
How to avoid windows being buried unter other ones? what tool pyqt give to define the window position and size?
I've realized that if we want user to take advantage of artview versatility we must first show them that versatility and give a starting point. I thing the best way of doing that is increasing the number of scripts and show they better, so I did it and also took some printscreens
(https://github.com/gamaanderson/artview/blob/ng-gatefilter/SCRIPTS.md)
let me know what you think. And FYI this is build over the ng-gatefilter branch
ARTview appears to only consider the possibility that valid radar.scan_type attributes can be 'rhi' or 'ppi'. In fact, valid, plottable scan types include 'sector' and possibly other names. I changed the 'ppi' test in the #CreatePlot section from
if self.radar.scan_type == 'ppi':
to
if self.radar.scan_type != 'rhi':
And I was able to plot up my 'sector' scan_type files just fine. I recommend more flexible checking for PPI vs. RHI style scans, or to just remove the tests and let ARTview fail with a warning if the scan type is really odd.
tl;dr - Right now, to avoid the possibility that the program might fail on an unusual volume, the program is failing on perfectly valid volumes.
With merge #84, documentation should be updated and an example added.
This is a project documentation noting I would like to implement a masking feature as a plugin.
It should:
Much of the masking can be done using a Pyart Gatefilter.
The Region of Interest plugin can be connected and a commit is proposed to make this possible.
So I will just write something here, as to force myself to get to work.
I have some experience in dynamical visualization and I believe to have some good ideas about it. I have this in bit-bucket (https://bitbucket.org/gamaanderson/radar_viewer), but I wouldn't recommend since it is really a mess, but it work really good for what I use.
Any way, what I learned is that visualization software should use classes to different functions, in your case that initially means manipulating files should be separated from visualizing, that would allow for example visualizing the same file with two windows side by side but with different moments but the same elevation. Handling this is a little trick, but there is a nice way for doing it.
Ok, this was terrible writing, I can't really explain what I mean, but I will do some work in the code to show it.
It would be nice to have the value where mouse is hovering printed either on the screen (say status bar) or in a separate window.
@pfhein
Paul added zoom and pan functionality into ARTView. Values were not being passed back.
I worked through this issue I believe and both functionalities seem to work. In addition, they now report back to the new popup window to modify all limits.
I generalized the code a bit more. Check out the new version I uploaded this morning and see what you think. You should be able to 'fetch' and then merge with your fork if you like the changes.
I am thinking that I would like to try to make the Zoom/Pan function a selectable option from a dropdown menu. What do you think?
I would like to know how easily can an inexperience user go around artview. For that I've made the following list
I don't know if we even have guinea pigs to try to go this check list, however I wanted to write it down!
@gamaanderson I want your opinion here. I need to bring the filename over from the menu (or Variable) class to the Display to assign in the new QLabel at top. Which do you think would be easiest?
I was thinking it may be as simple as passing a filename
keyword to the plot_radar.py
Display class?
Below is a list of potential features (some ongoing) that may be desirable in ARTview:
Unfortunately we introduced a minor problem with Python 2.6. Importlib is not in Python 2.6. Fortunately there is a very small module to download that adds importlib. My gut feeling on it is that Python 2.6 is not worth supporting. I am seriously considering moving away from Python 2.6. It is too old. Python 2.7 seems to be the current standard and python 3.4 seems to be the future.
Make plug-in for more advanced io operations. It must in any case have a tree view of directory. It could also have an optional cache of radar files.
Another tool to add would be the selectable area zoom. I would envision some sort of lasso or rubberband operation.
Additionally, a selectable area details would be useful (e.g. min, max, histogram plot...)
I have realized that it can happen that several V variables are changed at the same time, e.q. Vradar and Vfield because the new radar doesn't have the previous field. I'm not fully sure what would happen then, but I believe _update_plot() would be called twice, this would be wasted time to say the least.
I will present my thoughts bellow.
We should look into adding a secondary check to try pyart.aux_io
read routines to support a greater variety of radar files.
A Region of Interest class was introduced in Pull request #24 that allows the user to select a region. The next step is use this defined region to perform operations. I am thinking the following steps could be done:
Let me know what you think of these points. Do they seem reasonable?
Also, this is tangentially related to Issue #9 and I wondered if the rubberband idea has been outdated by the current zoom function or if it still has application.
With the previous PR #36 some improvements were implemented in the plot function. ARTView now loads the correct default limits. It also produces the correct plot. The issue is that the previous plot is not removed. I've tried multiple erase methods with no luck. I'm stumped right now.
ARTView does plot correclty on startup for PPI, RHI, and airborne.
We are currently dealing with some ugly hacks, for instance this one, that I believe the correct way of dealing would be using pyart configuration tools with the follow principle: "artview should always work with a unconfigured pyart, but a more smooth user experience does presuppose pyart is correctly configured"
However, for that to work we need pyart to give more focus to this possibility, this could be as simple as mentioning the PYART_CONFIG flag in README. What do you think of it @jjhelmus?
Related to this, we should also configure standard values for the scale (vmin and vmax) and colormap for every (pyart-)field. We are currently using values that are only useful in reflectivity-like fields, I have dificulties even for visualizing radial velocity. Couldn't pyart.graph also benefit from those standards?
When we select the Data Min and Data Max the artview changes the color bar, but it is not masking the data outside the interval (Data Min, Data Max). For example, if the data is under 10, when we set Data Min = 15, everything becomes blue, but it is still plotted. I realized it while using plot_ppi(), and in this case the solution is very simple. One of the options of plot_ppi() is mask_outside=False (this is the default), if we set mask_outside=True in /artview/artview/components/plot_radar.py, in line 741:
self.plot = self.display.plot_ppi(
self.Vfield.value, self.Vtilt.value,
vmin=cmap['vmin'], vmax=cmap['vmax'],
colorbar_flag=False, cmap=cmap['cmap'], mask_outside=True, #<------------------------
gatefilter=gatefilter, ax=self.ax, fig=self.fig, title=title)
then data outside (Data Min, Data Max) will be masked and artview will display only the data between Data Min and Data Max.
This option, mask_outside, also exists in plot_rhi(), and in plot_sweep_grid().
This is just a reminder to handle in the future.
Several parts of the new code were copied from the old one, So I believe we should revise it to make sure it has no bugs hidden, or if it could be done better. I'm specially concerned with the _update_plot() and how the zoom/pan is working.
@tjlang complained about the unexplanatory name of the variables in the ComponentControl, suggesting to use "longer" names instead of the internal name. Although I agree with him from the user side, I'm not inclined to do that, since this means that plug-ins writers would have to register their variables names inside artview. This is contradictory with our intention to make life as easy as possible for writing plug-ins and incrementing artview.
I much more inclined to change the name of the variables for more explicit ones. Here is a list of the shared variables names being used:
lims
and cmap
are the one that most worry me, better name could be zoom
and colormap
; container
is just an alias for radar
or grid
, so in theory it doesn't appear for the user
Python 2.6 does not support the -m option the same way as Python 2.7. python -m artview
does not work but python -m artview.__main__
works with an error. That is only a documentation change for Python 2.6.
I think it works but the error I believe is a different problem. The error it produces is the same as when I try to run it in the "installed folder". This may not be due to Python 2.6 but the error below is the error that I am getting.
denali:~$ python -m artview.__main__
/usr/local64/lib64/python2.6/site-packages/numpy/oldnumeric/__init__.py:11: ModuleDeprecationWarning: The oldnumeric module will be dropped in Numpy 1.9 warnings.warn(_msg, ModuleDeprecationWarning)
./
Traceback (most recent call last):
File "/usr/lib64/python2.6/runpy.py", line 122, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File "/usr/lib64/python2.6/runpy.py", line 34, in _run_code
exec code in run_globals
File "/home/software/usr/local64/lib64/python2.6/site-packages/artview/__main__.py", line 15, in <module>
artview.run(DirIn, filename, field)
File "/usr/local64/lib64/python2.6/site-packages/artview/scripts/standard.py", line 22, in run
MainMenu = Menu(DirIn, filename, name="Menu") #initiate Vradar
File "/usr/local64/lib64/python2.6/site-packages/artview/components/menu.py", line 47, in __init__
super(Menu, self).__init__(name=name, parent=parent)
File "/usr/local64/lib64/python2.6/site-packages/artview/core/core.py", line 95, in __init__
super(Component, self).__init__(parent=parent, flags=flags)
`AttributeError: 'flags' is not a Qt property or a signal``
I'm working on Issue #57 right now, creating a GateFilter plugin.
It seems that being able to get a list of just variables would be ideal for this application. I've tried a couple of different methods without success thus far. I've been concentrating on something like SelectRegion
where a display is chosen, but I'd like to drill down to the variables somehow from this.
The reason being that a list of variables is required to create the window output for potential masking on each field.
I open this to discuss how the Toolbox display should work.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.