ceholden / tstools Goto Github PK
View Code? Open in Web Editor NEWQGIS2 plugin tools for remote sensing timeseries
License: GNU General Public License v2.0
QGIS2 plugin tools for remote sensing timeseries
License: GNU General Public License v2.0
Passing around reference to the currently opened timeseries is stupid. Move this into a module which keeps track of storing reference. This would:
Add an option to draw a point or box in/around the pixel selected by plot tool
Stop using print statements and use logging module for debug and for things not shown in QgsMessageBar
Create series of tabs for each series of configuration options?
Abstract two things from controller to the time series drivers:
Add them...
CCDC v11 doesn't always work with the DOY plot for some of the simpler fits. Root cause of problem is TS driver which can feed empty np.array objects.
ipdb> ts.get_prediction(4)[0][0]
array([], dtype=float64)
ipdb> ts.get_prediction(4)[1][0]
array([], dtype=float64)
ipdb> ts.result[0]
{'category': 3, 'rmse': array([0, 0, 0, 0, 0, 0, 0], dtype=uint8), 'magnitude': array([ -340.09516338, -275.7719292 , -253.40984124, -329.00342285,
-423.18663196, -250.63092108, -2455. ]), 'pos': 48804583, 't_start': 728020, 'change_prob': -0.16666666666666666, 't_break': 0, 'num_obs': 1, 't_end': 728020, 'coefs': array([[ 768, 910, 866, 2623, 2421, 1351, 2455],
[ 0, 0, 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 0, 0, 0]], dtype=uint16)}
Check to make sure the result has nonzero length
We can actually fit a value for the timeseries example given above - just predict that one date. Problem comes from this line of code:
_mx = np.linspace(rec['t_start'],
rec['t_end'],
rec['t_end'] - rec['t_start'])
We need to check if t_end == t_start and specify "num" parameter as 1.
Add ability to include additional metadata for images in time series
When time series data is asked to be fetched for a given pixel, do so in a separate thread so the main GUI window doesn't freeze up.
Reference:
http://svn.reprojected.com/qgisplugins/trunk/threading_demo/threading_demo.py
Select between the original data and residuals (observations - model prediction)
So since we don't have the usual documentation for this yet, yank out the help
folder entirely. This isn't some kind of API so a Sphinx based documentation style that came with the Plugin Builder defaults isn't useful. Would prefer to just write documentation that is accessible via Github (gh-pages too?) as markdown, which gets built into HTML if anyone wants it within the deployment. So:
Allow user to click (with some x/y tolerance) a plotted value in the time series and add the corresponding image to the QGIS list of layers.
When you click a value on the plot:
Make that checkbox work so that the observation causing a break is highlighted in some manner
Create timeseries driver for BFAST monitor.
Plan:
rpy2
#2
to create a timeseries driver for running BFAST monitorTime series plot should have an option to automatically scale the Y axis according to the pixel max/min +/- some wiggle room for each band...
This option will disable manual configuration of min/max, but will update the QLineEdits to the appropriate value.
Refactor CCDCTimeSeries into more generic "LayerStackTimeSeries" which only does a "pass" to the results methods, or doesn't implement them at all (and thus wouldn't be an option in the QComboBox of drivers).
Culprits:
Simple fix for any contributed drivers, but should fix asap
Would be helpful to have more control over what images are used in timeseries
E.g. self.symbology_hint = [3, 2, 1]
On init of timeseries, if hasattr(ts, symbology_hint)
, then settings.p_symbol will change accordingly.
Example:
I load an 8 band dataset and then change to one that has 11 bands. The plot band QComboBox still shows 8 bands available but is referencing the new dataset
TODO:
Tear down plot options before switching to new timeseries
Use Python datetime & matplotlib's time module to show the year on the x-axis of the plot.
Reference:
http://matplotlib.org/examples/api/date_demo.html
http://stackoverflow.com/questions/13965740/converting-matlabs-datenum-format-to-python
Maybe put this in the options tab? See #14 as a feature that may accompany this within a pop-up
Cache requested pixel time series objects...
Can we use functools.lru_cache?
Reference:
http://docs.python.org/3.4/library/functools.html
Ask TS for attributes to add to table. Examples:
It is possible to deselect an active layer in QGIS and have it removed from self.mapCanvas().currentLayer(). If so, then we don't do a plot because the current layer is None (ccdctools.py line 106).
Suggestions:
Make this repository citable by adding a Digital Object Identifier (DOI)
Add some sort of mouse driven information to the time series plot i.e. a tooltip, side corner display, etc.
Have in some QComboBox a list of a variety of different strftime formats with example.
The question is, where do we want to put this? Toolbar configuration option? Popup configuration at the top of this tab?
Include number of bands in unique filename:
n500_x1_y1.npy --> n500_b8_x1_y1.npy
Create new tab on controls that includes various utilities like a zoom to row/col
IOError on existing file
Path doesn't save if you cancel from file dialog
IndexError or TypeError if user has a file extension in specified file name
etc...
When dealing with large datasets over the network, it may take a while to fetch the data. Implement a fix such that the mouse click event is disabled or ignored until the data collecting method has returned.
Optional?: add a spinning wheel, notice, etc.
Create a set of API tests for classes that inherit from AbstractTimeSeries
so that people can more quickly develop new drivers.
This should basically serve as a more exact set of documentation for what the class methods ought to return/yield/etc.
Restructure code according to guidelines in milestone 0.6
Line 623 in controller.plot_add_layer
AttributeErrpr: 'PathCollection' has no attribute 'get_data'
The "images" tab on the control panel widget should display the following for all images in time series:
cbox on/off | sensor type | date
where:
i.e.
[X] | LT5 | 2010-031
Reference:
http://stackoverflow.com/questions/12366521/pyqt-checkbox-in-qtablewidget
Qt & QGIS can help save plugin preferences, so we should take advantage of it. For example, we store the last opened image stack directory, last used plot and image display options, etc.
Reference:
http://www.qgis.org/pyqgis-cookbook/settings.html
It would be nice to be able to change the plot symbols to represent categorical information about each image in the time series. For example, you could use stars for LE7 imagery and squares for LT4/5.
Add dialog box in the plot tab to set symbology. Parse available metadata for each image in time series and add them as radio buttons. When one is selected, parse all unique values into a table that allows user to select the symbol type (circle, dot, square, etc) and the symbol color via combobox.
Much harder to accomplish - parse metadata and also include the image bands. Allow user to add categories with symbol and color. Categories are defined via expression system that allows logical operators to work on the variables and bands (e.g., sensor type == LT5 AND B5 < 3000). Big TODO since user has so much freedom with this.
Currently, QTableWidget stretches last column only by taking advantage of the option horizontalHeaderStretchLastSection.
Write a subclass of QTableWidget or something similar to enable auto-stretching of all columns, not just the last.
Submit request for TSTools to be added to QGIS repo:
Some requirements:
Eli suggested applying the marker symbology choice to the DOY plot so we retain color information ~ year.
We should be able to plot multiple timeseries on the same graph and be able to select which Y axis (btw we also now have multiple Y axes) to plot the data.
E.g.,
Bands menu is a pulldown, but we use tri-state checkboxes to indicate OFF, left Y axis, or right Y axis
We can also put QCheckBox into QComboBox (SO reference]
Long overdue...
Icons for configuration and plot tool
Move MapTool clicker from widget checkbox to item in toolbar and add in a button to toolbar that brings up configuration menu.
Issue: Once custom symbology has been applied to time series visualization, switching to new scene causes an error that prevents the new scene data/time series from being loaded.
Need a way to clear previous symbology file (manually or automatically) so new scene/symbology can be applied.
Process that generates the error:
Load time series for first scene (e.g. p012r031)
Click to display time series for a pixel
Plot > Symboloy > Attach metadata > Browse to csv (e.g. /projectnb/landsat/projects/IDS/p012r031/p012r031_attributes.csv) > Load > Attach metadata
Change colors/marker shapes and Apply (Note: attribute field for visualization in these files is "Score")
At this point, everything has been working fine.
Now, when I try to switch to a new scene, the following error message is generated 3-10 times:
An error has occured while executing Python code:
Traceback (most recent call last):
File "/project/earth/packages/CCDCTools/tstools/controller.py", line 174, in update_display
self.ctrl.symbology_controls.parse_metadata_symbology()
File "/project/earth/packages/CCDCTools/tstools/controls_symbology.py", line 321, in parse_metadata_symbology
self.md[row - 1] = getattr(tsm.ts, tsm.ts.metadata[row - 1])
AttributeError: 'YATSM_LIVE' object has no attribute 'SCORE
Then another error:
An error has occured while executing Python code:
Traceback (most recent call last):
File "/project/earth/packages/CCDCTools/tstools/tstools.py", line 137, in config_accepted
custom_options)
File "/project/earth/packages/CCDCTools/tstools/controller.py", line 151, in get_time_series
self.ctrl.init_plot_options()
File "/project/earth/packages/CCDCTools/tstools/controls.py", line 181, in init_plot_options
self.symbology_controls.setup_gui()
File "/project/earth/packages/CCDCTools/tstools/controls_symbology.py", line 57, in setup_gui
self.setup_tables()
File "/project/earth/packages/CCDCTools/tstools/controls_symbology.py", line 82, in setup_tables
self.md = [getattr(tsm.ts, _md) for _md in md]
AttributeError: 'YATSM_LIVE' object has no attribute 'DATE'
Then, there is a red message in the load bar area that "Time series failed to load". I need to close and re-open so I can load and work with next scene--very frustrating!
I'm hoping that there is a way to clear the custom symbology from the first scene before loading the next, either automatically or through an option in the symbology window.
Please let me know if you have any questions or need more info to recreate this sequence of errors.
Val
If you initialize a new timeseries that has a different projection AFTER the "query" layer has already been created, the "query" layer will be moved but the projection is not updated.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.