Clean up warning messages in base.py:

/Users/gtucker/Dev/landlab/landlab/grid/base.py:1239: SyntaxWarning: name 'azimuths_as_displacements' is used prior to global declaration
  global azimuths_as_displacements
/Users/gtucker/Dev/landlab/landlab/grid/base.py:1247: SyntaxWarning: name 'dummy_nodes_1' is used prior to global declaration
  global dummy_nodes_1
/Users/gtucker/Dev/landlab/landlab/grid/base.py:1248: SyntaxWarning: name 'dummy_nodes_2' is used prior to global declaration
  global dummy_nodes_2
/Users/gtucker/Dev/landlab/landlab/grid/base.py:1249: SyntaxWarning: name 'dummy_nodes_3' is used prior to global declaration
  global dummy_nodes_3

Grid[‘cell’][‘name’] == Grid[‘node’][‘name’][grid.core_nodes], and at the moment it isn’t. Both node & cell (and link & face) should point at the same addresses.

Add a fields keyword to write_netcdf that will allow a user to write only specific fields to the file. See issue #90 for a description of why this might be useful.

I wonder how we can modify the code to take a grid of rainfall data instead of one single value? If we can modify the code in a way to take one rainfall value for each DEM grid cell that would be a more realistic simulation.

Make an automated tool to make movies of LL output.

In the overland_flow_with_model_grid_dem.py example, the number of active links changes after the call to set_nodata_nodes_to_closed(). If you do

q = mg.create_active_link_array_zeros()

it recognizes the change. But

q = mg.add_zeros('active_link', 'name')

uses the original number of active links, not the modified value.

The two README files:

• README.rst
• README.md

are out-of-date. In addition, there are two README files.

Katy Barnhart has added GIS-style slope-aspect methods to LL. They're super useful, but slow (looped). Accelerate them.

The installation instructions for landlab needs to be updated. In addition, the developers guide need to be updated to reflect the move to git/github.

Some of the unit tests are failing. It looks like they are mostly related to doctests in the vegetation_ca and single_vegetation components but are not all that serious.

Grid[‘node’][‘name’] = 1. or array([1.]) should both be legitimate commands, since numpy should be happy to expand out these arrays whenever they get used.

A Wickert has a case (already!) where he wants to work with non-square raster grid. RMG can't do this. We need a separate grid class that can!

Having issues with any drivers from the examples folder that include stream power/ flow routing (coupled_driver, stream_power_driver). We've tried these on 4 different machines (3 OSX, 1 windows) and on only one (OSX) of them does the stream power component work to drive incision into the landscape. With the other three topography is essentially unchanged.

Same versions between all 3 macs, all using Canopy, same python/numpy versions.

As reported in #78, on some machines the unit tests are failing on functions that use scipy.weave. I've opened this issue to contain information specific to the scipy.weave problem.

I've tried running the unit tests on two different Mac's (10.9 and 10.10 - gcc 4.2.1) and Linux (Ubuntu) and see the same errors. I've also tried using both the Anaconda and Canopy distributions of scipy as well as building my own scipy from source but continue to get the same errors. However, @SiccarPoint runs the same tests without error on his machine (Mac 10.10, gcc 4.2.1, Canopy).

The good news. I've had some luck with the add_to_stack method in flow_accum_bw.py. Making the following changes to the code seems to eliminate the errors,

-        delta_l = numpy.take(delta,l)
-        delta_lplus1 = numpy.take(delta,l+1)
+        delta_l = int(numpy.take(delta,l))
+        delta_lplus1 = int(numpy.take(delta,l+1))
+        l = int(l)

The only changes I made were to explicitly make delta_l, delta_lplus1 and l ints (perhaps they had been unsigned ints of a different size?). Anyway, that seems to fix the problem for that one case.

One last note. When testing these things you may have to empty your cache. It appears scipy compiles some code in a cache so it doesn't have to compile it again. However, it doesn't seem smart enough to recompile even if a change was made. On my machine the cache is under,

~/.cache/scipy/python27_compiled/

I'm trying to use write.netcdf to get drainage area out of a driver I have built but get this error.

"ValueError: NetCDF 3 does not support type int64"

when I do get a netcdf file out it contains x and y coordinates and k_values but not the other two variables in the RasterModelGrid (topographic elevation or drainage area).

Need to clean up and standardise io interfaces for modules. "Belt and braces" approach to always passing the grid appears unnecessary, but there does need to be a grid update technique (method to call?) that can update the internal grid if necessary.

DEJH is going to do this for his modules, and see how it goes.

I have a suspicion that parts of LL might still not be using the right definitions of nodes, cells, and the derived "types" of those things ("core", "active"==not_boundary, "boundary"; "perimeter", "interior" for the specific case of a regular grid).

I'm particularly worried about calls to things like grid.number_of_active_cells.

Ensuring we have unit tests covering these cases seems essential. The real test will be small grids with interior boundary nodes & cells. Do we have these? I can add if we don't.

It seems as though tests are always being reported on Travis as passing, even when there are failures. Fortunately, I think all tests are currently passing in master so it shouldn't be a big deal yet. I believe it is due to .travis/run_travis_test.sh not returning the correct status code and should be an easy fix.

landlab.plot.imshow module functions need docstrings. Functions like plot_field are too opaque to use as-is.

I want to be able to do

mg.existing_data_fields
[ 'planet_surface__elevation',
'water_flow__discharge',
'planet_surface__gradient',
... ]

or something like it. We might already even have this functionality & I just don't know where it is? (Presumably there's a dictionary somewhere, that can be interrogated for its keys?)

(I'm still struggling with the fact that the code for the fields isn't intuitive, so spotting what functions are available is hard...)

It was suggested that the stream power component would be a good one to wrap with a basic modeling interface. This will mostly be a wrapping of the StreamPowerEroder class so that it exposes the BMI methods.

Edit instructions to have a single "recommended" method (GitHub GUI), moving alternative methods to private docs.

We hold a lot of lists that need to be updated every time the BCs are changed in the grid. These include

• active links
• active diagonals
• activecell_node
• corecell_node

At the moment, any function changing the BCs needs to also update all of these individually, and it's easy to forget. We need a bundled method that does all this in one go.

We should fix all of the outstanding broken unit and doc tests. It appears that the only remaining ones are under the flow_router component, which looks like it is owned by @gregtucker.

Make a generic interface for LinkCA models that reads states and transitions and other parameters from an input file.

The docstrings for the functions in raster_funcs.py need some updating.

I'm a newbie to Python and am having problems installing LandLab. I've followed the installation directions to run "python setup.py install" from the root directory of the landlab clone, but get an error each time. I've tried running it in Canopy and get this error:
File "", line 1
python setup.py install
^
SyntaxError: invalid syntax

When I try directly in the command line, this is the error I get:
File "setup.py", line 57
<<<<<<<<< HEAD
SyntaxError: invalid syntax

Any help would be appreciated.

LinkCA should implement file output (vtk and/or netCDF).

It would be nice if a ModelGrid had load and save methods that would write a grid along with it's fields to various formats. To start with, this wouldn't be much more that wrapping write_netcdf and similar functions.

The landlab documentation on readthedocs is not building and probably hasn't been for some time. It appears that Sphinx is unable to find the numpydoc extension. It looks like it broke about three weeks ago with bb3c99b although I can't see why that would have broken things. Most likely it has something to do with the way readthedocs is building the documentation.

In a linkCA model, the CA should run until either (1) it's time for plotting, (2) it's time to write output to file, or (3) it's time for some kind of external modification (such as "uplift"). So, the call to run should have a duration that represents the time to the soonest of these.

I am trying to install Landlab for the first time but I am getting the following error:
"Traceback (most recent call last):
File "<pyshell#18>", line 1, in
from setup.py import install
File "C:\Flash Flood\landlab\landlab-master\setup.py", line 10, in
from landlab import version
File "C:\Flash Flood\landlab\landlab-master\landlab__init__.py", line 45, in
from .testing.nosetester import LandlabTester
File "C:\Flash Flood\landlab\landlab-master\landlab\testing\nosetester.py", line 3, in
import nose
ImportError: No module named nose"

Any comments why I am getting the error? How can I solve it?

Thanks and regards

Run tests in Mac OS X.

Hi there,

I'm attempting to read in a discharge map into a driver using read_netcdf and then use this discharge in the stream power eroder. I've been following the code in the test_read_netcdf file and am not sure exactly how to get the data out of the files so that I can implement it into stream power. Would you be able to add a few comments to the read_netcdf file or look at my code and let me know what I'm missing? Thank you!

from landlab.components.flow_routing.route_flow_dn import FlowRouter
from landlab.components.stream_power.stream_power import StreamPowerEroder
from landlab.components.nonlinear_diffusion.Perron_nl_diffuse import PerronNLDiffuse
from landlab.components.diffusion.diffusion import DiffusionComponent #the two different diffusion formulations
import os, random #to manipulate path names
import time #timekeeping
import numpy as np #numerical operations
import pylab #to plot results
from landlab import RasterModelGrid #import grid class
from landlab import ModelParameterDictionary #handles input from input file
from landlab.plot.imshow import imshow_node_grid
from landlab.io.netcdf import read_netcdf, write_netcdf, NotRasterGridError, WITH_NETCDF4
from landlab.plot.imshow import imshow_node_grid 
from nose.tools import assert_equal

os.chdir('/Users/blynch/Research/Working_Directories/landlab/driver_bml')
#dem_name= 'DEM' #test_script_for_route_flow_dn.py
#outlet_row =  #test_script_for_route_flow_dn.py not sure if I need this outlet
#outlet_column =  #test_script_for_route_flow_dn.py

#get needed properties to build grid:
input_file = 'driver5_params.txt'


#initialize an object that will supply the parameters:
inputs = ModelParameterDictionary(input_file) #sp
nrows = inputs.read_int('nrows') #sp
ncols = inputs.read_int('ncols') #sp
dx = inputs.read_float('dx') #sp
dt = inputs.read_float('dt') #sp
run_time = inputs.read_float('run_time') #sp/diffusion
uplift = inputs.read_float('uplift_rate') #sp
init_elev = inputs.read_float('init_elev') #sp
#leftmost_elev = inputs.read_float('leftmost_elevation') #diffusion_driver
initial_slope = inputs.read_float('initial_slope') #diffusion_driver
nt = int(run_time//dt) #divide and truncate (number of timesteps) diffusion_driver

#Instatiate the grid object, 
mg = RasterModelGrid(nrows, ncols, dx) #sp

#create the fields in the grid
mg.create_node_array_zeros('topographic_elevation')
z = mg.create_node_array_zeros() + init_elev
mg['node']['topographic_elevation'] = z + np.random.rand(len(z))
mg.create_node_array_zeros('drainage_area')
a = mg.create_node_array_zeros() 
mg['node']['drainage_area'] = a + np.random.rand(len(a))
#ar = mg.node_vector_to_raster(mg['node']['drainage_area'])

#set boundary conditions
mg.set_fixed_value_boundaries_at_grid_edges(True, True, True, True) 

#make some K values in a field to test 
mg.at_node['K_values'] = 0.1+np.zeros(nrows*ncols)


print( 'Running ...' )
time_on = time.time()

#instantiate the components:
fr = FlowRouter(mg)
sp = StreamPowerEroder(mg, input_file)
diffuse = PerronNLDiffuse(mg, input_file)
lin_diffuse = DiffusionComponent(grid=mg, input_stream=input_file)


#perform the loop

uplifted_nodes = mg.get_core_nodes() #perform block uplift of interior of grid, leave boundary nodes at original elevations

#stream power loop
elapsed_time = 0. #total time in simulation
while elapsed_time < run_time:
    print elapsed_time
    if elapsed_time+dt>run_time:
        print "Short step!"
        dt = run_time - elapsed_time
    mg = fr.route_flow(grid=mg)

    # read in discharge map 
    randnum=random.randint(1,10)
    discharge_filename=('discharge' + str(randnum) +'.nc') 


    os.chdir('/Users/blynch/Research/Working_Directories/landlab/driver_bml/discharge_maps_nc_2') #go into directory of discharge_maps
    def test_read_netcdf3_64bit():
        read_netcdf('discharge_filename')
        assert_equal(mg.shape, (4, 3))


    mg.at_node['discharge_map']=discharge_filename

    os.chdir('/Users/blynch/Research/Working_Directories/landlab/driver_bml') #leave discharge_map directory


    mg,_,_, = sp.erode(mg, dt, node_elevs='topographic_elevation', node_drainage_areas='drainage_area', slopes_at_nodes= 'steepest_slope', K_if_used='K_values', Q_if_used='discharge_filename')
    #add uplift
    mg.at_node['topographic_elevation'][mg.core_nodes] +=uplift*dt
    elapsed_time += dt

    #diffusion driver loop
    for i in xrange(nt): # loop nt times, xrange is clever memory-saving way of prodcing consecutive integers to govern a loop)
    #this line performs functionality of componen,swap between these two
        mg = lin_diffuse.diffuse(mg, i*dt) #linear diffusion
        #mg = diffuse.diffuse(mg, i*dt) #nonlinear diffusion
        #mg = fr.route_flow(grid=mg)
        #mg = sp.erode(mg, dt, 'drainage_area')

    #plot N-S cross section from this stage in the run onto figure 1. 
    pylab.figure(1)
    elev_r = mg.node_vector_to_raster(mg['node']['topographic_elevation']) #turn the 1-D array of elevation values 
        #into a spatially accurate 2-D gridded format, for plotting
    drainage_area = mg.node_vector_to_raster(mg['node']['drainage_area'])
    discharge_filename=mg.node_vector_to_raster(mg['node']['discharge_filename'])
    im = pylab.plot(mg.dx*np.arange(nrows), elev_r[:,int(ncols//2)])
    im = pylab.plot(mg.dx*np.arange(nrows), drainage_area[:,int(ncols//2)])
   # im = pylab.plot(mg.dx*np.arange(nrows), discharge_filename[:,int(ncols//2)])
        #square brackets: subset of nodes to use
        #this type of data extraction from a larger data structure is called slicing/ fancy indexing 
time_off = time.time()
print 'Elapsed time: ', time_off-time_on

The documentation and unit tests for the landlab.field needs updating.

mg.node_slopes_using_patches() becomes very slow for very large (10^5 nodes) grids. I think this is a consequence of storing ragged lists as matrices, so a single many faced polygon (likely in really big grids) can really affect speed disproportionately.

Can we think of a way round this? ...remove long lines from the matrix, solve the rest, then put these "back in" row by row afterwards?? Also need to demonstrate it is this effect causing the slowdown in the first place before fixing it this way...

There are some broken unit tests in the landlab.utils.structured_grid module.

Sorry to land this on you Eric, but I still don't really follow the nuts and bolts of the fields!

landlab.field.grouped.ModelDataFields.has_field() appears to be giving wrong answers. I get this:

mg.has_field('node', 'a_field_that_isn't_defined_on_nodes_or_indeed_at_all')
True

Which is clearly wrong - mg.keys('node') confirms it. Could you debug? Thanks!

The plot.imshow method as is often does not plot some of the perimeter polygons. We either need it to colour all of these, or none of them.

This is a known display "issue" for voronoi_plot_2d, which we use. There is code to resolve it at the link I left in the code comments.

With a 7-node hex grid, the node in/out link matrices are incorrect. They seem to be rotated 90 degrees.

Running the unit tests from the Python interpreter like,

> python -c 'import landlab; landlab.test()'

gives different results than when running with nosetests

> nosetests

This is the error that was first reported by @nicgaspar.

Some differences are:

• Tests are run in example folders
• Typecasting warnings are reported (as they should be!)
• netCDF4 tests are run even if netCDF4 support is disabled

The typecasting warnings should be fixed as they are probably errors.

The model_grid.rst document needs to be cleaned up. The main problems are:

• unresolved conflicts still appear in the document
• toward the end of the document TeX still needs to be converted to reStructuredText

@SiccarPoint notes that the function boundary_cell_count should be deprecated as there are no boundary cells in the sense that this function thinks there is. However, @saisiddu disagrees. See landlab/utils/structured_grid.py for details.

It looks to me like boundary_cell_count makes sense but currently it returns the wrong number. Since perimeter nodes don't have cells, this function should return the number of outermost cells. For instance,

>>> boundary_cell_count((3, 4))
2
>>> boundary_cell_count((6, 5))
12

Another change might be to rename the function to perimeter_cell_count.

We need equivalents to rmg.number_of_patches, rmg.patch_nodes, and rmg.node_patches() for voronoi/irregular grids.

Once we have this, rmg.node_slopes_using_patches() can be promoted to base.py.

The current netCDF reader reads UGRID formatted netCDF but gives an error when trying to read netCDF files from GEBCO. It would be nice if landlab could read GEBCO netCDF files.

I am getting following error while I am trying to load "landlab.io":

Traceback (most recent call last):
File "C:\Flash Flood\landlab\landlab-master\docs\model_grid_guide\overland_flow_with_model_grid_dem.py", line 11, in
from landlab.io import read_esri_ascii
ImportError: No module named landlab.io

The unit tests are broken again. It appears they are failing on tests that involve functions like calculate_steepest_descent_across_cell_faces that now use masked arrays. @SiccarPoint, I think these are changes you added so I'm going to pass it over to you. Sorry.

It addition, scipy.weave is reporting warnings. They're not errors so maybe there's no problem there but it would be nice to clean those up, if possible.

The readthedocs master index and "index of key functions" are both missing some grid attributes. Missing attributes include: node_inlink_matrix, node_outlink_matrix, node_numinlink, node_numoutlink. There may be others.

All gradients are now measured such that a quantity decrease is a negative gradient (even if that quantity is elevation). The unit tests need to be updated to reflect this change.

In addition, function names that calculate gradients were changed. Unit tests need to be updated to reflect these changes.