jblindsay / whitebox-tools Goto Github PK
View Code? Open in Web Editor NEWAn advanced geospatial data analysis platform
Home Page: https://www.whiteboxgeo.com/
License: MIT License
An advanced geospatial data analysis platform
Home Page: https://www.whiteboxgeo.com/
License: MIT License
Hello,
First of all thank you very much for the Python API to your tools. We do tree species identification so we need to generate canopy height models using lidar DTM and DSM (I would love to be able to subtract the results of the two lidar_tin_gridding operations without saving the intermediate rasters but I digress!)
Is there a way to set the projection on the output rasters to the one from the input las files? I searched in the manual but could not find any information.
Hello!
There is one error when I use "FeaturePreservingDenoise".
./whitebox_tools -r=FeaturePreservingDenoise -v --wd=/home/keo/nnd_storage/thirdpart_software/whitebox-tools/testdata/test_dsm/ -i=DEM_gaps_filled.tif -o=DEM_smoothed.tif --filter=15 --norm_diff=20.0 --num_iter=4
thread 'main' panicked at 'Unrecognized tool name FeaturePreservingDenoise.', src/main.rs:66:21
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
Hello,
I am working with another Python package (pyfor) that exhibits similar behaviour so am recycling a large part of my post from there, I hope that is OK
I am trying to understand the behaviour of the interpolation when I am creating surfaces such as a DTM. As mentionned in my email discussion, our point clouds are already classified so I filter out the ground returns then use the following in pyyhon to create the DTM raster.
wbt.lidar_tin_gridding( i = name, output = normpath(DTM_dir + "/" + las_basefilename + "_DTM.tif"), parameter="elevation", returns="all", resolution=0.25, exclude_cls='1,3,4,5,7,8,9,18')
We are processing 1600 buffered 1km^2 las tiles (40m buffer) to create DTM, DSM and CHM which then gets used by our ITC segmentation script. The rasters are at 25cm resolution so the output raster tiles should be 4160*4160. Our previous method using the lidR package in R produced this result. We are super happy with the speed of wbt (20 seconds vs 20 minutes in R...), it is exactly what we need. I think my supervisor (Benoît St-Onge at UQAM) and you should have a chat one day by the way!
When I use whitebox (and pyfor), the rasters have uneven dimensions such as 4079*4159 for example. I understand what is happening, the ground las points do not reach the corner of the declared extent of the las file and the interpolation stops a bit short. Here are some screenshots that explain the issue
LAS ground points (1000m * 1000m)
lidR output (4160*4160)
pyfor output (4079*4079)
whitebox output for comparison (4080*4080)
wbt seems to miss the top pixels near the extent edge
Arcmap TIN display
I believe what may be happening is the interpolation (pyfor uses scipy, do not know what wbt uses) is not getting the extent from the LAS file but calculating it from the actual points which makes it fall short of the 'official' extent. Do not know what R does different in this respect but this would be the desired output.
Any way I can force this to be the case?
Thank you!
DEM Geotiffs with NAD 83 UTM Zone 10N projection produce derivative geotiffs with grids in the same coordinate system, but the metadata has changed to GCS NAD 1983 with decimal degree units, which causes difficulties in GIS.
Hello, whiteboxtools v1.02, it appears the tools run but wont write to a file whether I save it to temporary file or permanent file. Note I've reproduced the following error below with multiple tools. This is running on Windows through QGIS.
Reading data...
Saving data...
error while writing: Custom { kind: InvalidData, error: "Error while writing GeoTIFF file." }
Output file written
Elapsed Time (excluding I/O): 3.8
Several tools shift by half-pixel the output raster:
The problem occurs when going from FLT to FLT or TIFF. If the input raster is TIFF, the output raster (TIFF or FLT) has the correct extent.
The whitebox tool "VectorPolygonsToRaster" return a different result compare to the Whitebox GAT -win tool "Vector polygons to raster" . The output .tif has gaps!!!
from whitebox import WhiteboxTools
# Set up the environment
wbt = WhiteboxTools()
data_dir = "/../whitebox_issue"
wbt.work_dir = (data_dir)
i_vector = "poly.shp"
fieldZ = "Value"
out_raster = "poly2raster_PYTHON.tif"
wbt.vector_polygons_to_raster(
i=i_vector,
output=out_raster,
field=fieldZ,
nodata=True,
cell_size=1,
# base=None,
# callback=default_callback
)
The tool Convert Raster Format is adding a strip of NoData and slightly modified value on the right of the ouput raster when the input format is TIFF (NoData in purple in the image below).
The width of the strip varies: I got 55 pixels on a 10039 x 15151 raster and 63 pixels on a 2623 x 2651 raster (both TIFF) but has always the same look (one on two rows with repetitive diagonals). The values between two rows of NoData is not the original value, sometimes a bit higher sometimes a bit lower. If I convert the original TIFF to FLT with ArcMap and then convert those FLT to FLT or SDAT with Whitebox Tools, no NoData strip is added.
When using tool convert_raster_format
, if the name of the exported file contains it's extension, the companion file won't be named correclty.
wbt.convert_raster_format('infile.dep', '0.sdat_outfile.sdat')
# will give the files:
# 0.sdat_outfile.sdat
# 0.sgrd_outfile.sgrd instead of 0.sdat_outfile.sgrd
The problem is there also with .flt (companion file .hdr) and .rst (companion file .rdc). The problem is only with the Python API as the files exported from Whitebox GAT 3.4 doesn't experience this problem.
Is there an updated function to do the breach depressions (the not "fast") version?
When converting a DEP file to SDAT format, the exported raster is flipped horizontally. The problem is only with the Python API as the SDAT export form Whitebox GAT 3.4 doesn't have this problem.
wbt.convert_raster_format('infile.dep', 'outfile.sdat')
I tested with the lastest compiled binaries (0.16.0). Input and output exemple files can be downloaded here.
Hi @jblindsay
I was testing SplitColourComposite in WhiteboxTools 0.16.0, following this fix
https://github.com/alexbruy/processing-whitebox/issues/17#issuecomment-482542035
but now I'm getting the error below:
C:\>"C:\OSGeo4W64\WhiteboxTools_win_amd64\WBT\whitebox_tools.exe" --run=SplitColourComposite --input="D:\orto_teste.tif" --red="D:\orto_teste_r.tif" --green="D:\orto_teste_g.tif" --blue="D:\orto_teste_b.tif" -v
***********************************
* Welcome to SplitColourComposite *
***********************************
Reading data...
thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', src\libcore\option.rs:345:21
note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
Can this be a local problem?
Error message:
thread 'main' panicked at 'called
Option::unwrap()
on aNone
value', libcore\option.rs:335:21
Edit: might be simply that a base raster is required now, but it used to be optional in Whitebox.
Hello, John
I have been spending some time looking through the code and wanted to ask a seemingly simple question.
After reading though the FAQ and some of the closed issues raised about GDAL, it sounds like you are investing in writing a lot of the functionality from the ground up solely in Rust.
I was wondering if there was any use or plans for any of the crates in the geoRust group?. I didn't see it referenced in the Cargo.toml and was wondering what your thoughts were on those projects?
Thank you for your work, I hope to contribute once I get more comfortable with the language.
It would be a great feature if WhiteboxTools could support Numpy Arrays both as input and output. That would simplify integration with other Python librairies and offer a way to keep the data in memory for further processing.
GREAT WORK JOHN! woot.
Just curious if you plan to consolidate any common functions of whitebox-GAT to whitebox tools? Is this already the case that Whitebox-GAT calls rust whitebox-tools? I'm excited about this project -
myself I'd like to eventually be able to call rust these rust functions directly for go-code.. can just use go as a scripting language and call directly into the Whitebox-Tools binary for the time being... Do function calls return json output for non-file output? I imagine that there are a number of functions which have small amount of output, it'd be nice if this was machine parseable.
go john go!
The new RasterArea Tool is missing in the whitebox_tools.py.
I ran a test of LidarJoin using WhiteboxTools Runner on 5 files of total point record size 19753893. Version 1.2. These were non RGB points. The data produced in the output file was scrambled and grouped at the XY, ZX and ZY - see attached png file. I tried just 2 smaller files and returned the same result. I used lastools lasmerge and received the correct merge.
Hoping to have Watershed enhanced so that rasters can be used as pour_pts similar to Watershed in Whitebox. This is useful for generating “lakesheds” via scripts.
When running the 'FlowAccumulationFullWorkflow' with a geotiff DEM input, the flow pointer and flow accumulation processes produce data in the DEM 'NoData' areas. The 'NoData' areas are recognized in both ArcGIS and GDAL. This particular issue does not appear in WhiteboxGAT when using the same input DEM.
I did not test on the independent d8 flow pointer and d8 flow accumulation functions.
OS: Windows 10 Pro
Python Version: 3.60
Whitebox Tools Version: 0.70
I noticed small inconsistence in the manual URL's. Looks like for the wast majority of tools group name in the URL constructed by replacing spaces with underscores and converting text to the lower case, e.g.LiDAR Tools
-> lidar_tools
. But for Math and Stats Tools
this is not the case, this group converted to mathand_stats_tools
. Is this intentional or there is a minor bug in the URL construction?
Asking because in the WhiteboxTools QGIS plugin I create help links automaticaly using tool name and group and this inconsistence does not allow to generate valid urls for tools from the Math as Stats group. May I ask for some information how help URLs are generated, are my assumptions correct?
The Fill Single-Cell Pits and Breach Depressions tools seem to be unable to produce a GeoTIFF if the input is FLT or SDAT.
wbt = WhiteboxTools()
wbt.fill_single_cell_pits("input.tif", "fromTIF.flt")
# ...
# Saving data...
# error while writing: Custom { kind: InvalidData, error: StringError("Error while writing GeoTIFF file.") }
# Output file written
# ...
Are those tools trying to write spatial reference data to the TIFF file, but crash because they can't find any with FLT or SDAT files? I tried the same thing with Breach Single-Cell Pits and the TIFF file is correctly generated (without any spatial reference as expected).
Encountered a strange problem while trying to run Cost Distance tool. The output is consistently an integer data type instead of a float point. Currently working on v0.15.0
and here are the source and cost surfaces. I can't seem to find any issue with the raw data. Any ideas?
thread 'main' panicked at 'index 43 out of range for slice of length 34', libcore/slice/mod.rs:1971:5
note: Run with RUST_BACKTRACE=1
for a backtrace.
here is the tiff.
https://www.dropbox.com/s/wyt545n5mpqpy1m/governer_island_ns2.tif?dl=0
Tried to fill None with zero - did not help the issue,
Having identified several issues with the input parameters that can cause problems on Windows (#20), and also realised that --input is not actually the right parameter, I'm still unable to get NewRasterFromBase to run on Windows:
C:\Processing>whitebox_tools.exe --version
whitebox-tools v0.11.0 by Dr. John B. Lindsay (c) 2017-2018
C:\Processing>whitebox_tools.exe --run=NewRasterFromBase --wd="C:\Processing\" -i="simple.tif" --output="simple57.tif" --value=nodata --data_type=integer
thread 'main' panicked at 'Tool run with no paramters.', src\main.rs:76:21
stack backtrace:
0: <unknown>
1: <unknown>
2: <unknown>
3: <unknown>
4: <unknown>
5: <unknown>
6: <unknown>
7: <unknown>
8: <unknown>
9: <unknown>
10: <unknown>
11: <unknown>
12: <unknown>
13: <unknown>
14: BaseThreadInitThunk
15: RtlUserThreadStart
I do not believe the input file is the problem - it works with the Linux whitebox_tools, and I have tried a variety of files, all failing.
Is there anything else obvious I'm missing? Maybe it is just an issue with the order in which I've specified the parameters - there is probably some combination I haven't tried.
Would a proper debug build for Windows help?
Which seems a bit inconsistent when necessary pre processing tools like snap pour points don't output shapefiles and raster to vector points is not yet ported.
Some issues I've noticed trying to run whitebox_tools on Windows and comparing with Linux:
In general it seems that either --wd needs to be used, or paths need to be provided for input and output files (i.e. whitebox_tools never just looks in the current working directory), but
When i run breach depressions I get output that is identical to the input. I tested on Windows and Linux. 0.14 works correctly.
I wanted to update whitebox-tools but I failed to compile:
Compiling whitebox_tools v0.12.0 (file:///../Models/whitebox-tools)
error[E0658]: use of unstable library feature 'iterator_step_by': unstable replacement of Range::step_by (see issue #27741)
--> src/tools/gis_analysis/construct_vector_tin.rs:337:46
|
337 | for i in (0..result.triangles.len()).step_by(3) {
| ^^^^^^^
error[E0658]: use of unstable library feature 'iterator_step_by': unstable replacement of Range::step_by (see issue #27741)
--> src/tools/lidar_analysis/lidar_construct_vector_tin.rs:434:58
|
434 | for i in (0..result.triangles.len()).step_by(3) {
| ^^^^^^^
error: aborting due to 2 previous errors
For more information about this error, try `rustc --explain E0658`.
error: Could not compile `whitebox_tools`.
After running rustc --explain E0658
, it seems that I may use a nightly version of rustc to be able to use the iterator_step_by.
When trying to run the FillBurn tool it enters an infinite loop, and soon crashes from running out of RAM (6MB input raster ends up trying to use over 16GB RAM).
This is the loop that goes out of control:
https://github.com/jblindsay/whitebox-tools/blob/master/src/tools/hydro_analysis/fill_burn.rs#L503
It ends up reporting over 30000% progress.
Does this happen for anyone else?
trying to fill depressions in a DEM file. my code is below. the same thing happens when I call the 'slope' function. any thoughts on how to fix this?
wbts = wbt()
wbts.fill_depressions(dem='/Raster/DEM8m/apa8mdem',
output= 'apDEMfill.tif',
fix_flats=True)
returns:
./whitebox_tools --run="FillDepressions" --dem='/Raster/DEM8m/apa8mdem' --output='apDEMfill.tif' --fix_flats -v
Reading data...
thread 'main' panicked at 'called Option::unwrap()
on a None
value', src/libcore/option.rs:345:21
note: Run with RUST_BACKTRACE=1
environment variable to display a backtrace.
output: 0
Hi,
I set up the workflow as described but I cannot use the tool of Whitebox in QGIS within Processing.
These the steps I followed:
where WBT
is the folder containing the files of the tar
Every algorithm gives me the following error:
/bin/sh: 1: /home/matteo/Downloads/WBT/: Permission denied
I also tried to change the permission of the folder to 777 (but I did all the steps as normal user) and the error is still here.
OS: Linux Debian Sid, QGIS 3.9 cc @alexbruy
I have a script that uses both the elevation_above_stream_euclidean and the elevation_above_stream tools. The inputs are the same.
These are the lines of code:
wbt.elevation_above_stream(dem= path+name+tif, streams= name + '_'+ n + 'streams.tif', output= name + '' + n + 'HAND.tif')
wbt.elevation_above_stream_euclidean(dem= path+name+tif, streams=name + '_' + n + 'streams.tif', output= name + '' + n + 'HANDe.tif')
However, the wbt.elevation_above_stream_euclidean produces an empty .tif without throwing an error.
Hello
I tried to denoise a DEM using the following command
C:/WhiteboxTools/whitebox_tools.exe -r=FeaturePreservingDenoise -v --dem="/path/dem.tif" -o=demf.tif --filter=15 --norm_diff=20.0 --num_iter=4
the product is a DEM with a similar weight as the original one, however, I have been unable to read it with GDAL. Running gdalinfo on the output produces the following:
gdalinfo demf.tif
ERROR 1: demf.tif:Not a TIFF file, bad version number 10752 (0x2a00)
gdalinfo failed - unable to open 'demf.tif'.
The output of gdalinfo for the original file is the following
gdalinfo "/path/dem.tif"
Driver: GTiff/GeoTIFF
Files: /path/dem.tif
Size is 9001, 9001
Coordinate System is:
GEOGCS["WGS 84",
DATUM["WGS_1984",
SPHEROID["WGS 84",6378137,298.257223563,
AUTHORITY["EPSG","7030"]],
AUTHORITY["EPSG","6326"]],
PRIMEM["Greenwich",0],
UNIT["degree",0.0174532925199433],
AUTHORITY["EPSG","4326"]]
Origin = (21.999944444444445,45.000055555555555)
Pixel Size = (0.000111111111111,-0.000111111111111)
Metadata:
AREA_OR_POINT=Point
TIFFTAG_DATETIME=2015:05:26 21:46:30
TIFFTAG_RESOLUTIONUNIT=2 (pixels/inch)
TIFFTAG_SOFTWARE=[sensitive]
TIFFTAG_XRESOLUTION=50
TIFFTAG_YRESOLUTION=50
Image Structure Metadata:
INTERLEAVE=BAND
Corner Coordinates:
Upper Left ( 21.9999444, 45.0000556)
Lower Left ( 21.9999444, 43.9999444)
Upper Right ( 23.0000556, 45.0000556)
Lower Right ( 23.0000556, 43.9999444)
Center ( 22.5000000, 44.5000000)
Band 1 Block=9001x1 Type=Float32, ColorInterp=Gray
I hope this info is helpful, I tried to report this problem as instructed in the documentation
Kind regards,
Ignacio
I am trying to create mosaics of GeoTIFF files. I am obtaining a resulting GeoTIFF that looks ok, but the coordinate system is then gone in the resulting mosaic.
How can I georefence properly the resulting mosaic?
Inputs:
Driver: GTiff/GeoTIFF
Files: /tmp/out22_g_nd.tif
Size is 1024, 1024
Coordinate System is:
PROJCS["WGS 84 / UTM zone 29N",
GEOGCS["WGS 84",
DATUM["WGS_1984",
SPHEROID["WGS 84",6378137,298.257223563,
AUTHORITY["EPSG","7030"]],
AUTHORITY["EPSG","6326"]],
PRIMEM["Greenwich",0,
AUTHORITY["EPSG","8901"]],
UNIT["degree",0.0174532925199433,
AUTHORITY["EPSG","9122"]],
AUTHORITY["EPSG","4326"]],
PROJECTION["Transverse_Mercator"],
PARAMETER["latitude_of_origin",0],
PARAMETER["central_meridian",-9],
PARAMETER["scale_factor",0.9996],
PARAMETER["false_easting",500000],
PARAMETER["false_northing",0],
UNIT["metre",1,
AUTHORITY["EPSG","9001"]],
AXIS["Easting",EAST],
AXIS["Northing",NORTH],
AUTHORITY["EPSG","32629"]]
Origin = (579085.567641534260474,4738466.775685614906251)
Pixel Size = (1.502461069483093,-1.502458127183625)
Metadata:
AREA_OR_POINT=Area
TIFFTAG_RESOLUTIONUNIT=2 (pixels/inch)
TIFFTAG_SOFTWARE=WhiteboxTools
TIFFTAG_XRESOLUTION=72
TIFFTAG_YRESOLUTION=72
Image Structure Metadata:
INTERLEAVE=BAND
Corner Coordinates:
Upper Left ( 579085.568, 4738466.776) ( 8d 1'58.59"W, 42d47'40.80"N)
Lower Left ( 579085.568, 4736928.259) ( 8d 1'59.37"W, 42d46'50.93"N)
Upper Right ( 580624.088, 4738466.776) ( 8d 0'50.87"W, 42d47'40.22"N)
Lower Right ( 580624.088, 4736928.259) ( 8d 0'51.67"W, 42d46'50.35"N)
Center ( 579854.828, 4737697.517) ( 8d 1'25.12"W, 42d47'15.58"N)
Band 1 Block=1024x1 Type=Float32, ColorInterp=Gray
Computed Min/Max=11.000,251.000
NoData Value=0
Mosaic:
Driver: GTiff/GeoTIFF
Files: mosaic.tiff
Size is 1282, 1296
Coordinate System is:
LOCAL_CS["unnamed",
UNIT["unknown",1]]
Origin = (577458.933401877293363,4738466.775685614906251)
Pixel Size = (2.469379546168251,-2.469373084423899)
Metadata:
AREA_OR_POINT=Area
TIFFTAG_RESOLUTIONUNIT=2 (pixels/inch)
TIFFTAG_SOFTWARE=WhiteboxTools
TIFFTAG_XRESOLUTION=72
TIFFTAG_YRESOLUTION=72
Image Structure Metadata:
INTERLEAVE=PIXEL
Corner Coordinates:
Upper Left ( 577458.933, 4738466.776)
Lower Left ( 577458.933, 4735266.468)
Upper Right ( 580624.678, 4738466.776)
Lower Right ( 580624.678, 4735266.468)
Center ( 579041.806, 4736866.622)
Band 1 Block=1282x1 Type=Byte, ColorInterp=Red
Computed Min/Max=9.000,255.000
NoData Value=0
Band 2 Block=1282x1 Type=Byte, ColorInterp=Green
Computed Min/Max=6.000,255.000
NoData Value=0
Band 3 Block=1282x1 Type=Byte, ColorInterp=Blue
Computed Min/Max=2.000,255.000
NoData Value=0
Band 4 Block=1282x1 Type=Byte, ColorInterp=Alpha
Computed Min/Max=255.000,255.000
NoData Value=0
It looks like there are Rust bindings for GDAL. Is it possibly the offload the Raster/Vector IO to that library for more supported file options?
Are there any plans to allow the LiDAR interpolation functions to use any arbitrary field rather than what is currently available ('elevation', 'intensity', 'class', 'scan angle', 'user data'). The interpolation functions work great, but it would be helpful to be able to apply them to e.g., the number of returns per pulse (to get average number of returns per pulse in a cell), or to the RBG fields (in order to extract a color image) without having to write more code to bin these values. Right now the only way to do this is to trick the function by changing e.g., swapping the headers for Z and number of returns.
Seems output parameter in the Resampling tool defined incorrectly, it expects existing file to be passed as output, not new file
$ whitebox_tools --toolparameters=Resample
{"parameters":[{"name":"Input Files","flags":["-i","--inputs"],"description":"Input raster files.","parameter_type":{"FileList":"Raster"},"default_value":null,"optional":false},{"name":"Destination File","flags":["--destination"],"description":"Destination raster file.","parameter_type":{"ExistingFile":"Raster"},"default_value":null,"optional":false},{"name":"Resampling Method","flags":["--method"],"description":"Resampling method","parameter_type":{"OptionList":["nn","bilinear","cc"]},"default_value":"cc","optional":true}]}
Is this intentional or maybe a typo?
In the "Extract Valleys" tool there is a --variant
parameter which accoding to the description accepts one of the following values: LQ
(lower quartile), JandR
(Johnston and Rosenfeld) or PandD
(Peucker and Douglas), and uses LQ
as default. However in the JSON tool description available values listed in the short form, while defualt value listed in the long form Lower Quartile
.
{"name":"Variant","flags":["--variant"],"description":"Options include 'LQ' (lower quartile), 'JandR' (Johnston and Rosenfeld), and 'PandD' (Peucker and Douglas); default is 'LQ'.","parameter_type":{"OptionList":["LQ","JandR","PandD"]},"default_value":"Lower Quartile","optional":false}
Not sure is this bug or default value was just missed during values change.
I was attempting to build the latest version of whitebox_tools (commit df41d62) using the latest version of rust. I received the following errors. I'm not familiar with Rust. Is some other build option needed?
Thanks
--
rustup 1.19.0 (2af131cf9 2019-09-08)
rustc 1.37.0 (eae3437df 2019-08-13)
> cargo build --release
error: attempt to shift left with overflow
--> src/tools/lidar_analysis/lidar_idw_interpolation.rs:686:79
|
686 | frs.insert(p.x, p.y, ((255 << 24) | (clr.blue << 16) | (clr.green << 8) | clr.red) as f64);
| ^^^^^^^^^^^
|
= note: #[deny(exceeding_bitshifts)] on by default
error: attempt to shift left with overflow
--> src/tools/lidar_analysis/lidar_idw_interpolation.rs:686:93
|
686 | frs.insert(p.x, p.y, ((255 << 24) | (clr.blue << 16) | (clr.green << 8) | clr.red) as f64);
| ^^^^^^^^^^^^^^^^
error: attempt to shift left with overflow
--> src/tools/lidar_analysis/lidar_idw_interpolation.rs:806:52
|
806 | val = ((255 << 24) | ((val_blue.round() as u16) << 16) | ((val_green.round() as u16) << 8) | (val_red.round() a
s u16)) as f64;
| ^^^^^^^^^^^
error: attempt to shift left with overflow
--> src/tools/lidar_analysis/lidar_idw_interpolation.rs:806:66
|
806 | val = ((255 << 24) | ((val_blue.round() as u16) << 16) | ((val_green.round() as u16) << 8) | (val_red.round() a
s u16)) as f64;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
error: attempt to shift left with overflow
--> src/tools/lidar_analysis/lidar_idw_interpolation.rs:871:60
|
871 | val = ((255 << 24) | ((val_blue.round() as u16) << 16) | ((val_green.round() as u16) << 8) | (val_red.r
ound() as u16)) as f64;
| ^^^^^^^^^^^
error: attempt to shift left with overflow
--> src/tools/lidar_analysis/lidar_idw_interpolation.rs:871:74
|
871 | val = ((255 << 24) | ((val_blue.round() as u16) << 16) | ((val_green.round() as u16) << 8) | (val_red.r
ound() as u16)) as f64;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
error: attempt to shift left with overflow
--> src/tools/lidar_analysis/lidar_nn_gridding.rs:665:79
|
665 | frs.insert(p.x, p.y, ((255 << 24) | (clr.blue << 16) | (clr.green << 8) | clr.red) as f64);
Hello, I am getting these errors when running the following code, which works on other lidar datasets I have. The process continues for a few more tiles with this message interspersed with the progress messages and then the program hangs with no output.
Any idea where I can start troubleshooting these lidar tiles? Thank you. I tried resolution of 5m but same issues.
wbt.lidar_tin_gridding( parameter="elevation", returns="all", resolution=0.25, exclude_cls='0,1,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18')
Progress: 5%
Finished interpolating \\374000_5141000_suffix (9 of 138)
Finished interpolating \\375000_5143000_suffix (10 of 138)
Progress: 6%
thread 'unnamed' panicked at 'index out of bounds: the len is 604504064 but the index is 604504064',/rustc/9fda7c2237db910e41d6a712e9a2139b352e558b\src\libcore\slice\mod.rs:2463:10
note: Run withRUST_BACKTRACE=1
for a backtrace.
thread 'unnamed' panicked at 'index out of bounds: the len is 604504064 but the index is 604504064',/rustc/9fda7c2237db910e41d6a712e9a2139b352e558b\src\libcore\slice\mod.rs:2463:10
Finished interpolating \\375000_5139000_suffix (11 of 138)
Progress: 7%
thread unnamed' panicked at 'index out of bounds: the len is 604504064 but the index is 604504064',/rustc/9fda7c2237db910e41d6a712e9a2139b352e558b\src\libcore\slice\mod.rs:2463:10
thread 'unnamed' panicked at 'index out of bounds: the len is 604504064 but the index is 604504064',/rustc/9fda7c2237db910e41d6a712e9a2139b352e558b\src\libcore\slice\mod.rs:2463:10
There is new VectorAttributeField
parameter in WhiteboxTools which represents attribute field of the vector layer. May I suggest couple of improvements for it:
Not sure, if this is an issue or user error, but I am trying to use WBT clip tool to extract streams (polyline) inside a watershed (polygon) and I get the following error:
Reading data...
thread 'main' panicked at 'Unrecognized ShapeType', src\vector\shapefile\geometry.rs:483:18
note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
I checked the geometry types of my datasets: watershed = 'Polygon' and streams = 'LineString', however, after looking through geometry.rs I see no reference to LineString. Is there something I'm missing here?
Additional info. the watershed originally a DEM that is polygonized using rasterio & shapely, and finally written out using geopandas as ESRI Shapefile
. And the streams are NHD projected streams. Here is the snippet:
with rasterio.open("D:/dem.tif") as ds_dem:
arr_dem = ds_dem.read(1)
arr_dem[arr_dem > 0] = 100
mask = arr_dem == 100
results = (
{'properties': {'raster_val': v}, 'geometry': s}
for i, (s, v)
in enumerate(
shapes(arr_dem, mask=mask, transform=ds_dem.transform)))
poly = list(results)
poly_df = gpd.GeoDataFrame.from_features(poly)
poly_df.crs = in_crs
poly_df.to_file("D:/watershed_boundary.shp")
streams = "D:/NHD_streams.shp"
boundary = "D:/watershed_boundary.shp"
output = "D:/clipped_streams.shp"
wbt.clip(streams, tmp, output)
It would be useful to have a docker image of this, so you could pull-down and run a version of whitebox-tools
in environments that lack rust (typed from a Windowz desktop that has docker but no admin rights let alone a rust compiler). Happy to help with this if I get the 👍
Using the testdata/DEM.tif, I noticed that the resulting images from most tools shifted half a pixel towards the lower-right corner. The issue can be reproduced using the script below.
wbt = WhiteboxTools()
wbt.feature_preserving_denoise("DEM.tif", "smoothed.tif", filter=9)
wbt.breach_depressions("smoothed.tif", "breached.tif")
wbt.d_inf_flow_accumulation("breached.tif", "flow_accum.tif")
For example:
DEM.tif
CRS | EPSG:4326 - WGS 84 - Geographic
Extent | 664692.0589773271931335,4878903.5508630042895675 : 686018.1474491551052779,4895827.5998771572485566
Pixel Size | 89.9835,-90.0215
smoothed.tif
CRS | EPSG:4326 - WGS 84 - Geographic
Extent | 664737.0507251580711454,4878858.5400943495333195 : 686063.1391969859832898,4895782.5891085024923086
Pixel Size | 89.9835,-90.0215
diff = ~ 45 (half a pixel)
I want to override the default callback for all functions so it would be useful to have a set_callback
method in order to do this once rather than when each function is called.
When running any of the 'Stream Network' tools (e.g. long_profile, length_of_upstream_channels, etc.) I receive the following error:
thread 'main' panicked at 'Colour map not present in Paletted TIFF.', src\main.rs:49:21
Python call back:
.\whitebox_tools.exe --run="Longprofile" --wd="E:\Burnett_2018_Projects\End_of_Fish\Stream_Delin" --d8_pntr='hja_recon_v10_d8recon2_d8pntr.tif' --streams='hja_recon_v10_d8recon2_d8strm.tif' --dem='hja_recon_v10_d8recon2.tif' --output='hja_recon_v10_d8recon2_d8profile.html' -v
Workflow:
input a custom DEM
d8_pntr is output from 'flow_accumulation_full_workflow'
streams is output from 'extract_streams' and based on 'd8_flow_accum' from 'flow_accumulation_full_workflow'
Notes:
d8_pntr and d8_flow_accum rasters appear normal in GIS
System: Windows 10 x64
Ram: 64 GB
Python version: 3.6
WBT Version: 0.7
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.