Giter VIP home page Giter VIP logo

py3dtiles's Introduction

https://travis-ci.org/Oslandia/py3dtiles.svg?branch=master

py3dtiles

p3dtiles is a Python tool and library for manipulating 3D Tiles.

CLI Features

  • Convert pointcloud LAS and XYZ files to a 3D Tiles tileset (tileset.json + pnts files)
  • Merge pointcloud 3D Tiles tilesets into one tileset
  • Read pnts and b3dm files and print a summary of their contents

API features

  • Read/write pointcloud (pnts) and batched 3d model format

py3dtiles is distributed under the Apache 2 Licence.

GitHub repository: https://github.com/Oslandia/py3dtiles

Documentation : https://oslandia.github.io/py3dtiles

py3dtiles's People

Contributors

autra avatar delhomer avatar elemoine avatar jailln avatar jeremy-gaillard avatar ldgeo avatar mholthausen avatar nikosaul avatar pblottiere avatar peppsac avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

py3dtiles's Issues

Make a logo :-)

Let's have a logo for py3dtiles.

Some keywords:

3d, 3dtiles, pointclouds, buildings, streaming, conversion, multiprocessing...

Error after converting big las file (over 2GB)!

Hi,

I got this error when i tried to convert a big LAS file (over 2GB). With small file (less than 1GB), everything is fine. Can u explain for me about this and how can i fix it or is this affected to converted 3dtiles ?

My Computer has 4 CPU 64bit processors, RAM 16GB with No-GPU.

/DockerShared/py3dtiles/venv/lib/python3.5/site-packages/py3dtiles-1.1.0-py3.5.egg/py3dtiles/convert.py:336: RuntimeWarning: invalid value encountered in subtract
x_axis = br - bl
/DockerShared/py3dtiles/venv/lib/python3.5/site-packages/py3dtiles-1.1.0-py3.5.egg/py3dtiles/convert.py:338: RuntimeWarning: invalid value encountered in subtract
bl = bl - avg_min
/DockerShared/py3dtiles/venv/lib/python3.5/site-packages/py3dtiles-1.1.0-py3.5.egg/py3dtiles/convert.py:339: RuntimeWarning: invalid value encountered in subtract
tr = tr - avg_min
/DockerShared/py3dtiles/venv/lib/python3.5/site-packages/py3dtiles-1.1.0-py3.5.egg/py3dtiles/convert.py:83: RuntimeWarning: invalid value encountered in true_divide
v0 = z1 / np.linalg.norm(z1)
/DockerShared/py3dtiles/venv/lib/python3.5/site-packages/py3dtiles-1.1.0-py3.5.egg/py3dtiles/convert.py:354: RuntimeWarning: invalid value encountered in minimum
np.minimum(bl, tr),
/DockerShared/py3dtiles/venv/lib/python3.5/site-packages/py3dtiles-1.1.0-py3.5.egg/py3dtiles/convert.py:355: RuntimeWarning: invalid value encountered in maximum
np.maximum(bl, tr)
0.0 % in 0 sec [est. time left: 14151 sec]/DockerShared/py3dtiles/venv/lib/python3.5/site-packages/py3dtiles-1.1.0-py3.5.egg/py3dtiles/points/node.py:132: RuntimeWarning: invalid value encountered in greater_equal
indices = xyz_to_child_index(pending_xyz_arr, self.aabb_center)
1.19 % in 2 sec [est. time left: 169 sec]/DockerShared/py3dtiles/venv/lib/python3.5/site-packages/py3dtiles-1.1.0-py3.5.egg/py3dtiles/points/node.py:132: RuntimeWarning: invalid value encountered in greater_equal
indices = xyz_to_child_index(pending_xyz_arr, self.aabb_center)
7.85 % in 18 sec [est. time left: 211 sec]/DockerShared/py3dtiles/venv/lib/python3.5/site-packages/py3dtiles-1.1.0-py3.5.egg/py3dtiles/points/task/las_reader.py:118: RuntimeWarning: invalid value encountered in add
x = (x + offset_scale[0][0]) * offset_scale[1][0]
/DockerShared/py3dtiles/venv/lib/python3.5/site-packages/py3dtiles-1.1.0-py3.5.egg/py3dtiles/points/task/las_reader.py:119: RuntimeWarning: invalid value encountered in add
y = (y + offset_scale[0][1]) * offset_scale[1][1]
17.78 % in 41 sec [est. time left: 192 sec]/DockerShared/py3dtiles/venv/lib/python3.5/site-packages/py3dtiles-1.1.0-py3.5.egg/py3dtiles/points/task/las_reader.py:118: RuntimeWarning: invalid value encountered in add
x = (x + offset_scale[0][0]) * offset_scale[1][0]
/DockerShared/py3dtiles/venv/lib/python3.5/site-packages/py3dtiles-1.1.0-py3.5.egg/py3dtiles/points/task/las_reader.py:119: RuntimeWarning: invalid value encountered in add
y = (y + offset_scale[0][1]) * offset_scale[1][1]

Thanks.
Regards.

Pointcloud is in incorrect position and disappears after changing the camera

I processed a LAS file with py3dtiles to get the pnts file. The LAS SRS is WGS84, so I assumed no projection was needed when using this tool. When the camera flies to the bounding sphere of the tileset, the tileset zooms in from the side of the screen, and when the camera is changed by a fraction, it disappears. Also, the tileset is in the complete wrong area on the globe and is over the ocean. I've included some debugging information below, since I was unable to solve this. My guess is that it's an SRS issue, but not sure what the problem is exactly.

The command I ran is py3dtiles convert sample.las --out sample.

The Cesium Javascript code I'm using to load and fly to the tileset is below.

var viewer = new Cesium.Viewer('cesiumContainer');

var tileSet = new Cesium.Cesium3DTileset({ url: "Data/sample/tileset.json" });
viewer.scene.primitives.add(tileSet);

tileSet.readyPromise.then(function(tileset) {
    viewer.camera.flyToBoundingSphere(tileset.boundingSphere);
});

The output of running lasinfo sample.las is below.

---------------------------------------------------------
  Header Summary
---------------------------------------------------------

  Version:                     1.2
  Source ID:                   0
  Reserved:                    0
  Project ID/GUID:             '00000000-0000-0000-0000-000000000000'
  System ID:                   'libLAS'
  Generating Software:         'libLAS 1.8.1'
  File Creation Day/Year:      14/2020
  Header Byte Size             227
  Data Offset:                 1089
  Header Padding:              0
  Number Var. Length Records:  3
  Point Data Format:           3
  Number of Point Records:     9568586
  Compressed:                  False
  Number of Points by Return:  0 0 0 0 0 
  Scale Factor X Y Z:          0.00100000000000 0.00100000000000 0.00100000000000
  Offset X Y Z:                494000.000 7783000.000 0.000
  Min X Y Z:                   493987.195 7782854.536 112.952
  Max X Y Z:                   494474.386 7783254.889 145.192
  Spatial Reference:           
PROJCS["WGS 84 / UTM zone 55S",
    GEOGCS["WGS 84",
        DATUM["WGS_1984",
            SPHEROID["WGS 84",6378137,298.257223563,
                AUTHORITY["EPSG","7030"]],
            AUTHORITY["EPSG","6326"]],
        PRIMEM["Greenwich",0],
        UNIT["degree",0.0174532925199433],
        AUTHORITY["EPSG","4326"]],
    PROJECTION["Transverse_Mercator"],
    PARAMETER["latitude_of_origin",0],
    PARAMETER["central_meridian",147],
    PARAMETER["scale_factor",0.9996],
    PARAMETER["false_easting",500000],
    PARAMETER["false_northing",10000000],
    UNIT["metre",1,
        AUTHORITY["EPSG","9001"]],
    AUTHORITY["EPSG","32755"]]

Geotiff_Information:
   Version: 1
   Key_Revision: 1.0
   Tagged_Information:
      End_Of_Tags.
   Keyed_Information:
      GTModelTypeGeoKey (Short,1): ModelTypeProjected
      GTRasterTypeGeoKey (Short,1): RasterPixelIsArea
      GTCitationGeoKey (Ascii,22): "WGS 84 / UTM zone 55S"
      GeogCitationGeoKey (Ascii,7): "WGS 84"
      GeogAngularUnitsGeoKey (Short,1): Angular_Degree
      ProjectedCSTypeGeoKey (Short,1): PCS_WGS84_UTM_zone_55S
      ProjLinearUnitsGeoKey (Short,1): Linear_Meter
      End_Of_Keys.
   End_Of_Geotiff.


---------------------------------------------------------
  VLR Summary
---------------------------------------------------------
    User: 'LASF_Projection' - Description: 'GeoTIFF GeoKeyDirectoryTag'
    ID: 34735 Length: 64 Total Size: 118
    User: 'LASF_Projection' - Description: 'GeoTIFF GeoAsciiParamsTag'
    ID: 34737 Length: 30 Total Size: 84
    User: 'liblas' - Description: 'OGR variant of OpenGIS WKT SRS'
    ID: 2112 Length: 606 Total Size: 660
---------------------------------------------------------
  Schema Summary
---------------------------------------------------------
  Point Format ID:             3
  Number of dimensions:        16
  Custom schema?:              false
  Size in bytes:               34

  Dimensions
---------------------------------------------------------
  'X'                            --  size: 32 offset: 0
  'Y'                            --  size: 32 offset: 4
  'Z'                            --  size: 32 offset: 8
  'Intensity'                    --  size: 16 offset: 12
  'Return Number'                --  size: 3 offset: 14
  'Number of Returns'            --  size: 3 offset: 14
  'Scan Direction'               --  size: 1 offset: 14
  'Flightline Edge'              --  size: 1 offset: 14
  'Classification'               --  size: 8 offset: 15
  'Scan Angle Rank'              --  size: 8 offset: 16
  'User Data'                    --  size: 8 offset: 17
  'Point Source ID'              --  size: 16 offset: 18
  'Time'                         --  size: 64 offset: 20
  'Red'                          --  size: 16 offset: 28
  'Green'                        --  size: 16 offset: 30
  'Blue'                         --  size: 16 offset: 32
  
---------------------------------------------------------
  Point Inspection Summary
---------------------------------------------------------
  Header Point Count: 9568586
  Actual Point Count: 9568586

  Minimum and Maximum Attributes (min,max)
---------------------------------------------------------
  Min X, Y, Z: 		493987.195, 7782854.536, 112.952
  Max X, Y, Z: 		494474.386, 7783254.889, 145.192
  Bounding Box:		493987.195, 7782854.536, 494474.386, 7783254.889
  Time:			0.000000, 0.000000
  Return Number:	0, 0
  Return Count:		0, 0
  Flightline Edge:	0, 0
  Intensity:		0, 0
  Scan Direction Flag:	0, 0
  Scan Angle Rank:	0, 0
  Classification:	0, 0
  Point Source Id:	0, 0
  User Data:		0, 0
  Minimum Color (RGB):	0 256 0 
  Maximum Color (RGB):	65280 65280 65280 

  Number of Points by Return
---------------------------------------------------------
	(1) 9568586

  Number of Returns by Pulse
---------------------------------------------------------
	(0) 9568586

  Point Classifications
---------------------------------------------------------
	9568586 Created, never classified (0) 
  -------------------------------------------------------
  	0 withheld
  	0 keypoint
  	0 synthetic
  -------------------------------------------------------
```

Add B3DM Feature Table

The b3dm feature table is pretty simple compared to the point cloud feature table - in fact it almost seems unnecessary. Nonetheless, to make the generated b3dm + batch table to spec, it'd be nice to include the feature table.

There is one related closed ticket here without resolution. #75

I'm happy to contribute a fix for this - I'm wondering though if we need to tackle a bit of refactoring here by creating some sort of base FeatureTable that is shared between point clouds and b3dm?

Wrong transformation or rotation in b3dm export

There seems to be an error in the exporting of a b3dm in the transformation or rotation of the gltf object and the tileset boundingVolume box.
I have exported the Polyhedralsurface (in EPSG:4978) which is based on a DalunayTriangles TINZ of theses XYZ points (in EPSG:31468):

4577799, 5809394, 112
4566546, 5809394, 112
4623589, 5809394, 112
4623589, 5837277, 112
4566546, 5837277, 112
4577799, 5837277, 112

The area should be a "flat plane" at the height of 112 meters. But the resulting tileset has a rotation in the boundingVolume box and a boundingVolume extent that does not include all parts of the gltf object.

let boundingSphere = tileset.boundingSphere;
let cartographic = Cartographic.fromCartesian(boundingSphere.center);

let surface = Cartesian3.fromRadians(cartographic.longitude, cartographic.latitude, 0.0);
let matrix = Transforms.eastNorthUpToFixedFrame(surface);
tileset.root.transform = matrix;

Executing the above code on the tileset in the Cesium viewer fixes the rotation of the boundingVolume box, but the gltf object is tilted. When I view the gltf in a gltf Viewer I see also a tilted plane (note the attached scrrenshots).

tilted_boundingVolume

tilted_plane

gltf_viewer

  • the calculation of the boundingVolume must generate incorrect values that lead to a tilting of the box and an incorrect extent,
  • the glTF object also receives a tilt that does not correspond to the geometry of the initial values

Database connection

In line with two PRs (#70 and #72), we are manually passing arguments to the program so as it generates a connection to the database.

In order to makes the program more generalizable and the CLI less verbose, we could use a database connection configuration file (e.g. PostgreSQL comes with a pg_service.conf file).

Failure of some examples that are presented on the Pypi documentation page

How to reproduce the problem

I tried to reproduce the b3dm related example code of the py3dtiles Pypi documentation page.

You first install py3dtiles from the sources which goes

git clone https://github.com/Oslandia/py3dtiles
...
python setup.py install

and then (refer to "How to read a .b3dm file" section) try to run
a python script starting with

from py3dtiles import TileReader
from py3dtiles import B3dm
...

This will fail with a python error message of the form

ImportError: cannot import name 'TileReader' from 'py3dtiles'

Analysis of the problem

It seems that the Pypi documentation was correct prior to py3dtiles version incremental.
Indeed the git clone https://github.com/Oslandia/py3dtiles will (as of 13 may 2020)
checkout version 2.0.0 of py3dTiles, as opposed to version 1.0.2.

And alas the py3dtiles.TileReader, that existed in py3dtiles version 1.0.2 has disappeared (renamed?) from version 2.0.0.

Possible solution

The version of the documentation that is pushed on Pypi should be aligned with the
py3dTiles version number. Hence the git clone instruction should (for this Pypi version 1.0.2)
probably go:

git clone https://github.com/Oslandia/py3dtiles
cd py3dtiles
git checkout 8efa053986ad.      #  <--- SHA-1 of version 1.0.2
...

Just my 2cts.

task/las_reader.py Runtime error

D:\ProgramData\Anaconda3\envs\newpy3dtiles\python.exe E:/GitProject/py3dtiles/py3dtiles/points/task/las_reader.py
Traceback (most recent call last):
File "E:/GitProject/py3dtiles/py3dtiles/points/task/las_reader.py", line 8, in
import liblas
File "D:\ProgramData\Anaconda3\envs\newpy3dtiles\lib\site-packages\liblas_init_.py", line 2, in
from .core import get_version
File "D:\ProgramData\Anaconda3\envs\newpy3dtiles\lib\site-packages\liblas\core.py", line 139, in
las = ctypes.CDLL(lib_name)
File "D:\ProgramData\Anaconda3\envs\newpy3dtiles\lib\ctypes_init_.py", line 348, in init
self._handle = _dlopen(self._name, mode)
OSError: [WinError 126] 找不到指定的模块。

I use python3.6

(newpy3dtiles) C:\Users\Administrator>pip list
Package Version


alabaster 0.7.12
atomicwrites 1.2.1
attrs 18.2.0
Babel 2.6.0
certifi 2018.8.24
chardet 3.0.4
colorama 0.3.9
coverage 4.5.1
Cython 0.28.5
decorator 4.1.2
docutils 0.14
idna 2.7
imagesize 1.1.0
ipython 6.1.0
ipython-genutils 0.2.0
jedi 0.10.2
Jinja2 2.10
laspy 1.5.1
libLAS 1.8.1
line-profiler 2.0
llvmlite 0.20.0
lz4 0.10.1
MarkupSafe 1.0
more-itertools 4.3.0
numba 0.35.0
numpy 1.13.1
packaging 18.0
path.py 10.3.1
pickleshare 0.7.4
pip 18.1
pluggy 0.7.1
prompt-toolkit 1.0.15
psutil 5.4.7
psycopg2-binary 2.7.5
py 1.6.0
py-cpuinfo 4.0.0
Pygments 2.2.0
pyparsing 2.2.2
pyproj 1.9.5.1
pytest 3.8.2
pytest-benchmark 3.1.1
pytest-cov 2.6.0
pytz 2018.5
pyzmq 17.1.2
requests 2.19.1
setuptools 36.4.0
simplegeneric 0.8.1
six 1.10.0
snowballstemmer 1.2.1
Sphinx 1.8.1
sphinx-rtd-theme 0.4.2
sphinxcontrib-websupport 1.1.0
traitlets 4.3.2
triangle 20170429
urllib3 1.23
wcwidth 0.1.7
wheel 0.29.0
wincertstore 0.2

I don't know where the problem is

Support RGB information when converting .xyz files

When converting a .xyz file that contains 6 columns (x, y, z and r, g, b), one expects to build a colored 3dtile. However one gets the following trace:

Traceback (most recent call last):
  File "/home/rde/src/py3dtiles/py3dtiles/command_line.py", line 38, in main
    convert.main(args)
  File "/home/rde/src/py3dtiles/py3dtiles/convert.py", line 313, in main
    infos = fn(args)
  File "/home/rde/src/py3dtiles/py3dtiles/points/task/xyz_reader.py", line 38, in init
    points[i] = [float(s) for s in line.split(" ")]
ValueError: cannot copy sequence with size 6 to array axis with dimension 3

I read the xyz_reader.py script, and it seems that the color is filled with white triplets.

Considering the RGB info looks possible in init function (there is a RGB argument), however it is not straightforward in run function...

py3dtiles convert gives weird result if output projection format specified

If I run py3dtiles convert output.las --out xyz/pnts --srs_in 4326 --srs_out 32640 , I get the following output shown in code snippet. Without specifying output projection format atleast it create tiles in pnts format with default projection but I expect 32640 projection format. I feel the error is related with pyproj dependency. Eagerly waiting for reply.
crs.py:77: FutureWarning: '+init=:' syntax is deprecated. ':' is the preferred initialization method.
return _prepare_from_string(" ".join(pjargs))

Complete terminal output:

/home/tan/softwares/py3dtiles/venv/lib/python3.5/site-packages/pyproj/crs.py:77: FutureWarning: '+init=<authority>:<code>' syntax is deprecated. '<authority>:<code>' is the preferred initialization method.
  return _prepare_from_string(" ".join(pjargs))
/home/tan/softwares/py3dtiles/venv/lib/python3.5/site-packages/pyproj/crs.py:77: FutureWarning: '+init=<authority>:<code>' syntax is deprecated. '<authority>:<code>' is the preferred initialization method.
  return _prepare_from_string(" ".join(pjargs))
/home/tan/softwares/py3dtiles/py3dtiles/convert.py:402: RuntimeWarning: invalid value encountered in subtract
  x_axis = br - bl
/home/tan/softwares/py3dtiles/py3dtiles/convert.py:404: RuntimeWarning: invalid value encountered in subtract
  bl = bl - avg_min
/home/tan/softwares/py3dtiles/py3dtiles/convert.py:405: RuntimeWarning: invalid value encountered in subtract
  tr = tr - avg_min
/home/tan/softwares/py3dtiles/py3dtiles/points/task/las_reader.py:112: RuntimeWarning: invalid value encountered in add
  x = (x + offset_scale[0][0]) * offset_scale[1][0]
/home/tan/softwares/py3dtiles/py3dtiles/points/task/las_reader.py:112: RuntimeWarning: invalid value encountered in add
  x = (x + offset_scale[0][0]) * offset_scale[1][0]
/home/tan/softwares/py3dtiles/py3dtiles/points/task/las_reader.py:112: RuntimeWarning: invalid value encountered in add
  x = (x + offset_scale[0][0]) * offset_scale[1][0]
/home/tan/softwares/py3dtiles/py3dtiles/points/task/las_reader.py:113: RuntimeWarning: invalid value encountered in add
  y = (y + offset_scale[0][1]) * offset_scale[1][1]
/home/tan/softwares/py3dtiles/py3dtiles/points/task/las_reader.py:113: RuntimeWarning: invalid value encountered in add
  y = (y + offset_scale[0][1]) * offset_scale[1][1]
/home/tan/softwares/py3dtiles/py3dtiles/points/task/las_reader.py:113: RuntimeWarning: invalid value encountered in add
  y = (y + offset_scale[0][1]) * offset_scale[1][1]
/home/tan/softwares/py3dtiles/py3dtiles/points/task/las_reader.py:114: RuntimeWarning: invalid value encountered in add
  z = (z + offset_scale[0][2]) * offset_scale[1][2]
/home/tan/softwares/py3dtiles/py3dtiles/points/task/las_reader.py:114: RuntimeWarning: invalid value encountered in add
  z = (z + offset_scale[0][2]) * offset_scale[1][2]
/home/tan/softwares/py3dtiles/py3dtiles/points/task/las_reader.py:114: RuntimeWarning: invalid value encountered in add
  z = (z + offset_scale[0][2]) * offset_scale[1][2]
/home/tan/softwares/py3dtiles/py3dtiles/points/task/las_reader.py:112: RuntimeWarning: invalid value encountered in add
  x = (x + offset_scale[0][0]) * offset_scale[1][0]
/home/tan/softwares/py3dtiles/py3dtiles/points/task/las_reader.py:113: RuntimeWarning: invalid value encountered in add
  y = (y + offset_scale[0][1]) * offset_scale[1][1]
/home/tan/softwares/py3dtiles/py3dtiles/points/task/las_reader.py:114: RuntimeWarning: invalid value encountered in add
  z = (z + offset_scale[0][2]) * offset_scale[1][2]
/home/tan/softwares/py3dtiles/py3dtiles/points/node.py:131: RuntimeWarning: invalid value encountered in greater_equal
  indices = xyz_to_child_index(pending_xyz_arr, self.aabb_center)
  8.32 % in 1 sec [est. time left: 15 sec]/home/tan/softwares/py3dtiles/py3dtiles/points/node.py:131: RuntimeWarning: invalid value encountered in greater_equal
  indices = xyz_to_child_index(pending_xyz_arr, self.aabb_center)
  20.2 % in 2 sec [est. time left: 10 sec]/home/tan/softwares/py3dtiles/py3dtiles/points/node.py:131: RuntimeWarning: invalid value encountered in greater_equal
  indices = xyz_to_child_index(pending_xyz_arr, self.aabb_center)
 21.59 % in 3 sec [est. time left: 10 sec]/home/tan/softwares/py3dtiles/py3dtiles/points/task/las_reader.py:112: RuntimeWarning: invalid value encountered in add
  x = (x + offset_scale[0][0]) * offset_scale[1][0]
/home/tan/softwares/py3dtiles/py3dtiles/points/task/las_reader.py:113: RuntimeWarning: invalid value encountered in add
  y = (y + offset_scale[0][1]) * offset_scale[1][1]
/home/tan/softwares/py3dtiles/py3dtiles/points/task/las_reader.py:114: RuntimeWarning: invalid value encountered in add
  z = (z + offset_scale[0][2]) * offset_scale[1][2]
 31.09 % in 4 sec [est. time left: 8 sec]/home/tan/softwares/py3dtiles/py3dtiles/points/node.py:131: RuntimeWarning: invalid value encountered in greater_equal
  indices = xyz_to_child_index(pending_xyz_arr, self.aabb_center)
 34.66 % in 4 sec [est. time left: 7 sec]/home/tan/softwares/py3dtiles/py3dtiles/points/node.py:131: RuntimeWarning: invalid value encountered in greater_equal
  indices = xyz_to_child_index(pending_xyz_arr, self.aabb_center)
 34.66 % in 4 sec [est. time left: 8 sec]/home/tan/softwares/py3dtiles/py3dtiles/points/task/las_reader.py:112: RuntimeWarning: invalid value encountered in add
  x = (x + offset_scale[0][0]) * offset_scale[1][0]
/home/tan/softwares/py3dtiles/py3dtiles/points/task/las_reader.py:113: RuntimeWarning: invalid value encountered in add
  y = (y + offset_scale[0][1]) * offset_scale[1][1]
/home/tan/softwares/py3dtiles/py3dtiles/points/task/las_reader.py:114: RuntimeWarning: invalid value encountered in add
  z = (z + offset_scale[0][2]) * offset_scale[1][2]
  40.6 % in 4 sec [est. time left: 6 sec]/home/tan/softwares/py3dtiles/py3dtiles/points/node.py:131: RuntimeWarning: invalid value encountered in greater_equal
  indices = xyz_to_child_index(pending_xyz_arr, self.aabb_center)
 100.0 % in 10 sec [est. time left: 0 sec]

building 'triangle.core' extension error install

running build_ext
building 'triangle.core' extension
error: [Error 2] The system cannot find the file specified


Command "c:\python27\python.exe -u -c "import setuptools, tokenize;file='c:\windows\temp\pip-install-zso2cu\triangle\setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record c:\windows\temp\pip-record-gu3guo\install-record.txt --single-version-externally-managed --compile" failed with error code 1 in c:\windows\temp\pip-install-zso2cu\triangle\

Py3DTiles index structures vs Tiler

Use case: from py3dTiles to CityTiler

As software end-user I want to generate a tile set out of the following sources of information:

  1. some raw geometrical data source (e.g. a set of buildings each of which described as a triangle soup, or some point clouds)
  2. some (possibly optional and possibly hierarchical) semantic data
  3. some hierarchical (functional) criteria used as ordering principle for building the TileSet hierarchy

Low level building block analysis

Solving the use case requires the following software "building blocks" described below with the natural treatment pipeline order

  • Import-Tools that allow to walk on native data structures like

  • SRS-Tools : such Spatial Reference System tools can/must be used e.g. in order to remove the Cesium bias towards WGS84 data (and that doesn't seem to allow e.g. Lambert-93 based data)

  • Index-Tools : Data Structures that can be used as geometry based "sorting criteria/algorithms" in order to build the target (3DTiles) hierarchy e.g.

  • 3DTiles: a Data Structure (DS) to be used as target placeholder (and that knows how to serialize itself i.e. can be exported)

From building blocks to components

Given the above list of building blocks, and with separation of concerns design constraint in mind (that in this context translates to "don't pollute/bias 3DTiles with specific end-user data format or tiling strategy") what are their possible aggregation into software components ?

Current decomposition

The current content of Py3DTiles integrates all the blocks:

Note the already existing bias of the py3DTiles package toward the LAS data structure.

Possible component decomposition : extend current strategy

The idea is to pursue the current integrated strategy while

  • pulling aside (separate files) the index tools
  • extending it with other import tools (3D-CityDB)
  • extending it with SRS tools
  • pulling aside (separate sub-directory) the tilers

Pros and cons:

  • Pros: all in one python package
  • Cons: no separation of concerns, many (exploding number of) dependencies, code biased by py3Dtiles early contributions

Possible component decomposition : separation Tiles/Tiler

  • Possible target component architecture i.e. as Python packages
    1. Py3DTiles
      • 3DTiles (the DS)
      • Index-Tools
    2. Py3DTilers
      • Import-Tool
      • SRS tools
      • Tilers (conversion tools)

Pros and cons:

  • Pros: there can be as many Py3DTilers components as they are communities, Py3DTiles remains end-user data "agnostic"
  • Cons: requires more work in python packaging, harder to use, to diffuse (obfuscates the usage for the user)

export_tileset is not working

I have created the virtualenv and installed setup.py but still export_tileset is not working. Can any one tell me what might be the problem

Does it support trying to support PTS point cloud

I tried running it using PTS point cloud,The following error occurs:
Traceback (most recent call last): File "E:/GitProject/NEWpy3dtiles/py3dtiles/convert.py", line 803, in <module> main(args) File "E:/GitProject/NEWpy3dtiles/py3dtiles/convert.py", line 596, in main args.rgb) File "E:/GitProject/NEWpy3dtiles/py3dtiles/convert.py", line 69, in write_tileset for child in root_tileset['children']: KeyError: 'children'

Feature Table for b3dm files

I have a question regarding the export of a b3dm from xyz WKB representing a geological layer and the assignment of a feature table. Since the export is done without a feature table (because the corresponding lines are commented out, https://github.com/Oslandia/py3dtiles/blob/master/py3dtiles/b3dm.py), the 3D Tiles are not recognized as Cesium3DTileFeature in Cesium and can not be styled.
How can I add a feature table to the b3dm with py3dtiles?

Error at exporting from postgis: struct.error: unpack requires a buffer of 16 bytes

Wanted to test how it works with getting data from postgis but I receive a struct.error: unpack requires a buffer of 16 bytes

Offence happens here:

    166             for k in range(0, pointNb-1):
    167                 pt = np.array(struct.unpack(bo + pntUnpack, wkb[offset:offset
--> 168                               + pntOffset]), dtype=np.float32)
    169                 offset += pntOffset
    170                 line.append(pt)

As far as I can see psycopg2 returns a memoryview for the ST_AsBinary object from postgres. I've tried to convert this to bytes by using bytes(wkb) but that wouldn't help.

Testing it on the building.wkb file runs without error.

Cannot run the tests nor the samples.

Hi, I'm trying to compile py3Dtiles. It works, but I can't run the tests neither the example

(venv) LEU-XFHOUBIE01:py3dtiles fhoubie$ nosetests
.......EEEEE.E

ERROR: tests.test_points.test_grid_insert

Traceback (most recent call last):
File "/Users/fhoubie/Dev/github/Ostlandia/py3dtiles/venv/lib/python3.6/site-packages/nose/case.py", line 198, in runTest
self.test(*self.arg)
TypeError: test_grid_insert() missing 2 required positional arguments: 'grid' and 'node'

======================================================================
ERROR: tests.test_points.test_grid_insert_perf

Traceback (most recent call last):
File "/Users/fhoubie/Dev/github/Ostlandia/py3dtiles/venv/lib/python3.6/site-packages/nose/case.py", line 198, in runTest
self.test(*self.arg)
TypeError: test_grid_insert_perf() missing 3 required positional arguments: 'grid', 'node', and 'benchmark'

======================================================================
ERROR: tests.test_points.test_grid_getpoints

Traceback (most recent call last):
File "/Users/fhoubie/Dev/github/Ostlandia/py3dtiles/venv/lib/python3.6/site-packages/nose/case.py", line 198, in runTest
self.test(*self.arg)
TypeError: test_grid_getpoints() missing 2 required positional arguments: 'grid' and 'node'

======================================================================
ERROR: tests.test_points.test_grid_getpoints_perf

Traceback (most recent call last):
File "/Users/fhoubie/Dev/github/Ostlandia/py3dtiles/venv/lib/python3.6/site-packages/nose/case.py", line 198, in runTest
self.test(*self.arg)
TypeError: test_grid_getpoints_perf() missing 3 required positional arguments: 'grid', 'node', and 'benchmark'

======================================================================
ERROR: tests.test_points.test_grid_get_point_count

Traceback (most recent call last):
File "/Users/fhoubie/Dev/github/Ostlandia/py3dtiles/venv/lib/python3.6/site-packages/nose/case.py", line 198, in runTest
self.test(*self.arg)
TypeError: test_grid_get_point_count() missing 2 required positional arguments: 'grid' and 'node'

======================================================================
ERROR: tests.test_points.test_is_point_far_enough_perf

Traceback (most recent call last):
File "/Users/fhoubie/Dev/github/Ostlandia/py3dtiles/venv/lib/python3.6/site-packages/nose/case.py", line 198, in runTest
self.test(*self.arg)
TypeError: test_is_point_far_enough_perf() missing 1 required positional argument: 'benchmark'


Ran 14 tests in 0.523s

FAILED (errors=6)

(venv) LEU-XFHOUBIE01:tests fhoubie$ py3dtiles info tests/pointCloudRGB.pnts
Traceback (most recent call last):
File "/Users/fhoubie/Dev/github/Ostlandia/py3dtiles/venv/bin/py3dtiles", line 11, in
load_entry_point('py3dtiles==1.1.0', 'console_scripts', 'py3dtiles')()
File "/Users/fhoubie/Dev/github/Ostlandia/py3dtiles/venv/lib/python3.6/site-packages/pkg_resources/init.py", line 484, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "/Users/fhoubie/Dev/github/Ostlandia/py3dtiles/venv/lib/python3.6/site-packages/pkg_resources/init.py", line 2714, in load_entry_point
return ep.load()
File "/Users/fhoubie/Dev/github/Ostlandia/py3dtiles/venv/lib/python3.6/site-packages/pkg_resources/init.py", line 2332, in load
return self.resolve()
File "/Users/fhoubie/Dev/github/Ostlandia/py3dtiles/venv/lib/python3.6/site-packages/pkg_resources/init.py", line 2338, in resolve
module = import(self.module_name, fromlist=['name'], level=0)
File "", line 971, in _find_and_load
File "", line 955, in _find_and_load_unlocked
File "", line 656, in _load_unlocked
File "", line 626, in _load_backward_compatible
File "/Users/fhoubie/Dev/github/Ostlandia/py3dtiles/venv/lib/python3.6/site-packages/py3dtiles-1.1.0-py3.6.egg/py3dtiles/command_line.py", line 2, in
File "", line 971, in _find_and_load
File "", line 955, in _find_and_load_unlocked
File "", line 656, in _load_unlocked
File "", line 626, in _load_backward_compatible
File "/Users/fhoubie/Dev/github/Ostlandia/py3dtiles/venv/lib/python3.6/site-packages/py3dtiles-1.1.0-py3.6.egg/py3dtiles/convert.py", line 18, in
File "", line 971, in _find_and_load
File "", line 955, in _find_and_load_unlocked
File "", line 656, in _load_unlocked
File "", line 626, in _load_backward_compatible
File "/Users/fhoubie/Dev/github/Ostlandia/py3dtiles/venv/lib/python3.6/site-packages/py3dtiles-1.1.0-py3.6.egg/py3dtiles/points/node.py", line 8, in
File "", line 971, in _find_and_load
File "", line 955, in _find_and_load_unlocked
File "", line 656, in _load_unlocked
File "", line 626, in _load_backward_compatible
File "/Users/fhoubie/Dev/github/Ostlandia/py3dtiles/venv/lib/python3.6/site-packages/py3dtiles-1.1.0-py3.6.egg/py3dtiles/points/points_grid.py", line 5, in
File "", line 971, in _find_and_load
File "", line 955, in _find_and_load_unlocked
File "", line 656, in _load_unlocked
File "", line 626, in _load_backward_compatible
File "/Users/fhoubie/Dev/github/Ostlandia/py3dtiles/venv/lib/python3.6/site-packages/py3dtiles-1.1.0-py3.6.egg/py3dtiles/points/distance.py", line 5, in
File "/Users/fhoubie/Dev/github/Ostlandia/py3dtiles/venv/lib/python3.6/site-packages/numba/decorators.py", line 191, in wrapper
disp.enable_caching()
File "/Users/fhoubie/Dev/github/Ostlandia/py3dtiles/venv/lib/python3.6/site-packages/numba/dispatcher.py", line 566, in enable_caching
self._cache = FunctionCache(self.py_func)
File "/Users/fhoubie/Dev/github/Ostlandia/py3dtiles/venv/lib/python3.6/site-packages/numba/caching.py", line 614, in init
self._impl = self._impl_class(py_func)
File "/Users/fhoubie/Dev/github/Ostlandia/py3dtiles/venv/lib/python3.6/site-packages/numba/caching.py", line 349, in init
"for file %r" % (qualname, source_path))
RuntimeError: cannot cache function 'is_point_far_enough': no locator available for file '/Users/fhoubie/Dev/github/Ostlandia/py3dtiles/venv/lib/python3.6/site-packages/py3dtiles-1.1.0-py3.6.egg/py3dtiles/points/distance.py'

List of installed modules :
(venv) LEU-XFHOUBIE01:py3dtiles fhoubie$ pip list
Package Version Location


atomicwrites 1.2.1
attrs 18.2.0
Cython 0.28.5
laspy 1.5.1
libLAS 1.8.1
llvmlite 0.25.0
lz4 2.1.0
more-itertools 4.3.0
nose 1.3.7
numba 0.40.0
numpy 1.15.2
pip 18.0
pluggy 0.7.1
psutil 5.4.7
psycopg2-binary 2.7.5
py 1.6.0
py3dtiles 1.1.0 /Users/fhoubie/Dev/github/Ostlandia/py3dtiles/venv/lib/python3.6/site-packages/py3dtiles-1.1.0-py3.6.egg
pyproj 1.9.5.1
pytest 3.8.1
pyzmq 17.1.2
setuptools 40.4.3
six 1.11.0
triangle 20170429
wheel 0.31.1

Do you know if there is something wrong in the code or in my installation ?

Thanks
Frédéric

Incorrect winding order on some polygons

Polygons are triangulated in wkb_utils.py, after which the winding order of each triangle is checked to make sure it is the same as the winding order of the source polygon. However, this check is only done on the first vertices of the polygon, and if the start of the polygon happens to be convex, an incorrect winding order is assumed. This will lead to an inverted winding order on the resulting triangles.

See below for an example. The start of the roof polygon is convex. Because of this, py3dtiles inverts the winding order, leading to an inversion of the normals and the polygon not being rendered.

image

To fix this issue, the winding order should be calculated for the entire polygon, rather than the first few vertices, I will submit a pull request to fix this.

About: Cannot display PNTS after converting LAS with Input SRS WGS84 Projection

I'm trying to convert LAS to 3DTiles using Py3DTiles.

Hi, I'm succeed in installing Py3DTiles but I cannot display 3DTiles after installed.

I used LAS Viewer to compare output PNTS file with your tested PNTS and found problem relate to difference CRS. Your tested PNTS using projected CRS and my output using lonlat CRS. I dont know how to convert to the right CRS of Py3DTiles.

Please help me to fix this.
Thank you.

3dtiles from cityGML

Hello,

Can i create a 3dtiles from cityGML with this tool?

I have cityGMLs which contain 3D city models of paris and lyon in L93, i would like to create a 3dtiles of this models in wgs84. Please tell me how can i do that?

Thanks a lot!

Debian 8.11 seems too old to install Numba Py3Dtiles' dependency

Platform:

  • OS: Debian 8.11
  • Python: 3.4.2

When launching the pip install -e . command of py3dtiles' install, pip eventually pulls various python packages sub-dependencies that fail building:

  1. Numba (building wheel fail)
    Numba seems to depend on numpy but doesn't pull that dependency. This can be solved with a manual call to pip install numpy. Then pip install numba succeeds.
  2. llvmlite (building wheel fail)
    • In order to build, llvmlite requires the llvm-config command line utility (that seems to come packaged with llvm)
    • apt-get install llvm will install llvm-config version 3.5.0 (that is the version of llvm itself).
    • But the pip install lvmlite command will terminate with the message
      RuntimeError: Building llvmlite requires LLVM 6.0.x.
      

Note that this seems to be a known issue of Numba on Debian 8

I guess the conclusion is that Debian version 8.11 default package numbers are too old to support some straightforward install of Py3DTiles (because of its Numba sub-dependency).

Suggestion: maybe we should create some "Platforms where Py3DTiles install was reported to fail" chapter/entry in the Py3DTiles install notes that mentions Debian8 ? And then simply close this issue...

Release v2

I think cac2193 alone justify this release. It's also a good reason to release a major version :-)

TODO:

  • add metadata to setup.py for pypi website
    • link to doc
  • make a release note and a migration note
  • check the doc

@delhomer wdyt?

Two (minor) remarks about the install documentation

Is requirements.txt still used ?

The from sources installation commands currently require to launch a pip install -e ..
I don't know why (since I don't much about python install/distutils/setup thingies) but I can succeed with my py3dtiles installation without running that command.

Actually I can remove the requirements.txt file at all and the installation still succeeds.

Hence my question: are they some use cases or contexts where requirements.txt and the pip install -e . are required for things to run smoothly ?

I'm asking this because when you add a new module requirement to py3dtiles, you currently apparently seem to need to update both requirements.txt and the requirements section of setup.py (which doesn't look that DRY?).

Do we need to hard wire the path to python3 in command virtualenv -p /usr/bin/python3 venv ?

The installation commands also mention to virtualenv -p /usr/bin/python3 venv. Yet on OSX the python3 interpreter is not located by default in /usr/bin (both native and homebrew versions).
I thus had to run virtualenv -p python3 venv which looks more platform independent (it suffice to change your PATH to point to a specific version prior to launching the command).

convert fails with numba error

Hi, I tried running your conversion script in order to convert from a .las to a .pnts. I ran:
py3dtiles convert 20170811bszn.las --out ./tmp/

But then I get an error from the Numba package which hints towards function variable type incongruency and the process stays stalled forever. The output was:

1.52 % in 0 sec [est. time left: 11 sec]OH NO. b'6'
Failed in nopython mode pipeline (step: nopython frontend)
Invalid use of Function(<built-in function getitem>) with argument(s) of type(s): (array(float32, 2d, C), array(float64, 1d, C))
 * parameterized
In definition 0:
    All templates rejected with literals.
In definition 1:
    All templates rejected without literals.
In definition 2:
    All templates rejected with literals.
In definition 3:
    All templates rejected without literals.
In definition 4:
    All templates rejected with literals.
In definition 5:
    All templates rejected without literals.
In definition 6:
    All templates rejected with literals.
In definition 7:
    All templates rejected without literals.
In definition 8:
    All templates rejected with literals.
In definition 9:
    All templates rejected without literals.
In definition 10:
    All templates rejected with literals.
In definition 11:
    All templates rejected without literals.
In definition 12:
    TypeError: unsupported array index type array(float64, 1d, C) in [array(float64, 1d, C)]
    raised from /home/jgbs/.local/lib/python3.5/site-packages/numba/typing/arraydecl.py:71
In definition 13:
    TypeError: unsupported array index type array(float64, 1d, C) in [array(float64, 1d, C)]
    raised from /home/jgbs/.local/lib/python3.5/site-packages/numba/typing/arraydecl.py:71
This error is usually caused by passing an argument of a type that is unsupported by the named function.
[1] During: typing of intrinsic-call at /home/jgbs/.local/lib/python3.5/site-packages/py3dtiles/points/points_grid.py (32)

File "../../../../.local/lib/python3.5/site-packages/py3dtiles/points/points_grid.py", line 32:
def _insert(cells_xyz, cells_rgb, aabmin, inv_aabb_size, cell_count, xyz, rgb, spacing, shift, force=False):
    <source elided>

        return xyz[notinserted], rgb[notinserted], needs_balance
        ^

This is not usually a problem with Numba itself but instead often caused by
the use of unsupported features or an issue in resolving types.

To see Python/NumPy features supported by the latest release of Numba visit:
http://numba.pydata.org/numba-doc/latest/reference/pysupported.html
and
http://numba.pydata.org/numba-doc/latest/reference/numpysupported.html

For more information about typing errors and how to debug them visit:
http://numba.pydata.org/numba-doc/latest/user/troubleshoot.html#my-code-doesn-t-compile

If you think your code should work with Numba, please report the error message
and traceback, along with a minimal reproducer at:
https://github.com/numba/numba/issues/new

Traceback (most recent call last):
  File "/home/jgbs/.local/lib/python3.5/site-packages/py3dtiles/points/task/node_process.py", line 157, in run
    result, data = _process(node, octree_metadata, name, filenames, queue, begin, log_file)
  File "/home/jgbs/.local/lib/python3.5/site-packages/py3dtiles/points/task/node_process.py", line 109, in _process
    node.insert(node_catalog, octree_metadata.scale, data['xyz'], data['rgb'], halt_at_depth == 0)
  File "/home/jgbs/.local/lib/python3.5/site-packages/py3dtiles/points/node.py", line 73, in insert
    self._split(node_catalog, scale)
  File "/home/jgbs/.local/lib/python3.5/site-packages/py3dtiles/points/node.py", line 155, in _split
    self.insert(node_catalog, scale, xyz, rgb)
  File "/home/jgbs/.local/lib/python3.5/site-packages/py3dtiles/points/node.py", line 80, in insert
    self.aabb[0], self.inv_aabb_size, xyz, rgb)
  File "/home/jgbs/.local/lib/python3.5/site-packages/py3dtiles/points/points_grid.py", line 62, in insert
    int(self.cell_count[0] - 1).bit_length(), force)
  File "/home/jgbs/.local/lib/python3.5/site-packages/numba/dispatcher.py", line 401, in _compile_for_args
    error_rewrite(e, 'typing')
  File "/home/jgbs/.local/lib/python3.5/site-packages/numba/dispatcher.py", line 344, in error_rewrite
    reraise(type(e), e, None)
  File "/home/jgbs/.local/lib/python3.5/site-packages/numba/six.py", line 668, in reraise
    raise value.with_traceback(tb)
numba.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend)
Invalid use of Function(<built-in function getitem>) with argument(s) of type(s): (array(float32, 2d, C), array(float64, 1d, C))
 * parameterized
In definition 0:
    All templates rejected with literals.
In definition 1:
    All templates rejected without literals.
In definition 2:
    All templates rejected with literals.
In definition 3:
    All templates rejected without literals.
In definition 4:
    All templates rejected with literals.
In definition 5:
    All templates rejected without literals.
In definition 6:
    All templates rejected with literals.
In definition 7:
    All templates rejected without literals.
In definition 8:
    All templates rejected with literals.
In definition 9:
    All templates rejected without literals.
In definition 10:
    All templates rejected with literals.
In definition 11:
    All templates rejected without literals.
In definition 12:
    TypeError: unsupported array index type array(float64, 1d, C) in [array(float64, 1d, C)]
    raised from /home/jgbs/.local/lib/python3.5/site-packages/numba/typing/arraydecl.py:71
In definition 13:
    TypeError: unsupported array index type array(float64, 1d, C) in [array(float64, 1d, C)]
    raised from /home/jgbs/.local/lib/python3.5/site-packages/numba/typing/arraydecl.py:71
This error is usually caused by passing an argument of a type that is unsupported by the named function.
[1] During: typing of intrinsic-call at /home/jgbs/.local/lib/python3.5/site-packages/py3dtiles/points/points_grid.py (32)

File "../../../../.local/lib/python3.5/site-packages/py3dtiles/points/points_grid.py", line 32:
def _insert(cells_xyz, cells_rgb, aabmin, inv_aabb_size, cell_count, xyz, rgb, spacing, shift, force=False):
    <source elided>

        return xyz[notinserted], rgb[notinserted], needs_balance
        ^

This is not usually a problem with Numba itself but instead often caused by
the use of unsupported features or an issue in resolving types.

To see Python/NumPy features supported by the latest release of Numba visit:
http://numba.pydata.org/numba-doc/latest/reference/pysupported.html
and
http://numba.pydata.org/numba-doc/latest/reference/numpysupported.html

For more information about typing errors and how to debug them visit:
http://numba.pydata.org/numba-doc/latest/user/troubleshoot.html#my-code-doesn-t-compile

If you think your code should work with Numba, please report the error message
and traceback, along with a minimal reproducer at:
https://github.com/numba/numba/issues/new

OH NO. b'0'
Failed in nopython mode pipeline (step: nopython frontend)
Invalid use of Function(<built-in function getitem>) with argument(s) of type(s): (array(float32, 2d, C), array(float64, 1d, C))
 * parameterized
In definition 0:
    All templates rejected with literals.
In definition 1:
    All templates rejected without literals.
In definition 2:
    All templates rejected with literals.
In definition 3:
    All templates rejected without literals.
In definition 4:
    All templates rejected with literals.
In definition 5:
    All templates rejected without literals.
In definition 6:
    All templates rejected with literals.
In definition 7:
    All templates rejected without literals.
In definition 8:
    All templates rejected with literals.
In definition 9:
    All templates rejected without literals.
In definition 10:
    All templates rejected with literals.
In definition 11:
    All templates rejected without literals.
In definition 12:
    TypeError: unsupported array index type array(float64, 1d, C) in [array(float64, 1d, C)]
    raised from /home/jgbs/.local/lib/python3.5/site-packages/numba/typing/arraydecl.py:71
In definition 13:
    TypeError: unsupported array index type array(float64, 1d, C) in [array(float64, 1d, C)]
    raised from /home/jgbs/.local/lib/python3.5/site-packages/numba/typing/arraydecl.py:71
This error is usually caused by passing an argument of a type that is unsupported by the named function.
[1] During: typing of intrinsic-call at /home/jgbs/.local/lib/python3.5/site-packages/py3dtiles/points/points_grid.py (32)

File "../../../../.local/lib/python3.5/site-packages/py3dtiles/points/points_grid.py", line 32:
def _insert(cells_xyz, cells_rgb, aabmin, inv_aabb_size, cell_count, xyz, rgb, spacing, shift, force=False):
    <source elided>

        return xyz[notinserted], rgb[notinserted], needs_balance
        ^

This is not usually a problem with Numba itself but instead often caused by
the use of unsupported features or an issue in resolving types.

To see Python/NumPy features supported by the latest release of Numba visit:
http://numba.pydata.org/numba-doc/latest/reference/pysupported.html
and
http://numba.pydata.org/numba-doc/latest/reference/numpysupported.html

For more information about typing errors and how to debug them visit:
http://numba.pydata.org/numba-doc/latest/user/troubleshoot.html#my-code-doesn-t-compile

If you think your code should work with Numba, please report the error message
and traceback, along with a minimal reproducer at:
https://github.com/numba/numba/issues/new

Traceback (most recent call last):
  File "/home/jgbs/.local/lib/python3.5/site-packages/py3dtiles/points/task/node_process.py", line 157, in run
    result, data = _process(node, octree_metadata, name, filenames, queue, begin, log_file)
  File "/home/jgbs/.local/lib/python3.5/site-packages/py3dtiles/points/task/node_process.py", line 109, in _process
    node.insert(node_catalog, octree_metadata.scale, data['xyz'], data['rgb'], halt_at_depth == 0)
  File "/home/jgbs/.local/lib/python3.5/site-packages/py3dtiles/points/node.py", line 73, in insert
    self._split(node_catalog, scale)
  File "/home/jgbs/.local/lib/python3.5/site-packages/py3dtiles/points/node.py", line 155, in _split
    self.insert(node_catalog, scale, xyz, rgb)
  File "/home/jgbs/.local/lib/python3.5/site-packages/py3dtiles/points/node.py", line 80, in insert
    self.aabb[0], self.inv_aabb_size, xyz, rgb)
  File "/home/jgbs/.local/lib/python3.5/site-packages/py3dtiles/points/points_grid.py", line 62, in insert
    int(self.cell_count[0] - 1).bit_length(), force)
  File "/home/jgbs/.local/lib/python3.5/site-packages/numba/dispatcher.py", line 401, in _compile_for_args
    error_rewrite(e, 'typing')
  File "/home/jgbs/.local/lib/python3.5/site-packages/numba/dispatcher.py", line 344, in error_rewrite
    reraise(type(e), e, None)
  File "/home/jgbs/.local/lib/python3.5/site-packages/numba/six.py", line 668, in reraise
    raise value.with_traceback(tb)
numba.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend)
Invalid use of Function(<built-in function getitem>) with argument(s) of type(s): (array(float32, 2d, C), array(float64, 1d, C))
 * parameterized
In definition 0:
    All templates rejected with literals.
In definition 1:
    All templates rejected without literals.
In definition 2:
    All templates rejected with literals.
In definition 3:
    All templates rejected without literals.
In definition 4:
    All templates rejected with literals.
In definition 5:
    All templates rejected without literals.
In definition 6:
    All templates rejected with literals.
In definition 7:
    All templates rejected without literals.
In definition 8:
    All templates rejected with literals.
In definition 9:
    All templates rejected without literals.
In definition 10:
    All templates rejected with literals.
In definition 11:
    All templates rejected without literals.
In definition 12:
    TypeError: unsupported array index type array(float64, 1d, C) in [array(float64, 1d, C)]
    raised from /home/jgbs/.local/lib/python3.5/site-packages/numba/typing/arraydecl.py:71
In definition 13:
    TypeError: unsupported array index type array(float64, 1d, C) in [array(float64, 1d, C)]
    raised from /home/jgbs/.local/lib/python3.5/site-packages/numba/typing/arraydecl.py:71
This error is usually caused by passing an argument of a type that is unsupported by the named function.
[1] During: typing of intrinsic-call at /home/jgbs/.local/lib/python3.5/site-packages/py3dtiles/points/points_grid.py (32)

File "../../../../.local/lib/python3.5/site-packages/py3dtiles/points/points_grid.py", line 32:
def _insert(cells_xyz, cells_rgb, aabmin, inv_aabb_size, cell_count, xyz, rgb, spacing, shift, force=False):
    <source elided>

        return xyz[notinserted], rgb[notinserted], needs_balance
        ^

This is not usually a problem with Numba itself but instead often caused by
the use of unsupported features or an issue in resolving types.

To see Python/NumPy features supported by the latest release of Numba visit:
http://numba.pydata.org/numba-doc/latest/reference/pysupported.html
and
http://numba.pydata.org/numba-doc/latest/reference/numpysupported.html

For more information about typing errors and how to debug them visit:
http://numba.pydata.org/numba-doc/latest/user/troubleshoot.html#my-code-doesn-t-compile

If you think your code should work with Numba, please report the error message
and traceback, along with a minimal reproducer at:
https://github.com/numba/numba/issues/new

Could you shed any light on this error? Thank you.

numba.errors.UntypedAttributeError: Failed at nopython (nopython frontend)

(newpy3dtiles) E:\GitProject\py3dtiles\py3dtiles>py3dtiles convert ../exp/Tile_1.las --jobs 1 --overwrite 1 --out /tmp/destination
args: Namespace(benchmark=None, cache_size=1631, color_scale=None, command='convert', files=['../exp/Tile_1.las'], fraction=100, graph=True, jobs=1, out='/tmp/destination', overwrite=True, rgb=True, srs_in=None, srs_out=None, verbose=1)
Summary:

  • points to process: 15214249
  • offset to use: [ -37.52559269 -1.49362839 169.98965934]
  • root spacing: 0.9357871578755127
  • root aabb: [[ 0. 0. 0. ]
    [ 96.19355425 59.94030074 28.92638144]]
  • original aabb: [[ 0. 0. 0. ]
    [ 96.19355425 59.94030074 28.92638144]]

Submit next portion ('../exp/Tile_1.las', (15000000, 15214249))
0.0 % points in 0.7 sec [1 tasks, 1 nodes, 214249 wip]
0.0 % points in 0.7 sec [1 tasks, 2 nodes, 214249 wip]
OH NO. b'6'
Failed at nopython (nopython frontend)
Unknown attribute 'unique' of type Module(<module 'numpy' from 'd:\programdata\anaconda3\envs\newpy3dtiles\lib\site-packages\numpy\init.py'>)
File "points\points_grid.py", line 14
[1] During: typing of get attribute at e:\gitproject\py3dtiles\py3dtiles\points\points_grid.py (14)
Traceback (most recent call last):
File "e:\gitproject\py3dtiles\py3dtiles\points\task\node_process.py", line 156, in run
result, data = _process(node, octree_metadata, name, filenames, queue, begin, log_file)
File "e:\gitproject\py3dtiles\py3dtiles\points\task\node_process.py", line 108, in _process
node.insert(node_catalog, octree_metadata.scale, data['xyz'], data['rgb'], halt_at_depth == 0)
File "e:\gitproject\py3dtiles\py3dtiles\points\node.py", line 73, in insert
self._split(node_catalog, scale)
File "e:\gitproject\py3dtiles\py3dtiles\points\node.py", line 155, in _split
self.insert(node_catalog, scale, xyz, rgb)
File "e:\gitproject\py3dtiles\py3dtiles\points\node.py", line 80, in insert
self.aabb[0], self.inv_aabb_size, xyz, rgb)
File "e:\gitproject\py3dtiles\py3dtiles\points\points_grid.py", line 62, in insert
int(self.cell_count[0] - 1).bit_length(), force)
File "d:\programdata\anaconda3\envs\newpy3dtiles\lib\site-packages\numba\dispatcher.py", line 330, in _compile_for_args
raise e
File "d:\programdata\anaconda3\envs\newpy3dtiles\lib\site-packages\numba\dispatcher.py", line 307, in _compile_for_args
return self.compile(tuple(argtypes))
File "d:\programdata\anaconda3\envs\newpy3dtiles\lib\site-packages\numba\dispatcher.py", line 579, in compile
cres = self._compiler.compile(args, return_type)
File "d:\programdata\anaconda3\envs\newpy3dtiles\lib\site-packages\numba\dispatcher.py", line 80, in compile
flags=flags, locals=self.locals)
File "d:\programdata\anaconda3\envs\newpy3dtiles\lib\site-packages\numba\compiler.py", line 763, in compile_extra
return pipeline.compile_extra(func)
File "d:\programdata\anaconda3\envs\newpy3dtiles\lib\site-packages\numba\compiler.py", line 360, in compile_extra
return self._compile_bytecode()
File "d:\programdata\anaconda3\envs\newpy3dtiles\lib\site-packages\numba\compiler.py", line 722, in _compile_bytecode
return self._compile_core()
File "d:\programdata\anaconda3\envs\newpy3dtiles\lib\site-packages\numba\compiler.py", line 709, in _compile_core
res = pm.run(self.status)
File "d:\programdata\anaconda3\envs\newpy3dtiles\lib\site-packages\numba\compiler.py", line 246, in run
raise patched_exception
File "d:\programdata\anaconda3\envs\newpy3dtiles\lib\site-packages\numba\compiler.py", line 238, in run
stage()
File "d:\programdata\anaconda3\envs\newpy3dtiles\lib\site-packages\numba\compiler.py", line 452, in stage_nopython_frontend
self.locals)
File "d:\programdata\anaconda3\envs\newpy3dtiles\lib\site-packages\numba\compiler.py", line 865, in type_inference_stage
infer.propagate()
File "d:\programdata\anaconda3\envs\newpy3dtiles\lib\site-packages\numba\typeinfer.py", line 844, in propagate
raise errors[0]
File "d:\programdata\anaconda3\envs\newpy3dtiles\lib\site-packages\numba\typeinfer.py", line 137, in propagate
constraint(typeinfer)
File "d:\programdata\anaconda3\envs\newpy3dtiles\lib\site-packages\numba\typeinfer.py", line 519, in call
raise UntypedAttributeError(ty, self.attr, loc=self.inst.loc)
numba.errors.UntypedAttributeError: Failed at nopython (nopython frontend)
Unknown attribute 'unique' of type Module(<module 'numpy' from 'd:\programdata\anaconda3\envs\newpy3dtiles\lib\site-packages\numpy\init.py'>)
File "points\points_grid.py", line 14
[1] During: typing of get attribute at e:\gitproject\py3dtiles\py3dtiles\points\points_grid.py (14)

Boxes tilted over x-axes

I noticed that the bounding boxes created by py3dtiles are showing up in Cesium (debugShowBoundingVolume: true) as if they are tilted over the x-axis
(One might also assume that it is just the earth surface that is tilted and the boxes perfectly aligned to the cartesian grid)

image
(North is to the right of the image, latitude if roughly 51.84 degrees)

Here is a snippet of the tileset.json created by py3dtiles:

{"asset": {"version": "1.0", "gltfUpAxis": "Z"}, "geometricError": 500, "root": {"boundingVolume": {"box": [0.0, 0.0, 0.0, 1736.569, 0, 0, 0, 3140.645, 0, 0, 0, 1281.764]}, "geometricError": 500, "children": [...]
...
"refine": "add", "content": {"url": "tiles/1580.b3dm"}}], "refine": "add", "transform": [1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 3927692.935, 403020.138, 4992459.055, 1.0]}}

I am not sure how this is caused or what the consequences are. I am not used with working with the box as a boundingvolume and always used region up till now so it may even be normal behaviour or have nothing to do with the way py3dtiles works. I am seeing however unexpected disappearance of some tiles when I rotate the view around the Z-axis (perpend. to earth surface) that may have to do with how the boundingvolumes are defined.

Questions are:

  1. Is this normal or to be expected behaviour of a box as boundingVolume?
  2. Could it explain the unexpected disappearance from rendering of boxes when rotated to certain angles?

If needed I can provide a sample dataset and viewer.

WKB parsing error

Hello!

I'm trying to use this module to convert WKB -> B3dm and I always have an error when trying to parse a wkb file different than test/building.wkb.

At first I used code like that (it is an example from GDAL documentation), I just created a multipolygon feature and saved it as a wkb:

from osgeo import ogr

multipolygon = ogr.Geometry(ogr.wkbMultiPolygon)

# Create ring #1
ring1 = ogr.Geometry(ogr.wkbLinearRing)
ring1.AddPoint(0, 0)
ring1.AddPoint(5, 0)
ring1.AddPoint(5, 5)
ring1.AddPoint(0, 5)
ring1.AddPoint(0, 0)

# Create polygon #1
poly1 = ogr.Geometry(ogr.wkbPolygon)
poly1.AddGeometry(ring1)
multipolygon.AddGeometry(poly1)

wkb = multipolygon.ExportToWkb()

And the rest of this script I just copied from the How to write a .b3dm file section. It gives me an error:

Traceback (most recent call last):
  File "temp.py", line 77, in <module>
    ts = TriangleSoup.from_wkb_multipolygon(wkb)
  File "/.../py3dtiles/wkb_utils.py", line 27, in from_wkb_multipolygon
    multipolygons = [parse(bytes(wkb))]
  File "/.../py3dtiles/wkb_utils.py", line 170, in parse
    pntOffset]), dtype=np.float32)
struct.error: unpack requires a bytes object of length 16

This wkb file is valid, I can open it in FME viewer.
Then I tried to figure this out and I noticed, that multipolygons = [parse(bytes(wkb))] returns triangulated multipolygon (building.wkb):

[
    [
        [P1, P2, P3, P1],
        [Pi, Pj, Pk, Pi],
        ...
    ]
]

So I tried with a simple triangle. And the same error occured.

Finally I created a new method in the TriangleSoup class from_geojson_multipolygon, where instead of parsing a binary file I provide just a geojson geometry (nested lists and points as numpy arrays). This works, b3dm file is generated but it seems to be invalid.
I used a simple triangle: [[0,0,0], [5,0,0], [5,5,0], [0,0,0]]
And I can't display it in the Cesium. I also generated a tileset in FME with this single feature and it works. B3dm file looks a little bit different.

Any ideas what can be wrong?

When Point cloud zoom ou ,it disappeared

i try to convert txt format point cloud(e.g format: x y z r g b) to 3D Tiles (.pnts),it successed,but in Cesium,when I want to zoom out, it disappered.
and next problem is ,what is the format of xyz?when i convert ,the position is wrong ,totally strange.
i use WGS84Plane coordinate system,like
2552604.4 187598.9 0.0 6 18 24
2552604.4 187599.9 0.0 56 25 188
2552604.4 187600.9 0.0 232 77 222
2552604.4 187601.9 0.0 78 60 161
2552604.4 187602.9 0.0 180 215 55
2552604.4 187603.9 0.0 141 171 189
2552604.4 187604.9 0.0 142 216 27
this position is in China

Batch table support

I would like to style per point metadata in 3d tiles using the cesiumJS styling specification.

To do so (I believe) I need batch table support. Are there any plans for implementing batch table support?

[ENH] Support for batch table hierarchy extension

The need

Among the available 3Dtiles extensions, the batch table hierarchy extensions allows for "more complex metadata structures such as feature types or feature hierarchies".
The need for Py3DTiles is thus

  • to support this batch table hierarchy extension
  • to allow for the definition of other (possible user provided) extensions

Elements of reflection

  • As for the current version, the functionalities associated to a TileSet seem to be scattered (including the writing of a TileSet. This code structure makes it difficult to pinpoint the right location for introducing attaching some extension to a TileSet. Introducing a TileSet class would clarify the general code structure as well as the possible usage.
    For example we could write (something like):
     t = TileSet()
     for f in wkb_files:
         t.append_wkb(f)
     t.add_extension("3DTiles_batch_table_hierarchy")
     t.batch_table = ...
     t.write("tileset.json")
    
  • The concrete mean for a 3D Tiles JSON object is to add an optional extensions dictionary property. They are as many data structures for extensions dictionary entries as they are documented extensions. PY3Dtiles could thus provide a mechanism to declare (on the fly i.e. after creation) that a given TileSet instance should bear some given specific extension. An extension itself could possibly be defined as class respecting an extension API (defined as an abstract class). A user could then write
     class my_time_extension(Py3dTiles.Extension):
        def name(self):
            return "time_extension"
         def to_json(self):
            ....
    
     TileSet.register_extension(my_time_extension)
     my_time_extension_content = <...some info...>
     t = TileSet()
     t.add_extension("time_extension", my_time_extension_content)
     t.write("tileset.json")
    
  • PY3dTiles would provide the implementation of some standard extensions (e.g. batch table hierarchy) allowing a user to write (without having to register the "batch table hierarchy" standard extension)
    t = TileSet()
    ...
    my_batch_table_content = <...some info...>
    t.add_extension("bach_table_hierarchy", my_batch_table_content)
    t.write("tileset.json")
    

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.