mapbox / rio-rgbify Goto Github PK

In order to more easily stay under total and individual (tile) size limits, we should allow creation of mbtiles with 256x256 tiles in addition to 512x512.

The only blocker AFAIK being the rasterio.warp.RESAMPLE import.

Hi @dnomadb,

I am executing the following command to generate my own mbtiles:

rio rgbify -b -10000 -i 0.1 --max-z 19 --min-z 0 --format png input-tiff output-mbtiles

But the command cannot quit automatically after a long time execution and I have to interrupt it manually.

The input tiff file is small (10 MB) and it shouldn't take so long time (30 minutes) to perform it. Any suggestions? Thanks a lot!

Oliver

I am getting this error.
I have installed python 3.8 and pip install rio-rgbify.
When ever i run this command "rio rgbify --format png --min-z 0 --max-z 15 input.tif output.mbtiles

I am getting this error "_rasterio.err.CPLE_AppDefinedError: densify_pts must be at least 2 if the output is geograpic."

Kindly tell me what is the issue. I have checked all the online forums and also try your tif dataset.

I have a raster that covers a section of the map. The raster is does not cover a full tile [x,y,z] so the libray fills the remaining space to cover an entire tile. (sky blue on top is what rio-rgbify adds)

The transition from my raster, to the fill added by rio creates an unnatural edge to it which in a hillshade on the map looks like this:

The original raster does not have that intense margin.

Is this the expected behavior ? Do you have any suggestion how to avoid the prominent edge ?

rio rgbify --min-z 0 --max-z 14 --format png LHI.tiff LHI.mbtiles

Traceback (most recent call last):
  File "/rio-rgbify/rio_rgbify/scripts/cli.py", line 84, in rgbify
    tiler.run(workers)
  File "/rio-rgbify/rio_rgbify/mbtiler.py", line 355, in run
    tiles):
  File "/usr/lib/python2.7/multiprocessing/pool.py", line 673, in next
    raise value
CRSError: CRS is empty or invalid: u''

Traceback (most recent call last):
  File "/usr/local/bin/rio", line 11, in <module>
    sys.exit(main_group())
  File "/lib/python2.7/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/lib/python2.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/lib/python2.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/lib/python2.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/lib/python2.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/lib/python2.7/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/rio-rgbify/rio_rgbify/scripts/cli.py", line 84, in rgbify
    tiler.run(workers)
  File "/rio-rgbify/rio_rgbify/mbtiler.py", line 355, in run
    tiles):
  File "/usr/lib/python2.7/multiprocessing/pool.py", line 673, in next
    raise value
rasterio.errors.CRSError: CRS is empty or invalid: u''

LHI.tiff.zip

Hi,

We are trying to display our custom terrain / hillshade using RGB tiles. To do so, we went through the following procedure:

gdalwarp -t_srs EPSG:3857 -dstnodata None -novshiftgrid -co TILED=YES -co COMPRESS=LZW -co BIGTIFF=IF_NEEDED -ts 18000 18000 -r lanczos -wo SOURCE_EXTRA=1000 SRTM.tif SRTM.3857.tif

rio rgbify -b -10000 -i 0.1 SRTM.3857.tif SRTM.3857.RGB.tif

gdal2tiles.py --zoom=0-12 --processes=8 --srcnodata=-32768 -x SRTM.3857.RGB.tif tiles/

mb-util --image_format=png --scheme=tms tiles/ terrain_rgb.mbtiles

So, we ended up with a working MBTiles filled with RGB PNG files, though below the result is not satisfying as you can see different weird spots usually located at around 3000 meters height.

We tried the procedure with two different sources (SRTM and EUDEM) the result is the same.

Do you have any idea?

Thanks.

root@ubuntu:/mapbox/rio-rgbify# rio rgbify

Rio utility v1.07 - The Snowblind Alliance (c) 1999

command line switches available :-
-d display directory
-iy initialize with check for bad blocks
-in initialize without check for bad blocks
-x perform operations on external flash ram
-u specify file(s) to upload
-g specify file to download
-f specify text based playlist file which contains files to be upload
-z specify file to delete
-za delete all files
-o specify new playlist order in quotes
-p specify parallel port base IO address, default=0x378
-v enable verbose mode
-di specify initialization delay (default is 20000)
-dt specify tx delay (default is 100)
-dr specify rx delay (default is 2)

Hello, just wondering where things stood on getting a DEM TIF (16 bit signed) into mapbox studio. I used rgbify and gdal and it looks okay in Mapbox studio, but perhaps since its seeing it as a raster instead of raster-dem getting a bunch of noise when I looking at it as a terrain in Unity.

I've managed to generate a RGB DEM, but my problem is that the process generates the background values that I point in the image. When I try to simbolyze the dem in Maputnik this values are still present and I need to get rid of them. I suppose that is related to the base value, which is set to 10000, but I wonder if there is any way of removing these values once I have the MBTiles.

Using filter 3 results in ~30% smaller PNG files in some situations.

While processing a tif with very large no data value, i get this error:

ValueError: Data of 3.4028234663852886e+38 larger than 256 ** 3

A little unexpected since there is no documentation of this limitation.

Hi, I use my DEM generate from Drone using OpenDroneMap. After that, I use gdal_warp to re-project my DSM to EPSG:3857 and transform the greyscale data into the RGB data using rio-rgbify and last step i use gdal2tiles.py to generate xyz tiles. But i got error when add tiles layer to Mapbox GL JS as the picture shown below.

        this.map.addSource("mydem", {
          type: "raster-dem",
          tiles: [
            "https://url/tiles/dem-rgb-v2/{z}/{x}/{y}.png",
          ],
          tileSize: 512,
          maxzoom: 18
        });
        this.map.setTerrain({ source: "mydem", exaggeration: 0.7 });

I noticed that the created mbtiles missing metadata. But the metadata is useful for mapbox-gl ecosystem, such as mbview.

What I'm primarily trying to achieve is to be able to craft a 'Mars basemap' Mapbox style using Studio. The basemap is based on a hillshade layer derived from MOLA data (source: https://astrogeology.usgs.gov/search/map/Mars/GlobalSurveyor/MOLA/Mars_MGS_MOLA_DEM_mosaic_global_463m)

The following picture shows you as far as I managed to get, having imported the generated mbtiles file. It seems Studio doesn't recognized it as RGB-encoded mbtiles but as a simple raster... I haven't used any particular options to run rgbify:
rio rgbify mola-dem-webm-s.tif mola-dem.mbtiles --min-z 0 --max-z 4

Note: mola-dem-webm-s.tif is an Int16 GeoTiff file in EPSG:3857 projected coordinate system.

This is in case anyone else runs into the same issue, really.

I was having issues with artefacts at the edges of source images, caused by nodata values. Adding a call to numpy's nan_to_num in data_to_rgb (encoders.py) fixes it:

data /= interval
data = np.nan_to_num(data, True, 100000.0)
data = np.around(data / 2**round_digits) * 2**round_digits

(100000.0 is from the -b and -i values of my invocation and shouldn't be hardcoded, obviously.)

Hi,
I would like to create a AR application in Unity using Mapbox SDK and I need to import elevation data into Mapbox Studio.
It seems like i need to use rasterio rio-rgbify command but I don't know how to do it ...
Do i need to install python or anything else ?
Thank you for your help

Hi,

I'm trying to achieve the same results as in the blogpost https://blog.mapbox.com/realistic-terrain-with-custom-styling-ce1fe98518ab

I've managed to create both geotif and .mbtiles from my terrain geotif, using rio-rgbify. After uploading the files to mapbox as a raster-layer the rgb-values get messed up by the compression of the tiles, causing the elevation-styling to be corrupt.

I don't know if this is the right forum, but any ideas would be helpful!

Hey folks it looks like April 8th 2022 was the last release

https://pypi.org/project/rio-rgbify/#history

It contains e.g. #34

But looking at the source tarball and the wheel it seems like the wheel does not include that pull request.

Have a look e.g. at cli.py and search for the round-digits option; the source tarball has it, the wheel doesn't

• https://files.pythonhosted.org/packages/64/81/9fed37f150ad9b96f4fb5a8215d657ff37f2d37b5cf170786c0def3ad1eb/rio-rgbify-0.4.0.tar.gz
• https://files.pythonhosted.org/packages/99/5b/53107a991279957763ff877d3228ac9268d3e301a2572114424403016a1f/rio_rgbify-0.4.0-py3-none-any.whl

Could you folks cut a new release here, git-tagging it, and uploading consistent sources and wheels please!

Thank you!

Hello

Are DTED format or SRTM Supported?

With a DTED I run the following command rio rgbify -b 10000 -i 0.1 N43.dt0 n43.rgb.tif and I get the following message : rasterio._err.CPLE_NotSupportedError: DTED driver only uses the first band of the dataset

With a SRTM downloaded from http://dwtkns.com/srtm/ i run the same command rio rgbify -b 10000 -i 0.1 srtm_38_04.tif srtm_38_04_rgb.tif and I get this message Given nodata value, -32768.0, is beyond the valid range of its data type, <class 'numpy.uint8'>.

I may be doing something wrong somewhere but so far i've not seen where...
Regards

Hi,

I have multiple raster .mbtiles generated by command rio rgbify and all of them have the same zoom.
Each of them is a part of city and can be combined together to form the whole city. Is there any solution I can merge them into a single one so that I am able to manage them more efficiently? Thank you

Oliver

Have been trying to create an mbtiles with a 2 band data tif that i have :

rio rgbify india-pop-density.tif india-pop-density.mbtiles --max-z 10 --min-z 2

but I get this error:

rasterio.errors.CRSError: Invalid CRS: "init='epsg:3857'"

• Traceback: https://pastebin.com/EprQbvFJ
• TIF: India population density

cc @dnomadb @perrygeo

Hi there,

I am developing a 3D terrain using Maplibre GL, i want to use my own DEM data to visualize the terrain. I have converted DEM with EPSG:3857 into RGB terrain by utilizing water-gis tools. The tools comply with the MaplibreGL equation: height = -10000 + ((R * 256 * 256 + G * 256 + B) * 0.1).
Nonetheless, when i tried to visualize the raster-tiles into the raster-dem there is a vertical distortion occurred, it is shifted vertically, the picture as follows:

To debug this error, i have tried to change the base of the equation from (-10000) to (0) but it turns out that evoke other error to the terrain. The zoom level can only enable until 12, after that the map returns blank white screen. The base still shifted vertically but not as significant when the base is (-10000), picture as follows:

Other debugging effort is by customizing the encoding of maplibre, the code as follows:

 // Use a different source for terrain and hillshade layers, to improve render quality
              terrainSource: {
                type: 'raster-dem',
                // url: 'https://demotiles.maplibre.org/terrain-tiles/tiles.json',
                tiles: ['http://localhost:5173/BDGtiles/{z}/{x}/{y}.png'],
                tileSize: 256,
                encoding: "custom",
                baseShift: 10000,
                redFactor: 0,
                greenFactor:1,
                blueFactor:1
}

The terrain touches the ground with this method but the terrain surface getting rougher and abstract. image as follows:

How can i repair the base shifting issue from -10000 to 0 without causing any other bug?

Thank You

in

cc @dnomadb

Hey folks 👋

I gave this rio plugin a try to quickly encode a DEM raster into tiles the mapbox raster-dem source understands.

I then hit the following issue

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/rio_rgbify/scripts/cli.py", line 84, in rgbify
    tiler.run(workers)
  File "/usr/local/lib/python3.8/dist-packages/rio_rgbify/mbtiler.py", line 354, in run
    for tile, contents in self.pool.imap_unordered(self.run_function,
  File "/usr/local/lib/python3.8/dist-packages/riomucho/single_process_pool.py", line 15, in imap_unordered
    yield func(item)
  File "/usr/local/lib/python3.8/dist-packages/rio_rgbify/mbtiler.py", line 129, in _tile_worker
    reproject(
  File "/usr/local/lib/python3.8/dist-packages/rasterio/env.py", line 387, in wrapper
    return f(*args, **kwds)
  File "/usr/local/lib/python3.8/dist-packages/rasterio/env.py", line 612, in wrapper
    return f(*args, **kwds)
  File "/usr/local/lib/python3.8/dist-packages/rasterio/warp.py", line 368, in reproject
    _reproject(
  File "rasterio/_warp.pyx", line 474, in rasterio._warp._reproject
  File "rasterio/_warp.pyx", line 433, in rasterio._warp._reproject
  File "rasterio/_io.pyx", line 1778, in rasterio._io.InMemoryRaster.__init__
  File "rasterio/_base.pyx", line 1444, in rasterio._base._osr_from_crs
  File "/usr/local/lib/python3.8/dist-packages/rasterio/crs.py", line 496, in from_user_input
    return cls.from_string(value, morph_from_esri_dialect=morph_from_esri_dialect)
  File "/usr/local/lib/python3.8/dist-packages/rasterio/crs.py", line 400, in from_string
    return cls.from_proj4(string)
  File "/usr/local/lib/python3.8/dist-packages/rasterio/crs.py", line 421, in from_proj4
    obj._crs = _CRS.from_proj4(proj)
  File "rasterio/_crs.pyx", line 369, in rasterio._crs._CRS.from_proj4
rasterio.errors.CRSError: The PROJ4 dict could not be understood. PROJ: proj_create: cannot expand init='epsg:3857' +type=crs
Traceback (most recent call last):
  File "rasterio/_crs.pyx", line 367, in rasterio._crs._CRS.from_proj4
  File "rasterio/_err.pyx", line 193, in rasterio._err.exc_wrap_int
rasterio._err.CPLE_AppDefinedError: PROJ: proj_create: cannot expand init='epsg:3857' +type=crs

which seems to be related to rasterio/rasterio#1648 and due to us using the wrong way of specifying the epsg.

I fixed this issue locally and wanted to upstream it, but turns out we already have this commit 822aafc from March 29th 2018. And there is #32 upper casing EPSG. But we simply never rolled out a new release, meaning we're still affected by this issue

• https://github.com/mapbox/rio-rgbify/releases/tag/0.3.2
• https://pypi.org/project/rio-rgbify/

Could you folks merge #32 and tag & upload a new release, otherwise this project is pretty much unusable by now.

I install rasterio and rio-rgbify:

brew install gdal
sudo -H pip install rasterio
sudo -H pip install rio-rgbify

When I use:

rio rgbify aa.tiff aa.mbtiles

I get this error:

Usage: rio [OPTIONS] COMMAND [ARGS]...

Error: No such command "rgbify".

Anything am I missing?

can't fully understand the '-interval' params

I have a small tiff file(1.8M),it make MBTiles fine.
But another big file(400G),it quit without any error message output.
Any ideas?
python 2.7
rio-rgbify 0.3.2
rasterio 1.1.8
GDAL 2.4.1
OSX 10.15.7