Looks like the last two releases were tagged in the version control, but not released:
https://github.com/ElectronicChartCentre/java-vector-tile/releases

Thanks for this library!

Is it possible to use linestrings like so:

vtm.addFeature("isochrone", attributes, gf.createMultiLineString(lineStrings));

When I do this I get a result but when I use vtzero to show and check this file I get:

Error in layer 0 (feature 0): expected command 2 but got 1

For example, when a polygon's intersection of buffered tile envelope is a GeometryCollection which consists of a Point and a Polygon, this GeometryCollection will be omitted from the encoded bytes of tile.
Because the commands(geometry Geometry) method only process MultiPoint, MultiLineString, MultiPolygon, Point, LineString and Polygon, so we should convert the clipped mixed GeometryCollection to one of them.

Using this with the latest MapboxGL JS libraries (0.28) will give warnings that the tiles are not using v2 tile spec:
Vector tile source "gbif" layer "regression" does not use vector tile spec v2 and therefore may have some rendering errors.

Would you welcome a pull request to address this?

Hi !

I'm trying to have (on android) online map using sources from mapbox and offline maps using mbTiles. So i needed a decoder to get the data. The one I'm using is too slow (about few seconds to load the screen) so I tried with yours. Performance problems have gone away but now I have crash due to lack of memory after some seconds (it depends of the quantity of data on the screen).

Have you ever had this problem ?

I thought it's from the auto-generated VectorTile.java (which contains 4272 lines), so I've tried to generate a new one with the google.protobuf.nano lib (now it contains 781 lines) but now I have to modify every called methods in the decoder because they don't exists anymore or their name have changed.

Have you got an idea to improve the consumption of memory of you code. And can you tell us what kind of options have you used to generate the VectorTile.

Thank you !
Regards

Pierre

Thanks for the great work! I'm trying to encode and serve some geo data in vector tile format and this library has been super useful.

I'm wondering why VectorTileEncoder splits multi-polygons into multiple polygons and encodes them separately, though. According to the Vector Tile 2.0 spec, it seems like multi-polygons should just be encoded as VectorTile's POLYGON type with no splitting. The Encoder class also deals with multi-points and multi-linestrings appropriately, so I'm not sure why multi-polygons are the special case.

I have an issue displaying the points (JTS Vividsolutions, EPSG 4326) to tiles. The placement for all points on the map is wrong.

`
private final GeometryFactory gf = new GeometryFactory(new PrecisionModel(), 4326);
...

VectorTileEncoder encoder = new VectorTileEncoder(256);

    points.forEach(point -> {
        final Point location = point.getLocation();
        Map<String, Object> userDataMap = new HashMap<>();
        userDataMap.put("icon", "red");

        double[] coords = latLonToPixels(location.getY(), location.getX());
        Point reprojectedPoint = gf.createPoint(new Coordinate(coords[0], coords[1]));
        log.info("Reprojected point: Lat={} Lon={} to x ={}, y ={}", location.getY(), location.getX(),
                reprojectedPoint.getX(), reprojectedPoint.getY());
        encoder.addFeature("road", userDataMap, reprojectedPoint);
    });

// Finally, get the byte array
byte[] encoded = encoder.encode();
return encoded;

private double[] latLonToPixels(double lat, double lon) {
    double sinLat = Math.sin(lat * Math.PI / 180.0);
    double pixelX = ((lon + 180) / 360) * 256;
    double pixelY = (0.5 - Math.log((1 + sinLat) / (1 - sinLat)) / (Math.PI * 4)) * 256;

    double[] result = new double[2];
    result[0] = pixelX;
    result[1] = pixelY;
    return result;
}

`

Nano is not supported, the Google suggestion is to switch to the "lite" version:
protocolbuffers/protobuf#2730

It however seems lite will be released again only in Q3:
protocolbuffers/protobuf#4769

Features in the MVT spec may lack tags (if no properties need to be encoded): https://github.com/mapbox/vector-tile-spec/tree/master/2.1#42-features. This is uncommon, but possible. In this case java-vector-tile fails like:

	at java.base/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:64)
	at java.base/jdk.internal.util.Preconditions.outOfBoundsCheckIndex(Preconditions.java:70)
	at java.base/jdk.internal.util.Preconditions.checkIndex(Preconditions.java:266)
	at java.base/java.util.Objects.checkIndex(Objects.java:361)
	at java.base/java.util.ArrayList.get(ArrayList.java:427)
	at no.ecc.vectortile.VectorTileDecoder$FeatureIterator.parseFeature(VectorTileDecoder.java:233)
	at no.ecc.vectortile.VectorTileDecoder$FeatureIterator.findNext(VectorTileDecoder.java:186)
	at no.ecc.vectortile.VectorTileDecoder$FeatureIterator.hasNext(VectorTileDecoder.java:149)
	at no.ecc.vectortile.VectorTileDecoder$FeatureIterable.asList(VectorTileDecoder.java:106)

Here is a testcase:

layers {
  name: "layer"
  features {
    id: 1
    tags: 0
    tags: 0
    type: POINT
    geometry: 9
    geometry: 50
    geometry: 34
  }
  version: 2
}

Convert that text into a protobuf encoded vector tile by saving the above text as a file called point.txt and then doing:

git clone https://github.com/mapbox/vector-tile-spec
protoc vector-tile-spec/2.1/vector_tile.proto --encode vector_tile.Tile < point.txt > point.mvt

Mapbox GL js uses 512px x 512px tiles

https://www.mapbox.com/blog/512px-map-tile/

I noticed that java-vector-tile is creating 256px x 256px tiles .

Will this cause a problem with mapbox GL and is it possible to render 512px tiles? Does the tile pixel size even matter?

some map data have different show in different zoom level . for example I draw some walls, and the adjacent walls will fill the grey face between them;

how to relsove this problem ? i need help

VectorTileEncoder
List commands(Coordinate[] cs, boolean closePathAtEnd, boolean multiPoint)

   int _x = (int) Math.round(c.x * scale);
   int _y = (int) Math.round(c.y * scale);

“ Math.round ” Will undermine the geometry of the topology。
In most cases, this results in geometric destruction of the topology
Special circumstances, the line is greater than two, the ring is greater than three points, simplifying the data can not be decoded。

POLYGON((276 -9.971059, 275.694486 -11.216161,
275.790837 -11.470391, 275.858638 -11.682043,
274.721101 -12.110007, 274.274849 -12.936855,
274.428308 -11.687796, 274.201221 -10.983723,
274.746797 -9.867818, 274.394365 -9.660545,
274.463186 -9.491113, 274.960824 -9.819437,
274.565041 -10.965759, 274.645024 -11.502877,
275.117082 -11.614186, 275.52707 -11.349726,
275.928936 -9.91006, 276 -9.790078, 276 -9.971059))

Hi:

This is a wonderful tool for vector tile processing. However I found that the release jar have not been uploaded to maven center repository after 1.3.1, instead we have to use it from ecc mvn repo.

However we can not access ecc mvn repo directly, so the build job of the application will fail unless we create our own proxy repo.

So ss it possible to deploy this jar to maven center repository as before?

I am not sure if this is a bug, a feature or quite possibly a misunderstanding on my part.

When dealing with point data it appears that the extent is ignored and point geometries are clipped at the tile boundary here:
https://github.com/ElectronicChartCentre/java-vector-tile/blob/master/src/main/java/no/ecc/vectortile/VectorTileEncoder.java#L182

This seems undesirable in many circumstances since it is often the case that a point will end up on a rendering engine such as Mapnik, where the points are styled to be larger than a pixel. Unless a buffer can be provided, the renderer cannot handle tile boundaries correctly when e.g. rendering a 10pixel circle per point feature. The only option is then to pass 9 separate tiles to the renderer instead of 1 with a buffer.

Could you please confirm if my understanding is correct? It is nicely documented that this can be overridden, although I'm not sure this desirable default behaviour.

Attached is an example of the effect when a buffer can't be passed.

In certain cases clipping of a LineString will result in a MultiLineString containing a line which (after rounding) only contains multiple instances of the same point. This results in a single MOVE_TO command followed by 0 LINE_TO commands, which is not valid according to the vector tile specifications. A proposed fix can be found in #44.

Example of LineString causing the issue:
LINESTRING (3641 722,3642 720,3642 718,3642 716,3642 715,3642 713,3643 710,3643 706,3643 705,3643 704,3643 702,3644 700,3644 698,3644 696,3644 695,3644 694,3645 692,3645 691,3645 689,3645 687,3645 685,3645 684,3645 682,3645 680,3644 679,3644 678,3644 676,3644 675,3644 674,3643 673,3642 671,3642 668,3641 666,3640 664,3640 662,3639 660,3639 659,3638 658,3639 658,3641 657,3645 655,3646 655,3647 654,3648 653,3649 653,3649 652,3650 652,3650 650,3651 649,3651 647,3652 646,3653 644,3653 642,3654 641,3654 640,3655 640,3655 638,3656 638,3657 636,3658 635,3659 633,3661 630,3662 629,3663 628,3664 626,3666 624,3666 623,3669 620,3672 616,3673 614,3674 613,3675 612,3676 611,3678 609,3681 607,3682 606,3684 604,3685 603,3689 601,3694 596,3698 594,3700 592,3702 591,3704 588,3706 586,3707 584,3708 583,3708 582,3708 581,3708 579,3708 575,3707 569,3707 566,3707 564,3707 563,3708 562,3708 561,3709 559,3710 557,3712 554,3716 550,3720 546,3723 541,3725 538,3730 533,3734 528,3739 522,3743 517,3745 514,3748 511,3749 510,3750 508,3752 506,3755 503,3756 502,3759 499,3762 496,3766 491,3768 489,3769 488,3769 487,3770 487,3770 486,3771 484,3772 484,3773 481,3775 476,3776 473,3780 466,3782 460,3784 455,3786 450,3788 447,3789 444,3790 442,3791 439,3794 435,3797 432,3800 428,3803 426,3805 423,3806 420,3808 419,3810 415,3812 412,3814 408,3814 406,3816 402,3817 398,3818 396,3819 393,3819 390,3819 389,3819 388,3819 385,3819 383,3819 380,3820 379,3821 376,3821 374,3821 373,3821 372,3821 371,3824 371,3825 370,3826 370,3827 370,3828 368,3829 366,3829 364,3829 361,3829 360,3829 357,3829 355,3830 354,3830 353,3831 352,3831 351,3831 350,3832 350,3835 350,3837 349,3840 349,3842 349,3843 349,3845 349,3847 349,3851 349,3853 349,3854 349,3855 349,3858 350,3860 350,3864 350,3865 350,3868 350,3870 349,3871 349,3872 348,3875 347,3877 344,3879 343,3881 342,3883 342,3884 341,3885 341,3887 342,3889 343,3891 344,3892 345,3894 345,3898 344,3899 343,3900 343,3900 342,3900 341,3901 340,3901 339,3901 338,3902 337,3902 336,3903 336,3904 334,3906 333,3906 332,3908 331,3910 329,3912 327,3914 325,3915 324,3916 324,3920 322,3921 322,3924 320,3926 319,3927 318,3929 317,3931 315,3933 314,3935 312,3939 310,3940 309,3941 308,3942 308,3943 307,3946 307,3949 307,3951 306,3952 306,3955 306,3958 306,3961 306,3964 307,3965 307,3967 307,3970 307,3971 307,3973 307,3975 306,3976 306,3978 306,3982 304,3985 304,3987 303,3989 303,3991 302,3992 302,3996 301,4001 300,4002 300,4006 299,4008 299,4013 298,4016 298,4020 298,4024 297,4026 297,4031 297,4034 297,4036 296,4037 296,4039 296,4040 295,4043 295,4044 294,4045 294,4046 293,4047 293,4051 292,4056 289,4061 287,4066 284,4072 281,4075 279,4077 278,4079 277,4081 275,4083 273,4084 273,4083 273,4081 275,4079 277,4077 278,4075 279,4072 281,4066 284,4061 287,4056 289,4051 292,4047 293,4046 293,4060 287,4077 278,4083 273,4086 271,4087 270,4089 268,4091 267,4093 265,4095 264,4099 262,4102 260,4104 259,4108 257,4113 256,4118 254,4120 253,4122 252,4127 250,4130 249,4133 249,4137 248,4140 247,4144 246,4146 245,4148 245,4150 244,4154 242,4156 240,4159 239,4161 237,4163 237,4165 235,4166 235,4167 234)

after clipping:

(4046 293, 4047 293),
(4047 293, 4051 292),
(4051 292, 4056 289),
(4056 289, 4061 287),
(4061 287, 4066 284),
(4066 284, 4072 281),
(4072 281, 4074.5714285714284 279.2857142857143),
(4074.5714285714284 279.2857142857143, 4075 279),
(4075 279, 4077 278),
(4077 278, 4079 277),
(4079 277, 4081 275),
(4081 275, 4083 273),
(4083 273, 4084 273),
(4046 293, 4060 287, 4074.5714285714284 279.2857142857143),
(4074.5714285714284 279.2857142857143, 4077 278),
(4077 278, 4083 273),
(4083 273, 4086 271, 4087 270, 4089 268, 4091 267, 4093 265, 4095 264, 4096 263.5))```

The 9th line segment `(4074.5714285714284 279.2857142857143, 4075 279)` contains the same point twice (after rounding) and no further points, resulting in a single `MOVE_TO` command.

VectorTileEncoder allows for custom clipping, but then performs expensive validation operations, in terms of decoding the generated commands as well as calling Geometry.isValid, which is a pretty expensive operation in general.

Could you allow subclasses to opt-out of both these operations (the decoding and the validation)?

The specification says outer rings should be clockwise, and hole counter clockwise:
https://docs.mapbox.com/vector-tiles/specification/#winding-order

However the encoder code seems to be systematically generating the opposite order:

• for exterior rings, checks if not counter clock wise (aka clockwise), and if so, reverses it (making it CCW)
• for internal rings, check if they are CCW, and if so, reversed them (making them clockwise)

Am I missing anything here?

When decoding large tiles, the ability to pass in a filter is great. However, Filter is package private so cannot be customised by implementations and decoder only allows a single layer to be provided.

A useful addition would be the ability to filter by multiple layers.

I am trying to create a test implementation to serve vector tiles but am confused by the following:

• Add a feature with layer name (typically feature type name), some
  • attributes and a Geometry. The Geometry must be in "pixel" space 0,0
  • lower left and 256,256 upper right.
    Is the lower left origin part of the Vector Tile standard? Typically Canvas and screen coordinate systems have an upper left origin? Any recommendations for tools to go from Spherical Mercator coordinates (plus image size) to this lower left format? Also, why 256x256? Could these dimensions be made configurable?
    Thanks

Hi

We have been using the library to generate tiles since last year. Since a few weeks we have been trying to fix a bug in the generated tiles.
In the end, we found out, that the supplied geometries where not valid in some cases.
This resulted in situations, where the polygons where not clipped anymore and therefore the generated vector tile was not fully valid anymore.

The corresponding code is here: https://github.com/ElectronicChartCentre/java-vector-tile/blob/master/src/main/java/no/ecc/vectortile/VectorTileEncoder.java#L253:L274

As the exceptions are just silently ignored and no message is printed, it is quite hard to see this problem.

Do you have any opinions in adding logging or maybe rethink the exception handing?

Hi, I followed the constructions and add the dependences in pom. But when I turn my project to jar and submit it to spark it showed some errors like this:

User class threw exception: org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 1.0 failed 4 times, most recent failure: Lost task 0.3 in stage 1.0 (TID 4, 9.117.197.230): java.lang.VerifyError: Bad type on operand stack
Exception Details:
Location:
com/google/protobuf/GeneratedMessageV3$ExtendableMessage.getExtension(Lcom/google/protobuf/GeneratedMessage$GeneratedExtension;)Ljava/lang/Object; @2: invokevirtual
Reason:
Type 'com/google/protobuf/GeneratedMessage$GeneratedExtension' (current frame, stack[1]) is not assignable to 'com/google/protobuf/ExtensionLite'
Current Frame:
bci: @2
flags: { }
locals: { 'com/google/protobuf/GeneratedMessageV3$ExtendableMessage', 'com/google/protobuf/GeneratedMessage$GeneratedExtension' }
stack: { 'com/google/protobuf/GeneratedMessageV3$ExtendableMessage', 'com/google/protobuf/GeneratedMessage$GeneratedExtension' }
Bytecode:
0x0000000: 2a2b b600 23b0
at no.ecc.vectortile.VectorTileEncoder.encode(VectorTileEncoder.java:309)

So I wonder why it happens. Is it because the version of jdk? Mine is jdk1.8.
Looking forward to your reply.
Harriet

Thanks for the great library!

According to the Vector tile specification (https://docs.mapbox.com/vector-tiles/specification/) original GeoJSON data is encoded into the proper tags of the vector tile protobuf.

Do we have methods to translate Vector Tile file into GeoJSON files ? It would great to be able to do that in the future

This "issue" is intended to serve as a discussion around performance improvements that may be possible for the encoder. I am about to embark on profiling and exploring speed improvements and wish to see if others have either tried, or have ideas that could be made.

I am dealing with some pretty dense point tiles which are up to around 5-10MB in size. I am doing server side filtering on the tiles, which involves decoding a stored tile and rewriting it on the fly - sometimes aggregating the content of 2 or more tiles. I have found in early tests that rewriting a tile with approx 70% of pixels populated takes around 1.5 seconds on a fast laptop, and is proving insufficient for enjoyable slippy map use - this is in addition to the 1.5 seconds that mapnik requires to render the tile into a PNG.

The time is taken in calling ´´´encoder.addFeature()´´´ 100,000s of times and I will start by exploring if this can be improved, specifically for point tiles.

If anyone has comments, I'd greatly appreciate ideas!

Just to help illustrate, here is an example output:

First, thanks for the great work on this library . . . it has saved me a great deal of time!

I've noticed that, in a few places in the documentation, it's mentioned that the encoder pixel space starts at (0,0) in the lower-left and goes to (256,256) in the upper-right, e.g. from the README on the front page: // The Geometry uses (0,0) in lower left and (256,256) in upper right.. The Mapbox Vector Tile specification version 2.1 states: The upper left corner of the tile (as displayed by default) is the origin of the coordinate system. The X axis is positive to the right, and the Y axis is positive downward.

It appears that the implementation of the encoder class follows the spec correctly; however, the documentation has the Y-axis flipped. It's possible I'm misunderstanding something, since I'm new at using the MVT format, but when I rendered the tile output in Leaflet, it was indeed flipped vertically when I followed the documentation to convert my features to a 256x256 grid with (0,0) in the lower-left and (256,256) in the upper-right. Flipping the the tile so that (0,0) was in the upper-left and (256,256) was in the lower-right caused the tile to render correctly.

I'd suggest updating the documentation so that new users won't be confused as to how to map features to the tile grid space.

The constructor VectorTileEncoder(int, int, boolean) refers to a method GeometryFactory.createPolygon(Coordinate[]) which doesn't appear to exist in the current JTS API.

This results in a NoSuchMethodError when attempting to run with Maven dependencies on a current GeoTools release.

Can you please take a look or clarify the correct set of dependencies?

When reading polygons from MVT files, the rings are closed by pushing the first Coordinate instance to the end of the coordinates list. Since JTS Coordinate are mutable this may result in problems when using for instance JTS CoordinateFilter because they modify the coordinates in place, resulting in the same object instance being modified twice.

A new Coordinate instance should be used for closing the rings.

Thank you

How to specify the zoom level and how many tiles to generate?

I'm trying to write Java code to display the vector tile data, using OpenMap combined with the java-vector-tile decoder. When I pull the tile_data byte array out of the database, the java-vector-tile decoder doesn't read it correctly. I've compared the bytes with other tile data bytes I've received from servers, and it seems off - it looks really different, like it's encoded, compressed or something. I've tried running the bytes through Base64/zip decoding, but that doesn't help. Is there a trick to getting the java-vector-tile decoder to work on that data?