Breaking change between 1.5.3 and 1.6.0 affecting RASTER functions java.lang.NoSuchMethodError: void org.geotools.coverage.grid.GridGeometry2D about sedona HOT 5 CLOSED

The problem cannot be reproduced by simply launching pyspark locally using pyspark --packages org.apache.sedona:sedona-spark-shaded-3.4_2.12:1.6.0,org.datasyslab:geotools-wrapper:1.6.0-28.2. The java.lang.NoSuchMethodError exception raised by spark executor is really weird, since this constructor method do exist in org.datasyslab:geotools-wrapper:1.6.0-28.2. BTW, Sedona 1.6.0 is still using GeoTools 28.2, we reverted the GeoTools version upgrade before release.

After trying lots of possible misconfigurations, I finally reproduced this problem locally by putting both geotools-wrapper-1.6.0-28.2.jar and geotools-wrapper-1.6.0-31.0.jar to the classpath. Please make sure that only geotools-wrapper-1.6.0-28.2 is actually being used, and remove geotools-wrapper-1.6.0-31.0.jar from the cluster to avoid any misbehavior.

from sedona.

@golfalot In the first release candidate of Sedona 1.6.0, we wanted to upgrade Sedona to use geotools 31.0 and that's why geotool-wrapper 1.6.0-31.0 exists. However, given the current user base of Sedona, we decided to revert this breaking change in the final release candidate of Sedona 1.6.0 and fall back to geotools-wrapper 1.6.0-28.2.

The exact geotools-wrapper version to use is always mentioned here: https://sedona.apache.org/1.6.0/setup/maven-coordinates/

Unfortunately, Maven Central does not allow anyone to retract a published package so we cannot delete geotools-wrapper 1.6.0-31.0

from sedona.

I don't currently have the skills to debug this beyond this issue report. I hope the info provided is sufficiently comprehensive, do not hesitate to request further info from me.

An observation that is possibly material: geotools-wrapper release 1.6.0-28.2 Exclude GeographicLib here https://github.com/jiayuasu/geotools-wrapper/releases/tag/1.6.0-28.2

from sedona.

I also noted another change that could be material

https://docs.geotools.org/stable/userguide/welcome/upgrade.html

GeoTools 30.x
Replace constructor Envelope2D(crs,x,y,width,height) with ReferencedEnvelope.rect(x,y,width,height,crs)

from sedona.

Yes! thank you @Kontinuation , exactly right. geotools-wrapper-1.6.0-31.0.jar was present on the Synapse Spark Pool, removing it , and leaving only geotools-wrapper-1.6.0-28.2.jar fixes the issue.

• It was probably naive of me to leave it there and assume simply referencing the desired version in my config statement would suffice to disambiguate. I shall remember this important lesson!

• Why was 1.6.0-31.0 there in the first place ?
  This is my first time getting up and running with Sedona, once I figured out what jars I needed, I trawled Maven to find the latest versions of the two jars for my target versions of Spark and Scala. https://mvnrepository.com/artifact/org.datasyslab/geotools-wrapper/1.6.0-31.0

• How did I get the idea to downgrade to 1.6.0-28.2 ?
  When I headed over to https://github.com/jiayuasu/geotools-wrapper I saw that latest release was in fact 1.6.0-28.2 and NOT 1.6.0-31.0

• I wonder if it would be helpful for other users if version 1.6.0-31.0 were completely deleted from Maven ? As this version did not work for me.

Thank you once again for the great guidance, I appreciate the time you spent on this.

from sedona.