Data not always returned within bounding box about mapserver HOT 8 CLOSED

Thanks @geographika – it wasn't those, but after some more investigation, it looks like it is #6478 which I hadn't previously found – this was fixed in MapServer 7.6.5/8. Sadly Debian bullseye is 7.6.2 and Ubuntu LTS jammy is 7.6.4. I've done a full install of MapServer 8 from bullseye-backports (it looks like my previous attempt at this was not complete) and the issue looks to be resolved. Thanks for your assistance.

from mapserver.

Just as a reference because there may be no WFS server that behaves like they should by the WFS 1.1.0 standard:

14.3.3 Bounding box
The bounding box parameter, BBOX, is included in this specification for convenience as
a shorthand representation of the very common a bounding box filter which would be
expressed in much longer form using XML and the filter encoding described in [3]. A
BBOX applies to all feature types listed in the request.
The KVP encoding for a bounding box is defined in subclause 10.2.3 of normative
reference [15]. The general form of the parameter is:
BBOX=lcc1,lcc2,…,lccN,ucc1,ucc2,…uccN[,crsuri]
where lcc means Lower Corner Coordinate, ucc means Upper Corner Coordinate and
crsuri means the URI reference to the coordinate system being used. This encoding
allows N coordinates for each corner listed in the order of the optional crsuri. If the
crsuri is not specified then the 2-D coordinates shall be specified using decimal degrees
and WGS84 as described in [15].

from mapserver.

Thanks for your reply, but not sure I understand the relevance, sorry. MapServer does support bbox, it works fine in a previous version, and I get the same (different) results if I use it in a filter [URLs currently work as we have rolled back to buster]:

• 3857, 14 results: https://tilma.mysociety.org/mapserver/brent?SERVICE=WFS&VERSION=1.1.0&REQUEST=GetFeature&SRSNAME=urn%3Aogc%3Adef%3Acrs%3AEPSG%3A%3A3857&TYPENAME=Parks_and_Open_Spaces&filter=%3CFilter%3E%3CBBOX%3E%3Cgml:Box%20srsName=%22EPSG:3857%22%3E%3Cgml:coordinates%3E-24190.549273477,6716902.930357%20-23854.942946209,6717123.8811418%3C/gml:coordinates%3E%3C/gml:Box%3E%3C/BBOX%3E%3C/Filter%3E
• 27700, 29 results: https://tilma.mysociety.org/mapserver/brent?SERVICE=WFS&VERSION=1.1.0&REQUEST=GetFeature&SRSNAME=urn%3Aogc%3Adef%3Acrs%3AEPSG%3A%3A27700&TYPENAME=Parks_and_Open_Spaces&filter=%3CFilter%3E%3CBBOX%3E%3Cgml:Box%20srsName=%22EPSG:27700%22%3E%3Cgml:coordinates%3E523734.4667234342,183556.63528328092%20523940.1889593812,183699.01999836136%3C/gml:coordinates%3E%3C/gml:Box%3E%3C/BBOX%3E%3C/Filter%3E

from mapserver.

I know that my comment lacks relevance, in a way. But by the WFS 1.1.0 standard GetFeature with &bbox=-24190.549273477,6716902.930357,-23854.942946209,6717123.8811418 should not find anything because if bbox is expressed with 4 parameters then the coordinates should be in WGS84 units and lat-lon axis order and obviously those are not. The standard compliant way is to include SRID as a fifth parameter &bbox=-24190.549273477,6716902.930357,-23854.942946209,6717123.8811418,urn:ogc:def:crs:EPSG::3857. However, Mapserver does not interpret the 4-parameter bbox to mean WGS84 coordinates but either the units of the default coordinate system, or if &SRSNAME= is used then those units. Your usage of bbox is correct for Mapserver and obviously there is a bug somewhere.

EPSG:27700 makes often troubles. For getting more info, could you test also with for example UTM Zone 30N (EPSG:32630)?

from mapserver.

In EPSG:32630 with equivalent co-ordinates (692979.2296026859,5713254.965024991)-(693183.0386342288,5713400.188730145) it returns only 11 results, even fewer than EPSG:3857.

from mapserver.

@dracos - this (and #6958) sound like a projection/PROJ issue.

What version of PROJ is used in the deployment?
Is the PROJ_LIB pointing to the correct location?
Can you add the PROJ string directly to the Mapfile to see if there are any differences in results?

You could also try a WFS 2.0 request to see if you get the same results (if they are not the same, then it could be a MapServer issue).

Are you able to test the results with a MapServer 8.x deployment?

from mapserver.

Hi @geographika, thanks for your reply. The PROJ on the working server is debian buster 5.2.0, the non-working one is bullseye 7.2.1. Haven't had any other issues with projections, plus the fact it returns /some/ apparently correct information, so I think PROJ_LIB is pointing to the right place (/usr/share/proj), it doesn't change anything if I manually set it. We're not changing anything from the defaults really.

The source data is in 27700, the output is in 3857, the full proj conversion between the two (on the more up-to-date server) is, according to projinfo:

+proj=pipeline
  +step +inv +proj=tmerc +lat_0=49 +lon_0=-2 +k=0.9996012717 +x_0=400000
        +y_0=-100000 +ellps=airy
  +step +proj=hgridshift +grids=uk_os_OSTN15_NTv2_OSGBtoETRS.tif
  +step +proj=webmerc +lat_0=0 +lon_0=0 +x_0=0 +y_0=0 +ellps=WGS84

Which is what I would expect.

Can you add the PROJ string directly to the Mapfile to see if there are any differences in results?

I get some interesting results when, asking for epsg:3857, on the bullseye server, depending on the layer's PROJECTION:

• "init=epsg:27700" - 14 results
• "AUTO" - 16 results
• "+proj=tmerc +lat_0=49 +lon_0=-2 +k=0.9996012717 +x_0=400000 +y_0=-100000 +ellps=airy" - 16 results
• "+proj=tmerc +lat_0=49 +lon_0=-2 +k=0.9996012717 +x_0=400000 +y_0=-100000 +ellps=airy +towgs84=446.448,-125.157,542.06,0.15,0.247,0.842,-20.489 +units=m +no_defs" - 29 results!
• "+proj=tmerc +lat_0=49 +lon_0=-2 +k=0.9996012717 +x_0=400000 +y_0=-100000 +ellps=airy +nadgrids=uk_os_OSTN15_NTv2_OSGBtoETRS.tif +units=m +no_defs" - also 29 results! And these appear to differ by a couple of metres, as I'd perhaps expect of a more accurate projection.
• "+proj=pipeline +step +inv +proj=hgridshift +grids=uk_os_OSTN15_NTv2_OSGBtoETRS.tif +step +proj=tmerc +lat_0=49 +lon_0=-2 +k=0.9996012717 +x_0=400000 +y_0=-100000 +ellps=airy" - 0 results (assume I'm not specifying this correctly somehow? Or have got confused, or the 'modern' way doesn't work? Can't get grids to work at all)

Trying the same 29 entries with the map tiles #6958 does indeed 'fix' the missing stripes, there are slight discontinuities in the tiles at the point the stripes being, which I assume is correct and down to the projection mapping.

So yes, I agree it sounds like a projection issue, somehow, but I don't really see how it's happening. 27700 appears to be defined okay by proj, at worst it'd be the same as the "16 results" entry, so why does it only return 14? And proj seems to return results fine whatever I ask it directly. I think the datum shift in buster comes from GDAL, not proj, so perhaps that's removed in bullseye but something else in MapServer doesn't know it needs to pick up the conversion instead? It feels like there's still an issue here somehow.

Have also tried WFS 2 or mapserver 8 (though maybe not libmapserver2 8), same outputs. I would have thought others would have had this problem besides us!

from mapserver.

@dracos - there a few issues over at the PROJ lists about epsg:27700. OSGeo/PROJ#3079 (comment) might be a relevant comment? Maybe also OSGeo/gdal#3695.
Although it sounds like you've found the PROJ string that gives you the results you expect?

If you switch to a newer version of PROJ then the PROJ_LIB should also point to the location of the proj.db sqlite database rather than an epsg file for projections.

from mapserver.