mapproxy / mapproxy Goto Github PK

ResourceURL template in in WMTSCapabilities does not conform to WMTS standard. MapProxy uses double braces '{{' and '}}' to mark variable names whereas the standard uses single braces '{' and '}'. As far as I can see the standard does not explicitly forbid double braces, but it means that for instance QGIS is not able to consume WMTS from MapProxy.

MapProxy example:

Example from the spec:

We are using MapProxy 1.4.0

It would be handy to display the number of tiles for each level/resolution in the mapproxy-util grids output.

When doing a featureinfo request the source service returns a charset encoding in the http-headers like this:
Content-Type:application/vnd.ogc.gml;charset=UTF-8
mapproxy throws an NotImplementedError() if a xsl transformation on the result should be done.

Suggested fix in featureinfo.py:

change:
def create_featureinfo_doc(content, info_format):
if info_format in ('text/xml', 'application/vnd.ogc.gml'):
return XMLFeatureInfoDoc(content)
if info_format == 'text/html':
return HTMLFeatureInfoDoc(content)
to:
def create_featureinfo_doc(content, info_format):
if info_format.split(';')[0] in ('text/xml', 'application/vnd.ogc.gml'):
return XMLFeatureInfoDoc(content)
if info_format.split(';')[0] == 'text/html':
return HTMLFeatureInfoDoc(content)

I'm trying to proxy OSM source as WMS service with MapProxy.
I've set up the source for OSM tiles url, and the grid with origin: nw, but I get tiles in incorrect y-axis order.
What else I have to add to the configuration file to set up tiles in proper order?

My configuration:

services:
demo:
kml:
tms:
wmts:
wms:
srs: ['EPSG:4326', 'EPSG:900913']
image_formats: ['image/jpeg', 'image/png']

layers:

• name: osm_orig
  title: OpenStreetMap Original
  sources: [osm_orig_cache]

caches:
osm_orig_cache:
grids: [global_mercator_osm]
sources: [osm_orig_src]

sources:
osm_orig_src:
type: tile
url: http://c.tile.openstreetmap.org/%(tms_path)s.png

grids:
global_mercator_osm:
base: GLOBAL_MERCATOR
num_levels: 20
origin: nw

globals:
cache:
base_dir: './cache_data'
lock_dir: './cache_data/locks'

image:
resampling_method: nearest

A bit of an edge case here (apologies for the pun).

We've noticed an issue with requests from MapProxy that hit the boundary between eastern and western hemispheres. MapProxy is stopping the requests at the boundary, but in cases where the request is in the eastern hemisphere the upper right corner of the BBOX is being set to -180, not 180 (the eastern hemisphere's side of the boundary). This is causing some unfortunate display issues with our backend (MapServer).

For example:

Reprojected to EPSG:900913 (note the low quality of the image)
http://srss-ows-1.landgate.wa.gov.au/wxs/firewatch.php?styles=&format=image%2Fpng&height=1184&bbox=14636773.6723,-5400734.67052,20037508.3428,391357.58482&transparent=True&layers=layer37_MODIS_Global_Mosaic.ecw&service=WMS&width=1104&request=GetMap&srs=EPSG%3A900913&version=1.1.1

The same request in native EPSG:4326 fails because the BBOX is invalid
http://srss-ows-1.landgate.wa.gov.au/wxs/firewatch.php?styles=&format=image%2Fpng&height=1184&bbox=131.484375,-43.580391,-180,3.513421&transparent=True&layers=layer37_MODIS_Global_Mosaic.ecw&service=WMS&width=1104&request=GetMap&srs=EPSG:4326&version=1.1.1

But if we replace the -180 with 180 we get the desired outcome:
http://srss-ows-1.landgate.wa.gov.au/wxs/firewatch.php?styles=&format=image%2Fpng&height=1184&bbox=131.484375,-43.580391,180,3.513421&transparent=True&layers=layer37_MODIS_Global_Mosaic.ecw&service=WMS&width=1104&request=GetMap&srs=EPSG:4326&version=1.1.1

This is possibly something MapServer should deal with, but we thought we'd ask for a fix from MapProxy first since it seems like such an easy fix from your POV.

We've worked around it for the time being by reducing our meta_size to 3,3 from 4,4 so only New Zealand is being pixelated, but a longer-term fix would be great.

If our Mapproxy-running server has more IP adresses it would be nice to be able use them all to fight traffic shapers on remote WMS.

http://stackoverflow.com/questions/1150332/source-interface-with-python-and-urllib2

Has anyone tried to implement utfgrid output?
Where to start?

I have added support within MapProxy for a new SQLite-based cached data format called geopackage. There is talk of initiating a pull request to the MapProxy repo in order to facilitate adoption and spread the word about MapProxy software.

Are you open to merging support for new cache data formats within MapProxy if I were to generate a pull request? This would hinge on approval from my stakeholders to give this code back, but so far everyone seems to be on-board to move forward.

Here are some links talking about geopackage:
http://www.opengeospatial.org/projects/groups/geopackageswg
http://www.weogeo.com/blog/The_GeoPackage.html

maproxy config source:

layers:
  - name: cadaster
    title: cadaster layer
    sources: [ wms_cadaster]
  - name: cadaster1
    title: cadaster layer
    sources: [wms_cadaster1]

sources:
  wms_cadaster:
    type: wms
    wms_opts:
      version: 1.1.1
      featureinfo: true
    req:
      url: http://localhost/cgi-bin/mapserv?map=map/wms_cadaster.map&
      layers: cadaster
      transparent: true
      format: image/png
      radius: 10
  wms_cadaster1:
    type: wms
    wms_opts:
      version: 1.1.1
      featureinfo: true
    req:
      url: http://localhost/cgi-bin/mapserv?map=map/wms_cadaster.map&
      layers: cadaster1
      transparent: true
      format: image/png
      radius: 10

Openlayers code:

cadaster = new OpenLayers.Layer.WMS("cadaster", "/mapproxy/service?", {layers: "cadaster", format: 'image/png'});

cadaster1 = new OpenLayers.Layer.WMS("cadaster1", "/mapproxy/service?", {layers: "cadaster1", format: 'image/png'});

clickCadasterGood = new OpenLayers.Control.WMSGetFeatureInfo({
    layers: [cadaster1],
    queryVisible: true,
    infoFormat: 'application/vnd.ogc.gml',
    maxFeatures: 1,
    autoActivate: true,
    vendorParams: {
        radius: 15
    },
    eventListeners: {
    }
}); 

clickCadasterBad = new OpenLayers.Control.WMSGetFeatureInfo({
    layers: [cadaster1,cadaster],
    queryVisible: true,
    infoFormat: 'application/vnd.ogc.gml',
    maxFeatures: 1,
    autoActivate: true,
    vendorParams: {
        radius: 15
    },
    eventListeners: {
    }
}); 

Hello,

I use MapProxy with Autocad Map to view a tile source..
At initialisation the client zoom to bbox of wms and request quadkey((0,0,0)) who return an empty string, and fail to initialyse the wms layer.
My whorkaround is to mydify the quadkey function (mapproxy/client/tile.py) to return "0"

def quadkey(tile_coord):
    """
    >>> quadkey((0, 0, 1))
    '0'
    >>> quadkey((1, 0, 1))
    '1'
    >>> quadkey((1, 2, 2))
    '21'
    """
    x, y, z = tile_coord
    quadKey = ""
    for i in range(z,0,-1):
        digit = 0
        mask = 1 << (i-1)
        if (x & mask) != 0:
            digit += 1
        if (y & mask) != 0:
            digit += 2
        quadKey += str(digit)
    # workaround for (0,0,0) tile
    if quadkey == "":
        quadkey = "0"
    return quadKey

It is a bug or Is there any way to set min level to 1 in the configuration file ?

The problem is mapnik-specific and is well described here:
https://groups.google.com/forum/?hl=ru&fromgroups=#!topic/mapnik/NVpQHfICdas

and here:
mapnik/mapnik#1550

This is MapProxy output:
http://db.tt/m2K8mBR3

So the problem is that Map element in mapnik config file is configured with srs="+init:epsg:3857"
which is then reprojected somewhere in MapProxy to EPSG:3857 (while it shouldn't). If raster reprojection occurs, then it triggers mapnik bug with corrupted raster edges.

Fixed that in mapnik OGCServer by changing the long proj string to just +init:epsg:3857, so it doesn't reproject. Hoped that helps with MapProxy too, but unfortunately it didn't.

I created two caches:

1. PNG Geodetic TMS cache created via gdal2tiles.py, source is []
2. PNG/JPEG mixed-mode mbtiles cache using Geodetic grid, source is the TMS cache above

Conf is here: https://gist.github.com/stevendlander/5782462

I created a seed for cache 2, specifying the Geodetic grid and a custom coverage for my data in EPSG:4326

Seed conf: https://gist.github.com/stevendlander/5782483

I then kick off a seed task for this job: mapproxy-seed -f conf.yaml -s seed.yaml

I show the following stack trace: https://gist.github.com/stevendlander/5782514

While trying to use the mbcheck program (from the node-mbtiles package), I've discovered that the metadata table is empty in the MBTiles databases produced by MapProxy.

There is indeed a method called update_metadata() in cache/mbtiles.py but it doesn't seem to be called.
Furthermore, it doesn't support the bounds property, which is recommended by the MBTiles spec.

Producing fully populated MBTiles databases would help when working with third-party tools.

All throughout the documentation the http.ssl_no_cert_check is referred to in its singular form, whereas the name of the property in reality is the plural http.ssl_no_cert_checks.

Affected pages:

http://mapproxy.org/docs/1.4.0/configuration.html
http://mapproxy.org/docs/1.4.0/sources.html
http://mapproxy.org/docs/1.4.0/configuration_examples.html

I made the following cache:

caches:
kabul:
sources: []
grids: [GLOBAL_GEODETIC]
format: image/png
cache:
type: file
directory_layout: tms
directory: '/data/result'

Tiles are valid and appear when zoomed into the extent of the cached data. However, when changing the grid to the following:

grids:
kabul_grid:
srs: "EPSG:4326"
bbox: [69.0007369,34.3603824,69.4629450,34.6855883]
bbox_srs: "EPSG:4326"

No tiles are rendered. So, in order to specify an extent of the data AND have it render correctly, I must keep the kabul cache at GLOBAL_GEODETIC and then make an additional cache with kabul as the source, applying the kabul_grid to it. Is this working as intended?

EDIT: This was tested in both GLOBAL_GEODETIC and in GLOBAL_MERCATOR. Both produce this result.

I have a cleanup task configured to delete all tiles from level 10 onwards for a cache that's located on another machine. globals.cache.base_dir is set to a UNC path and when I run mapproxy-seed I get this error http://twitpic.com/ab44s1/full.

All of the paths I've configured are UNC paths, so I'm not sure where it's getting confused with non-UNC paths.

MapProxy 1.4.0 on Windows Server 2008.

The image on the left is the original tile image. The image on the right is the same tile returned by MapProxy using MapProxy's WMS service. Both tiles are 256x256 pixels, but it can be seen that the last row of the tile on the right is transparent.

The configuration file required to reproduce the bug is https://gist.github.com/1851b26a9ed0438feb9a, it should work out-of-the box (the original tiles are stored in S3).

The WMS request is:

http://localhost:8080/service/?FORMAT=image%2Fpng&LAYERS=VelolandRoutenNational&TRANSPARENT=TRUE&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&STYLES=&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&SRS=EPSG%3A21781&BBOX=586400,196400,752800,362800&WIDTH=256&HEIGHT=256

The original tile is

http://schweizmobil-test.s3.amazonaws.com/tpayne-20121206/VelolandRoutenNational/14/1/1.png

Useful information:

• The computed tile coordinate is z=14, x=1, y=1
• The resolution of the requested tile is 650m/pixel, this corresponds exactly to one of the resolutions listed in the grid definition. Other resolutions do not seem to be affected.
• This error occurs for all tiles with the same Y coordinate, resulting in a horizontal line across all tiles in the map. Tiles with different Y coordinates are not affected.

Could this be a rounding error somewhere?

cc @fredj

grids:
 swiss_grid_new:
   srs: 'EPSG:21781'
   bbox: [420000, 30000, 900000, 360000]
   bbox_srs: 'EPSG:21781'
   res: [250, 100, 50, 20, 10, 5, 2.5, 2, 1.5, 1, 0.5 ,0.25, 0.125, 0.0625]
   origin: 'nw'

The WMTS GetCapabilities document that MapProxy generates includes
TileMatrix blocks that look like this:

<TileMatrix>
<ows:Identifier>00</ows:Identifier>
<ScaleDenominator>892857.142857</ScaleDenominator>
<TopLeftCorner>420000.0 40000.0</TopLeftCorner>
<TileWidth>256</TileWidth>
<TileHeight>256</TileHeight>
<MatrixWidth>8</MatrixWidth>
<MatrixHeight>6</MatrixHeight>
</TileMatrix>

Mapnik often doesn't load many fonts by default, so most mapnik-using applications have a config option for a directory to load fonts from.

For example, in mod_tile:
https://github.com/openstreetmap/mod_tile/blob/master/renderd.conf#L9

In nik2img.py:
http://code.google.com/p/mapnik-utils/source/browse/trunk/nik2img/nik2img.py#153

In Tilemill, a bunch of OS-specific places are checked automatically, which is an alternative approach
https://github.com/mapnik/node-mapnik/blob/master/lib/mapnik.js#L33

The code in nik2img that registers a directory of fonts recursively:

http://code.google.com/p/mapnik-utils/source/browse/trunk/nik2img/mapnik_utils/fonts.py

Something similar for mapproxy would be very useful.

If I try to transform POINT(71 50) from EPSG:4326 to EPSG:32644 in PostGIS or cs2cs utility I get the POINT(-216025.317866142 5586720.84493467). But when I set up grid in mapproxy config:

services:
demo:
tms:

sources:
my_mapnik_source:
  type: mapnik
  use_mapnik2: True
  mapfile: /home/rykovd/mapnik-config.xml

grids:
utfgrid:
  srs: EPSG:32644
  tile_size: [256, 256]
  origin: ll
  res_factor: sqrt2
  bbox: [71,50,87,54]
  bbox_srs: EPSG:4326

caches:
ex1_cache:
  cache:
      type: file
      directory_layout: tms
  sources: [my_mapnik_source]
  grids: [utfgrid]

layers:
- name: merged_layer
  sources: [ex1_cache]

and try to query TileMap resource of TMS using http://10.22.0.9:8080/tms/1.0.0/merged_layer/EPSG32644 URL I get response which contains the following bbox information:

<BoundingBox minx="-216025.317866" miny="5540547.36994" maxx="929865.502423" maxy="6029925.90671"/>

As you can see minx value is the same as in PostGIS, but miny is different - 5540547.36994 vs 5586720.84493467.

Tile access (TMS/KML/WMTS) is not possible when using the watermark feature. I guess the image_opts get lost when the watermarks are added.

services:
  demo:
  tms:

layers:
    - name: osm
      title: OpenStreetMap
      sources: [osm_cache]

caches:
  osm_cache:
    grids: [global_mercator_osm]
    sources: [osm_tms]
    disable_storage: true
    watermark:
      text: '@ OpenStreetMap contributors'

sources:
  osm_tms:
    type: tile
    grid: global_mercator_osm
    url: http://c.tile.openstreetmap.org/%(tms_path)s.png

grids:
  global_mercator_osm:
    base: GLOBAL_MERCATOR
    num_levels: 18
    origin: nw

Initialising a service with a large number of layers takes a long time. Follow on from this post on the mailing list (with data and timings included): http://osgeo-org.1560.n6.nabble.com/MapProxy-GetCapabilities-request-very-slow-td5039049.html

Gists of a config file with 200 layers that was taking 60s to init via 'mapproxy-utils serve-develop':

https://gist.github.com/keithamoss/d06b55e44ae98cbc083a
https://gist.github.com/keithamoss/48d3080a093f619fb94b

Another approach to Issue 80 would be to allow the seeder to cleanup a specific items from the cache, or to remove all items newer than a specific date. We have a situation where images are updated throughout the day, but may already have been requested and cached. We need a mechanism to directly remove an image (and its related pyramid of images) from the cache, or to just flush everything that's newer than 2 hours, for example.

Add either a custom or auto-generated index to the mbtiles caches created by MapProxy. This would make the mbtiles created by MapProxy usable to libraries that need an index to perform searches.

We've just noticed what seems to be a a minor issue with multiple service initialisations happening when a bunch of requests for tiles for an uninitialised (or changed) service come in simultaneously (log attached).

I'm thinking ideally only the first call should initialise the service and the others should wait around until it's initialised. Not sure how hard that would be, though. Open to other ideas if you have any though, Oliver. Thanks.

For reference, we're running MapProxy as MultiApp via WSGI in Apache.

2013-04-03 14:40:51,005 - INFO - mapproxy.multiapp - initializing project app fwew with e:\mapproxy\keith\projects\fwew.yaml
2013-04-03 14:40:51,005 - INFO - mapproxy.config - reading: e:\mapproxy\keith\projects\fwew.yaml
2013-04-03 14:40:53,427 - INFO - mapproxy.config - reading: e:\mapproxy\keith\projects\base-config\base-mapproxy.yaml
2013-04-03 14:40:54,895 - INFO - mapproxy.multiapp - initializing project app fwew with e:\mapproxy\keith\projects\fwew.yaml
2013-04-03 14:40:54,911 - INFO - mapproxy.config - reading: e:\mapproxy\keith\projects\fwew.yaml
2013-04-03 14:40:56,848 - INFO - mapproxy.config - reading: e:\mapproxy\keith\projects\base-config\base-mapproxy.yaml
2013-04-03 14:40:58,677 - INFO - mapproxy.multiapp - initializing project app fwew with e:\mapproxy\keith\projects\fwew.yaml
2013-04-03 14:40:58,677 - INFO - mapproxy.config - reading: e:\mapproxy\keith\projects\fwew.yaml
2013-04-03 14:41:00,615 - INFO - mapproxy.config - reading: e:\mapproxy\keith\projects\base-config\base-mapproxy.yaml
2013-04-03 14:41:02,115 - INFO - mapproxy.multiapp - initializing project app fwew with e:\mapproxy\keith\projects\fwew.yaml
2013-04-03 14:41:02,115 - INFO - mapproxy.config - reading: e:\mapproxy\keith\projects\fwew.yaml
2013-04-03 14:41:04,459 - INFO - mapproxy.config - reading: e:\mapproxy\keith\projects\base-config\base-mapproxy.yaml
2013-04-03 14:41:05,802 - INFO - mapproxy.multiapp - initializing project app fwew with e:\mapproxy\keith\projects\fwew.yaml
2013-04-03 14:41:05,802 - INFO - mapproxy.config - reading: e:\mapproxy\keith\projects\fwew.yaml
2013-04-03 14:41:07,522 - INFO - mapproxy.config - reading: e:\mapproxy\keith\projects\base-config\base-mapproxy.yaml

Hey there,
I have configured mapnik rendering like this:

layers:
  - name: overlay_mapnik
    title: overlay_mapnik
    sources: [overlay_mapnik]
caches:
  overlay_mapnik:
    sources: [overlay_mapnik]
sources:
  overlay_mapnik:
    type: mapnik
    mapfile: osm_overlay.xml
    use_mapnik2: true
    transparent: true

If I access the MapProxy WMS server through the demo-page, I can see the tiles being rendered correctly and the server returns HTTP 200 with the correct images.
However, when I try to access an (uncached) area via TMS (or by the tiles/ URL for OSM-like zoom levels), the tiles are not being rendered. Previousely cached tiles that were created by the WMS access will be returned, but no rendering will be done.
The request hangs for a while and apache will finally return HTTP 500.

Version 1.5.0, SuSE 12.2, running on mod_wsgi.

EDIT: This only seems to happen when running through mod_wsgi.
Accessing via localhost:8080 seems to work correctly and triggers the rendering.

regards,
Adrian

I have a PNG TMS cache in geodetic from zoom levels 0-9 and I want to convert it to mbtiles as PNG/JPEG mixed mode.

I setup a cache for the TMS folder and then setup a cache for the mbtiles using the TMS cache as the source. When I try to view this mbtiles cache using mapproxy-util serve-develop, I receive the error listed in the title.

Tested thoroughly and this only occurs when the grids match. Using GLOBAL_GEODETIC for each cache will produce this error, however giving it a custom grid (global_geodetic_sqrt2 for instance) will generate the cache as expected.

I will provide Gists if requested, thanks!

Often imagery source is very slow and cannot handle many requests at same time. So when someone wants to warm up it's own cache, he cannot control download speed and imagery source can crash, especially this happens with ESRI products. That's why mapproxy-seed needs some throttling mechanism. It could be based on request rate per second or downloaded data size per second, or maybe time pauses between requests could help.
Throttling would be very useful option for mapproxy-seed.

[imported from https://bitbucket.org/olt/mapproxy/issue/35/transparency-and-opacity-fix]
This proposal for enhancement is relative to mapproxy 1.2.0.

Mapproxy does not work well when adding opacity to layers which also have transparent area's. The transparent area's become either semi-transparent black or semi-transparent-white.
The problem is described in: http://lists.osgeo.org/pipermail/mapproxy/2012-January/000913.html
and http://lists.osgeo.org/pipermail/mapproxy/2012-January/000914.html.

I've created a patch which solves the problem:

#!python

mapproxy/image/__init__.py:
    layer_img = layer.as_image()
    if (layer.image_opts and layer.image_opts.opacity is not None
        and layer.image_opts.opacity < 1.0):
-        layer_img = layer_img.convert(img.mode)
-        img = Image.blend(img, layer_img, layer.image_opts.opacity)
+        if layer_img.mode == 'RGBA':
+            alpha = layer_img.split()[3]
+            alpha = ImageEnhance.Brightness(alpha).enhance(layer.image_opts.opacity)
+            layer_img.putalpha(alpha)
+            img.paste(layer_img, (0, 0), layer_img)
+        else:
+            layer_img = layer_img.convert(img.mode)
+            img = Image.blend(img, layer_img, layer.image_opts.opacity)
    else:
        if layer_img.mode == 'RGBA':
            # paste w transparency mask from layer

mapproxy/platform/image.py:
Everywhere I've substituted:
Image, ImageColor, ImageDraw, ImageFont, ImagePalette, ImageChops
by:
Image, ImageColor, ImageDraw, ImageFont, ImagePalette, ImageChops, ImageEnhance 

I propose to add this to mapproxy.

Hi,

I've been looking at implementing an AWS S3 Cache backend based on the FileCache, I now have a working prototype which uses boto and I'd appreciate any thoughts and feedback. The code can be found here in the s3 branch: https://github.com/walkermatt/mapproxy/tree/s3.

If you'd like to set up an S3 cache then you'll need the boto package and a cache of type 's3' and a valid bucket name. For instance to configure the demo osm_cache to use an S3 cache:

caches:
  osm_cache:
    grids: [GLOBAL_MERCATOR, global_geodetic_sqrt2]
    sources: [osm_wms]
    cache:
      type: s3
      bucket: uniquebucketid

The bucket ID must be unique across all of S3 as described here so choose something unique. At present new buckets are created in the EU zone, I'll no doubt make that configurable soon.

Even though you can store any arbitrary data in S3 against a key I've chosen to effectively store files with the keys being the effective file system path. This may allow an existing cache to be uploaded to S3 using something like s3fs or even offline import then used / maintained by the S3 cache.

BTW The original motivation for looking at an S3 cache is to set up a redundant shared cache used by several MapProxy instances all running the same configuration behind a load balancer. At present I'm using file based cache stored on EBS volumes which works well but results in duplication of the cache data. I've experimented with using s3fs with a file based cache but I've had several stability issues where the s3fs mount disappeared periodically.

Cheers,

Matt.

When MapProxy caches the response of a WMS request it should use either max-age within the Cache-Control header (preferable), or the Expires header, to compute a timestamp to store in its cache. When a second request is made for the same tile the stored timestamp should be consulted first to determine if the request needs to be reissued to the service providing the WMS.

Note: This was unabashedly borrowed from a MapCache issue request, but is equally as valid (MapServer/mapcache#36).

MapProxy tries to remove tiles from cache when it fails to load it, to allow re-retrieving of broken tiles.
https://github.com/mapproxy/mapproxy/blob/master/mapproxy/image/tile.py#L70

But this only works for filebased caches but not MBTiles, etc.

[Imported from https://bitbucket.org/olt/mapproxy/issue/43/mapproxy-crashes-when-built-against-os-x]

I've got an OS X 10.7 system with Python 2.7.3 installed as a Framework (that's default for OS X). I've installed PIL, PyYAML and MapProxy. Serve-develop starts with default configuration (base-config). Server crashes on the first WMS GetMap or TMS request with no error message - it just returns to the command line.

The last message is:

mapproxy.source.request - INFO - GET http://osm.omniscale.net/proxy/service?layers=osm&width=512&version=1.1.1&bbox=-20037508.3428,-20037508.3428,20037508.3428,20037508.3428&service=WMS&format=image%2Fpng&styles=&srs=EPSG%3A900913&request=GetMap&height=512 200 23.3 166

MapProxy doesn't crash when built against Homebrew installed python. This could be ok, but I'm planning to use mapnik source with mapproxy. Mapnik requires boost library, and boost library builds against OS default python only. So I don't get mapnik bindings in my homebrew python. So the only solution left is to recompile boost against non-default python on OS X, but this isn't a trivial task at all.

Maybe something can be changed in MapProxy code, so it won't crash when getting wms/tms requests on OS X?

When I issue http://myserver/mapproxy/demo

The apache2 error log shows the error from the title twice:

[Tue Jan 15 22:11:49 2013] [error] No handlers could be found for logger "mapproxy.service.wmts"
[Tue Jan 15 22:11:49 2013] [error] No handlers could be found for logger "mapproxy.service.wmts"

There is an easy fix for this which I have tested and will send in through a pull request

i've got a wms-server http://gis.susu.ac.ru/wms1? in EPSG:28411 but i can't see it in MapProxy viewer, e. g. in EPSG:4326 or EPSG:900913. What do you think about a probably reasons of that problem? Thank you.

We have a .yaml configuration file with ~600 layers in it and have noticed that each time it changes (every 5 minutes in our case) that httpd.exe adds ~18MB of memory.

Steps to reproduce:

1. Pop https://gist.github.com/keithamoss/6cb5e5a5a8664731f094 and https://gist.github.com/keithamoss/223dc0273ee74ce25e9a in a directory
2. Hit http://server/mapproxy/firewatch/service
3. Change firewatch.yaml (e.g. change one of the layer titles)
4. Hit http://server/mapproxy/firewatch/service again, httpd.exe memory increases

This is what the log looked like after 4 iterations with httpd.exe going from ~14MB to ~94MB

2013-05-03 15:01:39,039 - INFO - mapproxy.multiapp - initializing project app firewatch with e:\mapproxy\keith\projects\firewatch.yaml
2013-05-03 15:01:39,039 - INFO - mapproxy.config - reading: e:\mapproxy\keith\projects\firewatch.yaml
2013-05-03 15:01:40,631 - INFO - mapproxy.config - reading: e:\mapproxy\keith\projects\base-config\base-mapproxy.yaml
2013-05-03 15:01:40,803 - INFO - mapproxy.system - using pyproj for coordinate transformation
2013-05-03 15:01:55,882 - INFO - mapproxy.multiapp - initializing project app firewatch with e:\mapproxy\keith\projects\firewatch.yaml
2013-05-03 15:01:55,882 - INFO - mapproxy.config - reading: e:\mapproxy\keith\projects\firewatch.yaml
2013-05-03 15:01:57,648 - INFO - mapproxy.config - reading: e:\mapproxy\keith\projects\base-config\base-mapproxy.yaml
2013-05-03 15:02:08,414 - INFO - mapproxy.multiapp - initializing project app firewatch with e:\mapproxy\keith\projects\firewatch.yaml
2013-05-03 15:02:08,414 - INFO - mapproxy.config - reading: e:\mapproxy\keith\projects\firewatch.yaml
2013-05-03 15:02:10,071 - INFO - mapproxy.config - reading: e:\mapproxy\keith\projects\base-config\base-mapproxy.yaml
2013-05-03 15:02:58,321 - INFO - mapproxy.multiapp - initializing project app firewatch with e:\mapproxy\keith\projects\firewatch.yaml
2013-05-03 15:02:58,321 - INFO - mapproxy.config - reading: e:\mapproxy\keith\projects\firewatch.yaml
2013-05-03 15:03:00,023 - INFO - mapproxy.config - reading: e:\mapproxy\keith\projects\base-config\base-mapproxy.yaml

Tested with MapProxy 1.5.0 (with bac0464 integrated) and 1.6.0a with the same result.

We're running MapProxy as MultiApp via WSGI in Apache on Windows Server 2008.

Have confirmed that it happens when using mapproxy-util serve-multiapp-develop, too.

If you create a source and use the seed:only parameter MapProxy do not use the coverages in the services. In this case MapProxy creates a dummy source without coverages. So the bounding box in the capabilities covers the whole world.

If seed:only is set MapProxy only use the coverages for seeding.

We use 2 mapproxy Nodes behind a Loadbalancer without sticky sessions.
mappproxy cache directory is storage to the same NFS share, is locatet at a Netapp device.
The NFS is mounted under this options: rw,noatime,nodiratime,noacl,noac
When mapproxy write a tile, lockfiles are storage to the NFS share directory under /cache/mapproxy/locks, that is configured on both nodes.
The format of file is eb543c83f5f73bc5abf13e6573dba30d.lck0
This format is always the same....
We to suspect that on node delete the lck file from the other node...
Here ist the issue from the apache error logfile:

[Thu Feb 28 11:19:08 2013] [error] [client _] Traceback (most recent call last):, referer: http://isk.geobasis-bb.de/BrandenburgViewer/basiskarte.html
[Thu Feb 28 11:19:08 2013] [error] [client *] File "/usr/lib/pymodules/python2.6/mapproxy/wsgiapp.py", line 158, in call, referer: http://isk.geobasis-bb.de/BrandenburgViewer/basiskarte.html
[Thu Feb 28 11:19:08 2013] [error] [client *] resp = self.handlers[handler_name].handle(req), referer: http://isk.geobasis-bb.de/BrandenburgViewer/basiskarte.html
[Thu Feb 28 11:19:08 2013] [error] [client *] File "/usr/lib/pymodules/python2.6/mapproxy/service/ows.py", line 38, in handle, referer: http://isk.geobasis-bb.de/BrandenburgViewer/basiskarte.html
[Thu Feb 28 11:19:08 2013] [error] [client *] return self.services[service].handle(req), referer: http://isk.geobasis-bb.de/BrandenburgViewer/basiskarte.html
[Thu Feb 28 11:19:08 2013] [error] [client *] File "/usr/lib/pymodules/python2.6/mapproxy/service/base.py", line 30, in handle, referer: http://isk.geobasis-bb.de/BrandenburgViewer/basiskarte.html
[Thu Feb 28 11:19:08 2013] [error] [client *] return handler(parsed_req), referer: http://isk.geobasis-bb.de/BrandenburgViewer/basiskarte.html
[Thu Feb 28 11:19:08 2013] [error] [client *] File "/usr/lib/pymodules/python2.6/mapproxy/service/wms.py", line 111, in map, referer: http://isk.geobasis-bb.de/BrandenburgViewer/basiskarte.html
[Thu Feb 28 11:19:08 2013] [error] [client *] renderer.render(merger), referer: http://isk.geobasis-bb.de/BrandenburgViewer/basiskarte.html
[Thu Feb 28 11:19:08 2013] [error] [client *] File "/usr/lib/pymodules/python2.6/mapproxy/service/wms.py", line 371, in render, referer: http://isk.geobasis-bb.de/BrandenburgViewer/basiskarte.html
[Thu Feb 28 11:19:08 2013] [error] [client *] layer_merger), referer: http://isk.geobasis-bb.de/BrandenburgViewer/basiskarte.html
[Thu Feb 28 11:19:08 2013] [error] [client *] File "/usr/lib/pymodules/python2.6/mapproxy/util/async.py", line 204, in _single_call, referer: http://isk.geobasis-bb.de/BrandenburgViewer/basiskarte.html
[Thu Feb 28 11:19:08 2013] [error] [client *] result = func(_args), referer: http://isk.geobasis-bb.de/BrandenburgViewer/basiskarte.html
[Thu Feb 28 11:19:08 2013] [error] [client _] File "/usr/lib/pymodules/python2.6/mapproxy/service/wms.py", line 412, in _render_layer, referer: http://isk.geobasis-bb.de/BrandenburgViewer/basiskarte.html
[Thu Feb 28 11:19:08 2013] [error] [client *] layer_img = layer.get_map(self.query), referer: http://isk.geobasis-bb.de/BrandenburgViewer/basiskarte.html
[Thu Feb 28 11:19:08 2013] [error] [client *] File "/usr/lib/pymodules/python2.6/mapproxy/layer.py", line 289, in get_map, referer: http://isk.geobasis-bb.de/BrandenburgViewer/basiskarte.html
[Thu Feb 28 11:19:08 2013] [error] [client *] result = self._image(query), referer: http://isk.geobasis-bb.de/BrandenburgViewer/basiskarte.html
[Thu Feb 28 11:19:08 2013] [error] [client *] File "/usr/lib/pymodules/python2.6/mapproxy/layer.py", line 322, in _image, referer: http://isk.geobasis-bb.de/BrandenburgViewer/basiskarte.html
[Thu Feb 28 11:19:08 2013] [error] [client *] tile_collection = self.tile_manager.load_tile_coords(affected_tile_coords), referer: http://isk.geobasis-bb.de/BrandenburgViewer/basiskarte.html
[Thu Feb 28 11:19:08 2013] [error] [client *] File "/usr/lib/pymodules/python2.6/mapproxy/cache/tile.py", line 126, in load_tile_coords, referer: http://isk.geobasis-bb.de/BrandenburgViewer/basiskarte.html
[Thu Feb 28 11:19:08 2013] [error] [client *] created_tiles = creator.create_tiles(uncached_tiles), referer: http://isk.geobasis-bb.de/BrandenburgViewer/basiskarte.html
[Thu Feb 28 11:19:08 2013] [error] [client *] File "/usr/lib/pymodules/python2.6/mapproxy/cache/tile.py", line 227, in create_tiles, referer: http://isk.geobasis-bb.de/BrandenburgViewer/basiskarte.html
[Thu Feb 28 11:19:08 2013] [error] [client *] created_tiles = self._create_meta_tiles(meta_tiles), referer: http://isk.geobasis-bb.de/BrandenburgViewer/basiskarte.html
[Thu Feb 28 11:19:08 2013] [error] [client *] File "/usr/lib/pymodules/python2.6/mapproxy/cache/tile.py", line 293, in _create_meta_tiles, referer: http://isk.geobasis-bb.de/BrandenburgViewer/basiskarte.html
[Thu Feb 28 11:19:08 2013] [error] [client *] return self._create_threaded(self._create_meta_tile, meta_tiles), referer: http://isk.geobasis-bb.de/BrandenburgViewer/basiskarte.html
[Thu Feb 28 11:19:08 2013] [error] [client *] File "/usr/lib/pymodules/python2.6/mapproxy/cache/tile.py", line 243, in _create_threaded, referer: http://isk.geobasis-bb.de/BrandenburgViewer/basiskarte.html
[Thu Feb 28 11:19:08 2013] [error] [client *] for new_tiles in async_pool.imap(create_func, tiles):, referer: http://isk.geobasis-bb.de/BrandenburgViewer/basiskarte.html
[Thu Feb 28 11:19:08 2013] [error] [client *] File "/usr/lib/pymodules/python2.6/mapproxy/util/async.py", line 54, in _result_iter, referer: http://isk.geobasis-bb.de/BrandenburgViewer/basiskarte.html
[Thu Feb 28 11:19:08 2013] [error] [client *] for result in results:, referer: http://isk.geobasis-bb.de/BrandenburgViewer/basiskarte.html
[Thu Feb 28 11:19:08 2013] [error] [client *] File "/usr/lib/pymodules/python2.6/mapproxy/util/async.py", line 191, in map_each, referer: http://isk.geobasis-bb.de/BrandenburgViewer/basiskarte.html
[Thu Feb 28 11:19:08 2013] [error] [client *] for value in self._get_results(next_result, results, raise_exceptions):, referer: http://isk.geobasis-bb.de/BrandenburgViewer/basiskarte.html
[Thu Feb 28 11:19:08 2013] [error] [client *] File "/usr/lib/pymodules/python2.6/mapproxy/util/async.py", line 235, in _get_results, referer: http://isk.geobasis-bb.de/BrandenburgViewer/basiskarte.html
[Thu Feb 28 11:19:08 2013] [error] [client *] for i, value in self._fetch_results(raise_exceptions):, referer: http://isk.geobasis-bb.de/BrandenburgViewer/basiskarte.html
[Thu Feb 28 11:19:08 2013] [error] [client *] File "/usr/lib/pymodules/python2.6/mapproxy/util/async.py", line 144, in run, referer: http://isk.geobasis-bb.de/BrandenburgViewer/basiskarte.html
[Thu Feb 28 11:19:08 2013] [error] [client *] result = func(_args), referer: http://isk.geobasis-bb.de/BrandenburgViewer/basiskarte.html
[Thu Feb 28 11:19:08 2013] [error] [client _**] File "/usr/lib/pymodules/python2.6/mapproxy/cache/tile.py", line 304, in _create_meta_tile, referer: http://isk.geobasis-bb.de/BrandenburgViewer/basiskarte.html
[Thu Feb 28 11:19:08 2013] [error] [client *] with self.tile_mgr.lock(main_tile):, referer: http://isk.geobasis-bb.de/BrandenburgViewer/basiskarte.html
[Thu Feb 28 11:19:08 2013] [error] [client *] File "/usr/lib/pymodules/python2.6/mapproxy/platform/cpython/lock.py", line 42, in enter, referer: http://isk.geobasis-bb.de/BrandenburgViewer/basiskarte.html
[Thu Feb 28 11:19:08 2013] [error] [client *] self.lock(), referer: http://isk.geobasis-bb.de/BrandenburgViewer/basiskarte.html
[Thu Feb 28 11:19:08 2013] [error] [client ] File "/usr/lib/pymodules/python2.6/mapproxy/platform/cpython/lock.py", line 65, in lock, referer: http://isk.geobasis-bb.de/BrandenburgViewer/basiskarte.html
[Thu Feb 28 11:19:08 2013] [error] [client *] self._lock = self.try_lock(), referer: http://isk.geobasis-bb.de/BrandenburgViewer/basiskarte.html
[Thu Feb 28 11:19:08 2013] [error] [client *] File "/usr/lib/pymodules/python2.6/mapproxy/platform/cpython/lock.py", line 56, in _try_lock, referer: http://isk.geobasis-bb.de/BrandenburgViewer/basiskarte.html
[Thu Feb 28 11:19:08 2013] [error] [client *] return LockFile(self.lock_file), referer: http://isk.geobasis-bb.de/BrandenburgViewer/basiskarte.html
[Thu Feb 28 11:19:08 2013] [error] [client *] File "/usr/lib/pymodules/python2.6/mapproxy/util/ext/lockfile.py", line 124, in init, referer: http://isk.geobasis-bb.de/BrandenburgViewer/basiskarte.html
[Thu Feb 28 11:19:08 2013] [error] [client *] pid = fp.read().strip()[:20], referer: http://isk.geobasis-bb.de/BrandenburgViewer/basiskarte.html
[Thu Feb 28 11:19:08 2013] [error] [client *] IOError: [Errno 116] Stale NFS file handle, referer: http://isk.geobasis-bb.de/BrandenburgViewer/basiskarte.html

is mapproxy Loadbalancer ready?
There are a LB howto for mapproxy?

many thx
Karsten

According to the WMS GetFeatureInfo Request specification the QUERY_LAYERS parameter has a "comma-separated list of one or more layers to be queried". When firing such a request to Mapproxy only the first layer will be queried, the others are ignored.

Since many clients e.q. OpenLayers, will combine layers which are accessable through the same service url GetFeatureInfo through Mapproxy isn't working as excpected.

--Gerard

UPDATE: lxml package solved this issue. My mistake to not testing this first. May be the optional install in the doc could be changed to (highly) recommended.

Hi
Mapproxy doesn't offer acces to root resource with TMS client.
Some softwares particularly needs Services information :
< Services > < TileMapService title="Example Static Tile Map Service" version="1.0.0" href="http://www.osgeo.org/services/tilemapservice.xml" / >< /Services >

as described here http://wiki.osgeo.org/wiki/Tile_Map_Service_Specification

Currently does not works. Throws exception on start:

Traceback (most recent call last):
  File "./fcgi-socket", line 4, in <module>
    application = make_wsgi_app('/etc/mapproxy/mapproxy.yaml')
  File "/usr/local/lib/python2.6/dist-packages/mapproxy/wsgiapp.py", line 99, in make_wsgi_app
    conf = load_configuration(mapproxy_conf=services_conf, ignore_warnings=ignore_config_warnings)
  File "/usr/local/lib/python2.6/dist-packages/mapproxy/config/loader.py", line 1412, in load_configuration
    conf_dict = load_configuration_file([os.path.basename(mapproxy_conf)], conf_base_dir)
  File "/usr/local/lib/python2.6/dist-packages/mapproxy/config/loader.py", line 1430, in load_configuration_file
    current_dict = load_yaml_file(conf_file)
  File "/usr/local/lib/python2.6/dist-packages/mapproxy/util/yaml.py", line 29, in load_yaml_file
    return load_yaml(f)
  File "/usr/local/lib/python2.6/dist-packages/mapproxy/util/yaml.py", line 38, in load_yaml
    return yaml.load(doc, Loader=yaml.CLoader)
  File "/usr/local/lib/python2.6/dist-packages/yaml/__init__.py", line 73, in load
    loader.dispose()
AttributeError: 'CLoader' object has no attribute 'dispose'

mapserver source does not allow to set environment variables for mapserv executable within yaml file. It is necessary for handy setting mapserver PATH, PROJ_LIB, GDAL_DATA, MS_ERRORFILE, etc.

I have a working mapproxy setup that has a RGBA PNG cache which works fine when accessed through the web.

When I try to run mapproxy-seed for the cache I get the error

An error occured. Retry in 2 seconds: IOError('decoder zip not available',)
An error occured. Retry in 2 seconds: IOError('decoder zip not available',)
An error occured. Retry in 4 seconds: IOError('decoder zip not available',)
...

I have mapproxy installed in a virtualenv.

LayerMerge currently only supports RGBA, not 8-bit paletted imagery.

It's not possible to set the image format with mapproxy-util export.

When MapProxy receives a GetFeatureInfo request for a WMS source that contains the FEATURE_COUNT property it doesn't get passed on to the source service. As the default for FEATURE_COUNT is usually 1 this is resulting in single features being returned.

MapProxy 1.4.0.

Installed MapProxy in Osgeo4w on Windows (tried both Windows7 and Windows Server 2008R2)

setting following environment variables:

OSGEO4W_ROOT=c:/osgeo4w
PYTHONHOME=%OSGEO4W_ROOT%\apps\Python27
PATH = %PATH%;%OSGEO4W_ROOT%\bin;%OSGEO4W_ROOT%\apps\Python27\Scripts;

If you create a wsgi app as written in docs, the demo apps are not working if called from osgeo4w apache. The maps are not showing because MapProxy cannot use PIL

But if wsgi is called via serve-develop all is working fine

To me it appears that something is wrong with loading PIL module if running in apache/mod_wsgi mode. Looking into error logs of apache immediately after starting there is a user warning that PIL is not available (raised in image.py). And when a map is served a real import error about PIL is shown.

The export tool tries to use existing tiles (without opening them) when the cache and export girds are compatible. This fails when the image formats are different:

Exception in thread Thread-1:
Traceback (most recent call last):
File "D:\Python27\Lib\threading.py", line 551, in __bootstrap_inner
self.run()
File "d:\mapproxy_venv\lib\site-packages\mapproxy\seed\seeder.py", line 92, in run
self.work_loop()
File "d:\mapproxy_venv\lib\site-packages\mapproxy\seed\seeder.py", line 104, in work_loop
exceptions=(SourceError, IOError), ignore_exceptions=(LockTimeout, ))
File "d:\mapproxy_venv\lib\site-packages\mapproxy\seed\util.py", line 216, in exp_backoff
result = func(*args, **kw)
File "d:\mapproxy_venv\lib\site-packages\mapproxy\cache\tile.py", line 126, in load_tile_coords
created_tiles = creator.create_tiles(uncached_tiles)
File "d:\mapproxy_venv\lib\site-packages\mapproxy\cache\tile.py", line 213, in create_tiles
created_tiles = self._create_single_tiles(tiles)
File "d:\mapproxy_venv\lib\site-packages\mapproxy\cache\tile.py", line 237, in _create_single_tiles
created_tiles.extend(self._create_single_tile(tile))
File "d:\mapproxy_venv\lib\site-packages\mapproxy\cache\tile.py", line 253, in _create_single_tile
source = self._query_sources(query)
File "d:\mapproxy_venv\lib\site-packages\mapproxy\cache\tile.py", line 274, in _query_sources
return self.sources[0].get_map(query)
File "d:\mapproxy_venv\lib\site-packages\mapproxy\source\tile.py", line 95, in get_map
return CacheMapLayer.get_map(self, query)
File "d:\mapproxy_venv\lib\site-packages\mapproxy\layer.py", line 390, in get_map
self._check_tiled(query)
File "d:\mapproxy_venv\lib\site-packages\mapproxy\layer.py", line 409, in _check_tiled
raise MapError("invalid tile format, use %s" % self.tile_manager.format)
MapError: invalid tile format, use jpeg

There is a report that MapProxy does processes cascaded GeoTIFF requests from MapServer sources and thus removing the georeference information.

http://lists.osgeo.org/pipermail/mapproxy/2012-August/001173.html

When looking at layers, rendered from source

mpn:
type: mapnik
mapfile: /work/mapnik/mpn.xml

the background is black.

But in the xml file, I have

When rendering with just a python script or OGCServer, the background is transparent.

Is this a bug or just an incorrect setup?