osmlab / editor-layer-index Goto Github PK

The "Mapnik" name made sense when the alternative was "Osmarender"; but now that there are multiple Mapnik-based renderings just on osm.org it doesn't make a whole lot of sense. Users get understandably confused; there's been at least one example of a mapper logging a bug against Mapnik (the piece of software) for a perceived shortcoming in OSM Carto because iD's imagery layer talks about that layer as "OpenStreetMap (Mapnik)".

The code in scripts does not have a license attached, preventing forking of it.

Assigning @tmcw since he wrote the code and he's the only one who can put on a license.

The license (both of the imagery database and the items in that database) is a separate issue.

@pnorman it makes lots of sense to have an attribute for the vintage of the
imagery. We should probably add that as an optional attribute.

How do we want to do this? I am leaning towards two tags, one for the start, one for the end. start_date and end_date are the obvious names. JSON schema includes a date-time definition already but it's not well-suited for imagery where frequently the only date is a year and we want to allow 2013 as a valid date.

I can see 4 cases

1. Imagery where a year is known
2. Imagery where a date range is known, but not always days. This could span years for imagery taken in Dec/Jan
3. Composite layers spanning multiple years, but each year corresponding to a different area
4. Sat imagery taken at a specific date and time (e.g. landsat). The full date and time is important information here

Having a root>states.yml file is kind of weird - it would be better if this was either totally individual files, or one big file. Everywhere in between complicates tools and testing.

Could we use some continuous integration setup like iD has to verify that all pull requests meet the schema and that running make doesn't change the final XML/JSON files?

I put up source definitions for the imagico.de OSM images for mapping on https://github.com/imagico/osmim-imagery-index.

Included there are files for the individual layers as well as a global file for the aggregate containing all the individual bounding boxes as polygons.

Not sure how this best fits into the structure here so no concrete PR.

Editing the partially sorted 6.1k line imagery.xml file is messy and prone to errors.

I propose splitting it into about 4-6 files based on global regions.

The breakdown would be about

lines  region
  130  global
 1500  americas
 4000  europe
  500  other

The build scripts could build a combined image.xml file. Right now editing the files is unnecessarily difficult due to their size, this would only be for editing ease.

I got this with make check. I knew which file it was, but it would be best to report it.

$ make check
Traceback (most recent call last):
  File "scripts/check.py", line 7, in <module>
    source = json.load(open(file))
  File "/usr/lib/python2.7/json/__init__.py", line 278, in load
    **kw)
  File "/usr/lib/python2.7/json/__init__.py", line 326, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode
    obj, end = self.scan_once(s, idx)
ValueError: Expecting , delimiter: line 9 column 5 (char 271)
make: *** [check] Error 1

https://github.com/osmlab/editor-imagery-index/blob/gh-pages/sources/world/OpenStreetMap-Mapnik.json doesn't make use of a/b/c

Hello, I'm using the iD-Openstreetmap editor whose background Imagery are pulled from editor-imagery-index.

I would recommend to use basemap.at (maptype "Orthofoto") Imagery instead of the former Geoimage.at for Austria. For general information about this map, have a look at http://basemap.at/index_en.html
Both are from the same organisation (geoland.at) with the same licensing when using in Openstreetmap. By the way, we already use basemap.at within iD (maptype "Standard", so maybe you should change its name within iD's background map list to "basemap.at Standard")

Basemap.at is the latest, newest free mapping portal of Austrian federal states. It is kind of successor of the former geoland.at WMS-service. Even if the most of the Imagery is the same within both services, the ones of basemap.at are of better image-quality (JPEG-compression, brightness...), in some areas newer (e.g. Vienna), and have less image-artefacts (like zig-zag lines at a straight edge of a building).

The Tilemap-URL template is:
http://maps{s}.wien.gv.at/basemap/bmaporthofoto30cm/normal/google3857/{z}/{y}/{x}.jpeg
where {s} could be 1,2,3,4
Maxzoom level=19

Name: Japan GSI Ortho Imagery
source tag: GSImaps/ort
url: http://cyberjapandata.gsi.go.jp/xyz/ort/{z}/{x}/{y}.jpg

Reference: http://wiki.openstreetmap.org/wiki/GSImaps#Ortho_Photo_Tiles

Bing has some serious distortion and should not be used in Japan https://www.openstreetmap.org/user/PlaneMad/diary/36262

General consensus per #134 is that it is better to leave these sources in and broaden the scope of this project.
(aka revert #124)

Have been playing around with redeploying iD with a set of custom imagery, as documented in this blog post and in the imagery schema.json.

Was getting errors until I updated my imagery json file to replace "url" with "template", see here.

Is this an error in the schema json here, or am I missing something?

I recently compared the editor-imagery-index and the JOSM imagery wiki and saw that they have diverged quite a lot in the last 2 years. Both had a significant amount of additions and corrections, but apparently not the same for both. In fact, each database would grow by about 50% if they pulled the updates from the other one!

There is a certain threshold for users to report a new source of imagery, we can be happy if the users contribute new sources at all. It would be too much to ask for an iD user to edit the JOSM wiki in addition to a change request in the editor-imagery-index. Likewise a JOSM user will usually just care about the JOSM repository.

But as far as imagery goes, the choice of editor shouldn't really matter: Any background that is useful in iD will also be helpful in JOSM and vice versa (TMS). So as a common repository for both project seems currently out of reach, we should aim for the next best thing, which is two databases with the same content. As long as the entries get synchronized on a regular basis, this is the same as a shared database from the user perspective. Of course it is more work for the maintainers.

The JOSM team will care for the IIE -> JOSM update. This ticket is basically a reminder that we have lots of good new stuff and you are welcome to take it! :)

JOSM ticket for reference (includes some basic stats): http://josm.openstreetmap.de/ticket/10760

The best attribute to flag imagery sources that are the best layer to display by default in a given region doesn't always work well – for example when an imagery is only better than another imagery in a fraction of its coverage area (e.g. see #112 (comment) ff.).

Let's discuss possible improvements here.

3973ef8 changed the significant figures/rounding on all sorts of imagery output

e.g.

diff --git a/imagery.geojson b/imagery.geojson
index 59befee..c4bc96a 100644
--- a/imagery.geojson
+++ b/imagery.geojson
@@ -7,121 +7,121 @@
                 "coordinates": [
                     [
                         [
-                            -17.9291733, 
-                            28.8910589
+                            -17.929173299999999, 
+                            28.891058900000001
                         ], 
                         [
-                            -18.0333424, 
-                            28.7998146
+                            -18.033342399999999, 
+                            28.799814600000001
                         ], 
                         [
                             -18.0374275, 
-                            28.7317767
+                            28.731776700000001
                         ], 
                         [
                             -17.87811, 
-                            28.4322434
+                            28.432243400000001
                         ], 
                         [
                             -17.8311317, 
-                            28.4178731
+                            28.417873100000001
                         ], 

From #88 (comment):

It would be great to have someone add a slippy-map interface to http://osmlab.github.io/editor-imagery-index/.

Could You add Geoportal imagery for countrywide Poland from the following link: http://wms.misek.pl/geoportal.orto/tms/{zoom}/{x}/{y}/
This link has been provided on Polish Openstreetmap forum http://forum.openstreetmap.org/viewtopic.php?id=22525

Please tell if any more info is needed to complete this.
Thank You
Tomek

From @astoff in openstreetmap/iD#2256:

Two extremely useful raster layers have been made available for use in Brazil: "IBGE Mapa de Setores Rurais" and "IBGE Mapa de Setores Urbanos". They are listed in JOSM's imagery preferences, and the TMS addresses can be found there.

These options should appear in the background settings panel when editing in Brazil, just like the USGS topographic map appears there when editing in the USA.

This repo has fallen behind the JOSM wiki list.

The Yahoo imagery (actually, the API that feeds it to us) has been removed long ago. This effectively means that we are not allowed to use Yahoo imagery anymore. With it goes the 'html' type.

So far we've had @jfirebaugh myself and at least one other forget to run `make' before commit.

Can anyone think of a way to check this with travis?

I was thinking of md5summing the output files, running make and comparing the results. The problem is that although the JSON output is equivalent across python versions, it's text representation is not.

This project now exists for a while, but I don't see that its main goal to unify the editor templates will ever be reached.

From my point as JOSM maintainer it is a step backwards. We moved from source-code based project to a community based approach and this again is source-code based.

Why couldn't the JOSM infrastructure be used? It allows community access and wiki style reduces the entry-level for contributors. JOSM tool support for shapes exists and also if XML is not soo easy, most contributors simply use copy and paste from other templates. I did develop that wiki stuff based on the user requirements and it is accepted by the users. The wiki already does data validity checks, so users get feedback when structure is bad.

If you really want to reach unification, my suggestion would be to expand the JOSM infrastructure instead of doing a separate project.

• We can add any missing or additional information in the XML
• We can directly output the geojson format or anything else from JOSM server (using direct integration into the python code which already generates the map files).

To have unique templates, iD editor e.g. could use that output and import it into their git repository from time to time.

Advantages:

• Directly edits by users (which for JOSM is the main data source)
• Immediate changes (no request, simply do it)
• One central place

Right now there isn't a way to specify what imagery should be loaded by default for a region, preventing iD from appropriately selecting background imagery for users.

See openstreetmap/iD#1310

Moved from openstreetmap/iD#1960 (@RM87):

Please add Estonian Land Board ortophotos as custom layer for the Estonia area.
Name: Estonian Land Board ortophoto
source tag: Estonian Land Board orto
url: http://kaart.maakaart.ee/tiles/1.0.0/maaamet_orto_EPSG900913/{z}/{x}/{y}.jpeg?origin=nw
extent: 21.7,57.45,28.25,59.85

When using imagery.xml in JOSM country codes are not listed in the available entries list.

The current link for MapBox terms sends to an About Mapbox Streets page.

Mapbox Streets is licensed under our Terms of Service. OpenStreetMap contributors create and improve its map data, licensed under the ODbL. DigitalGlobe acquires and processes our commercial imagery. All other imagery is derived from public sources and processed by Mapbox.

The DigitalGlobe link is probably not what is relevant for the imagery given.

RIght now we have layers which are purely processed OpenStreetMap data. These layers can't be used in an OpenStreetMap editor to add anything new to OSM, because all the data in them is already in OSM.

What should we do with them?

The standard way right now to create an imagery bounding polygon is to draw one in JOSM and convert the way (or multipolygon) using the imagery XML plugin.

What will be the easiest way to get a single-file JSON? Do we need to modify the plugin?

For reference, I need WKT files for my imagery for various purposes. To generate these I use JOSM, then run osm2poly.pl bounds.osm | poly2wkt.pl and copy the result to my remote imagery server.

These need to be cross-checked against current P2/JOSM lists:

• Port au Prince - GeoEye Jan 2010 (http://gravitystorm.dev.openstreetmap.org/imagery/haiti/{z}/{x}/{y}.png)
• Haiti - IOM Drone Imagery, 2012-13 (http://wms.openstreetmap.fr/tms/1.0.0/iomhaiti/{zoom}/{x}/{y})
• NAIP (http://cube.telascience.org/tilecache/tilecache.py/NAIP_ALL/{z}/{x}/{y}.png)
• Austria - geoimage.at (http://geoimage.openstreetmap.at/4d80de696cd562a63ce463a58a61488d/{z}/{x}/{y}.jpg)
• Russia - Kosmosnimki.ru IRS Satellite (http://irs.gis-lab.info/?layers=irs&request=GetTile&z={z}&x={x}&y={y})
• Belarus - Kosmosnimki.ru SPOT4 Satellite (http://irs.gis-lab.info/?layers=spot&request=GetTile&z={z}&x={x}&y={y})
• Switzerland - Canton Solothurn - SOGIS 2007 (http://mapproxy.sosm.ch:8080/tiles/sogis2007/EPSG900913/{z}/{x}/{y}.png?origin=nw)
• Poland - Media-Lab fleet GPS masstracks (http://masstracks.media-lab.com.pl/{z}/{x}/{y}.png)

There should be a util.py for some common routines.

I'm interested in the idea of adding a simple geometry-extraction API to "rendered" layers, such as the US Forest Service one I've recently added.

This would work pretty simply:

1. User clicks location in editor
2. Editor requests http://osm.cycle.travel/extract-forest.geojson?lat=-107.255916&lon=37.725650
3. Server returns GeoJSON (or whatever) for all features at that location

It could also be of use for the TIGER 2014 tiles, CORINE, etc. etc., encouraging people to bring third-party content in manually rather than through a troublesome bulk import.

This would require one or potentially more new attributes in imagery sources - at the least, an API endpoint. Currently the list of attributes carried through to http://osmlab.github.io/editor-imagery-index/imagery.json (etc.) is limited and hardcoded.

What would be the preferred way of adding an additional attribute? Simply patching the build scripts, or...?

When converting an XML file convert.py creates JSON like

"bbox": {
    "min_lon": "-119.6011924", 
    "max_lon": "-119.3095139", 
    "min_lat": "49.7714941", 
    "max_lat": "50.0326418"
}, 

The lat/lon shouldn't be quoted

Sometimes in same area is more than one imagery source available. Priority property could help to sort these sources by most valuable, most accurate or any other characteristic.

We should use Transifex for translating imagery names and descriptions. Follow what iD does for presets:

• Assign each source an id
• make builds an imagery.yml source for Transifex
• make translations pulls down translations into locale directory, split into separate files
• Downstream consumers merge translations into their own

The schema states that the default attribute defines

Whether this is a good default imagery for its area

Yet we have "default": true on Mapnik, MapQuest Open Aerial, and MapBox Satellite.

We're also converting it to <default>true</default> in the JOSM XML, which has a completely different meaning. There it is about if the imagery is shown in the list by default.

I would like to add mapping layers of buildings and parcels to the selection map layers in OSM editor iD.

• name buildings map layers: "RUIAN budovy"
  url: http://tile.poloha.net/budovy/{z}/{x}/{y}.png
  zoom: 13 - 19
• name parcels map layers: "RUIAN parcely"
  url: http://tile.poloha.net/parcely/{z}/{x}/{y}.png
  zoom: 13 - 19

Definition for JOSM (with Czech Republic polygon):
http://josm.openstreetmap.de/wiki/Maps/Czech%20Republic

What can I do?

For a planned imagery offset database each imagery source should have a unique identifier. While it usually can be deduced from TMS/WMS URL, for some sources (e.g. Bing) the dictionary entry is used instead. In such cases it would be nice to have a key for that in lists.

News item here:
http://www.scoop.co.nz/stories/PA1404/S00416/aerial-imagery-now-online-for-reuse.htm

imagery and licences here (CC):
https://data.linz.govt.nz/x/vuozYk

There are lots of layers, I think 42. I'm not entirely conversant with json, but if someone could explain the format clearly, I should be able to take part in making the json files myself.

Let's say there is an imagery that supports more than one type of service. For example WMS as well as tiles. Would it make sense to add both to this repository? Because while WMS may support more different projections, tiles would be more suitable for applications (e.g. the iD editor).

How exactly should one add such imagery?

Some sources of imagery have boundaries that are too broad, resulting in useless items in the imagery menu in iD.

These include:

• MapBox Satellite, no bounds specified but not available in the UK.
• MapQuest Open Aerial, no bounds specified but not available in the UK.
• Surrey Air Survey, bounding box without polygon.
• 7th Series (OS7), not available at http://www.openstreetmap.org/#map=19/53.04769/-1.60237

There is obviously a need to coordinate with imagery sources if they may be expanded in the future,

"QA No Address" is rather inscrutable. @simonpoole what would be a more descriptive name?

It doesn't look like this is working anymore. Should it be removed?

Some of the imagery in the index is not legally usable except for editing OpenStreetMap.

As a courtesy to others who may want to use the index for their own projects, should we indicate this?