I'm quite ambivalent about this, but it was a tad bit harder to review changesets from uploaders that did not use English in the comment. For example, changeset1 and changeset2.

Description + workflow copy to be used on tasks.openstreetmap.us. Post after peer review is finalized.

tasks.openstreetmap.us supports arbitrary polygons as tasking areas. We'll use election district polygons to chunk up tasks.

https://gist.github.com/lxbarth/6418814

Needs fix before continuing import.

There are areas in which what looks to be the same address appears multiple times. What's very likely going on is that we're missing a significant attribute from the source data in our addressing scheme.

Investigating now.

Now that latest data is published, we should update the data sources of the script.

Ref: http://www.mapbox.com/blog/nyc-and-openstreetmap-cooperating-through-open-data/

On 13th Street in Brooklyn there are diverging addresses, for instance:

addr:street=East 13th Street
addr:street=Sheepshead Bay Road

http://www.openstreetmap.org/#map=19/40.58887/-73.95594

.

Bit of a strange one from the source data, and it could be a one-off that doesn't need fixing, but node 2494076774 has addr:housenumber = 174 REAR. Is there a better formatting for this?

e.g. http://www.openstreetmap.org/browse/way/247745456

addr:street = Grand Central Parkway Sr North

TIGER way has Grand Central Parkway North Service Road

This building has 3 different address nodes in it. Two of them have address ranges.

Download the area here.
http://www.openstreetmap.org/browse/node/2555442506

For some reason these houses have an address such as ###-## and then one right next to it like ###.

Not sure what is going on, any advice? It is just this street, the rest of the import area looks good.

• Document import in WIki
• Link from http://wiki.openstreetmap.org/wiki/New_York,_New_York

Duplexes are two building polygons in the source data, but probably should be one building polygon in OSM with two address nodes.

Example: duplex half 1, duplex half 2

Did not expand "Ft Hamilton Parkway" to "Fort Hamilton Parkway".

example

I've noticed that many of the buildings are composed of extra nodes.

They don't detract from the import in any way but they are completely useless and if there was an easy way to perform an automated edit to remove them it may want to be examined.

Looks like this area was redeveloped at this is old data that's still in the address DB.

link to extraneous addr node

Found when addressing #24

Investigating.

To the casual, Bing-using importer, this building might appear to be gone because it's not in the Bing imagery, but looking at the USGS imagery shows that it is there and Streetview shows that at the time the car drove past there was a sign for new construction.

This isn't necessarily a bug, but something to think about when reviewing.

There are a series of buildings with single separate address nodes where the address information should be sitting on the building polygon (see #15).

Write a bot that iterates through those buildings, copies address data from address nodes to building polygon and then saves back to OSM?

@emacsen - got some good starter code for this?

@MateoV - can you turn off changewithin for the time of the import and give the recipients over at NYC DoITT a heads up that it's off as long as we're importing?

There are 150+ changesets uploaded before #8 generating overlapping building warnings.

Needs automated fix.

http://www.openstreetmap.org/browse/way/241855753
http://www.openstreetmap.org/browse/way/241855467

@MateoV -

I assume there are two buttons because we can't open this in one go, right?

In my mind ideally we'd have one button that:

• Downloads OSM data into JOSM
• Downloads import data in a separate layer into JOSM

Presuming that this is not possible, we should at least flip the buttons around, as if you download OSM data first, loading the import data second will merge it into the same layer which is not preferable.

Colin Reilly told me DoITT has tried to place address points as close as possible to the actual entrance to a building.

Right now we're merging addresses into buildings where possible, losing any address location information.

Should we stop merging addresses into buildings?

The PS390 building was tagged with amenity=school and I think nowadays the community likes building=school with the land area around it tagged amenity=school (kind of like leisure=park).

Ran into a couple changesets where the street "St Johns Place" was not expanded correctly to "Saint John's Place".

• http://www.openstreetmap.org/changeset/19308015
• http://www.openstreetmap.org/changeset/19308088

Example objects (since fixed):

• http://www.openstreetmap.org/way/250331380
• http://www.openstreetmap.org/way/250330549

I can't join the mapping party on October 12th but I'd like to help. I can be a guinea pig for the import process, or whatever else makes sense. Let me know.

(Opening a ticket because that's what the instructions on OSM Task Manger said to do).

@emacsen - based on your review, what are common mistakes importers make? Looking for a list here that we can use as a basis for a daily script or something that creates a report that we can use for cleanups.

Hey,

Is anyone else haveing trouble accessing the tasking site? I can get on the main openstreetmap.us site but not the subdomain?

Once we've implemented a script, write up a proposal for peer review #1 by import-us working group.

65076 shows odd overlaps:

That's at Essex / Houston / Rivington:

Next action

• Review NYC source data, inspect for similar problem.

The tasking server has a review function.

Who has access to review? And if they do, does it provide feedback?

Not only are there dashes in Queens addresses, but there are twice as many in Queens address ranges like this one.

Related to #25.

Two overlapping questions:

1. Should we discount existing NYC buildings from OSM files prepared for import? I can go either way here because it could be fine to do this manually.
2. How can we generate quick diffs between OSM buildings and NYC data for future updates from NYC to OSM?

This question pertains not only to buildings but also to addresses.

Some buildings have addr:street tags with incorrect street name (e.g. 108 Avenue instead of 108th Avenue)

example

(This appears to have been fixed in a later generation of the OSM files, but the older stuff needs to be fixed.)

Ordinal suffixes are missing. This is a critical error and I'll stop import effective immediately until I've fixed this.

There are a series of areas affected. I'll need to find a way to update them.

Some BIDs are 4xxxxxx, and they are unaddressed. Sarah thought this had some special meaning in the dataset.

Some of these buildings have no height (height of 0.0 meters), which tells me that something is very odd with that.

If we know what these are, we should tag them. The same does for any other feature like this, where the ID or other indicator in the original dataset can tell us something about the data other than building.

We export data with much higher precision and we validate data in JOSM at much higher precision than what OSM handles (see example below).

The result is that while buildings validate before upload, they don't validate after they've been downloaded from OpenStreetMap again.

@MateoV - what options do you see for a fix here? Should we round all coordinates to 7 places after the decimal point when we read in data before we merge nodes?

# Node before upload  
Node: -14546
  Data Set: 26e15b1d
  Edited at: <new object>
  Edited by: <new object>
  Version: 0
  In changeset: 0
  Coordinates: 40.68840054832918, -73.99240044050097
  Coordinates (projected): -8236796.339608559, 4966488.890780375
  Part of: 
    Way: -14556

# Node after upload - note the precision lost in Coordinates.
Node: 2524540191
  Data Set: 55a7e5ae
  Edited at: 2013-11-08T18:26:37Z
  Edited by: lxbarth_nycbuildings (1764427)
  Version: 1
  In changeset: 18787119
  Coordinates: 40.6884005, -73.9924004
  Coordinates (projected): -8236796.335100011, 4966488.883685272
  Part of: 
    Way: 245220992

We were discussing the issue of dashes in addresses, e.g. 632 - 636 5 Avenue, Brooklyn.

The wiki currently states this is equivalent to all housenumbers in the inclusive range being there. Of course the wiki does not make it clear if this is only for a-b or also for a - b, or how to deal with housenumbers that do have a dash in them.

The obvious question was, what do data consumers support.

Richard Welty thought that mkgmap and OSMand didn't support dashes to indicate ranges in housenumbers. Nominatim interprets more complicated schemes than other geocoders, so I investigated what it supports by quering ___ 5th Avenue, Brooklyn, New York, USA

___    Match type        Nominatim place_id
646    Exact match       9151589306
640    No exact match    8213537384
636    No exact match    8213537307
634    No exact match    8213537303
632    No exact match    8213537298

It's safe to say that Nominatim doesn't handle - in addresses but falls back to matching on the street which it knows is close.

This is actually good for NYC, because there are addresses that have dashes in them, and you wouldn't want those dashes specially interpreted.

Given that the data consumers we checked don't special-case -, nor do we really want them to, we should look at what to better do with these addresses.

Converting ranges covering more than two housenumbers to interpolation ways would be ideal and the correct way to represent the data. It's not obvious to me that this conversion could be done within code, and there's a substantial number of these.

Looking at some of the housenumbers, I see padding, eg 38 becomes 038. Look at:

http://www.openstreetmap.org/browse/node/2494176588

for an example.

housenumbers with leading zeros, or address ranges with leading zeros, should have leading zeros striped.

sort of like chicago:building_id, dug up this previous discussion:

https://groups.google.com/forum/#!topic/openstreetmap/kH_DwuYEWRU

I'm not super familiar with doing imports and on cursory inspection it looks like this isn't included right now, are there plans for this? It would be useful for joining with PLUTO/PAD, and i imagine this could be used as a stepping stone to crowdsourcing a better building age dataset (see comments on http://bdon.org/2013/09/12/building-age-nyc/)

What up New Yorkers, you have an official name no one uses? :)

What do you want to do here? I'm just going to leave at "Avenue of the Americas" as this is the official name but wanted to flag.

http://en.wikipedia.org/wiki/Sixth_Avenue_(Manhattan)

Issue: it's hard to determine whether an upload is properly done simply by looking at the ratio between time and number of objects created. In fact, it is possible to create large and clean uploads in short amounts of time. While some slower ones can be still quite messy.

Here are a couple of ideas for scripted changeset analysis that would help us flag uploads for manual inspection:

• Run JOSM validators on an upload (can you run JOSM from CLI?), flag on any non-role related errors.
• Flag if only building and address nodes where modified in an upload, but no other geometries were touched

None of the above thresholds are our measures of quality, but they are useful thresholds to raise a flag at.

/cc @iandees per our conversation last night
/cc @emacsen

There appears to be an incomplete export. I need to investigate more.

Area in question: City Island

Exported file:

buildings-addresses-82115.osm

Original data:

Needs more investigation.

From https://lists.openstreetmap.org/pipermail/imports/2013-September/002154.html

The sample OSM file is not clean in the JOSM validator. The row houses are not being merged correctly, they are missing shared nodes.

Sounds a bit like osmlab/dcbuildings#11

There were a couple nodes imported from GNIS (e.g. this fire station) that probably should have been merged with buildings during import.

This can be laborious, because you can't assume the building directly underneath the node is the one it should be merged with. For example, there were two nodes for a fire station that was a couple buildings down the block. I had to look up the name on the GNIS nodes to figure out which building they belonged on.

Normally I would assume that these are multiple entrances to the same building with different addresses, but they're located right next to each other. Without a survey, it's hard to know what to do with these.

node1 and node2

Sometimes there are extra nodes on an otherwise perfectly rectangular building. I checked for these by scanning up a block and comparing the position of the "meta node" handles -- they should all be pretty close together except in the case where there's an extra node that doesn't need to be there

You might consider running a simplification algorithm that only gets rid of completely superfluous nodes (i.e. where the angle between the line segments on either side of the node isn't very very close to 180°).

Before #28 was fixed, a series of changesets were uploaded where ordinal suffixes were missing:

East 4 Street instead of East 4th Street

Add missing ordinal suffixes on data that was uploaded before #28 was applied. Based on the fact that detecting and fixing ordinals is programmatically reliable, this could be an automated edit.

The OSM file for Rat Island loads a bunch of buildings that aren't in the bounding box.

The data imported by this changeset seems to be kind of crummy in the source NYC data. For example, this house (which looks like a single-family house) has two address nodes: node1 and node2.

The other houses in this area seem to have similar problems, but not consistently.

• Model after dcbuildings
• Capture entire process from download to output as chunked .osm files in the scripts
• Keep in mind that address data will change and might need a spatial join and not an id join to buildings.
• Find an appropriate shapefile for chunking out data (for DC we used census tracts, they're a bit big, there maybe something smarter in NYC)