Giter VIP home page Giter VIP logo

openaddresses's Introduction

OpenAddresses

Brief

A global collection of address, cadastral parcel and building footprint data sources, open and free to use. Join, download and contribute. We're just getting started.

This repository is a collection of references to address, cadastral parcel and building footprint data sources.

Contributing addresses

  • Open an issue and give information about where to find more address data. Be sure to include a link to the data and a description of the coverage area for the data.
  • You can also create a pull request to the sources directory.
  • More details in CONTRIBUTING.md.

Why collect addresses?

Street address data is essential infrastructure. Street names, house numbers, and post codes combined with geographic coordinates connects digital to physical places. Free and open addresses are rocket fuel for civic and commercial innovation.

Contributors

Code Contributors

This project exists thanks to all the people who contribute. [Contribute].

Financial Contributors

Become a financial contributor and help us sustain our community. [Contribute]

Individuals

Organizations

Support this project with your organization. Your logo will show up here with a link to your website. [Contribute]

License

The data produced by the OpenAddresses processing pipeline (available on batch.openaddresses.io) is not relicensed from the original sources. Individual sources will have their own licenses. The OpenAddresses team does its best to summarize the source licenses in the source JSON for each source. For example, the source JSON for the County of San Francisco contains a link to the County of San Francisco's open data license.

The source JSON in this repo (in the sources/ directory) is licensed under the CC0 1.0 Universal (CC0 1.0) Public Domain Dedication as described in the license file there. The rest of the repository is licensed under the BSD 3-Clause License.

openaddresses's People

Contributors

albarrentine avatar andrewharvey avatar ansis avatar arch0345 avatar astoff avatar bertday avatar davidchiles avatar garrett-w avatar geobrando avatar iandees avatar ingalls avatar jalbertbowden avatar josephkuchar avatar kelvinabrokwa avatar khobbs3 avatar lxbarth avatar mal315 avatar migurski avatar minicodemonkey avatar mmdolbow avatar orangejulius avatar palewire avatar pmacmaps avatar pre-commit-ci[bot] avatar riordan avatar sbma44 avatar slibby avatar stefanb avatar tlpinney avatar trescube avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

openaddresses's Issues

Datasets that cover multiple counties

Right now, when a dataset covers multiple counties it will be downloaded multiple times because it is listed multiple times. For example, Carver, Dakota, and Scott are all covered by the ftp://gisftp.metc.state.mn.us/Parcels2009.zip file. I think each source file should have its own JSON blob and we should do a better job specifying which areas are covered by that source file.

Enable travis

The config is here and it should 'just work' but travis-ci is slow in syncing my organization memberships today

Use tape's assertions

Right now this is console.logging in the style of tape - if we use assertions, it'll be wirable to travis and simpler. Will grab this.

Some servers do not support HEAD and fail in npm test

Will need to do a GET for tests for these particular sites...

var request = require('request');

var url = "https://sites.integritygis.com/Geocortex/Essentials/REST/sites/Adams_County_IA_Public/map/mapservices/0/layers/5";

var r = request.head(url, function(err, response) {
  console.log(response.statusCode);
});

var r2 = request.get(url, function(err, response, body) {
  console.log(response.statusCode);
});

One approach is to explicitly assign a server type. The other is to change the behavior based on "features" that the url has.

Split + move

Before we go live we should move everything to openaddresses/.

As we're doing this, I propose we also split the address repo from the tool and create a:

  • openaddresses/addresses // current /sources
  • openaddresses/tools // all code for downloading, testing and caching

@tmcw - looking for guidance on namespaces here. Ideally both repo names jive with node.js naming conventions as we'll want them to be npm modules too.

Chapel Hill NC address file is broken

When downloading file below, you'll get 22 bytes. Just sent an email to county.

city: Chapel Hill
county: Orange
data: "http://gis.townofchapelhill.org/GIS_Data/addresses.zip"
license: ""
year: ""

npm publish

The README advises to require('addresses') but this isn't published to the (available) addresses npm namespace. Someone want to push it?

Statewide dataset for Maine

I found a data source that has parcel data for the state of Maine.

http://www.maine.gov/megis/catalog/

Would it make sense to add something to the yaml like this?

- state: Maine 
  data: "http://www.maine.gov/megis/catalog/shps/state/parcelss.zip
  comment: organized towns
  date: 2014-01-28
- state: Maine
  data: "http://www.maine.gov/megis/catalog/shps/state/parcels_uts.zip"
  comment: unorganized towns
  date: 2014-01-23

Cache assets

  • Hash file for a fingerprint in manifest
  • If a fingerprint is different from an existing one, update
  • Add version numbers to manifest
  • Cache all assets to s3
  • Store Link to s3 in manifest

Most consumers of address data would use the cached file instead of the original files as they are more reliably available and faster to download. Updating cached files would be a more involved maintenance task.

Update / remove failing downloads

Run npm test and

  • go through all failing endpoints - update or remove (?)
  • go through all endpoints that don't list a data URL - update or remove (?)

Need to figure out whether we really want to remove, move to archive files or just flag addresses as out of date.

Multiple Sources that are not accessible

Allamakee County, IA
Ames, IA
Appanoose County, IA
Audrain County, MO
Audubon County, IA
Benton County, IA
Benton County, MN
Berrien County, MI
Blackford County, IN
Blue Earth County, MN
Boone County, IA
Bremer County, IA
Brookings County, SD
Brown County, SD
Carroll County, IN
Cass County, IA
Cass County, IL
Cass County, MO
Cerro Gordo County, IA
Cherokee County, IA
Chester County, SC
Chickasaw County, IA
City of Buffalo, MN
City of Foley, AL
City of Greenfield, IN
City of Luverne, MN
City of Valparaiso, IN
City of Wildwood, MO
Clay County, SD
Clayton County, IA
Clinton County, IA
Clinton County, IN
Corson County, SD
Cottonwood County, MN
Crawford County, AR
Crawford County, IN
Custer County, SD
Davis County, IA
Dearborn County, IN
Decatur County, IN
DeKalb County, IN
Delaware County, IA
Delaware County, IN
Dickinson County, IA
Dodge County, MN
Dubuque County, IA
Emmet County, IA
Faribault County, MN
Fayette County, IA
Floyd County, IA
Franklin County, AR
Franklin County, IA
Freeborn County, MN
Gibson County, IN
Grant County, IN
Greene County, MO
Grundy County, IA
Guthrie County, IA
Hancock County, IN
Hardin County, IA
Harding County, SD
Harrison County, IA
Hendricks County, IN
Henry County, IA
Henry County, IN
Houston County, MN
Howard County, IN
Huntington County, IN
Jackson County, IA
Jackson County, MN
Jasper County, IA
Jasper County, MO
Jefferson County, AR
Johnson County, IA
Johnson County, IN
Jones County, IA
Kanabec County, MN
Keokuk County, IA
Knox County, IN
Kosciusko County, IN
LaGrange County, IN
Lake County, SD
Lancaster County, SC
LaPorte County, IN
Le Sueur County, MN
Lee County, IA
Lincoln County, AR
Louisa County, IA
Lyon County, KS
Madison County, AR
Madison County, IA
Mahaska County, IA
Marion County, IA
Marshall County, IN
Martin County, MN
Mason City, IA
McCook County, SD
Miami County, KS
Mills County, IA
Montgomery County, IA
Montgomery County, IN
Moody County, SD
Morgan County, IL
Morrison County, MN
Newton County, IN
Nicollet County, MN
Noble County, IN
Nobles County, MN
Ogle County, IL
Page County, IA
Pettis County, MO
Pike County, MO
Pine County, MN
Pocahontas County, IA
Poinsett County, AR
Poweshiek County, IA
Putnam County, IN
Rice County, MN
Rush County, IN
Sac County, IA
Sebastian County, AR
Shelby County, IA
Sherburne County, MN
Sioux City, IA
Spink County, SD
Stearns County, MN
Steuben County, IN
Story County, IA
Taney County, MO
Tipton County, IN
Town of Avon, IN
Town of Brownsburg, IN
Town of Danville, IN
Town of McCordsville, IN
Town of Plainfield, IN
Union County, IA
Union County, SD
Vigo County, IN
Wabash County, IN
Wabasha County, MN
Warren County, IA
Watonwan County, MN
Wayne County, IN
Wells County, IN
Whiteside County, IL
Whitley County, IN
Wilson County, KS
Winnebago County, IA
Winneshiek County, IA
Woodbury County
Wright County, IA
Wright County, MN
Yankton County, SD

Notes on places

I have been searching for various data sources, and even though I am not able to find the exact one, I can find a point of contact, or I run into an issue where you need to sign up for an account. To limit duplicating of effort, it would be nice to have a way to attach notes as a way to build a simple workflow.

Is there going to be a way to add comments on places such as counties or cities?

Possible bad ssl certificate in one of the sources

openaddresses/node_modules/tape/index.js:75
        throw err
              ^
Error: UNABLE_TO_VERIFY_LEAF_SIGNATURE
    at SecurePair. (tls.js:1370:32)
    at SecurePair.EventEmitter.emit (events.js:92:17)
    at SecurePair.maybeInitFinished (tls.js:982:10)
    at CleartextStream.read [as _read] (tls.js:469:13)
    at CleartextStream.Readable.read (_stream_readable.js:320:10)
    at EncryptedStream.write [as _write] (tls.js:366:25)
    at doWrite (_stream_writable.js:223:10)
    at writeOrBuffer (_stream_writable.js:213:5)
    at EncryptedStream.Writable.write (_stream_writable.js:180:11)
    at write (_stream_readable.js:583:24)
npm ERR! Test failed.  See above for more details.
npm ERR! not ok code 0

Launching

Let's launch this sucker. Here's what I'm seeing outstanding:

  • Create a downloadable package #20
  • One source per file - See #64
  • Ideally cache assets #17
  • Split + move to openaddresses organization #62
  • Do a blog post

Anything I'm missing?

Broken files

Tested all files yesterday, fixed some. There are at least three that remain broken.

  • egis3-lacounty-gov-dataportal-wp-content-uploads-2012-06-lacounty_address_points
  • ewebmap-ci-lubbock-tx-us-currentdata-parcels-parcels.zip
  • manzano-cabq-gov-gisshapes-base.zip

Error when unzipping them with unzip:

End-of-central-directory signature not found.  Either this file is not
a zipfile, or it constitutes one disk of a multi-part archive

Simple web site

Add a very simple web site.

  • List available datasets
  • Form to submit address datasets (power with Google forms)

hardcode types.

The fuzzy type detection that looks for FeatureServer is inevitably going to come across a non-esri URL with that in it. Better to be explicit than vague here.

Store mime type and compression in manifest

Store the mime type and compression in the address manifest so data consumers can do basic stuff with data. E. g. decompressing, generating a concave hull and rendering it to a map for a coverage map.

Updating Cache

@lxbarth @iandees @tlpinney
Hey guys, just getting things up and running for our launch. If you guys could hold off on making any additions or redactions to the cache for the next little while that would be great!

1 source per file

Having a root>states.yml file is kind of weird - it would be better if this was either totally individual files, or one big file. Everywhere in between complicates tools and testing.

Store attributes of each file that correspond to a location element that would be useful to a geocoder

Different shapefiles and other data types can use different naming conventions for location information.

Using this as a reference
https://github.com/geocommons/geocoder/blob/master/doc/parsing.rst

Adding id for unique id of the row, fullstreet for cases like "1111 Long Trail Rd" and fulladdr for cases like "1111 Long Trail Rd, Anyville, TX, 99999"

Here is an example

https://github.com/tlpinney/addresses/blob/attributes/sources/us/Colorado.yaml#L26-L35

Should be easy to switch if we decide on a different naming convention.

One source is throwing a ETIMEDOUT error in npm test

One source is not being handled correctly when running npm test.

    openaddresses/node_modules/tape/index.js:75
        throw err
              ^
Error: ETIMEDOUT
    at null._onTimeout (openaddresses/node_modules/openaddresses-download/node_modules/request/request.js:677:15)
    at Timer.listOnTimeout [as ontimeout] (timers.js:110:15)
npm ERR! Test failed.  See above for more details.
npm ERR! not ok code 0

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.