Giter VIP home page Giter VIP logo

postcodesior's Introduction

PostcodesioR

PostcodesioR

Travis-CI Build Status Package-License Project Status: Active – The project has reached a stable, usable state and is being actively developed. CRAN_Status_Badge DOI Downloads

An API wrapper around postcodes.io - free UK postcode lookup and geocoder. This package helps to find and transform information about UK administrative geography like postcodes, LSOA, MSOA, constituencies, counties, wards, districts, CCG or NUTS.

The package is based exclusively on open data provided by postcodes.io. PostcodesioR can be used by data scientists or social scientists working with geocoded UK data. A common task when working with such data is aggregating geocoded data on different administrative levels, e.g. turning postcode-level data into counties or regions. This package can help in achieving this and in many other cases when changing the aggregation of geographic data is required.

Installation

This package can be installed from GitHub (developmental version) or CRAN (stable).

In order to install PostcodesioR use one of the following commands:

# stable version
install.packages("PostcodesioR")

or

# developmental version
if(!require("devtools")) {
  install.packages("devtools")
}
devtools::install_github("ropensci/PostcodesioR")

Loading

Load the package by typing

library(PostcodesioR)

Examples

Where possible, I tried to return a data frame. Unfortunately, a lot of API calls return more complex data and in those cases it is safer to use lists. The API limits the number of returned calls. Check functions’ documentation for more details.

For additional information about the returned data and the function calls see the original documentation.

The main function of this package provides information related to a given postcode

lookup_result <- postcode_lookup("EC1Y8LX")

#overview
str(lookup_result)
## 'data.frame':    1 obs. of  38 variables:
##  $ postcode                       : chr "EC1Y 8LX"
##  $ quality                        : int 1
##  $ eastings                       : int 532544
##  $ northings                      : int 182128
##  $ country                        : chr "England"
##  $ nhs_ha                         : chr "London"
##  $ longitude                      : num -0.0909
##  $ latitude                       : num 51.5
##  $ european_electoral_region      : chr "London"
##  $ primary_care_trust             : chr "Islington"
##  $ region                         : chr "London"
##  $ lsoa                           : chr "Islington 023D"
##  $ msoa                           : chr "Islington 023"
##  $ incode                         : chr "8LX"
##  $ outcode                        : chr "EC1Y"
##  $ parliamentary_constituency     : chr "Islington South and Finsbury"
##  $ admin_district                 : chr "Islington"
##  $ parish                         : chr "Islington, unparished area"
##  $ admin_county                   : logi NA
##  $ date_of_introduction           : chr "198001"
##  $ admin_ward                     : chr "Bunhill"
##  $ ced                            : logi NA
##  $ ccg                            : chr "NHS North Central London"
##  $ nuts                           : chr "Haringey and Islington"
##  $ pfa                            : chr "Metropolitan Police"
##  $ admin_district_code            : chr "E09000019"
##  $ admin_county_code              : chr "E99999999"
##  $ admin_ward_code                : chr "E05013699"
##  $ parish_code                    : chr "E43000209"
##  $ parliamentary_constituency_code: chr "E14000764"
##  $ ccg_code                       : chr "E38000240"
##  $ ccg_id_code                    : chr "93C"
##  $ ced_code                       : chr "E99999999"
##  $ nuts_code                      : chr "TLI43"
##  $ lsoa_code                      : chr "E01002704"
##  $ msoa_code                      : chr "E02000576"
##  $ lau2_code                      : chr "E09000019"
##  $ pfa_code                       : chr "E23000001"

Check the vignette to see all functions in action.

Citation

Please cite this package if it is used in a publication

Walczak, E. J., (2023). PostcodesioR: An R package for UK geocoding. Journal of Open Source Software, 8(84), 5334, https://doi.org/10.21105/joss.05334

BibTeX entry is here:

@article{postcodesior:2023,
  title = {{PostcodesioR: An R package for UK geocoding}},
  author = {Eryk J. Walczak},
  journal = {Journal of Open Source Software},
  volume = {8},
  number = {84},
  pages = {5334},
  year = {2023},
  doi = {10.21105/joss.05334},
  note = {R package version 0.3.1},
  url = {https://cran.r-project.org/web/packages/PostcodesioR/},
}

Notes

Currently, there is a limit to the number of API calls that can be made. However, postcodes.io provides full list of geolocation data that can be used locally without limitations. The original data is sourced from Office for National Statistics Data Portal. That file is rather large so I didn’t include it in the package.

Go to the package’s website or to my blog for more examples.

Code of Conduct

Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms.

ropensci_footer

postcodesior's People

Contributors

andreassoteriades avatar erzk avatar layik avatar robinlovelace avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

postcodesior's Issues

Help vignette for bulk_postcode_lookup is inaccurate

The help vignette for bulk_postcode_lookup() states that argument postcodes 'Accepts a list of postcodes'. This is untrue - I've just spent 15 minutes working out that it accepts a list of length one, whose sole element is a character vector of postcodes.

## This will work:
bulk_postcode_lookup( list( postcodes = c("PR3 0SG", "M45 6GN", "EX165BL")))

## This will not work:
bulk_postcode_lookup( list("PR3 0SG", "M45 6GN", "EX165BL"))

The documentation (with the exception of the examples) implies the latter is the correct syntax.

I'm not sure why the character vector needs to be wrapped in a list, but I'm treating this as a documentation error rather than a missing feature.

postcode_lookup failing unexpectedly

For some postcodes, I think the postcode_lookup() function is failing in (what I think) is an unexpected way.

Take the postcode "IM26BG" as an example. The postcode appears to be valid:

PostcodesioR::postcode_validation("IM26BG")
#> [1] TRUE

However, the postcode_lookup function fails in R:

PostcodesioR::postcode_lookup("IM26BG")
#> Error in (function (..., row.names = NULL, check.rows = FALSE, check.names = TRUE, : arguments imply differing number of rows: 1, 0

Passing the postcode into the 'Lookup a postcode' API endpoint on the official postcodes.io site reveals that this postcode returns null in many of the fields:

{
    "status": 200,
    "result": {
        "postcode": "IM2 6BG",
        "quality": 9,
        "eastings": null,
        "northings": null,
        "country": "Isle of Man",
        "nhs_ha": "Isle of Man",
        "longitude": null,
        "latitude": null,
        "european_electoral_region": null,
        "primary_care_trust": "Isle of Man",
        "region": null,
        "lsoa": null,
        "msoa": null,
        "incode": "6BG",
        "outcode": "IM2",
        "parliamentary_constituency": null,
        "admin_district": null,
        "parish": null,
        "admin_county": null,
        "admin_ward": null,
        "ced": null,
        "ccg": "Isle of Man",
        "nuts": null,
        "codes": {
            "admin_district": "M99999999",
            "admin_county": "M99999999",
            "admin_ward": "M99999999",
            "parish": "M99999999",
            "parliamentary_constituency": "M99999999",
            "ccg": "M01000001",
            "ccg_id": "YK1",
            "ced": "M99999999",
            "nuts": null
        }
    }
}

I believe this explains the R error, as it appears the lookup_postcode() function is failing when it finds null values (creating differing number of rows).

I think the lookup_postcode() function should convert null values to NA or fail with a more informative error message.

codecov.io push suggestion/issue

Might want to check ropensci/stats19 travis YML to send active pushes to codecov.io. I can see the covr package check locally being 89.51% would do with few more tests if need be.

Not urgent or necessary at all. Close it if you wish.

package is empty

After following your installation instructions and loading the library, no new functions are loaded.

> devtools::install_github("erzk/PostcodesioR")
> ls(2)
character(0)

bulk_postcode_lookup vignette returns 400 error

What

When user follows the vignette for bulk_postcode_lookup, Bad Request (HTTP 400) is returned, which is unexpected.

Steps to reproduce

  1. Install PostcodesioR
  2. Load library(PostcodesioR)
  3. Follow steps here:
    pc_list <- list(postcodes = c("PR3 0SG", "M45 6GN", "EX165BL"))
    bulk_lookup_result <- bulk_postcode_lookup(pc_list)
  4. Run
  5. See Warning message: In extract_results(r) : Bad Request (HTTP 400). printed in RStudio console

Other information

postcode_lookup("PR3 0SG"), postcode_lookup("M45 6GN") and postcode_lookup("EX165BL") returns the expected data for each single postcode

postcode_validation returns Bad Request error rather than FALSE

This is happening for the postcode F93 HF23.
Obviously this is not a real postcode but when I use postcode_validation() to check if it is valid I get the following warnings message:

Warning message: In extract_results(r) : Bad Request (HTTP 400).

I think it happens with any 'postcode' with 'H' as the first letter of the second section of the postcode. I think it should be returning FALSE instead?

bulk_postcode_lookup does not ignore spaces

The help file for bulk_postcodes_lookup() says that spaces are ignored. Spaces are ignored for postcode_lookup(), but not for bulk_postcode_lookup().

library(PostcodesioR)

postcode_lookup("S7 1FL")
#>   postcode quality eastings northings country                   nhs_ha
#> 1   S7 1FL       1   434790    385119 England Yorkshire and the Humber
#>   longitude latitude european_electoral_region primary_care_trust
#> 1 -1.478695 53.36182  Yorkshire and The Humber          Sheffield
#>                     region           lsoa          msoa incode outcode
#> 1 Yorkshire and The Humber Sheffield 042D Sheffield 042    1FL      S7
#>   parliamentary_constituency admin_district                     parish
#> 1          Sheffield Central      Sheffield Sheffield, unparished area
#>   admin_county date_of_introduction              admin_ward ced
#> 1           NA               198001 Nether Edge and Sharrow  NA
#>                   ccg      nuts             pfa admin_district_code
#> 1 NHS South Yorkshire Sheffield South Yorkshire           E08000019
#>   admin_county_code admin_ward_code parish_code parliamentary_constituency_code
#> 1         E99999999       E05010875   E43000173                       E14001467
#>    ccg_code ccg_id_code  ced_code nuts_code lsoa_code msoa_code lau2_code
#> 1 E38000146         03N E99999999     TLE32 E01008112 E02001652 E08000019
#>    pfa_code
#> 1 E23000011

bulk_postcode_lookup(list(postcodes = c("S7 1FL")))
#> Warning in extract_results(r): Bad Request (HTTP 400).

bulk_postcode_lookup(list(postcodes = c("S7 1FL", "S71FL"))) |> 
  lapply(\(x) is.null(x[["result"]]))
#> [[1]]
#> [1] TRUE
#> 
#> [[2]]
#> [1] FALSE

Created on 2024-03-05 with reprex v2.1.0

Radius parameter not working in nearest_postcode

Hi

I noticed the radius parameter was not having the desired effect in the nearest_postcode function. Possibly due to the extra ? in the query string below?

r <- GET(paste0("https://api.postcodes.io/postcodes/", postcode, "/nearest", "?limit=", limit, "?radius=", radius)) extract_results(r)

Missing postcode

I attempted a lookup using the list c("EC1Y8LX", "SW1X 7XL", "DE3 5LF", "DE3 9LR").
All were successful except for "DE3 5LF" for which a "Not Found (HTTP 404)" message was returned. I have since confirmed that "DE3 5LF" is a valid postcode.

Best regards,
Paul

Is there a 100 bulk limit?

Awesome package to start with!

I discovered via using the package that there is a 100 bulk lookup limit. Correct? If so, I am happy to contribute to make that clear rather than returning a In extract_results(r) : Bad Request (HTTP 400). error which lead me to sanity check my postcodes.

Thanks and hoping it is a valid "issue".

Submit to CRAN?

👋 Any plans to submit to CRAN? Anything we can do to help?

Only centroids available as opendata?

As an Openstreetmap mapper, use of postcode datasets comes up frequently, and usually ends up with people saying there's a doubt about licence compatibility. The underlying Codepoint Open is OGL, which is fine for Openstreetmap ODBL and others, but this is only for centroids of postcode areas. The Postcode Address File remains closed. I guess for a lot of uses this isn't an issue, but as many postcode areas are really odd shapes, and in some cases heights - I used to live in a tower block with 3 postcodes comprising different floors, so identical centroids, there could be some use cases that are problematic.

Feature request: bulk_postcode_lookup should accept a character vector and/or `...`

bulk_postcode_lookup() requires postcodes to be a length-one list that contains only a character vector, as exemplified in the function's help page:

pc_list <- list(postcodes = c("PR3 0SG", "M45 6GN", "EX165BL"))
bulk_postcode_lookup(pc_list)

I'm continually caught off-guard by this counter-intuitive requirement. What is the purpose of the list?

Please implement support for accepting a character vector and/or ..., like so:

bulk_postcode_lookup(c("PR3 0SG", "M45 6GN", "EX165BL"))
bulk_postcode_lookup("PR3 0SG", "M45 6GN", "EX165BL")

Those inputs could be coerced into list form under the hood so that the functionality of old code is preserved.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.