Giter VIP home page Giter VIP logo

spew2synthia's Introduction

SPEW to Synthia

What you'll need

  • access to host spew.olympus.psc.edu or similar
  • read access to path /mnt/lustre0/machines/data.olympus.psc.edu/srv/apache/data/syneco/spew_1.2.0

Stack

  • Python[tested with 3.5.1]
  • Shapely: to translate from WKBto Latitude and Longitude. [tested with 1.5.17]

Run on spew.olympus.psc.edu

To load python:

module load python/3.5.1

To go to the script directory:

cd spew2synthia

Only for the first time

To install Shapely as a user if it is not available globally:

pip3 install --user shapely

For USA states

To translate all states in the USA:

python3 state.py

Note:

  • Output files will be generated in directory ./populations/2010_ver1_<state FIPS #>
  • Log files will be generated in ./logs
    • Overall stdout will be logged into file states.< timestamp >.
    • All stdout of each state will be logged into file <state FIPS #>.out. Note: if the .out file exists, the script will skip the state.
    • If there is any error, it will be logged into file <state FIPS #>.err.

For USA counties

To translate all counties in the USA:

python3 county.py

Note:

  • Output files will be generated in directory ./populations/2010_ver1_<county FIPS #>
  • Log files will be generated in ./logs
    • Overall stdout will be logged into file counties.< timestamp >.
    • All stdout of each state will be logged into file <state FIPS #>/<county FIPS #>.out. Note: if the .out file exists, the script will skip the county.
    • If there is any error, it will be logged into file <state FIPS #>/<county FIPS #>.err.

For other countries (excluding USA)

To translate all countries using IPUMS and Canada data sources:

python3 country.py

Note:

  • Output files will be generated in directory ./populations/spew_1.2.0_<country ISO 3166-1 alpha-3>
  • Log files will be generated in ./logs
    • Overall stdout will be logged into file countries.< timestamp >.
    • All stdout of each state will be logged into file <country ISO 3166-1 alpha-3>.out. Note: if the .out file exists, the script will skip the county.
    • If there is any error, it will be logged into file <country ISO 3166-1 alpha-3>.err.

spew2synthia's People

Contributors

johngrefenstette avatar gallowdd avatar sahawut avatar fred-publichealth-pitt-edu avatar maverickloneshark avatar mgk8 avatar terencesperringerjr avatar

Watchers

James Cloos avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.