The Housing Data Coalition (HDC) has received housing court filings data from the New York State Office of Court Administration (OCA). In this repository we manage the Extract-Transform-Load process for getting raw XML filings data from OCA via SFTP, parsing the nested XML data into a set of tables, and making those CSV files publicly available for download.

To work with these data you can use the NYCDB to automatically load all of the tables into a PostgreSQL database for analysis. You can also find documentation about the data, included a comprehensize data dictionary on the NYCDB wiki.

• oca_index
• oca_causes
• oca_addresses
• oca_parties
• oca_events
• oca_appearances
• oca_appearance_outcomes
• oca_motions
• oca_decisions
• oca_judgments
• oca_warrants

The data we receive from OCA is an extract of all landlord and tenant cases in NYC housing court, without personally identifying information. For more details about the raw data and the final parsed tables, see /docs.

For information about the details of various components, see /lib

First, you will only be able to run this yourself if you have HDC's credentials to access to the SFTP to get the raw data transfered from OCA and access to the private AWS S3 where those files are stored.

You will need Docker.

First, you'll want to create an .env file by copying the example one:

cp .env.example .env     # Or 'copy .env.example .env' on Windows

Take a look at the .env file and fill in the AWS S3 credentials.

To run the whole process in the docker container run:

docker-compose run app