nycplanning / recipes Goto Github PK
View Code? Open in Web Editor NEW🍣 🍥 🍙 🍘 🍚 🍜 🍲 🍢 🍡 🥚 🍞 🍩
recipes's People
Contributors
Stargazers
Watchers
recipes's Issues
[ztl] Zoning Tax Lots Bulk Load
Make sure all source files are already uploaded to S3, also make sure you close out the issue after a successful run! Thanks
Fix dpr_capitalprojects json to pandas
Each of the records contained in 'Locations' should be a row. Currently, a row is an FMSID, with json for all records in the Locations column. We are only pulling the first location, in lines 16-17
time change for DOB weekly pull
change the pull time to Monday at midnight each week
dof cama file cannot load
issues:
- gdal couldn't recognize txt file
- no headers
solution:
- consider preprocessing
- both pts and cama might need special treatment
[ztl] Zoning Tax Lots Bulk Load
This is Adrian and I confirm that I uploaded all the files needed for this loading as of 12/25/2020
Make sure all source files are already uploaded to S3, also make sure you close out the issue after a successful run! Thanks
Maintenance: Spatial boundary standardization
Determine which include water, and standardize how we refer to it
| Schema | Water Included |
|---|---|
| dcp_boroboundaries_wi | Yes |
| dcp_cdboundaries | Yes |
| dcp_cdboundaries_clipped | No |
| dcp_censusblocks | Yes |
| dcp_censustracts | Yes (see note) |
| dcp_congressionaldistricts | No |
| dcp_councildistricts | Yes |
| dcp_puma | No |
| dcp_policeprecincts | No |
| dcp_school_districts | No |
| dcp_firecompanies | No |
| dcp_healthareas | No |
Notes:
- dcp_censustracts.”20c” actually has data for no-water-included blocks data!
- dcp_congressionaldistricts does not have latest view, and is versioned by upload date
Go through workflows to make sure we’re pulling the correct one
Maintenance: Clean up recipes.csv for out-of-date and no-longer-relevant recipes
| Schema name | Uses python script to load | Requires dispatch |
|---|---|---|
| dcp_zoningmapamendments | ||
| dcp_limitedheight | ||
| dcp_specialpurpose | ||
| dcp_specialpurposesubdistricts | ||
| dcp_zoningdistricts | ||
| dcp_commercialoverlay | ||
| dof_dtm | ||
| dof_shoreline | ||
| dof_condo | ||
| dcp_mih | ||
| dcp_edesignation | ||
| dcp_zoningtaxlots | ||
| dcp_ntaboundaries | ||
| doitt_buildingfootprints | ||
| dcas_ipis | ||
| dcp_school_districts | ||
| dcp_boroboundaries_wi | ||
| dcp_councildistricts | ||
| dcp_censusblocks | ||
| dcp_zoningmapindex | ||
| dcp_cdboundaries | ||
| lpc_historic_districts | ||
| lpc_landmarks | ||
| dcp_firecompanies | ||
| dcp_policeprecincts | ||
| dcp_healthareas | ||
| dcp_healthcenters | ||
| dcp_zoning_maxfar | ||
| pluto_input_bsmtcode | ||
| pluto_input_landuse_bldgclass | ||
| pluto_input_condo_bldgclass | ||
| pluto_input_cama_dof | ||
| dof_shoreline | ||
| dof_condo | ||
| fema_firms2007_100yr | ||
| fema_pfirms2015_100yr | ||
| pluto_input_condolot_descriptiveattributes | ||
| dsny_frequencies | ||
| dcp_censustracts | ||
| dcas_facilities_colp | ||
| dob_permitissuance | ||
| dob_jobapplications | ||
| dob_cofos | ||
| housing_input_lookup_occupancy | ||
| housing_input_lookup_status | ||
| housing_input_dcpattributes | ||
| housing_input_hny | ||
| dcp_mappluto | ||
| dcp_mappluto_unclipped | ||
| dcp_pluto | ||
| pluto_corrections | ||
| housing_input_hny_job_manual | ||
| housing_input_removals | ||
| hpd_hny_units_by_building | ||
| hpd_hny_units_by_project | ||
| transitzones | ||
| inclusionary_housing | ||
| waterfront_access_plan | ||
| fresh_zones | ||
| coastal_zone_boundary | ||
| ldgma | ||
| upland_waterfront_areas | ||
| appendixj_designated_mdistricts | ||
| doe_all_proposals | ||
| sca_capital_projects | ||
| ctpp_journey_to_work | ||
| ctpp_mode_split_ny | ||
| uscensus_ct_shp | ||
| uscensus_nj_shp | ||
| uscensus_ny_shp | ||
| sca_bluebook | ||
| melissa_input | ||
| melissa_corrections | ||
| melissa_outsideofnyc | ||
| nysdec_title_v_facility_permits | ||
| nysdec_state_facility_permits | ||
| dep_cats_permits | ||
| dcp_lion | ||
| doitt_elevation_points | ||
| doitt_transportation_structures | ||
| doitt_railroad_structures | ||
| nysdot_functional_class | ||
| nysdot_aadt | ||
| nysdot_traffic | ||
| dot_traffic_cameras | ||
| cscl | ||
| sca_e_projections | ||
| sca_e_pct | ||
| nysdec_air_monitoring_stations | ||
| doe_school_locations | ||
| nysparks_historicplaces | ||
| nysparks_parks | ||
| doitt_zipcodeboundaries | ||
| doitt_buildingcentroids | ||
| facilities_classification | ||
| dcp_facilities | ||
| tunnel_ventilation_towers | ||
| dcp_areas_of_concern | ||
| zipcode_city_lookup | ||
| dcp_municipalcourtdistricts | ||
| nypd_policeprecincts | ||
| dcp_stateassemblydistricts | ||
| dcp_statesenatedistricts | ||
| dpr_capitalprojects | ||
| dot_projects_bridges | ||
| dot_projects_streets | ||
| dot_projects_intersections | ||
| ddc_capitalprojects_infrastructure | ||
| ddc_capitalprojects_publicbuildings | ||
| edc_capitalprojects | ||
| edc_capitalprojects_ferry | ||
| fisa_capitalcommitments | ||
| dcp_cpdb_agencyverified | ||
| dcp_congressionaldistricts | ||
| fdny_firecompanies | ||
| acs_daycareheadstart | ||
| bpl_libraries | ||
| dca_operatingbusinesses | ||
| dcas_colp | ||
| dcla_culturalinstitutions | ||
| dcp_pops | ||
| dcp_sfpsd | ||
| dep_wwtc | ||
| dfta_contracts | ||
| doe_busroutesgarages | ||
| doe_lcgms | ||
| doe_universalprek | ||
| dohmh_daycare | ||
| dot_bridgehouses | ||
| dot_ferryterminals | ||
| dot_mannedfacilities | ||
| dot_pedplazas | ||
| dot_publicparking | ||
| dpr_parksproperties | ||
| dsny_mtsgaragemaintenance | ||
| dycd_afterschoolprograms | ||
| fbop_corrections | ||
| fdny_firehouses | ||
| foodbankny_foodbanks | ||
| hhc_hospitals | ||
| hra_centers | ||
| moeo_socialservicesiteloactions | ||
| nycdoc_corrections | ||
| nycha_communitycenters | ||
| nycha_policeservice | ||
| nycourts_courts | ||
| nypl_libraries | ||
| nysdec_lands | ||
| nysdec_solidwaste | ||
| nysdoccs_corrections | ||
| nysdoh_healthfacilities | ||
| nysdoh_nursinghomes | ||
| nysed_activeinstitutions | ||
| nysoasas_programs | ||
| nysomh_mentalhealth | ||
| nysopwdd_providers | ||
| nysparks_historicplaces | ||
| nysparks_parks | ||
| qpl_libraries | ||
| sbs_workforce1 | ||
| uscourts_courts | ||
| usdot_airports | ||
| usdot_ports | ||
| usnps_parks | ||
| zipcode_borough_lookup | ||
| dcp_atomicpolygons | ||
| dcp_addresspoints | ||
| acs_demographic | ||
| acs_economic | ||
| acs_housing | ||
| facilities_input_research | ||
| cbbr_submissions | ||
| sca_housing_by_sd | ||
| housing_input_research | ||
| dcp_pad | ||
| dcp_knownprojects | ||
| doe_eszones | ||
| doe_mszones | ||
| doe_hszones | ||
| doe_school_subdistricts | ||
| sca_capacity_projects_current | ||
| sca_capacity_projects_prev | ||
| em_buildings_composite | ||
| esd_projects | ||
| edc_projects | ||
| edc_sca_inputs | ||
| edc_dcp_inputs | ||
| dcp_n_study | ||
| dcp_n_study_future | ||
| dcp_n_study_projected | ||
| hpd_rfp | ||
| hpd_pc | ||
| dcp_rezoning | ||
| cbbr_agency_updates | ||
| dcp_planneradded | ||
| test | ||
| dpr_access_zone |
Maintenance: Clarify which recipes require a dispatch and which are app-loadable
dcas_colp load nycCOLPall_2018.shp from zip file
Download ESRI Shapefiles
https://www1.nyc.gov/assets/planning/download/zip/data-maps/open-data/nyc-colp-shp-201811.zip
There will be 10 shapefiles in the unzipped folder
Use nycCOLPall_2018.shp
[ztl] Zoning Tax Lots Bulk Load
Make sure all source files are already uploaded to S3, also make sure you close out the issue after a successful run! Thanks
1125 record have a value in the geom source filed but no geometry. This shouldn't happen.
SELECT * FROM cpdb_dcpattributes where geomsource is not null and geom is null;
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.