betanyc / budgetbuddy Goto Github PK
View Code? Open in Web Editor NEWAn API for more intuitive access to NYC Budget Data, current and historical
Home Page: http://budgetbuddy.herokuapp.com
License: MIT License
An API for more intuitive access to NYC Budget Data, current and historical
Home Page: http://budgetbuddy.herokuapp.com
License: MIT License
Taken from: https://github.com/BetaNYC/budgetBuddy/blob/master/apiCalls.txt
Our Hierarchy looks like:
year/operatingBudget/agency/unitOfAppropriation/responsibilityCenter/budgetCode/objectclass/object
GET /:year/op
- citywide totals with agency totals rolled up.
GET /:year/op/detail
- full drill down - this is basically the whole database... no?
GET /:year/op/:agency
- just has the totals for the agency and roll ups for the next lower element in the hierarchy
GET /:year/op/:agency/detail
- full drill down
GET /:year/op/:agency/:uoa
- just this UOA and its children (responsibility centers)
GET /:year/op/:agency/:uoa/detail
- full drill down
...and so on...
Year to Year comparisons?
GET /:year/op/:agency/history?since=2012
Rollup Object Classes at the agency level:
GET /:year/op/:agency/?objectClass=01
This will return the sum of all object class 01 values in this agency
//how do we query multiples?
Hi, I've been using the scraper as I downloaded it June 29 to work with the adopted budget files, but I've been running into some problems where in some places the comma is misplaced and it looks like this in some places in the original csv:
This is from the adopted 2014 budget:
841,DEPARTMENT OF TRANSPORTATION,4000,"EXECUTIVE MANAGEMENT, TRAFFIC",Z030,PlaNYC Congestion Mitigation,10 SUPPLYS&MATL,1,00 S,UPPLIES + MATERIALS - GENERAL,ADOPTED BUDGEFY15,,AMOUNT,11881478,ss6_14.txt,128315
I was trying to compare the two years but there were mistakes because it didn't recognize that the value in one corresponded with the other.
Related, in another case it seems to have ignored a corresponding value in the modified budget column in that file, so for example it only listed this :
002,MAYORALTY,,,0244,Central Insurance Program,04 ADD GRS PAY,,042,LONGEVITY DIFFERENTIAL,ADOPTED BUDGET FY15,,AMOUNT,342,ss6_14.txt,295
when that same value also exists in the Modified section but it doesn't register for some reason.
This is only for a class (project due Thursday), so if this can't be fixed easily, I can do a more basic exercise with the existing converted data, but ideally I wanted to work with the adopted budget files and compare. My main objective at the moment is to get the files into CSV.format and then run some basic operations in Python/Pandas. Thanks!
Ideas are in /apiCalls.txt
Heroku app? What database? What backend framework?
Should we allow for CSV and JSON results?
FYI, the City will adopt a new budget by the end of this month (they had a handshake deal last Thursday. Should be adopted this week). So new data will be coming in shortly.
... one that works on "supporting schedules" budget PDFs for current and previous years.
I tried the scraper on converted to text versions of both this year's and last year's adopted budgets , and in each case got the error:Traceback (most recent call last):
File "parse.py", line 228, in scrape
'file_name': f.name
File "parse.py", line 188, in parse
data = self.line2dict(line)
File "parse.py", line 136, in line2dict
value = self.process_value(value)
File "parse.py", line 37, in process_value
return Decimal(v.replace(',', ''))
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/decimal.py", line 548, in new
"Invalid literal for Decimal: %r" % value)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/decimal.py", line 3872, in _raise_error
raise error(explanation)
InvalidOperation: Invalid literal for Decimal: '-'
Traceback (most recent call last):
File "parse.py", line 236, in
scrape(open(sys.argv[1], 'r'))
File "parse.py", line 232, in scrape
sys.stderr.write(e + u'\n')
TypeError: unsupported operand type(s) for +: 'InvalidOperation' and 'unicode'
Any thoughts?
-- Miranda N.
Volkan has suggested Swagger for API docs and console.
From an email thread:
The way I look at it, it's going to end up in PostgreSQL anyway when we deploy to Heroku. (We can swap it out later.) So we might as well keep the sqlite in the repo and consider that our development server. That would also make it easy for newcomers to get it up and running on their own.
(Volkan)
I meant to ask about that (we should have had this thread in an issue). What are the implications of leaving it in sqlite? What are the benefits of moving it to PostGres, especially since it's not that big and won't change at all except for adding more data a year from now.
(Chris)
We should do some work to get it on master without moving over the DB, as I'd prefer not to commit the sqlite binary (instead, we should have something that generates the DB from the pdf/text, most likely storing it in Postgres.)
(John)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.