Giter VIP home page Giter VIP logo

nflgame's Introduction

Downloads

I decided to abandon my efforts on this project and have begun looking for a new owner. You can read up on the details here, but tl;dr; I'm a bitter coder looking for a non-bitter coder to give this project love.

A python3 fork of... nflgame

Currently released under nflgame-redux drop-in replacement for nflgame so as to easily use with projects like nfldb.

Purpose

nflgame is an API to retrieve and read NFL data feeds, HOWEVER, the nfl has nuked the feeds this project relies on effectively killing it over night. There has been a concerted effort to include the historic data, but going forward, unless this project gets a new maintainer, it will remain in this state.

Installation

this project is no longer python2 compatible.. The old python2 (<1.2.20) branch should work but... come on now... just upgrade.

  1. Create a && activate a python3 venv

    python3 -m venv ~/python3
    source ~/python3/bin/activate
    
  2. In your python 3 virtualenv...

pip install nflgame-redux
  1. Update players
nflgame-update-players

Documentation and getting help

If you need help, please come visit us in discord on channel https://discord.gg/3cHXbRp.

If you aren't a programmer, then the tutorial for non programmers is for you!

Also, nflgame has decent (but not perfect)API documentation. If you're just looking around, make sure to look at the submodules as well.

Feel free to open a new issue on the tracker, which is currently the most expedient way to get support.

How it works

nflgame works by parsing the same JSON data that powers NFL.com's live GameCenter. Therefore, nflgame can be used to report game statistics while a game is being played.

The package comes pre-loaded with game data from every pre- and regular season game from 2009 up until the present (I try to update it every week). Therefore, querying such data does not actually ping NFL.com.

However, if you try to search for data in a game that is being currently played, the JSON data will be downloaded from NFL.com at each request (so be careful not to inspect for data too many times while a game is being played). If you ask for data for a particular game that hasn't been cached to disk but is no longer being played, it will be automatically cached to disk so that no further downloads are required.

Here's a quick teaser to find the top 5 running backs by rushing yards in the first week of the 2013 season:

import nflgame

games = nflgame.games(2013, week=1)
players = nflgame.combine_game_stats(games)
for p in players.rushing().sort('rushing_yds').limit(5):
    msg = '{} {} carries for {} yards and {} TDs'
    print( msg.format(p, p.rushing_att, p.rushing_yds, p.rushing_tds))

And the output is:

L.McCoy 31 carries for 184 yards and 1 TDs
T.Pryor 13 carries for 112 yards and 0 TDs
S.Vereen 14 carries for 101 yards and 0 TDs
A.Peterson 18 carries for 93 yards and 2 TDs
R.Bush 21 carries for 90 yards and 0 TDs

Or you could find the top 5 passing plays in the same time period:

import nflgame

games = nflgame.games(2013, week=1)
plays = nflgame.combine_plays(games)
for p in plays.sort('passing_yds').limit(5):
    print p

And the output is:

(DEN, DEN 22, Q4, 3 and 8) (4:42) (Shotgun) P.Manning pass short left to D.Thomas for 78 yards, TOUCHDOWN. Penalty on BAL-E.Dumervil, Defensive Offside, declined.
(DET, DET 23, Q3, 3 and 7) (5:58) (Shotgun) M.Stafford pass short middle to R.Bush for 77 yards, TOUCHDOWN.
(NYG, NYG 30, Q2, 1 and 10) (2:01) (No Huddle, Shotgun) E.Manning pass deep left to V.Cruz for 70 yards, TOUCHDOWN. Pass complete on a fly pattern.
(NO, NO 24, Q2, 2 and 6) (5:11) (Shotgun) D.Brees pass deep left to K.Stills to ATL 9 for 67 yards (R.McClain; R.Alford). Pass 24, YAC 43
(NYG, NYG 20, Q1, 1 and 10) (13:04) E.Manning pass short middle to H.Nicks pushed ob at DAL 23 for 57 yards (M.Claiborne). Pass complete on a slant pattern.

I want a database!

Great news! I've pulled nfldb into python3 and it comes with nflgame 2.0 already installed. It even has an example docker image that should get everyone up and running pretty quick.

Other related projects of Andrew Gallant's...

nflvid with nfldb to search and watch video of plays.

Updating the player json (e.g., rosters)

Since player meta data (like a player's team, position or status) changes throughout the season, the JSON database included with nflgame needs to be updated occasionally. While I try to update it and push out new releases weekly, you can also update the database by running the following command:

nflgame-update-players

It will send at least 32 requests (and usually not much more than that) to NFL.com and update the JSON player database in place by default. I tend to run it every 12 hours or so. This is the only piece of nflgame that relies on web scraping.

Loading data into Excel

Every sequence of players can be easily dumped into a file formatted as comma-separated values (CSV). CSV files can then be opened directly with programs like Excel, Google Docs, Open Office and Libre Office.

You could dump every statistic from a game like so:

game.players.csv('player-stats.csv')

Or if you want to get crazy, you could dump the statistics of every player from an entire season:

nflgame.combine(nflgame.games(2010)).csv('season2010.csv')

Contributing (WIP)

  • All active development takes place on the "dev" branch. This is where pull requests should be submitted against.
  • Bug fixes for released versions should be submitted against "master" branch and will get merged accordingly.
  • This project will stick to Semantic Versioning
  • Tests are greatly encouraged but not required.

Authored by Andrew Gallant maintained by Derek Adair

nflgame's People

Contributors

benschroeder avatar burntsushi avatar cjmcdonn avatar cminton avatar cmulcrone avatar derek-adair avatar front9tech avatar hugovk avatar jclapp23 avatar jthomm avatar leighlondon avatar mickeelm avatar mlderes avatar ochawkeye avatar panzarino avatar sebasttj avatar smehlhoff avatar snake-plissken avatar stchiang avatar tddouglas avatar weixiyen avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

nflgame's Issues

Issue Template

Anyone have a template they can recommend from another project?

Can this fork be used with nfldb?

I'm trying to get nfldb working but as you already know, it's broken with current season. I saw this new repo and got excited, but I see it is only handling nflgame, and not nfldb yet. I did notice that you forked nfldb as well, but there are new commits there yet. Are you planning on updating it to work with this anytime soon?

Thanks!

Finding the data of next plays.

Does anyone have an idea how to find say the yardline of next play given the play_id? Of course looping through drive is a solution but I would like to use play only.

having issues with find_all error line 415

Hello,

I recently replaced all of the burnt sushi files with dereks, everything on the schedule side is good but whenever I try to update the players, I get this message:

File "Z:\Python27\lib\site-packages\nflgame\update_players.py", line 415, in run
tbodys = soup.find(id='result').find_all('tbody')
AttributeError: 'NoneType' object has no attribute 'find_all'

and then python FCs any idea on how to fix?

Thanks

Creating a full season game log

I'm trying to become familiar with this package, but am having some issues creating a full season game log (individual stats for each player for each week of the season). Here is the code I have, which seems like it would work but it takes a very long time. If anybody could help that would be great

emptydf = pd.DataFrame(columns = uniquekeys)
week1 = 1
year1 = 2009
numrecords = 0

for y in range(year1, 2018):
  for w in range(week1, 18):
    games = nflgame.games(y,w)
    for g in games:
      hometeam = g.home
      awayteam = g.away
      for p in g.max_player_stats():
        numrecords = numrecords + 1
        playername = p.name
        playerteam = p.team
   # playerpos = p.position
    #print(type(p.stats))
        tempdf = pd.DataFrame(p.stats, index = [numrecords])
        tempdf.insert(0,'Name',playername)
        tempdf.insert(1,'Week',week)
        tempdf.insert(2,'Year',year)
        tempdf.insert(3,'Team',playerteam)
        tempdf.insert(4,'Home',hometeam)
        tempdf.insert(5,'Away',awayteam)
    #tempdf.insert(6,'Position',playerpos)
        emptydf = pd.concat([emptydf,tempdf]).fillna(0)
  print('Year % Complete' % y)
 

Include update_sched.py in pypi?

Currently only update_players is being exposed as executable in the pypi package. Should we add update_schedule? Is this a moot point in light of the other scheduling changes in the works?

Switching from nflgame to nflgame-redux

If I already have nflgame installed and I install nflgame-redux, will running nflgame-update-players pull in the 2018 rosters? Just trying to get a sense of how much configuration is needed to switch over from nflgame to nflgame-redux. Thanks for all of the work on this by the way. I'd love to try to contribute on the python3 port.

PG_Config Executable not found

Hi All,

I've taken upon the task of setting up this database and using it for my fantasy league. I have zero experience programming, not familiar with python, but thought it would be fun to learn.

I followed this guide and have gotten to pretty much the final steps. The issue I'm having however, is that when I try to install nfldb, it spits back an error at the end of the install:

Error: pg_config executable not found.

pg_config is required to build psycopg2 from source. Please add the directory

containing pg_config to the $PATH or specify the full executable path with the

option:

python setup.py build_ext --pg-config /path/to/pg_config build ...

or with the pg_config option in 'setup.cfg'.

If you prefer to avoid building psycopg2 from source, please install the PyPI

'psycopg2-binary' package instead.

For further information please check the 'doc/src/install.rst' file (also at

http://initd.org/psycopg/docs/install.html).

I have no idea what that means at all. I've tried googling the subject, and I can't seem to grasp exactly what is going on here.

I've also attached the piplog so you can see the log for yourself.

Any advice for a newbie?

Thanks everyone
pip.log

Player records not updating

When I run the nflgame-update-players I'm not updating players with their new 2018 teams. This is the response I get:

Downloading team rosters...
32/32 complete. (100.00%)
Done!
Fetching GSIS identifiers for players not in nflgame...
1/1 complete. (100.00%)
Done!


There were some errors during the download. Usually this is a
result of an HTTP request timing out, which means the
resulting "players.json" file is probably missing some data.
An appropriate solution is to re-run the script until there
are no more errors (or when the errors are problems on 
NFL.com side.)
-------------------------------------------------------------------------------
Could not get GSIS id at http://www.nfl.com/player/cedwilson/2560936/profile
-------------------------------------------------------------------------------
Could not find gsis_id for {'status': u'RES', 'last_name': u'Wilson', 'weight': 197, 'profile_id': 2560936, 'number': 16, 'height': 74, 'college': u'Bowie State', 'full_name': u'Ced Wilson', 'first_name': u'Ced', 'years_pro': 0, 'birthdate': u'3/10/1995', 'team': 'DAL', 'position': u'WR', 'profile_url': 'http://www.nfl.com/player/cedwilson/2560936/profile'}

Is the non-existent GSIS id for Ced Wilson preventing other players from being updated?

New data feeds found

So after researching a bit, I (my girlfriend actually, give credit where credit is due) stumbled upon some interesting feeds on nfl.com.

https://feeds.nfl.com/feeds-rs/schedules.json - great for nflgame. This file contains the whole of the 2018 season (from HOF to REG17). This means that we could still be restrictive with calls to NFL.com for what it's worth (once a day or so) but we'll get the whole season every time we call so any changes made to a game ten weeks ahead would be included. Maybe they'll fill up with postseason games in the same file, we don't know yet. This one also has more info for each game than its old counterpart. A great addition is the ISO-time-field for kickoffs which is awesome for our purposes. Today's feeds only gives the 12-hour time which is a bit tricky when it comes to Pacific time vs London time games, as can be seen in the code. With ISO-times there's no confusion whatsoever.

https://feeds.nfl.com/feeds-rs/scores.json - feeds the score bar on nfl.com it seems. Was previously http://www.nfl.com/liveupdate/scorestrip/scorestrip.json and an equivalent for the postseason. Despite the name, these weren't valid JSON. The new one is, and contains more info (for instance, current position of the ball, points in each quarter and remaining timeouts). This one doesn't matter too much for nflgame in the way it is written now, the game data is taken from the respective game's pages anyway (which contain more detail and every play, not just the current). If one is interested in a lighter live-function that gives you pretty much all info of right now in all games without the underlying player stats it could be interesting (one call vs up to 16 calls) to develop. Depends on the use case.

https://feeds.nfl.com/feeds-rs/bigPlayVideos.json - havent looked into Andrew's nflvid-project myself but yeah, the big play highlights are all collected here with URLs to the videos and other stuff. Easy to link if that should be desirable.

A thing to consider with all these feeds is that they all support the Last-Modified header (including the game feeds that we use today). None of the files mentioned are massive, but it's possible to just check with a HEAD request if a resource is modified or not before doing an actual (possibly unnecessary) GET of it. They all support gzip too (the impact of enabling it probably isn't gonna be huge but still, wouldn't hurt).

Also, since there seems to be some confusion on whether what we are using is an API, if the NFL has an API or not...I thought this'd be a good place to clear that out:

Does the NFL have an API? Yes they do. Can we use it for nflgame? No we cant.

The yes part: api.nfl.com does exist and whenever you visit a page on nfl.com then API calls will be made there (as well as to the feeds mentioned above, that's how we found them, just use an inspector tool in your browser and you'll see it).

The no part: If you try to make a call yourself, for example via curl or just posting it in the browser, you'll get a permission denied (there are some open calls, but nothing of value that I've found). There is a public documentation of the API. https://api.nfl.com/docs/getting-started/index.html. From what I understand after reading around a bit and checking Stack Overflow-threads on the matter, it seems like it is not easily available but more a commercial or special case-thing. Nothing you can use straight away at the moment and for it to be used here, everyone would probably need their own account and I can't really rule out if it's even possible to get one.

So what are we using? We are using data feeds/resources that simply are available from NFL.com. They are not part of an API, they are just there. Why the NFL has chosen to not lock them down (like the actual API) we don't know.

Calculating score during historical games

Hey folks!

I'm new here, so sorry if the answer to this is obvious.

I'm trying to use nflgame to do some statistical analysis on historical games. Part of the analysis requires knowing the score of the game at different times. For instance, I need to know what the score was at halftime, and also what it was after the end of the first drive after the half.

Looking at a particular game, I can easily find the score at the end of the game. However, it doesn't seem to be recorded for individual quarters or the half. Looking at drives, I can even determine the drive.result (TD, punt, interception, etc). However, there isn't any data recording the actual score at the end of the drive.

I thought about calculating this myself using the result field of all drives that occurred before the one I'm interested in, but I noticed that the field is fairly limited. A result is still a TD even if the extra point is missed. I also wonder what would happen with pick-sixes, safeties, or extra points that get returned the other way.

Any ideas on how I can find this information?

Play data doesn't update after game closes

Looking over the data from 2018 week 1, I'm seeing some things that don't quite line up. Check out play 3673 in 2018091001.json. This play was a 2 yard gain for Todd Gurley. However, he is listed further down in one of the sequence nodes with 32 yards.

While this may not be visible in nflgame, that data flows into nfldb as incorrect data, so if you're trying to do any aggregate queries there, they will not be correct.

Any ideas here?

image

Should we remove historical data from nflgame?

Just spitballing. Serious implications with this idea, but what if we leave the historic data / records / searching PURELY to nfldb.

This project becomes more focused on what its good at, digging out data from NFL's shitty api's.

This allows us to;

  1. Separate some concerns between nflgame and nfldb. Currently nflgame and nfldb have overlap. This is confusing to end users unfamiliar with these projects and... more importantly... is duplicated work.
  2. Make nflgame teeny tiny

Automatic schedule updates

I would like to be lazy and not have to run update_schedule.py --year every time the schedule changes, a game is added or a new season starts.

This is one of most common issues submited to burnsushi/nflgame, "Why cant i see this years games?"I need to drudge up the tickets, but at some point I brought this up and there were some surface level discussion about how this may work. Just getting the ball rolling on this because i feel like it could have a domino effect on a lot of this project.

Any thoughts on this? Does anyone have a clever solution that I am missing?

git conventions / contribution guidelines

Also looking for suggestions here.

Things to agree upon;

  • .gitignore
  • Are we picky about how pull requests work?
  • Any python conventions?
  • Should we start asking for tests w/ pull requests? ๐Ÿ˜ˆ

PULL_LATEST_SCHEDULE config option?

A very common question throughout all nfl projects is, "why can't i see the next seasons schedule??".*

A way to eliminate this is to just have nflgame provide the latest schedule given a flag PULL_LATEST_SCHEDULE. This would allow new users to query future seasons games without manually running update_sched.py --year 2018.

I suspect this was done to prevent accident abuse of the nfl.com json stream, so some planning/research is needed to make sure this wont set off any red flags for nfl.com.

feeds-rs on nfl.com contains all you could possibly ask for

So after some friendly help over at Reddit, turns out that what I posted in #28 really was only the tip of the iceberg. Not even that.

http://www.nfl.com/feeds-rs?_wadl

If you check that link, you'll get all the endpoints that are under feeds-rs. There are dozens and dozens more. Just some examples:

#The 1983 NFL teams (goes back to 1970)
https://feeds.nfl.com/feeds-rs/teams/1983.json

#The current roster of the Arizona Cardinals
https://feeds.nfl.com/feeds-rs/roster/3800.json

#All draft picks in the 4th round of 2015
https://feeds.nfl.com/feeds-rs/draft/byRound/4/2015.json

#Player stats on Khalil Mack (one of the many player-related endpoints, depending on what kind of info you want and how you want it presented/filtered)
https://feeds.nfl.com/feeds-rs/playerStats/2543463.json

#Player transactions for the Denver Broncos in september:
https://feeds.nfl.com/feeds-rs/transactions/team/1400/2018/9.json

#Colts@Patriots 2 weeks ago
http://www.nfl.com/feeds-rs/boxscorePbp/2018100400.json

And the list goes on, and on, and on. You can get info on officials, blog posts, free agency...

In the way nflgame is used now, this doesn't really change things drastically. There are things that could be simplified though, especially the update-player logic which is pretty advanced and relies on web scraping. So that would benefit from this.

I gladly admit when I'm wrong, I think it is safe to say that the NFL does have an API (apart from api.nfl.com), this is way more than a few "accidental" feeds.

Install Issues

It's giving me the following error when trying to install on a Raspberry Pi. I'm not the most advanced experienced programmer so something simple may be the issue. I have installed the original NFLGame in the past, looking to upgrade to this version for the 2018 schedule updates
screen shot 2018-09-09 at 11 11 11 am

Expand updating of schedule logic

Today the update schedule-logic checks the current week, if not explicitly called with arguments/flags to get other weeks.

As per the discussion in #12, we wish to expand this functionality in an attempt to decrease the need for having to run this script manually with flags/arguments. Like this:

  • Check which schedules we have in schedule.json for the current season.
  • If we don't have any schedules for the season, check if PRE0 exists. If not, abort. If PRE0 exists, store PRE0. Check for PRE1. If PRE1 does not exist, abort. If PRE1 exists, store PRE1. Continue like this until all schedules* are fetched (if not aborted).
  • If we have some schedules for the season but not all, try to get all missing schedules for the season. Regardless if existing or not, get the current week.
  • If we have all schedules*, only check the current week.

*) During the period 1st of March to Monday morning (when the late games should have ended) after week 17's games which are all on the Sunday of week 17, "all schedules" means PRE0-PRE4 and REG1-REG17. Not the postseason. The morning after regular season is concluded, "all schedules" includes the postseason.

Jacksonville 3 Letter Abbreviation

So I have a script that uses a team's full name (i.e. Jacksonville) and grabs it's 3 letter abbreviation. For Jacksonville, this returns JAC, but the games use JAX which means I can never find Jacksonville in a game. Here is some relevant code showing my issue.

nflTeams = nflgame.teams
#teams is a list of full names I read in from a file earlier in the code, names include "Jacksonville" and "New England"
for team in teams:
    # Loop through all NFL teams i.e. all 32 teams
    for nflTeam in nflTeams:
        # Check if this NFL team is our fantasy team
        if team == nflTeam[1]:
            # We have found our team, now loop through all
            # games since games use team abbreviations
            for game in week:
            # Here nflTeam[0] will have "JAC" and game.away or game.home will have "JAX"
                if nflTeam[0] == game.away:
                # Some code here...
                elif nflTeam[0] == game.home:
                # More code here...

So should both of these use JAX or JAC and not one and the other?

Fastest way to calculate fantasy points for nfl.com scoring?

I'm very comfortable with Python, but I'm just starting to play around with nflgame, so very much a beginner in that sense. I'll be doing an nfl.com fantasy league and am hoping to use this to make myself a custom dashboard. I found this script that calculates fantasy scores for all players in a given week. However, it looks like the nfl.com scoring might be a little more complicated, especially for defense and special teams. This seems like something someone out there has probably already done, and I'm not big on reinventing wheels, but I'm not seeing any examples more specific to nfl.com.

Are there any further resources out there for what I'm looking to do, or is my best bet to just figure out how to calculate each stat one by one and add those to the script?

Thanks in advance!

AttributeError: 'NoneType' object has no attribute 'position'

I have been running this bit of code for the first 4 weeks of the nfl season and it has been working great. but as of this week I am getting the error:

AttributeError: 'NoneType' object has no attribute 'position'

here is my code:

import nflgame
import sys
import csv

week = 5
games = nflgame.games(2018, week=week)
play = nflgame.combine_plays(games)

path = "C:\\Users\\Documents\\nfl\\Current\\WRDepth\\Incomplete\\Week"  + str(week)+".csv"
w = csv.writer(open(path, 'wb'))
w.writerow(["Name", "Team", "Position", "Yds"])

for play in play.filter(passing_incmp__gt=0):
    for player in play.players:
        if player.player.position in ['WR', 'TE', 'RB']:
            row = [player, player.player.team, player.player.position,  play.passing_incmp_air_yds] 
            w.writerow(row)

it seems for some reason the position attribute is not there or it gets hung up if a player does not have a position listed. Your help in figuring out the problem would be welcomed!

Is there a way to get the real-world time of a play?

I've been working on a project that shows all the plays, but I want to display them in the order in which they occurred. Is there anything in nflgame or nfldb that has this information, or is it just not available via the JSON we parse?

The way I've been working around it is using the time_inserted value in NFLDB and making sure nflgame is running live during Sunday. But if any of my processes crash during the game, there's no way to determine when each play actually occurred in the real world.

Export nfldb output to CSV

Hi -

I saw that you recently commented in the original, and forked this - in case you are looking to actively take over, I am copying my question over from there:

"_I'd like to export all data from a given nfldb query into a CSV file so i can load in Tableau Public - I remember this was possible in nflgame I think.

Could anyone provide an example of how to do this? I'm fairly basic at this stuff.

Appreciate it_."

I am pretty new to python so I wasn't sure if there was a function in the query tool to get csv or whether I'd need to use some other python functionality. I understand this may be a really newb question so feel free to just say "you need to learn how to use a Panda" or something like that :)

Docker

I'm thinking a docker image will be a FANTASTIC way to negate low level questions. A lot of times a working example is all I've needed to grok a project.

It should;

  • Include runnable version of tests
  • Include demo apps
  • PyPi Example
  • Local build example
  • demonstrate how/when requests are made to nfl.com
  • A fully baked and ready to go version released via hub.docker.com

Which branch is the active one?

Sorry I'm a bit confused by all the copy/paste, merging and branches. I had expected to see a branch for dev and somewhere where the dev branch merges into the main branch (where i would typically look for the latest working version). Is this the case in this project? It doesn't look like it given the way the branches look to me.

Python 3 Discussion

I'm opening this issue to have an open discussion about if this is worth the effort and if so, what that may look like. I will be updating this issue as the discussion evolves.


Starting this off w/ a simple pro/con list as i see it... which is no doubt incomplete.

Upgrade Pros

  1. Security - This is probably my primary reason. I've tightly coupled this into my application so its a bit concerning to be stuck on python 2.7.

  2. Newest python features - async stuff could be quite useful in live

Upgrade Cons

  1. Work needs to be done (lol)

Questions

  1. Will using python 2 to 3 work?

Players not updating nflgame-redux

I have run nflgame.update_player, but I have noticed the player data is not being updated. Specifically, Amari Cooper is still listed as a Raider. I have received this error, which I think may be causing the issue.

Downloading team rosters...
32/32 complete. (100.00%)
Done!
Fetching GSIS identifiers for players not in nflgame...
1/1 complete. (100.00%)
Done!

There were some errors during the download. Usually this is a
result of an HTTP request timing out, which means the
resulting "players.json" file is probably missing some data.
An appropriate solution is to re-run the script until there
are no more errors (or when the errors are problems on
NFL.com side.)

Could not get GSIS id at http://www.nfl.com/player/cedwilson/2560936/profile

Could not find gsis_id for {'status': u'RES', 'last_name': u'Wilson', 'weight': 197, 'profile_id': 2560936, 'number': 16, 'height': 74, 'college': u'Bowie State', 'full_name': u'Ced Wilson', 'first_name': u'Ced', 'years_pro': 0, 'birthdate': u'3/10/1995', 'team': 'DAL', 'position': u'WR', 'profile_url': 'http://www.nfl.com/player/cedwilson/2560936/profile'}

Determining change week to week

What is the best way going about finding a "delta" between the previous two games? For example, if I want to see the change in Saquon Barkley's rushing yards from week 3 to 4 in the 2018 season, what is the best way to go about that? What about finding a delta for each week throughout the season?

nfldb not updating from nflgame

Players are not being update on the nfldb database. Schedule and plays are. I'm not receiving any errors when I run nfldb-update. Anyone experience this before?

Remove hard coded team object in __init__.py

A dicitonary of the various ways to refer to a team is stored in init.py

This seems odd and clunkyy. I think the team objects should be deduced so we dont have to update it if/when a team moves.

I am unsure of what side effects this has and it is probably not even worth doing xD

2017 Super Bowl Missing Overtime Data

I noticed that the 2017 Super Bowl seems to be incorrect. There's no data regarding the overtime.

>>> pp.pprint(game[0].schedule)
{   u'away': u'PHI',
    u'day': 4,
    u'eid': u'2018020400',
    u'gamekey': u'57500',
    u'home': u'NE',
    u'meridiem': u'PM',
    u'month': 2,
    u'season_type': u'POST',
    u'time': u'6:30',
    u'wday': u'Sun',
    u'week': 4,
    u'year': 2017}
>>> pp.pprint(game[0].data['qtr'])
u'Final'
>>> pp.pprint(game[0].score_home_q5)
0
>>> pp.pprint(game[0].score_away_q5)
0

'pip install nflgame-redux' is installing version 1.2.20 instead of 1.2.22a2

โžœ pip install -U  nflgame-redux
Collecting nflgame-redux
  Using cached https://files.pythonhosted.org/packages/41/ab/aa4eb49758cea59d331f43b546cb05928b50eaf007bc40085d6325f77e02/nflgame_redux-1.2.20-py2-none-any.whl
Requirement already satisfied, skipping upgrade: pytz in /Users/JoeNo/.asdf/installs/python/2.7.15/lib/python2.7/site-packages (from nflgame-redux) (2018.5)
Requirement already satisfied, skipping upgrade: beautifulsoup4 in /Users/JoeNo/.asdf/installs/python/2.7.15/lib/python2.7/site-packages (from nflgame-redux) (4.6.3)
Requirement already satisfied, skipping upgrade: httplib2 in /Users/JoeNo/.asdf/installs/python/2.7.15/lib/python2.7/site-packages (from nflgame-redux) (0.11.3)
Installing collected packages: nflgame-redux
Successfully installed nflgame-redux-1.2.20

What is the process involved in upgrading via pip? If I use 'pip install -U git+https://github.com/derek-adair/nflgame.git', upgrades properly.

โžœ pip install -U git+https://github.com/derek-adair/nflgame.git
Collecting git+https://github.com/derek-adair/nflgame.git
  Cloning https://github.com/derek-adair/nflgame.git to /private/var/folders/xb/5lcldfjd7wzckc3z9t1csmdh0000gn/T/pip-req-build-5RRGYl
Requirement already satisfied, skipping upgrade: pytz in /Users/JoeNo/.asdf/installs/python/2.7.15/lib/python2.7/site-packages (from nflgame-redux==1.2.22a2) (2018.5)
Requirement already satisfied, skipping upgrade: httplib2 in /Users/JoeNo/.asdf/installs/python/2.7.15/lib/python2.7/site-packages (from nflgame-redux==1.2.22a2) (0.11.3)
Requirement already satisfied, skipping upgrade: beautifulsoup4 in /Users/JoeNo/.asdf/installs/python/2.7.15/lib/python2.7/site-packages (from nflgame-redux==1.2.22a2) (4.6.3)
Installing collected packages: nflgame-redux
  Found existing installation: nflgame-redux 1.2.20
    Uninstalling nflgame-redux-1.2.20:
      Successfully uninstalled nflgame-redux-1.2.20
  Running setup.py install for nflgame-redux ... done
Successfully installed nflgame-redux-1.2.22a2

Doubly counted plays

Last week (2018 week 2) several players got credited for an extra completion in their player._stats.

When looking through the game plays, it appears that the plays were entered twice into the JSON despite being the same, and when aggregated for player._stats, both were counted.

For example, looking at all completed passes to Javorius Allen gives:

(BAL, CIN 44, Q3, 4 and 2) (6:45) (Shotgun) J.Flacco pass short middle to J.Allen to CIN 43 for 1 yard (D.Dennard). {u'playerName': u'J.Flacco', u'clubcode': u'BAL', u'yards': 1.0, u'statId': 15, u'sequence': 2}
(BAL, BAL 38, Q3, 2 and 4) (3:21) (Shotgun) J.Flacco pass short left to J.Allen to BAL 44 for 6 yards (N.Vigil) [G.Atkins]. {u'playerName': u'J.Flacco', u'clubcode': u'BAL', u'yards': 6.0, u'statId': 15, u'sequence': 2}
(BAL, CIN 33, Q4, 1 and 10) (11:15) (Shotgun) J.Flacco pass short middle to J.Allen to CIN 17 for 16 yards (S.Williams). {u'playerName': u'J.Flacco', u'clubcode': u'BAL', u'yards': 16.0, u'statId': 15, u'sequence': 2}
(BAL, BAL 40, Q4, 3 and 10) (2:14) (Shotgun) J.Flacco pass short middle to J.Allen to BAL 47 for 7 yards (D.Dennard; H.Nickerson). {u'playerName': u'J.Flacco', u'clubcode': u'BAL', u'yards': 7.0, u'statId': 15, u'sequence': 2}
(BAL, CIN 44, Q4, 2 and 6) (1:15) (Shotgun) J.Flacco pass short middle to J.Allen to CIN 38 for 6 yards (D.Dennard). CIN-S.Williams was injured during the play. {u'playerName': u'J.Flacco', u'clubcode': u'BAL', u'yards': 6.0, u'statId': 15, u'sequence': 1}
(BAL, CIN 44, Q4, 2 and 6) (1:15) (Shotgun) J.Flacco pass short middle to J.Allen to CIN 38 for 6 yards (D.Dennard). CIN-S.Williams was injured during the play. {u'playerName': u'J.Flacco', u'clubcode': u'BAL', u'yards': 6.0, u'statId': 15, u'sequence': 4}

Where the last two plays are the same play entered twice, differing only in the sequence assigned.

And when looking for all completed passes thrown to Trey Burton:

(CHI, SEA 3, Q1, 1 and 3) (4:59) (Shotgun) M.Trubisky pass short middle to T.Burton for 3 yards, TOUCHDOWN. {u'playerName': u'M.Trubisky', u'clubcode': u'CHI', u'yards': 3.0, u'statId': 16, u'sequence': 2}
(CHI, CHI 23, Q3, 1 and 10) (10:39) (Shotgun) M.Trubisky pass short left to T.Burton to CHI 26 for 3 yards (E.Thomas). {u'playerName': u'M.Trubisky', u'clubcode': u'CHI', u'yards': 3.0, u'statId': 15, u'sequence': 1}
(CHI, CHI 46, Q3, 2 and 9) (3:46) (Shotgun) M.Trubisky pass short right to T.Burton to SEA 45 for 9 yards (J.Coleman). {u'playerName': u'M.Trubisky', u'clubcode': u'CHI', u'yards': 8.0, u'statId': 15, u'sequence': 1}
(CHI, CHI 46, Q3, 2 and 9) (3:46) (Shotgun) M.Trubisky pass short right to T.Burton to SEA 45 for 9 yards (J.Coleman). {u'playerName': u'M.Trubisky', u'clubcode': u'CHI', u'yards': 9.0, u'statId': 15, u'sequence': 4}
(CHI, CHI 10, Q4, 2 and 15) (10:04) (Shotgun) M.Trubisky pass short middle to T.Burton to CHI 15 for 5 yards (A.Calitro). {u'playerName': u'M.Trubisky', u'clubcode': u'CHI', u'yards': 5.0, u'statId': 15, u'sequence': 1}

the third and fourth plays are the same play counted twice, differing by one yard.

Similarly G.Bernard got credited in his player._stats for a completion even though the play was called back.

T.Lockett and J.Conner also have incorrect reception stats when compared to the count of plays they have with receptions.

2016 Season Super Bowl listed as week 5, querying POST week 4 gives error

This seemed wrong to me

>>> pp.pprint(nflgame.games(2016, kind="POST", week=4)[0].schedule)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/cruffenach/Library/Python/2.7/lib/python/site-packages/nflgame/__init__.py", line 225, in games
    return list(games_gen(year, week, home, away, kind, started))
TypeError: 'NoneType' object is not iterable
>>> pp.pprint(nflgame.games(2016, kind="POST", week=5)[0].schedule)
{   u'away': u'NE',
    u'day': 5,
    u'eid': u'2017020500',
    u'gamekey': u'57167',
    u'home': u'ATL',
    u'meridiem': u'PM',
    u'month': 2,
    u'season_type': u'POST',
    u'time': u'6:30',
    u'wday': u'Sun',
    u'week': 5,
    u'year': 2016}

Is there an up-to-date repository that connects nflgame-redux to nfldb with all of the db value fixes?

Sorry as this isn't directly a nflgame issue per-se but this is the only active community that I'm aware of.

Everytime I uninstall/reinstall nfldb, I've got to re-jump through all of the hurdles to get the data updated. I know @ochawkeye is the nfldb guru, but the head of his fork isn't set up in conjunction with this either.

Does this mythical nfldb repository exist already, because it's definitely something that should be set up if it hasn't already.

Gracefully handle player record when its None

Update players does not gracefully handle missing player records;

Could not get GSIS id at http://www.nfl.com/player/cedwilson/2560936/profile

Could not find gsis_id for {'status': u'RES', 'last_name': u'Wilson', 'weight': 197, 'profile_id': 2560936, 'number': 16, 'height': 74, 'college': u'Bowie State', 'full_name': u'Ced Wilson', 'first_name': u'Ced', 'years_pro': 0, 'birthdate': u'3/10/1995', 'team': 'DAL', 'position': u'WR', 'profile_url': 'http://www.nfl.com/player/cedwilson/2560936/profile'}

Releases

Need the ability to release improvements, either @BurntSushi can hook up permissions, we can find someone who has them OR we create nflgame-redux.

I'd rather wait for burnt sushi but if something doesn't happen soon i will just use redux until we get ahold of him.


UPDATE:

  • I've reached out to burnt Sushi to take over the pypi listing
  • I've uploaded 1.2.22-rc1 to nflgame-redux, which includes the initial 2018 schedule data.

My thoughts on Andrew's nfl* code and where to go from here

There are two other projects that are driven by nflgame... nfldb and nflfan. This is something I was unaware of upon discovery and well into the development of one of my projects. Initially my thoughts were to fork all projects and unify them under one code base, in an entirely new namespace. However, i'm not so sure that solves much of anything and it would certainly bloat the package. I'm shelving this notion for now, but i'm definitely considering . Anyways nflfan seems to be the culmination of Andrew's work.

One of the reasons Andrew has abandoned this project is the copious amount of VERY low level issue's filed. There are numerous issue's/questions/suggestions that are either a) filed in the wrong repository altogether or b) a direct result of not being aware of the suite of projects contributed by Andrew.

This is a documentation problem...
There are no references to nfldb or nflfan ANYWHERE in the documentation for this project. In the NFLdb repository there is mention of nflgame, but man i feel like this should HIT YOU OVER THE FREAKING HEAD. Something as simple as a table of contents in each description.md would go a long way.

Week 16 2018 BAL @ LAC game causes `AssertionError: More than one game matches the given criteria`

When week 16 2018 Baltimore Ravens at Los Angeles Chargers game was flexed into Saturday time slot, it created a second record in schedule.json for the game.

Python 2.7.14 (v2.7.14:84471935ed, Sep 16 2017, 20:19:30) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import nflgame
>>> game = nflgame.one(2018, 16, 'BAL', 'BAL', kind='REG')

AssertionError: More than one game matches the given criteria.
[{u'week': 16, u'meridiem': u'PM', u'gamekey': u'57792', u'season_type': u'REG', u'away': u'BAL', u'year': 2018, u'month': 12, u'eid': u'2018122201', u'time': u'8:20', u'home': u'LAC', u'wday': u'Sat', u'day': 22},
 {u'week': 16, u'meridiem': u'PM', u'gamekey': u'57792', u'season_type': u'REG', u'away': u'BAL', u'year': 2018, u'month': 12, u'eid': u'2018122301', u'time': u'4:05', u'home': u'LAC', u'wday': u'Sun', u'day': 23}]

There are probably other ways to fix this, but one hammer for this is to just remove the offending game from schedule.json.

  [
   "2018122301",
   {
    "away": "BAL",
    "day": 23,
    "eid": "2018122301",
    "gamekey": "57792",
    "home": "LAC",
    "meridiem": "PM",
    "month": 12,
    "season_type": "REG",
    "time": "4:05",
    "wday": "Sun",
    "week": 16,
    "year": 2018
   }
  ],

Hightlight & delete.

Python 2.7.14 (v2.7.14:84471935ed, Sep 16 2017, 20:19:30) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import nflgame
>>> game = nflgame.one(2018, 16, 'BAL', 'BAL', kind='REG')
>>> print game
BAL (22) at LAC (10)
>>>

Play by Play video nflvid-slice

I understand nflvid is not maintained but reaching out to see if anyone has been able to get 2018 video and sliced. I have been able to download a full game coaches file but do not have the xml file.

I have made changes to init.py to try and get the xml path for 2018. Assuming the path is different. Has anyone discovered a new working path? Or is this just obsolete? Any better ideas?

Game time remaining

Is there a way to get the remaining game time with nflgame? I noticed that nfldb does not store this information.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.