Notes on future development w/ `cfbd` API about ap-cfb-xc HOT 8 OPEN

Looks like you tagged me in here along with my gist (possibly accidentally!), but I dug around very quickly on the ESPN API endpoint you pulled out from there for rankings and found this: http://sports.core.api.espn.com/v2/sports/football/leagues/college-football/seasons/2023/types/2/weeks/1/rankings/1?lang=en&region=us. You can change the seasons and weeks values appropriately to get historical data, and based on other projects I've done with the API, you can get data on bowl/playoff games with a types value of 3 and weeks value of 1.

A few notes:

• Looks like a rankings value 1 in the URL is the AP poll -- you can manipulate this value for different polls if you want them.
• You won't need to use BeautifulSoup for this either -- just a requests.get() to grab the JSON and the json package to parse it.
• You'll have to make GET requests to the URLs provided in the $ref key of each record in the ranks array to get detailed team information.
• It does seem like there is receiving-votes data in there for your needs under the others array.

Hope this helps!

from ap-cfb-xc.

This may be a good API: https://api.collegefootballdata.com/api/docs/?url=/api-docs.json#/. One would have to request a key here though: https://collegefootballdata.com/key.

from ap-cfb-xc.

I believe this API has a GitHub link: https://github.com/CFBD/cfb-api. They also have more scripts in their repo: https://github.com/CFBD. Python specifics of this API here: https://github.com/CFBD/cfbd-python. This may be a good starting point for coding up the v2 of AP_XC.

from ap-cfb-xc.

cfbd python API Update

As noted in eda4b69, the cfbd python API does have historical AP poll data that is accessible back to 1939 (s/o '39 Texas A&M Aggies). It is updated quickly: 2 days after the Week 2 AP poll was released, the data came through on this API. May return faster in the future as well.

Limitation

However, a bit issue with this is that it seems to only have the top 25 for every poll, not the "others receiving votes." The CFB XC methodology is not un-workable without these other teams, but the 25 number is an arbitrary cutoff; there are other teams that rank 26 to deep in the 40s, especially in early season polls, that help flesh out finishers for 5-team races.

ESPN API Alternative

There may be another alternative available. ESPN has "hidden API endpoints," according to this gist document by @akeaswaran. The rankings API does have an "others" item that may be the ticket to success. It is also updated quickly: 2 days after the Week 2 AP poll was released, the data came through on this API. May return faster in the future as well. The rankings API specifically is at http://site.api.espn.com/apis/site/v2/sports/football/college-football/rankings.

Limitation

This seems to only be for the current season. So while it seems current, it may be limited to non-historical seasons, which would be a real bummer.

Solution

See if the API URL can be tweaked to induce a historical season. The public URL the v1 of this code was built on uses these for BeautifulSoup to scrape. The formatting of the URL is accounted for in the follwoing code. ESPN formats its rankings URL like so: https://www.espn.com/college-football/rankings/_/week/1/year/2023/seasontype/2

from ap-cfb-xc.

@akeaswaran, thank you very much for the head start on the ESPN API! It's most helpful.

I am slowly working on a new script using it in an ESPN feature branch that incorporates your notes.

from ap-cfb-xc.

The espn_cfb_api repo may have some good examples for how to efficiently use the ESPN API for python and retrieve conference information. The main script, espn_cfb_api.py, has useful information as to how to access conference information:

• https://github.com/sorna-matthew/espn_cfb_api/blob/ff0494cc2b32c00505a21359f25c96ffdf637d29/espn_cfb_api.py#L8-L12
• https://github.com/sorna-matthew/espn_cfb_api/blob/ff0494cc2b32c00505a21359f25c96ffdf637d29/espn_cfb_api.py#L28-L58

The big drawback is that it is only for the current year, so there is no direct application. This will need to serve as inspiration rather than a plug-and-play source.

from ap-cfb-xc.

Explore using pydantic for API calls. It can help you with request and response validation, data serialization, and making your API calls more predictable and type-safe.

When you're receiving data from a website using requests.get() and you want to parse and validate the response data, Pydantic can still be a valuable tool. Although .json() is convenient for parsing JSON data, Pydantic can provide additional benefits such as data validation and type checking.

Here's how you can use Pydantic to read and validate the response from a website:

1. Define a Pydantic Model: Create a Pydantic model that represents the structure of the expected response data.

from pydantic import BaseModel

class WebsiteResponse(BaseModel):
    status: int
    content: str

In this example, we're defining a WebsiteResponse model with two fields: status (for the HTTP status code) and content (for the response content).

2. Make the Request and Deserialize: Use requests.get() to fetch data from the website and then deserialize the response using your Pydantic model.

import requests
from your_module import WebsiteResponse  # Import your Pydantic model

response = requests.get('https://example.com/some_endpoint')

if response.status_code == 200:
    website_data = WebsiteResponse(status=response.status_code, content=response.text)
else:
    website_data = WebsiteResponse(status=response.status_code, content='')

# Now website_data is a validated Pydantic model

Here, we're creating a website_data instance of the WebsiteResponse model and populating it with data from the response. If the response status code is not 200, we provide an empty string for the content field. You can handle error cases according to your needs.

3. Access Validated Data: Once you have website_data, you can access its fields like any other Python object:

print(website_data.status)   # Access the status code
print(website_data.content)  # Access the content

4. Validation and Error Handling: Pydantic will automatically validate that the data in website_data matches the structure defined in the model. If the response data doesn't match, a ValidationError will be raised, which you can catch and handle.

from ap-cfb-xc.