https://www.youtube.com/watch?v=hHZ3YeWKcCM
Ever wanted to create a calendar or event for a concert, sport team, artist tour, trash collections give Canden a try. Canden creates calendar events from a webpage so you never miss what is important.
In the example, we create a calendar from the team USA basketball schedule https://www.usab.com/teams/5x5-mens-world-cup/schedules.
✅: represents the approach used in code
- Fork the repo
- Create a virtual env and install the dependencies in the requirements.txt file
- In your Google Developer Console, create a new project named Canden with access to the Calendar API.
- Add OAuth 2.0 credentials to the Canden project and copy the
client_secrets.json
to the root of the project. Do not commit the secrets file. - Cmd to add to calendar:
python3 main.py
- Customize the date scrapper for another website and repeat step 5 😄
Stanza is an existing product that allows users to sync events, sports, shows, etc schedules to their calendars. However not all events are in Stanza such as the Team USA mens basketball schedule. Canden is another to quickly add events to calendar once you know which HTML elements hold the date and info on the websites.
- Try to build in java
- Setting up maven on my own is too complex
- Scope the effort
- Design
- Test the Google Calendar API
- Find website tags
- Roadmap
- Set up GoogleCalendar and Fetch all calendars
- Create a calendar
- Create an event on the calendar
- Storing the calendar key as “name” vs “id”
- Parse webpage for dates and event name
- BS4
- Selenium
- Create a main file to seal the deal
- Move the timezone to the main file
- Google Calendar API scopes | Google for Developers
- Selenium docs
- Beautiful Soup docs
- How to S***pe JavaScript-Rendered Web Pages with Python
Allow users to add events from a webpage to a calendar. Write some python scripts to get the schedule from the website and insert events to Google Calendar.
- Get events from a webpage
- Create a google calendar from a list of events
Get started with Google Calendar Client in python.
API Auth connections are not supported so we need to set up a Oauth Client.
googleapiclient.errors.HttpError: <HttpError 401 when requesting https://www.googleapis.com/calendar/v3/users/me/calendarList?key=<API-KEY>&alt=json returned "API keys are not supported by this API. Expected OAuth2 access token or other authentication credentials that assert a principal. See https://cloud.google.com/docs/authentication". Details: "[{'message': 'Login Required.', 'domain': 'global', 'reason': 'required', 'location': 'Authorization', 'locationType': 'header'}]"
- Create a project in GCP and add Google Calendar to the project
- Set up an OathClient and copy the client_secrets.json to the root of the project
- python oauth2client will look for the secrets file to connect to the project
Read the game schedule from the team website - Schedules - USA Basketball Men's World Cup Team - USA Basketball
Approaches
-
⚠️ Use chat gpt prompt to do the grunt work -use python and beautifulsoup to get the game schedule from https://www.usab.com/teams/5x5-mens-world-cup/schedules
does not work. Check github commits for solution. -
⚠️ ChatGPT chose the wrong element so specify the elements in the prompt. Using a simple chat gpt prompt such asuse python and beautifulsoup to get the game schedule from the sub elements of div.font-title on https://www.usab.com/teams/5x5-mens-world-cup/schedules website
-
✅ Inspecting the result, it seems the dates info is loaded using a script tag. On page load the HTML body may not contain the desired element and class. This conclusion comes from inspecting the python
request.content
. We can use selenium to load the page and then get the schedule using bs4.