Giter VIP home page Giter VIP logo

OpenCourseAPI

An open-source API to scrape, process, and serve college course and class data 📒

(Also, a fresh take on OwlAPI =D)

Features

  • 🔍  Scrapes terms, departments, courses, and classes
  • 🧩  Campus-agnostic architecture allows easy extensibility
  • 📅  Currently serves 10+ years of just FHDA data
  • 🏫  Scrapes a total of 4 colleges (adding more is just a matter of time)

Coming Soon:

  • 📙 Scrape course catalogs to get more course info, such as descriptions
  • 🔗 Link classes to professors for advanced analytics
  • 🌐 GraphQL API (proof-of-concept finished)

Data Explorer => opencourse.dev

URL is subject to change.. but for now, the frontend and the API are hosted on the same domain.

API Docs

Currently, the following data for the following campuses exists:

id name
fh Foothill College
da De Anza College
wv West Valley College
mc Mission College

All endpoints (except for /:campus) support the following query parameters:

name format default
year a valid year 2020
quarter fall, winter, spring, or summer fall

Example: /da/depts?year=2019&quarter=fall

GET /:campus

Example: /fh

Get campus metadata information.

Sample Response 
{
  "id": "fh",
  "terms": [
    {
      "year": 2020,
      "term": "summer",
      "code": "202111"
    },
    {
      "year": 2020,
      "term": "spring",
      "code": "202041"
    },
    {
      "year": 2020,
      "term": "winter",
      "code": "202031"
    },
    {
      "year": 2019,
      "term": "fall",
      "code": "202021"
    },
    {
      "year": 2020,
      "term": "fall",
      "code": "202121"
    }
  ]
}

GET /:campus/courses

Example: /fh/courses

Get all courses at a campus for the selected term and year.

Sample Response 
[
  {
    "dept": "ACTG",
    "course": "1A",
    "title": "Financial Accounting I"
  },
  {
    "dept": "ACTG",
    "course": "1B",
    "title": "Financial Accounting II"
  },
  {
    "dept": "MATH",
    "course": "1A",
    "title": "Calculus"
  }
]

GET /:campus/classes

Example: /fh/classes

Get all classes at a campus for the selected term and year.

Sample Response 
[
  {
    "CRN": 20238,
    "raw_course": "ACTG F001A01W",
    "dept": "ACTG",
    "course": "1A",
    "section": "01W",
    "title": "Financial Accounting I",
    "units": 5,
    "start": "10/19/2020",
    "end": "12/11/2020",
    "seats": 2,
    "wait_seats": 15,
    "status": "open",
    "times": [
      {
        "days": "TBA",
        "start_time": "TBA",
        "end_time": "TBA",
        "instructor": [
          "Joe L  Mayer (P)"
        ],
        "location": "FC ONLINE"
      }
    ]
  },
  { "...": "..." }
]

POST /:campus/classes

Get selected classes in a batch request.

Body Schema
{
  resources: Resource[]
}
Resource Schema

Combinations: CRN, dept, or dept + course

{
  CRN: number,
  dept: string,
  course: string
}
Response Schema
{
  resources: [
    {
      status: 'success' | 'error',
      error: string,
      data: Class | Class[] | null
    }
  ]
}
Sample Body & Response

Body:

{
  resources: [
    { dept: 'CS', course: '1A' },
    { dept: 'CS', course: '2B' },
    { CRN: 20211 },
    {} // Invalid
  ]
}

Response:

{
  "resources": [
    {
      "status": "success",
      "data": [
        {
          "CRN": 30239,
          "raw_course": "C S F001A01Z",
          "dept": "CS",
          "course": "1A",
          "section": "01Z",
          "title": "Object-Oriented Programming Methodologies in Java",
          "units": 4.5,
          "start": "01/04/2021",
          "end": "03/26/2021",
          "times": ["..."],
          "status": "open",
          "seats": 40,
          "wait_seats": 10,
          "wait_cap": 10
        },
        {"...": "..."}
      ]
    },
    {
      "status": "success",
      "data": [
        {
          "CRN": 30239,
          "raw_course": "C S F001A01Z",
          "dept": "CS",
          "course": "1A",
          "section": "01Z",
          "title": "Object-Oriented Programming Methodologies in Java",
          "units": 4.5,
          "start": "01/04/2021",
          "end": "03/26/2021",
          "times": ["..."],
          "status": "open",
          "seats": 40,
          "wait_seats": 10,
          "wait_cap": 10
        },
        {"...": "..."}
      ]
    },
    {
      "status": "success",
      "data": {
        "CRN": 30324,
        "raw_course": "ACTG F001C03W",
        "dept": "ACTG",
        "course": "1C",
        "section": "03W",
        "title": "Managerial Accounting",
        "units": 5,
        "start": "01/04/2021",
        "end": "03/26/2021",
        "times": ["..."],
        "status": "open",
        "seats": 40,
        "wait_seats": 15,
        "wait_cap": 15
      }
    },
    {
      "status": "error",
      "error": "At least \"CRN\" or \"dept\" have to be specified."
    }
  ]
}

GET /:campus/classes/:crn

Example: /fh/classes/20238

Get a class by its CRN (in a given term and year).

Sample Response 
{
  "CRN": 20238,
  "raw_course": "ACTG F001A01W",
  "dept": "ACTG",
  "course": "1A",
  "section": "01W",
  "title": "Financial Accounting I",
  "units": 5,
  "start": "10/19/2020",
  "end": "12/11/2020",
  "seats": 2,
  "wait_seats": 15,
  "status": "open",
  "times": [
    {
      "days": "TBA",
      "start_time": "TBA",
      "end_time": "TBA",
      "instructor": [
        "Joe L  Mayer (P)"
      ],
      "location": "FC ONLINE"
    }
  ]
}

GET /:campus/depts

Example: /fh/depts

Get all departments at a campus for the selected term and year.

Sample Response 
[
  {
    "id": "GIST",
    "name": "Geospatial Tech & Data Sci"
  },
  {
    "id": "GLST",
    "name": "Global Studies"
  },
  {
    "id": "GID",
    "name": "Graphic and Interact Desig"
  }
]

GET /:campus/depts/:dept

Example: /fh/depts/GLST

Get a department at a campus for the selected term and year.

Sample Response 
{
  "id": "GLST",
  "name": "Global Studies"
}

GET /:campus/depts/:dept/classes

Example: /fh/depts/MATH/classes

Get all classes in a department for the selected term and year.

Sample Response 
[
  {
    "CRN": 20086,
    "raw_course": "MATH F001A01V",
    "dept": "MATH",
    "course": "1A",
    "section": "01V",
    "title": "Calculus",
    "units": 5,
    "start": "09/21/2020",
    "end": "12/11/2020",
    "seats": 0,
    "wait_seats": 7,
    "status": "waitlist",
    "times": [
      {
        "days": "MW",
        "start_time": "07:30 AM",
        "end_time": "09:45 AM",
        "instructor": [
          "Diana Monica  Uilecan (P)"
        ],
        "location": "FH ONLINE"
      }
    ]
  },
  { "...": "..." }
]

GET /:campus/depts/:dept/courses

Example: /fh/depts/MATH/courses

Get all courses in a department for the selected term and year.

Sample Response 
[
  {
    "dept": "MATH",
    "course": "1A",
    "title": "Calculus",
    "classes": [
      20086,
      20087,
      20088,
      20257,
      20761,
      20762,
      20305,
      21310
    ]
  },
  {
    "dept": "MATH",
    "course": "1B",
    "title": "Calculus",
    "classes": [
      20672,
      20089,
      20769,
      20773
    ]
  }
]

GET /:campus/depts/:dept/courses/:course

Example: /fh/depts/MATH/courses/1A

Get a course in a department for the selected term and year.

Sample Response 
{
  "dept": "MATH",
  "course": "1A",
  "title": "Calculus",
  "classes": [
    20086,
    20087,
    20088,
    20257,
    20761,
    20762,
    20305,
    21310
  ]
}

GET /:campus/depts/:dept/courses/:course/classes

Example: /fh/depts/MATH/courses/1A/classes

Get all classes for a course in the selected term and year.

Sample Response 
[
  {
    "CRN": 20086,
    "raw_course": "MATH F001A01V",
    "dept": "MATH",
    "course": "1A",
    "section": "01V",
    "title": "Calculus",
    "units": 5,
    "start": "09/21/2020",
    "end": "12/11/2020",
    "seats": 0,
    "wait_seats": 7,
    "status": "waitlist",
    "times": [
      {
        "days": "MW",
        "start_time": "07:30 AM",
        "end_time": "09:45 AM",
        "instructor": [
          "Diana Monica  Uilecan (P)"
        ],
        "location": "FH ONLINE"
      }
    ]
  },
  { "...": "..." }
]

Examples

Python

Install requests, and use the API as follows:

import requests

API_URL = 'https://opencourse.dev'
req = requests.get(f'{API_URL}/fh/courses')

if req.ok:
    courses = req.json()

    for course in courses:
        print(course['dept'], course['course'], course['title'])
View Output 
ACTG 1A Financial Accounting I
ACTG 1B Financial Accounting II
ACTG 1C Managerial Accounting
ACTG 51A Intermediate Accounting I
ACTG 52 Advanced Accounting
ACTG 53 Financial Statement Analysis
ACTG 54 Accounting Information Systems
ACTG 58 Auditing
ACTG 59 Fraud Examination
ACTG 60 Accounting for Small Business
ACTG 64A Computerized Accounting Practice Using Quickbooks
ACTG 64B Computerized Accounting Practice Using Excel
ACTG 65 Payroll & Business Tax Accounting
ACTG 66 Cost Accounting
ACTG 67 Tax Accounting
...

Development

For running the API server, install python 3.8, pip, and pipenv.

Note: the following is subject to change (especially the scraper modules)

git clone https://github.com/OpenCourseAPI/OpenCourseAPI.git
cd OpenCourseAPI

pipenv install # install all python dependencies

python -m campus.fhda.fhda_scrape # scrape Foothill / De Anza College
python -m campus.wvm.wvm_scrape # scrape West Valley / Mission College

python server.py # start the server

To run the frontend, install Node.js (preferably v12+) and yarn. Afterwards, run the following:

cd frontend
yarn install # install NPM packages
yarn start # start dev server

To build for production, use:

yarn run build

The generated static files are in frontend/build. To run a static server to serve the built files, use:

yarn run start:static

To run tests, use the following:

yarn test # run all tests
yarn test:e2e # only run end-to-end tests

Contribute

All contributions are welcome! A contribution guide is TBA.. but you can start by opening an issue and looking at the development guide above. Thanks!

Core Team

License

MIT License

Open Course API's Projects

docs icon docs

[WIP] Experimental docs for opencourse.dev

live-fhda-class-data icon live-fhda-class-data

Course data extracted from FHDA MyPortal and automatically updated every 5 - 10 minutes 📈

opencourseapi icon opencourseapi

An open source API to scrape, process, and serve college course and class data 📒

opencoursebot icon opencoursebot

A handy discord bot for college-related information such as classes, professors, and statistics ✨

owlapi icon owlapi

An open source REST API written in Python to scrape and serve Foothill / De Anza course data :ledger:

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.