Giter VIP home page Giter VIP logo

scrapify's Introduction

Scrapify

A script to create a small snapshot of Spotify in a SQLite database, pivoted aruond your users Saved Tracks and playlists. Useful for doing analysis or for hosting mock-spotify servers for testing.

Schema

Currently, Scrapify populated the following object models, along with their constituent sub-objects, i.e., copyrights and images.

  • Artists
  • Artists Albums / Albums Artists
  • Albums
  • Album Tracks
  • Tracks
  • Audio Features
  • Playlists
  • Playlist Tracks
  • Users
  • User's Followed Artists
  • User's Saved Tracks
  • User's Saved Albums

For a complete view of the schema, see the schemaonly.sql file.

Usage

Dependencies

  • Python 3
  • Spotipy

Replace the user field with your username.

./scrapify.py

The final output is a tpy.db sqlite file in the same directory containing a lot of spotify information. Scrapify grabs first and second order information about artists and albums from your library and playlists.

Rate limiting note

If you get rate limited, just run the tool again after the limit expires. It can pick up from places it left off from provided you do not delete any of the files created under src/raw.

When you feel like you have a sufficently populated tpy.db, you are free to delete the whole src/raw directory.

Bugs

  • Double parses artists and albums on multiple runs. This is mitigated by the fact that it doesn't multiple-insert into the sqlite database.

TODO's

  • Categories
  • Category Playlists
  • Featured Playlists
  • Audio Analyses

scrapify's People

Watchers

James Cloos avatar  avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.