Obtain song lyric data about ttds HOT 4 CLOSED

Information: Each new line in the Genius Lyrics denotes as a new document

[FOR INDEXER]
Unique ID should be an integer instead of a string - look into it
Create two collections: { line_id: song_id} and {song_id: lyrics_document_line_str }
If time allows, look into MongoDB projection (based from a TTDS Report)

Data Collection:

1. Add sample collection data from Eminem for 500+ songs [COMPLETED]
2. Look into Dave's code - LyricsGenius problems [COMPLETED]
3. Automating data collection [IN PROGRESS]
4. Compile all the JSON files into one CSV file [IN PROGRESS]
5. Update README file to run data collection on U of Edinburgh server
6. Find better solution?
7. Review DevOps practices and check code before release

from ttds.

Tried out the guide in the description but I don't think it can work because you need to manually copy each track's URI from Spotify.

Looking into this library called 'lyricsgenius' (GitHub - johnwmillr/LyricsGenius: Download song lyrics and metadata from Genius.com 🎶🎤

from ttds.

I've pushed a script to the auto_data_collection branch to automate the data collection completely. Please refer to scrape_ad.py. Just run it on longjob with a subset of letters as normal and you can monitor its progress by looking at its log file on DICE.

from ttds.

Currently scraping data for artists whose names start with I, J, K and L

from ttds.