timrogers / rapgenius Goto Github PK

I was thinking about adding a basic CLI for this gem, what do you think? 👍

Since the gem uses screen scraping extensively, it's highly likely to break, even in the short term.

We current have specs which test against included examples (Big Sean's "Control" specifically, as downloaded on 17th August) but there should be a process to check against live data, so we can easily know whether changes to Rap Genius have broken the parser.

RubyGems.org doesn't report a license for your gem. This is because it is not specified in the gemspec of your last release.

via e.g.

spec.license = 'MIT'
# or
spec.licenses = ['MIT', 'GPL-2']

Including a license in your gemspec is an easy way for rubygems.org and other tools to check how your gem is licensed. As you can imagine, scanning your repository for a LICENSE file or parsing the README, and then attempting to identify the license or licenses is much more difficult and more error prone. So, even for projects that already specify a license, including a license in your gemspec is a good practice. See, for example, how rubygems.org uses the gemspec to display the rails gem license.

There is even a License Finder gem to help companies/individuals ensure all gems they use meet their licensing needs. This tool depends on license information being available in the gemspec. This is an important enough issue that even Bundler now generates gems with a default 'MIT' license.

I hope you'll consider specifying a license in your gemspec. If not, please just close the issue with a nice message. In either case, I'll follow up. Thanks for your time!

Appendix:

If you need help choosing a license (sorry, I haven't checked your readme or looked for a license file), GitHub has created a license picker tool. Code without a license specified defaults to 'All rights reserved'-- denying others all rights to use of the code.
Here's a list of the license names I've found and their frequencies

p.s. In case you're wondering how I found you and why I made this issue, it's because I'm collecting stats on gems (I was originally looking for download data) and decided to collect license metadata,too, and make issues for gemspecs not specifying a license as a public service :). See the previous link or my blog post about this project for more information.

A new Rap Genius iOS app launches in the next couple of days - I'm running the beta on my phone, and have been intercepting the requests. It uses a fully-featured API hosted at https://api.rapgenius.com.

Upgrade the gem to use this.

Repro with the following:
RapGenius::Song.find(161630).description

It's due to the following empty description in the json
{"dom"=>{"tag"=>"root"}}

Hey guys. I'm currently developing an iOS application that I need to pull lyrics for songs that I specify. I'm currently researching different apis that I can use to retrieve song lyrics. How can I use this within my iOS application? Thanks

Great gem! I wonder if there's a way to check if multiple artists are in RG. I've got about 400+ artists from the Twin Cities and I'm trying to determine the extent of their representation in RG. Any ideas would be great!

At the moment, Song quite deliberately has a #annotations method rather than #lyrics because up 'til now I've only been parsing annotated lines which are contained in <a> tags.

The markup doesn't make it easily to sanely parse the rest of the plain text lyrics.

Potentially, the best way to do this would be to loop through the .lyrics element, taking every line and converting it into a new Lyric object which would represent a lyric, whether annotated or not, replacing Annotation.

That Lyric object would then respond to methods like #annotated? and #annotation. There are challenges in this idea though - for instance, unannotated parts of the song don't have unique IDs.

When getting annotations we have to add a space between each one. In the line.rb file I added a space for explanations in end.join(" ")
Not sure if this is the optimal solution

Awesome gem!

Was wondering if you'd consider supporting additional fields in the json. Some of the fields I find valuable are:

• Whether an artist is primary or featured for a song (@type)
• Description dom (currently it just gets the children field)

Thanks!

I was going to start working on the Search that's mentioned in the README but wanted to see what your vision for the API was.

My first thought was

songs = RapGenius.search("Big Sean") # or '.find' to be consistent.
songs.class       # => Array
songs.first.class # => SearchResult

song = songs.first
song.id    # => 203961
song.title # => "Big Sean (Ft. Jay Electronica & Kendrick Lamar) – Control"
song.url   # => "/Big-sean-control-lyrics"

lyrics = RapGenius::Song.find(song.url)
# ...

You can see the kind of results returned by their server for their autocomplete by running this:

curl "http://rapgenius.com/search/quick?q=Big+Sean+Cont" -H "X-Requested-With:XMLHttpRequest" -H "Referer:http://rapgenius.com/" -H "Accept:application/x-javascript, text/javascript"

which is easily parseable:

Big Sean (Ft. Jay Electronica & Kendrick Lamar) – Control (French Version)|/Big-sean-control-french-version-lyrics|205998
Big Sean (Ft. Jay Electronica & Kendrick Lamar) – Control|/Big-sean-control-lyrics|203961
Sa-roc (Ft. Big Sean - No I.D., Jay Electronica, Kendrick Lamar & Sa-roc) – CONTROL|/Sa-roc-control-lyrics|204686
C3 – Control ( Disses Kendrick Lamar , Jay-Z, Tyler The Creator, Big Sean, Meek Mill & More )|/C3-control-disses-kendrick-lamar-jay-z-tyler-the-creator-big-sean-meek-mill-and-more-lyrics|205362

You can omit the "Accept" header and get back an HTML partial that we could parse directly in Nokogiri; whichever's easier.

The Terms of Use on Rap Genius (more generally genius.com) seem to explicitly prohibit this.

Specifically:

Except as expressly authorized by Genius in writing, you agree not to modify, copy, frame, scrape, rent, lease, loan, sell, distribute or create derivative works based on the Service or the Genius Content, in whole or in part, except that the foregoing does not apply to your own User Content (as defined above) that you legally upload to the Service. In connection with your use of the Service you shall not engage in or use any data mining, robots, scraping or similar data gathering or extraction methods. Any use of the Service or the Genius Content other than as specifically authorized herein is strictly prohibited.

I only bring it up because I'd like to pull lyric data from them as well and most of these sites tend to have a clause that includes that kind of language. Have you had any pushback?

Hey Tim,

I currently have three apps running that use this gem as a dependency, all of which seem to have broken within the past week. It looks like your guess the song example is also not working. Based on a cursory look, it appears that your gem is no longer returning any results. Did Genius shut off access to their private API endpoint? I wonder if this has something to do with their release of a public API which requires authentication via Oauth?

http://genius.com/discussions/161769-The-genius-api-is-now-public

By the way, thanks so much for building this gem to begin with! This was such an amazing resource and enabled me to build projects that would not have been possible otherwise.

Hey—just wanted to give you a heads up that the RG API now supports a text_format query param (to any endpoint) that lets you choose what representation formatted text should have. E.g. http://api.rapgenius.com/songs/70324?text_format=html will return formatted text (lyrics, description) in HTML. Valid options are dom (default), html, markdown, and plain, and you can pass multiple formats separated by a comma.

Thought that might help simplify things on the client end!

README.md contains a fair bit of sample code which gives a good idea on how to get started, but it'd be really nice to have a sample app using this stuff.

My current idea is to build something that will let you embed Rap Genius annotations in your own page with an embed code, e.g.:

<script type="text/javascript" src="http://rapgeniusapi.herokuapp.com/annotation/1234.js"></script>

...but there are challenges there since performance is slow from Nokogiri, and thus we'd need it to do some asynchronous loading to avoid killing people's pageloads...

now that you're working with the Genius API, are lines from songs no longer available? Gem is no longer retrieving lyrics for my app