panuganti / nreadability Goto Github PK

For example, http://www.nytimes.com/2010/11/09/books/09book.html and 
http://www.nytimes.com/2010/11/14/world/asia/14myanmar.html?hp

I've attached a patch that fixes the problem, along with a new unit test.  (I 
generated the patch with cygwin svn, so let me know if the line endings are 
messed up or anything.)

The fix itself was simple; readability.js, when determining which siblings to 
append to the main content body, gives a bonus when a sibling has the same 
class name.  I added that logic to CreateArticleContentElement().

And thanks for porting Readability to C#!  It's really been a great help on a 
few projects.

One thing that's great about readability.js is that it can often fetch 
subsequent pages from the first page of a multi-page article.  It would be 
great if nreadability could do that as well.

The meat of this is findNextPageLink(), which I've already implemented as a 
method in NReadabilityTranscoder for my own use.  It seems to work well, but 
before I go any further I'd like to know (1) if this is a feature you'd like to 
include in nreadability, and (2) if so, what you want its interface to look 
like.

There are a couple of options that I can see.  First, a new, public method:

  TranscodeFromWeb(string url, out bool mainContentExtracted) { }

potentially with the overload 

  TranscodeFromWeb(string url, out bool mainContentExtracted, IPageFetcher fetcher) { }

so the http requests can be mocked for testing (or a public property). 

Another option is to create a new class NReadabilityWebTranscoder that calls 
NReadabilityTranscoder as many times as necessary.  NReadabilityTranscoder 
would be updated to set the nextPage parameter as a public property (or output 
parameter), if appropriate.