using Godep

Currently the user cannot customize option values.

Not sure if that's helpful, just thought I'll dump this here as an reproducible example.

package main

import (
	"fmt"

	"github.com/mmcdole/gofeed"
	readability "github.com/philipjkim/goreadability"
)

func main() {
	fp := gofeed.NewParser()
	feed, err := fp.ParseURL("https://roadsandkingdoms.com/feed/")
	if err != nil {
		return
	}
	opt := readability.NewOption()
	opt.DescriptionExtractionTimeout = 4000

	opt.ImageRequestTimeout = 3000
	for _, i := range feed.Items {
		fmt.Println(i.Link)
		content, err := readability.Extract(i.Link, opt)
		if err != nil {
			return
		}
		fmt.Println(content.Title)
		fmt.Println(content.Description)
		break
	}
}

Output:

Desktop|⇒ go run thing.go
https://roadsandkingdoms.com/2018/rk-insider-lets-talk-about-brazil/
R&K Insider: Let's talk about Brazil
 Frustration, despair, and heightened risk: those are the common themes in conversations with friends and colleagues who have been covering the protests in Gaza and the West Bank.
Desktop|⇒

https://github.com/mauidude/go-readability works correctly so I'm going to switch to that one for now.

For example:

func bodyStr(res *http.Response) (s *string, err error) {
    defer res.Body.Close()
    body, err := ioutil.ReadAll(res.Body)
    if err != nil {
        return nil, fmt.Errorf("ioutil.ReadAll failed: %v", err)
    }
    result := string(body)
    _, cs, _ := charset.DetermineEncoding(body, res.Header.Get("Content-Type"))
    if cs == "windows-1252" {
        cs = "euc-kr"
    }
    if cs != "utf-8" {
        result, err = iconv.ConvertString(result, cs, "utf-8")
        if err != nil {
            return nil, fmt.Errorf("Converting string from %v to utf-8 failed, url: %v",
                cs, res.Request.URL.String())
        }
    }
    return &result, nil
}

Should enhance documentation for https://godoc.org/github.com/philipjkim/goreadability .

hi, I saw this on the Readme,

Command Line Tool
TODO

Any chance you could actualy do this and also release it as a binary on github ?
Your goreadablity would make a great little portable, easy to install cli tool to combine as a filter/pipe to other tools.

1. A a "reader mode"/"pocket mode" in the browser (Qutebrowser)
   An example (based on the old python-readability), https://github.com/qutebrowser/qutebrowser/blob/master/misc/userscripts/readability

2. As a filter/pipe for other cli tools, ex, newsboat (rss reader), shell scripts, text editor (vim), cli web browsers, hs etc....

Not everyone has the entire go distribution installed or as a simple user is familiar with compiling.Besides that releasing a binary, would make it easier for others to create a package for a Linux distro, based on the binary (ex. Archlinux AUR).

Give it a url or html file or read from standard input, just dump the cleaned up html(with?without pictures?) to a file or standard output. Additionaly, you could:

• add the option to dump it as plain text
• add the option to dump it out to the terminal (ANSI escaped, color, bold) ready to be piped to 'less' etc..
  (See also https://github.com/lunny/html2md, https://github.com/mitchellh/go-wordwrap)

IMPORTANT: dont forget UTF8 Unicode encoding and east asian languages (CJK). For some reason almost everyone who releases these html parsing things, always forgets about foreign language pages/sites.

Extremely useful in the Linux world.

PS:
These guys have made something similar, but doesnt seem maintained/updated anymore,
https://github.com/feeeper/newspaper/blob/readability-support/main.go

The difference is that wihle yours derives from the original Arc90/ruby,
theirs seems to be derived from go-shiori/go-readability (mozilla pocket/readability port)