ycombinator returns 403 about enlive-tutorial HOT 3 CLOSED

The web server wants to see a User-Agent that looks like a browser.

from enlive-tutorial.

I found a fix here: https://stackoverflow.com/questions/18683565/handling-response-code-403-for-url-with-clojure-enlive/18693667#18693667

I forked the tutorial and made some changes that seem to fix the problem, but as I am SUPER new to Clojure, I'm hesitant to submit a pull request. Could someone take a look at what I did and let me know how I can polish it? In particular, I broke the User-Agent spoofing into a separate function, and I don't know enough to make sure connections get closed/garbage-collected properly.

from enlive-tutorial.

@backerman your code isn't working for me either. Instead of error 403, now I receive error 522

tutorial.scrape1=> (fetch-url *base-url*)
IOException Server returned HTTP response code: 522 for URL: https://news.ycombinator.com/  sun.net.www.protocol.http.HttpURLConnection.getInputStream (HttpURLConnection.java:1459)

I was able to get the code to work using the following user-agent:

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1944.0 Safari/537.36

from enlive-tutorial.