Comments (3)
The web server wants to see a User-Agent that looks like a browser.
from enlive-tutorial.
I found a fix here: https://stackoverflow.com/questions/18683565/handling-response-code-403-for-url-with-clojure-enlive/18693667#18693667
I forked the tutorial and made some changes that seem to fix the problem, but as I am SUPER new to Clojure, I'm hesitant to submit a pull request. Could someone take a look at what I did and let me know how I can polish it? In particular, I broke the User-Agent spoofing into a separate function, and I don't know enough to make sure connections get closed/garbage-collected properly.
from enlive-tutorial.
@backerman your code isn't working for me either. Instead of error 403, now I receive error 522
tutorial.scrape1=> (fetch-url *base-url*)
IOException Server returned HTTP response code: 522 for URL: https://news.ycombinator.com/ sun.net.www.protocol.http.HttpURLConnection.getInputStream (HttpURLConnection.java:1459)
I was able to get the code to work using the following user-agent:
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1944.0 Safari/537.36
from enlive-tutorial.
Related Issues (11)
- links to source and html are (still) broken HOT 2
- scrape1 HN url needs to be updated with https
- It seems scrape2.clj produces unexpected results HOT 1
- Templates can be set to autoreload as of 1.1.2
- scrape1 does not match titles and points correctly HOT 1
- Scraping nytimes.com outdated
- Random RuntimeException: Unable to resolve symbol... HOT 1
- Update this tutorial HOT 1
- REPL will not start.
- :data should be :links or vice verse
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from enlive-tutorial.