Comments (2)
已提供基于htmlUtil下载器
from xxl-crawler.
你好,感谢关注哈!
已支持JS渲染方式采集数据。得益于模块化结构设计,可自由扩展其他 "PageLoader" 实现,如 "Selenium" 方式等;
from xxl-crawler.
Related Issues (20)
- 爬取到的页面可能出现"截断"问题-----网瘾少年徐志摩
- CrawlerThread的process方法里判断当前链接是否是白名单链接逻辑有问题 HOT 1
- 建议使用jdk1.8 HOT 1
- ajax请求爬取
- [新需求]针对post请求,相同的url,根据参数不同返回不同结果的页面抓取实现 HOT 1
- maven引入1.2.2版本,测试07报错 HOT 1
- 【需求】VO嵌套 HOT 1
- 线程安全问题
- 扩散全站功能异常问题. HOT 1
- [issue] 多线程情况下,tryFinish()很小的概率会误判当前运行状态 HOT 1
- setWhiteUrlRegexs正则传参不起作用
- 发送post请求时返回400 HOT 1
- 请问一下,有登录后再爬取内容的功能吗?
- com.xuxueli.crawler.thread.CrawlerThread#processPage问题
- connect timeout超时处理
- 使用SeleniumPhantomjsPageLoader后,jsoup解析后document对象中的baseUri为空
- JsoupUtil工具类loadPageSource()方法里Connection没有调用requestBody
- 请问该项目还维护和更新吗 HOT 1
- 是否允许基于身份认证的爬虫
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from xxl-crawler.