Giter VIP home page Giter VIP logo

scrapy's Introduction

感谢关注,zone7 公众号,与您一起学习分享后端知识。本公众号涉及的知识点将会有:nodejs,python,docker,kubernetes,后端架构 zone7

scrapy

关于 scrapy 的各种初体验,本项目会持续更新,直到 scrapy 系列推文结束更新。

doubanTop250 | 豆瓣爬虫

此项目为 scrapy 初体验源码,关于豆瓣电影 Top 250 的爬虫文章, 相应的文章地址为:

rentHouseScrapy | 租房爬虫

入口:直接运行 main.py 即可

此项目为 scrapy 实战文章。

如何启动项目?

此项目为 scrapy 实战文章,关于房天下广州租房数据的爬取与分析 相应的文章地址为:

开始抓取:

clone 当前项目到你的电脑,安装好相应的依赖包,配置好数据库 ip 地址、账号、密码。

分析数据:

数据抓取完毕之后,进入到 analysis 目录下,直接运行文件,即可开始数据分析。

如何修改成自己所在城市的爬虫?`

修改 baseUrl = "http://gz.zu.fang.com/" 成你对应城市的地址就行。

girlScrapy | 妹子图

关于 scrapy 爬取图片的教程。此项目爬取了妹子图网站。

splashScrapy | 爬取QQ音乐周杰伦歌词与专辑

入口:直接运行 main.py 即可

相应的文章地址为:

doubanScrapyRedis | 分布式爬虫

入口:直接运行 main.py 即可(请配置好你本地的 redis 数据库)

jingDongScrapyRedis | 京东图书,分布式爬虫

入口:直接运行 main.py 即可(请配置好你本地的 redis 数据库)

scrapy's People

Contributors

zonezoen avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.