Giter VIP home page Giter VIP logo

springboot-solr's Introduction

简介

SpringBoot + solr + webmagic 学习solr写的demo

1 概述

根据https://github.com/ameizi/solrj-example 改的,原项目是Spring,改成Springboot solrj-example的webmagic价格和图片有点小问题,图片存取URL即可,爬取的数据从存入数据库改成直接放入solr

2 webmagic使用自定义Downloader

chromedrive是2.25版本的,支持的Chrome版本v53-55,如果版本不一致自行替换chromedrive或修改chrome版本

3 Solr启动

本人用的solr6.5.1,使用自带的jetty,无需配置,tomcat启动需8或者更高版本,jdk1.8或以上 windows启动 cd solr目录/bin + solr start
linux启动 cd cd solr目录/bin + ./solr start -force solr windows tomcat配置参考http://blog.csdn.net/liuzhen917/article/details/70328214?utm_source=itdadao&utm_medium=referral solr CentOS tomcat配置参考http://blog.csdn.net/l1028386804/article/details/70199983

4 Solr配置core:

  1. jetty 在\server\solr中添加product文件夹, tomcat 在web.xml中配置的 F:\solr_home 路径F:\solr_home下创建product
  2. 复制solr目录\server\solr\configsets\basic_configs 下的conf至上面的product目录下
  3. 启动solr在core admin 中添加core product
  4. 添加field ,把Product.java 中的属性都添加入core product中,可在页面scheme中添加或修改配置文件

5 爬虫运行

运行爬虫DemoApplicationTests.main获取数据

6 SpringBoot启动后界面主页

http://localhost:8111/jd

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.