Giter VIP home page Giter VIP logo

predictor-word's Introduction

predictor-word

elasticsearch 扩展 ikpinyin 插件的 docker 镜像,适合直接对生产环境进行部署。

Version

  • elasticsearch -> 5.1.1
  • ik -> 5.1.1
  • lc-pinyin -> 5.1.1

Require

  • docker
  • docker-compose
  • npm (因为作者是 nodejs 使用者,所以习惯使用 npm 进行命令管理,也可以直接使用 docker-compose cli)

Init

  • Clone project
$ git clone https://github.com/TalkWIthKeyboard/predictor-word.git
  • Build image
$ npm run build

Create Index

PUT localhost:9200/index

{
  "settings": {
    "analysis": {
      "analyzer": {
        // 汉语转拼音首字母分词
        "ik_letter_smart": {
          "type": "custom",
          "tokenizer": "ik_max_word",
          "filter": [
            "lc_first_letter"
          ]
        },
        // 汉语转拼音全拼分词
        "ik_py_smart": {
          "type": "custom",
          "tokenizer": "ik_max_word",
          "filter": [
            "lc_full_pinyin"
          ]
        },
        // 汉语分词
        "ik_china_english_smart": {
          "type": "custom",
          "tokenizer": "ik_max_word",
          "filter": [
            "stemmer"
          ]
        }
      }
    }
  }
}

Example

GET localhost:9200/index/_analyze?analyzer=ik_china_english_smart&text=hello你好

{
    "tokens": [
        {
            "token": "hello",
            "start_offset": 0,
            "end_offset": 5,
            "type": "ENGLISH",
            "position": 0
        },
        {
            "token": "你好",
            "start_offset": 5,
            "end_offset": 7,
            "type": "CN_WORD",
            "position": 1
        }
    ]
}
GET localhost:9200/index/_analyze?analyzer=ik_letter_smart&text=hello你好

{
    "tokens": [
        {
            "token": "hello",
            "start_offset": 0,
            "end_offset": 5,
            "type": "ENGLISH",
            "position": 0
        },
        {
            "token": "nh",
            "start_offset": 5,
            "end_offset": 7,
            "type": "CN_WORD",
            "position": 1
        }
    ]
}
GET localhost:9200/index/_analyze?analyzer=ik_china_english_smart&text=hello你好

{
    "tokens": [
        {
            "token": "hello",
            "start_offset": 0,
            "end_offset": 5,
            "type": "ENGLISH",
            "position": 0
        },
        {
            "token": "nihao",
            "start_offset": 5,
            "end_offset": 7,
            "type": "CN_WORD",
            "position": 1
        }
    ]
}

Tips

  • 因为作者是将该镜像部署到廉价腾讯云服务器使用,所以对 elasticsearch 的内存使用进行了限制(默认是2G),如果有充分的资源可以注释掉 docker-compose.yml 中的 ES_JAVA_OPTS: -Xms512m -Xmx512m
  • 关闭了访问需要登录的设置:xpack.security.enabled
  • 添加新的分析器:
    • 关闭 index
      POST localhost:9200/index/_close
      
    • 添加分析器
      PUT localhost:9200/index/_settings
      
      {
        "analysis": {
          "analyzer": {
            "newAnalyzer": {
              "type": "custom",
              "tokenizer": "ik_max_word",
            }
          }
        }
      }
      
    • 开启 index
      POST localhost:9200/index/_open
      

predictor-word's People

Contributors

talkwithkeyboard avatar

Watchers

 avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.