Giter VIP home page Giter VIP logo

feathers-solr's Introduction

feathers-solr

CI Coverage Status Known Vulnerabilities Download Status

A Feathers Solr Adapter. Tested with Solr 8.x, require at least >= Solr 5.x.

Installation

$ npm install feathers-solr --save

Important: feathers-solr implements the Feathers Common database adapter API and querying syntax. It use sthe native node http and https module.

API

service([options])

Returns a new service instance initialized with the given options.

const service = require('feathers-solr');
app.use('/search', service({host, core}));

Options:

  • host - The name of the Solr core / collection.
  • core - The name of the Solr core / collection.
  • events (optional) - A list of custom service events sent by this service
  • paginate (optional) - A pagination object containing a default and max page size
  • whitelist (DEPRECATED) - renamed to allow
  • allow (optional) - A list of additional query parameters to allow
  • multi (optional) - Allow create with arrays and update and remove with id null to change multiple items. Can be true for all methods or an array of allowed methods (e.g. [ 'remove', 'create' ])
  • id (optional, default: 'id') - The name of the id field property.
  • commitStrategy - (optional, default: { softCommit: true, commitWithin: 10000, overwrite: true }) - Define how Index changes are stored Solr Commits.
  • defaultParams (optional default: { echoParams: 'none' })- This params added to all Solr request.
  • defaultSearch - (optional, default: { defType: 'edismax', qf: 'name^10 age^1 gender' }) - Search strategy if query contains the param $search The Extended DisMax Query Parser.
  • queryHandler (optional default: '/query') - This params defines the Solr request handler to use.
  • updateHandler (optional default: '/update/json') - This params defines the Solr update handler to use.
  • createUUID (optional default: true) - This params add a UUID if not exist on data. Id's generated by crypto
  • escapeFn (optional default: (key: string, value: any) => { key, value }) - To apply escaping.
  • requestOptions (optional default: { timeout: 10 }) - The options passed to http.request.

Getting Started

The following example will create a Service with the name and endpoint solr.

const feathers = require('@feathersjs/feathers');
const express = require('@feathersjs/express');
const socketio = require('@feathersjs/socketio');

const Service = require('feathers-solr').Service;

// Create an Express compatible Feathers application instance.
const app = express(feathers());
// Turn on JSON parser for REST services
app.use(express.json());
// Turn on URL-encoded parser for REST services
app.use(express.urlencoded({ extended: true }));
// Enable REST services
app.configure(express.rest());
// Set up default error handler
app.use(express.errorHandler());

// Create a service
const options = {
  host: 'http://localhost:8983/solr',
  core: 'gettingstarted',
  paginate: {},
  events: ['testing']
};
app.use('gettingstarted', new Service(options));

// Start the server.
const port = 3030;

app.listen(port, () => {
  console.log(`Feathers server listening on port ${port}`)
});

Start Solr

 bin/solr start -e gettingstarted

Run the example with node app and go to localhost:3030/gettingstarted.

Querying

Feathers Docs Database Querying

Supported Solr specific queries

This Adapter uses the Solr JSON Request API.

The following params passed in raw to Solr. This gives the full access to the Solr JSON Request API.

  • $search (alias to query)
  • $params (alias to params)
  • $facet (alias to facet)
  • $filter (alias to filter)

To avoid full query (read) access, just whitelist only $search and add your query strategy into a Hook.

$search

An alias to Solr param query (string) - Solr Schemaless Mode

Simple Search in default field _text_.

query: {
  $search: 'John';
}

The Standard Query Parser - Some Search Examples:

  • Exact match: { $search: "John" }
  • Fuzzy match: { $search: "John~" }
  • Phrase match: { $search: "John Doe" }
  • Starts with: { $search: "Jo*" }
  • Ends with: { $search: "*n" }
  • Contains: { $search: "(John AND Doe)" }
  • Contain one: { $search: "(John OR Doe)" }

Define a default search query.

service.options.defaultSearch = {
  defType: 'edismax',
  qf: 'name^10 age^1 gender'
};

const response = await service.find({
  query: {
    $search: 'Doug 20 male'
  }
});

See $parmas example how query advanced search

$facet

An alias to Solr param facet

Get Min, Max and a Range Facet

query: {
    $facet: {
        age_min : "min(age)",
        age_max : "max(age)",
        age_ranges: {
            type: "range",
            field: "age",
            start: 0,
            end: 100,
            gap: 10
        }
    }
}

The response should look like this:

{
    QTime: 0,
    total: 50,
    limit: 10,
    skip: 0,
    data: [...],
    facet: {
        age_min: 1,
        age_max: 104,
        count: 54,
        age_ranges: {
            buckets: [{
                val: 0,
                count: 4
            }, {
                val: 25,
                count: 17
            }, {
                val: 50,
                count: 15
            }, {
                val: 75,
                count: 14
            }]
        }
    }
}

Get a Range Multi Facet

query:{
  $search:'blue',
  '{!tag=COLOR}color':'Blue',
  $facet:{
      sizes:{type:terms, field:size},
      colors:{type:terms, field:color, domain:{excludeTags:COLOR} },
      brands:{type:terms, field:brand, domain:{excludeTags:BRAND}
  }
}

$params

An alias to Solr param params. Allows you to access all Solr query (read) features like:

Spellchecker - Solr Spell Checking

const response = await service.find({
  query: {
    $search: 'John !Doe +age:[80 TO *]',
    $params: {
      'defType': 'edismax',
      'qf': 'name^10 city^5 age',
      'mm': '2<99% 7<80% 10<50%',
      'q.op': 'OR',
      'sow': true,
      'spellcheck': true,
      'spellcheck.accuracy': 0.7,
      'spellcheck.extendedResults': true,
      'spellcheck.collate': true,
      'spellcheck.count': 10,
      'spellcheck.maxCollations': 1,
      'spellcheck.maxCollationTries': 10,
      'spellcheck.collateExtendedResults': true,
      'spellcheck.onlyMorePopular': true,
      'spellcheck.dictionary': 'LANG_X_text_spell_token'
    }
  }
});

Suggester - Solr Suggester

const response = await service.find({
  query: {
    $suggest: 'john'
  }
});
const response = await service.find({
  query: {
    $params: {
      'group': true,
      'group.field': 'gender',
      'group.format': 'simple'
    }
  }
});

Highlight - Solr Highlighting

const response = await service.find({
  query: {
    $search: 'doug',
    $params: {
      'hl': true,
      'hl.field': 'name'
    }
  },
  paginate: { max: 10, default: 3 }
});

MoreLikeThis - Solr MoreLikeThis

const response = await service.find({
  query: {
    $search: 'male',
    $params: {
      'mlt': true,
      'mlt.fl': 'gender'
    }
  },
  paginate: { max: 10, default: 3 }
});
const response = await service.find({
  query: {
    $select: ['*', 'score', '_dist_:geodist()'],
    $params: {
      'sfield': 'location_p',
      'pt': '40.649558, -73.991815',
      d: 50,
      distanceUnits: 'kilometers',
      sort: 'geodist() asc'
   }
  },
  paginate: { max: 10, default: 3 }
});

$filter

An alias to Solr filter passed in raw. It's recomanded to go with the common Querying.

See more query variants JSON Facet API,Solr Facet Functions and Analytics, Solr Subfacets, Multi-Select Faceting

Service Methods

All service methods provide the multi options.

Service.create

The options.commitStrategy.override is true in default. This allow to override an existing id by service.create. Add the field _version_ to the $select params will return the document content with its version. Create with an existing id and _version_ for optimistic concurrency

Service.update

Will overide the complete Document. If the _version_ field is part of update content, it will be removed.

Service.patch

Use the Solr Updating Parts of Documents

Simple usage

service.patch(id, {age: 30});

Atomic Updates - Increment field age + 1

service.patch(id, {age: {inc:1}});

All Solr methods provided:

  • set - Set or replace the field value(s) with the specified value(s), or remove the values if 'null' or empty list is specified as the new value. May be specified as a single value, or as a list for multiValued fields.
  • add - Adds the specified values to a multiValued field. May be specified as a single value, or as a list.
  • add-distinct - Adds the specified values to a multiValued field, only if not already present. May be specified as a single value, or as a list.
  • remove - Removes (all occurrences of) the specified values from a multiValued field. May be specified as a single value, or as a list.
  • removeregex - Removes all occurrences of the specified regex from a multiValued field. May be specified as a single value, or as a list.
  • inc - Increments a numeric value by a specific amount. Must be specified as a single numeric value.

Service.remove

Provide delete by id ans query Delete all documents at once:

service.remove(null, {});

Performance considerations

Data mutating operations in Solr do not return document data. This is implemented as additional queries to return deletd or modified data.

To avoid this overhead use the client directly for bulk operations.

const options = {
  host: 'http://localhost:8983/solr',
  core: 'gettingstarted',
}

const client = solrClient('http://localhost:8983/solr');

await client.post('/update/json', { data: [] })

Maniging Solr

Using the solrClient for raw communication with solr. See adapter test how to:

  • create and delete a Solr core
  • add, 'update' and delete the Solr core schema
  • add and delete the Solr core config request handler and components
const options = {
  host: 'http://localhost:8983/solr',
  core: 'gettingstarted',
}

const client = solrClient('http://localhost:8983/solr');
await client.post('/admin/cores', { params: {...createCore, name: name} })
await client.post(`/${core}/schema`, { data: addSchema });
await client.post(`/${core}/config`, { data: addConfig });

License

Copyright (c) 2022

The MIT License (MIT)

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

feathers-solr's People

Contributors

dependabot[bot] avatar idaho avatar sajov avatar sassninja avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

feathers-solr's Issues

Example doesn't work

The "Getting Started" Example in the readme file doesn't work ("SolrClient is not a function").

Basic authorize

Hi
I want to improve to support authorize with Basic Authorization.
I have edited in jsonRequestApi.js as below :
headers : {
"Authorization" : "Basic " + new Buffer( opts.user + ":" + opts.pass).toString("base64")
}
but it didn't work. It's time out. Please tell me where i can edit to support Basic Authorization

Remove does not work correctly

service.remove(null,{query: { foo: "bar"}}) throws an error. Service.remove(null, {foo:"bar"}) works, but if I understand the feathers docs correctly, the first should be correct.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.