Giter VIP home page Giter VIP logo

pantheon-systems / solr-power Goto Github PK

View Code? Open in Web Editor NEW
125.0 40.0 60.0 4.46 MB

A WordPress plugin to connect to Pantheon's Apache Solr search infrastructure, or your own!

Home Page: https://wordpress.org/plugins/solr-power/

License: GNU General Public License v2.0

PHP 89.74% CSS 2.09% JavaScript 3.69% Shell 3.59% Gherkin 0.10% SCSS 0.79%
apachesolr solr wordpress wordpress-plugin

solr-power's Introduction

Solr Search for WordPress

Contributors: getpantheon, Outlandish Josh, 10up, collinsinternet, andrew.taylor, danielbachhuber, mattleff, mikengarrett, jazzsequence, jspellman, pwtyler
Tags: search
Requires at least: 4.6
Requires PHP: 7.1
Tested up to: 6.5.2
Stable tag: 2.5.3
License: GPLv2 or later
License URI: http://www.gnu.org/licenses/gpl-2.0.html

Improve your user experience with the Apache Solr search engine for your WordPress website.

Description

Actively Maintained Lint and Test

Search is critical for your site, but the default search for WordPress leaves a lot to be desired. Improve your user experience with the Apache Solr search engine for your WordPress website.

  • Fast results, with better accuracy.
  • Enables faceting on fields such as tags, categories, author, and page type.
  • Indexing and faceting on custom fields.
  • Drop-in support for WP_Query with the solr_integrate parameter set to true.
  • Completely replaces default WordPress search, just install and configure.
  • Completely integrated into default WordPress theme and search widget.
  • Very developer-friendly: uses the modern Solarium library

Installation

The Solr Power plugin can be installed just like you'd install any other WordPress plugin.

Because Solr Power is intended to be a bridge between WordPress and the Apache Solr search engine, you'll need access to a functioning Solr 3.6 instance for the plugin to work as expected. This plugin does not support other versions of Solr. The plugin also requires PHP 7.1 or higher.

If you're using the Solr Power plugin on Pantheon, setting up Apache Solr is as easy as enabling the Apache Solr add-on in your Pantheon dashboard. Once you've done so:

  1. Configure which post types, taxonomies and custom fields to index by going to the Indexing tab of the Solr Power settings page.
  2. Index your existing content by going to the plugin options screen and selecting the applicable Actions:
      • Index Searchable Post Types
  3. Search on!
  4. See the examples/templates directories for more rich implementation guidelines.

If you're using the Solr Power plugin elsewhere, you'll need to install and configure Apache Solr. On a Linux environment, this involves four steps:

  1. Install the Java Runtime Environment.
  2. Run ./bin/install-solr.sh to install and run Apache Solr on port 8983.
  3. Configuring Solr Power to use this particular Solr instance by setting the PANTHEON_INDEX_HOST and PANTHEON_INDEX_PORT environment variables.
  4. Copying schema.xml to the Solr configuration directory (a path similar to solr/conf/schema.xml).

Alternatively, there are a couple of community-maintained Docker containers you may be able to use: kalabox/pantheon-solr, kshaner/solr.

In a local development environment, you can point Solr Power to a custom Solr instance by creating a MU plugin with:

<?php
/**
 * Define Solr host IP, port, scheme and path
 * Update these as necessary if your configuration differs
 */
putenv( 'PANTHEON_INDEX_HOST=192.168.50.4' );
putenv( 'PANTHEON_INDEX_PORT=8983' );
add_filter( 'solr_scheme', function(){ return 'http'; });
define( 'SOLR_PATH', '/solr/wordpress/' );

** Note for Lando users **

If you are using lando for development, the MU plugin is not needed. Lando auto configures everything for your local environment to connect to the docker index it maintains and if you overrite the ENV variables it will mess with that configuration.

Development

This plugin is under active development on GitHub:

https://github.com/pantheon-systems/solr-power

Please feel free to file issues there. Pull requests are also welcome! See CONTRIBUTING.md for information on contributing.

For further documentation, such as available filters and working with the SolrPower_Api class directly, please see the project wiki:

https://github.com/pantheon-systems/solr-power/wiki

WP-CLI Support

This plugin has WP-CLI support.

All Solr Power related commands are grouped into the wp solr command, see an example:

$ wp solr
usage: wp solr check-server-settings
   or: wp solr delete [<id>...] [--all]
   or: wp solr index [--batch=<batch>] [--batch_size=<size>] [--post_type=<post-type>]
   or: wp solr info [--field=<field>] [--format=<format>]
   or: wp solr optimize-index
   or: wp solr repost-schema
   or: wp solr stats [--field=<field>] [--format=<format>]

See 'wp help solr <command>' for more information on a specific command.

You can see more details about the commands using wp help solr:

**NAME**

  wp solr

**DESCRIPTION**

  Perform a variety of actions against your Solr instance.

**SYNOPSIS**

  wp solr <command>

**SUBCOMMANDS**

  check-server-settings      Check server settings.
  delete                     Remove one or more posts from the index.
  index                      Index all posts for a site.
  info                       Report information about Solr Power configuration.
  optimize-index             Optimize the Solr index.
  repost-schema              Repost schema.xml to Solr.
  stats                      Report stats about indexed content.

WP_Query Integration

Use Solr in a custom WP_Query instead of querying a database. Add 'solr_integrate' => true to the query arguments.

NOTE: Currently, only basic queries, tax_query, meta_query and date_query are supported. See examples/example.custom_WP_Query.php for an example.

A meta_query can use the following compare operators:

  • '='
  • '!='
  • '>'
  • '>='
  • '<'
  • '<='
  • 'LIKE'
  • 'NOT LIKE'
  • 'IN'
  • 'NOT IN'
  • 'BETWEEN'
  • 'NOT BETWEEN'
  • 'EXISTS'
  • 'NOT EXISTS'

('REGEXP', 'NOT REGEXP', and 'RLIKE' are not supported.)

Configuration Tips

Searching by author name

To support searching by author name (e.g. where "Pantheon" would return posts authored by the "Pantheon" user), add the following to your custom schema.xml:

<copyField source="post_author" dest="text"/>

Boosting relevancy score by publish date

The following guidance can be used to extend the Solr index and modify boosts beyond just this example.

To support math functions on dates, you must add a custom schema.xml to Solr and reindex with the new schema.

Add the following to schema.xml:

  <!-- Add to <types> -->
  <!-- See: https://lucene.apache.org/solr/6_2_0/solr-core/org/apache/solr/schema/TrieDateField.html -->
  <fieldType name="tdate" class="solr.TrieDateField" omitNorms="true" precisionStep="6" positionIncrementGap="0"/>

  <!-- Add to <fields> -->
  <field name="post_date_iso" type="tdate" indexed="true" stored="true" required="true" />

Add the following to your functions.php file.

  <?php
  /**
   * Hooks into the document build process to add post date field in proper format.
   */
  function my_solr_build_document( $doc, $post_info ) {
        $post_time = strtotime( $post_info->post_date );
        // Matches format required for TrieDateField
        $doc->setField( 'post_date_iso', gmdate( 'c\Z', $post_time ) );
        return $doc;
  }
  add_filter( 'solr_build_document', 'my_solr_build_document', 10, 2 );

  /**
   * Hooks into query processor, Dismax, to add publish date boost.
   * See: https://www.metaltoad.com/blog/date-boosting-solr-drupal-search-results
   */
  function my_solr_dismax_query( $dismax ) {
        $dismax->setQueryParser( 'edismax' );
        $dismax->setBoostQuery( 'recip(abs(ms(NOW/HOUR,post_date_iso),3.16e-11,1,1))' );
        return $dismax;
  }
  add_filter( 'solr_dismax_query', 'my_solr_dismax_query' );

Common issues

  • Failing to post the schema.xml will result in an error during indexing, "Missing post_date_iso field."
  • If you have the field and type in the schema, but don't add the solr_build_document filter, you will get a similar error.
  • If the post_date_iso field is missing from the index, Solr will ignore this boost and return regular results.
  • Trying to use a regular date field for the boost query will result in an error in the request instead of results.

Explicit Commit vs Autocommit

Once solr has sent the data to the solr server, solr must COMMIT the data to the index and adjust the index and relevancy ratings accordingly before that data can appear in search results.

By default, Solr Search for WordPress has auto-commit disabled. The index is committed when the uncommitted item is two minutes old, or the cron runs. By default, the cron runs on the Pantheon platform every hour.

When autocommit is enabled, Solr Search for WordPress commits data when it sends every post. When running on Pantheon, we recommend leaving autocommit disabled to aid overall site performance.

To enable autocommit, add the following to wp-config.php or an mu-plugin.

define( 'SOLRPOWER_DISABLE_AUTOCOMMIT', false );

To force-commit data outside of a normal cron run, from the command line, you can run the command below or simply force a cron-run.

wp solr commit

Security Policy

Reporting Security Bugs

Please report security bugs found in the Solr Power plugin's source code through the Patchstack Vulnerability Disclosure Program. The Patchstack team will assist you with verification, CVE assignment, and notify the developers of this plugin.

solr-power's People

Contributors

ahebrank avatar allan23 avatar ataylorme avatar bamadesigner avatar chriswiegman avatar cjbos avatar danielbachhuber avatar defunctl avatar dependabot[bot] avatar erik-pantheon avatar ghamok avatar iamntz avatar jazzsequence avatar jhned avatar jimkeller avatar kporras07 avatar kshaner avatar mattleff avatar mgburns avatar mikengarrett avatar mikevanwinkle avatar ocean90 avatar patelutkarsh avatar pavellishin avatar pwtyler avatar richard-decastongrene-electro avatar ritesh-patel avatar scottbuscemi avatar stevector avatar yuri-pochepko avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

solr-power's Issues

Rename Project: "solr-power"

facepalm - http://martinfowler.com/bliki/TwoHardThings.html

In preparation for release, I shortened the name of this project, which created a conflict with an existing plugin upstream, as well as creating issues with the admin UI. Mea culpa.

Alternate Names

  • Apache Solr for WordPress / apachesolr-for-wordpress
  • SolrPress / solrpress

Feel free to add your own in the comments.

Create a public interface for getting server info

Rather than having the following throughout:

$server = array(
    'ping_status' => $ping ? 'successful' : 'failed',
    'ip_address'  => getenv( 'PANTHEON_INDEX_HOST' ),
    'port'        => getenv( 'PANTHEON_INDEX_PORT' ),
    'path'        => SolrPower_Api::get_instance()->compute_path(),
);

We should have a method on the API class that returns this data, for use by the options page and CLI.

syntax error

wp-content/plugins/solr-power/vendor/solarium/solarium/library/Solarium/Core/Client/Adapter/Curl.php:156

and i am not able to index products in live environment

Unit and system tests

We need to get some unit tests and system tests in place for this project to be able to move fast going forward.

Starting this issue to collect/prioritize test criteria.

Document local setup steps

For users who want to work with the plugin locally, we should have a short "how-to" guide for getting Solr running locally.

Implement basic faceting widget

One of the best features of a Solr backend is the ability to easily "facet" the search query. We should make it easy for users to discover this power. Using the built-in categories to power a basic "Faceted Search" widget would go a long way towards this.

Other primitive facets to consider:

  • Post type
  • Author
  • Date (month?)

Configurability for search order?

Right now the defauly search query will still return posts sorted by created order. I think the default should be to sort them by their relevance (score). It would make for a pretty nifty demo:

  • Before: Here are your Vanilla search results in date posted. (sad trombone)
  • After: Behold! Solr Powered search results, which pulled an older post to the top because it's clearly more relevant. (gong!)

Add wp-cli support

We should support actions from the plugins settings page in wp-cli.

  • Check Server Settings
  • Optimize Index
  • Delete All
  • Repost schema.xml
  • Index Searchable Post Types

We should also consider informative commands.

  • info should return ping status, Solr server IP address, Solr server port and Solr server path.
  • stats should return a table of indexing status by post type

I think these two would create a minimum viable product. Additional commands can be items from the indexing tab in the plugin settings such as setting the default search operator.

Include search results out of the box

To go along with #12 (a search widget), we need to include out-of-the-box search results.

The initial vanilla results should be equivalent to WP core search results except the default sort should be by relevance rather than date, and of course they would be fetched from the Solr index.

This will prevent developers from needing to do a theme-override to experience the power of search results.

Improved status on Admin screen

We should provide an improved status page as part of the first screen of admin UI.

  • It should do a ping and report the status (which will surface if something is wrong right away)
  • We can also show the current count of indexed posts, comments, pages, etc.

pager bug

Page-links for paginated posts produce 404s

Disabling the Output Result Pager option does not seem to remove page-links as assumed (not sure that's the intent behind the option though?)

screen shot 2016-03-25 at 4 15 06 pm


screen shot 2016-03-25 at 3 59 58 pm

Example query

  • examples/example.custom_field_query.php
  • edit readme.txt

Split admin interfaces

Currently the single admin page has severak sections:

  1. Solr server status and configuration
  2. Index options
  3. Result options
  4. Actions
  5. Debug (solr query)

We should separate these out into their own tabs and forms, so they can be documented and utilized separately, and it will be easier for customers to set up solr.

Custom Fields Bug

Index is attempted on custom fields but they are not included within results

Multisite Indexing

How should indexing be handled in a multisite environment?
Should each site have their own admin page or just one main page for the network admin?

CC: @joshkoenig

Handle un-posted schema.xml file

This is a Pantheon specific thing, but I want to handle it better for a good first time experience.

When the plugin is first enabled, the developer needs to post the schema.xml file before search can actually work. Otherwise it will 404, leading to an unhandled exception from Solarium.

I think we should make posting the default schema.xml a part of enabling the plugin.

We also need to figure out how to catch the 404 and provide a helpful error message to the admin. This will happen when users first push the plugin into a new environment, and if they are cloning the DB they may not go through the enable step again. Catching the 404 and having a friendly error message would make for a better experience.

De-fork Solarium

Pre-requisite for #14 - we have a little shim in Solarium. We should subclass instead like the proper OOPers that we are.

Parse Error in Curl.php

[31-Aug-2015 12:22:21 UTC] PHP Parse error: syntax error, unexpected '[' in /srv/bindings/89d174f37673485283ee1f869b9d854d/code/wp-content/plugins/solr-for-wordpress/vendor/solarium/solarium/library/Solarium/Core/Client/Adapter/Curl.php on line 156

if ($endpoint->getOption('ssl')['local_cert']) {
           curl_setopt($handler, CURLOPT_SSLCERT, realpath($endpoint->getOption('ssl')['local_cert']));
       }

This site is using PHP 5.3

Case:
#44014

@joshkoenig @erik-pantheon

Enable full query interface

Currently, the interface for the WP Solr plugin only allows searches for string values. While this is useful for the layman, most developers who are using Solr require the ability to test more complicated queries. The Pantheon's Drupal Solr Module does this well. One can run something like:

/select?fl=label,bundle,ts_ol_full_text&fq=bundle:bill%20AND%20ts_ol_full_text:property&wt=json

...and return results with the Drupal Solr module. Being able to do the same in Solr Powr would be invaluable.

Script to go from GitHub repo to WordPress SVN

We need a script for converting this github project to publish via SVN to wp.org/plugins. There are a few already out there, eg:

http://scribu.net/blog/deploying-from-git-to-svn.html

https://danielbachhuber.com/2012/09/30/git-in-my-subversion/

And some tools that could help:

https://github.com/benbalter/WP-Readme-to-Github-Markdown

It would be great to make something that we can share w/others on this. Would be an excellent best-practice to share.

The goal is to have a "scripts" directory in the plugin that includes a shell script to do the translation/release in a repeatable way. We can rely on the local script-running user having all appropriate access credentials.

Bonus Points it would be great to lay out a pattern for doing development work on Pantheon as well, pulling changes from a Pantheon site repo into this repo via git subtree merge. That would be a separate item in the scripts directory.

Use WPCS to ensure the codebase adheres to WordPress Coding Standards

WPCS (https://github.com/WordPress-Coding-Standards/WordPress-Coding-Standards) is a series of PHP Codesniffer Sniffs to ensure a codebase adheres to WordPress' coding standards. We should set it up for the project, as we already have for https://github.com/pantheon-systems/wp-redis et al.

Setting up WPCS involves:

Once it's set up, we'll need to:

  • Update all files to adhere to WPCS

Options not saving properly

The options page is not saving preferences. Checkboxes are going un-saved, and key text fields are being filled-in by Array. This is the result after un-checking "index pages" and then clicking "Save Changes":

screen shot 2015-08-12 at 11 37 35 am

Not "index pages" is still checked, and the two text fields below have Array as their value.

Could be related to #19 as a name-spacing issue?

WP-CLI Integration

When you've got a lot of context to index, doing it via the web UI is le suck.

Better would be this:

> wp-cli solr index --count=100
Indexing the next 100 posts
..........
Done! 100 posts sent to index in 7.16s

I think this plus a few other things could be enough for a 0.3 release!

Use Behat to test Solr Power on Pantheon

On a couple other Pantheon plugins now, we're using Behat to verify the plugin works as expected on Pantheon (e.g. pantheon-systems/wp-redis#103). It would be cool if Solr Power did this too.

At a high-level, here's how it works:

  • We're using Circle (in addition to Travis) to automate the creation and destruction of a Pantheon environment for an existing Pantheon site. In accordance to our existing naming scheme, you'd use solr-power as the site name.
  • bin/behat-prepare.sh creates the environment, pushes the current plugin copy up, activates it, and performs any other configuration steps needed.
  • bin/behat-test.sh runs the Behat tests against the created environment
  • bin/behat-cleanup.sh removes the environment after the tests are done running.

You can likely copy and lightly modify most of the referenced pull request. I'd be keen to get your input on the documentation, and implementation generally.

cc @stevector

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.