Giter VIP home page Giter VIP logo

redmine_full_text_search's Introduction

Full text search plugin

This plugin provides the following features:

  • Fast and rich full text search to Redmine
  • Display similar issues on an issue page

Requirements

If you're using MySQL or MariaDB, you need Mroonga 9.03 or later.

If you're using PostgreSQL, you need PGroonga 2.2.0 or later.

Mroonga and PGroonga uses Groonga as full text search engine. You need Groonga 9.0.3 or later.

Optional requirements

  • ChupaText server: Text extractor server.

How to use

Install PGroonga or Mroonga

See PGroonga document

See Mroonga document

Install ChupaText server (optional)

You can choose one of them to install ChupaText:

  1. Use chupa-text-docker
  2. Use chupa-text-vagrant
  3. Use chupa-text-http-server

chupa-text-docker is recommended. See chupa-text-docker document to install chupa-text-docker.

See chupa-text-vagrant document to install chupa-text-vagrant.

chupa-text-http-server is a normal Ruby on Rails application like Redmine itself. You can deploy chupa-text-http-server as a normal Ruby on Rails application.

Install this plugin

$ cd redmine
$ git clone https://github.com/clear-code/redmine_full_text_search.git plugins/full_text_search
$ bundle install
$ RAILS_ENV=production bin/rails redmine:plugins:migrate

Restart Redmine.

NOTE for PGroonga:

If you use normal user for Redmine. You must run the following query as a super user before run RAILS_ENV=production bin/rails redmine:plugins:migrate:

CREATE EXTENSION IF NOT EXISTS pgroonga;

Configure this plugin

Open https://YOUR_REDMINE_SERVER/settings/plugin/full_text_search and configure items in the page. If you install ChupaText server, you must configure "ChupaText server URL". If you install your ChupaText server by chupa-text-docker or chupa-text-vagrant on the same host, it's http://127.0.0.1:20080/extraction.json.

Synchronize data

You need to create index for existing data. You need to run full_text_search:synchronize task until no more synchronize target data.

$ cd redmine
$ RAILS_ENV=production bin/rails full_text_search:synchronize
$ RAILS_ENV=production bin/rails full_text_search:synchronize
$ RAILS_ENV=production bin/rails full_text_search:synchronize
...

Synchronize query expansion list

This plugin supports query expansion. You can use this feature to implement synonym search.

You can administrate query expansion list by Web UI in administration page or data file.

You can use the following format for data file:

  • CSV
  • JSON

If you use CSV, use the following format:

SOURCE1,DESTINATION1
SOURCE2,DESTINATION2
...

Example:

MySQL,MySQL
MySQL,MariaDB
MariaDB,MySQL
MariaDB,MariaDB

If you use JSON, use one of the following formats:

[
["SOURCE1", "DESTINATION1"],
["SOURCE2", "DESTINATION2"],
...
]
[
{"source": "SOURCE1", "destination": "DESTINATION1"},
{"source": "SOURCE2", "destination": "DESTINATION2"},
...
]

Examples:

[
["MySQL",   "MySQL"],
["MySQL",   "MariaDB"],
["MariaDB", "MySQL"],
["MariaDB", "MariaDB"]
]
[
{"source": "MySQL",   "destination": "MySQL"},
{"source": "MySQL",   "destination": "MariaDB"},
{"source": "MariaDB", "destination": "MySQL"},
{"source": "MariaDB", "destination": "MariaDB"}
]

You can synchronize query expansion list with the data file by the following command:

$ cd redmine
$ RAILS_ENV=production bin/rails full_text_search:query_expansion:synchronize INPUT=query-expansion.csv

You can confirm the current query expansion list in administration page.

How to recover broken database

Mroonga

Mroonga isn't crash safe. If MySQL is crashed while updating data in Mroonga, Mroonga data may be broken.

Here is the instruction to recover from broken Mroonga data.

If you're using Redmine plugin Delayed Job, you need to stop workers and delete jobs for this plugin:

$ sudo -H systemctl stop [email protected]
$ cd redmine
$ RAILS_ENV=production bin/rails runner 'Delayed::Job.where(queue: "full_text_search").delete_all'

Stop MySQL:

$ sudo -H systemctl stop mysqld

Remove Mroonga related files:

$ cd redmine
$ database_name=$(RAILS_ENV=production bin/rails runner 'puts ActiveRecord::Base.configurations[Rails.env]["database"]')
$ sudo -H sh -c "rm -rf /var/lib/mysql/${database_name}.mrn*"

Start MySQL:

$ sudo -H systemctl start mysqld

Check that Mroonga has been properly installed based on the Mroonga manual. If Mroonga isn't installed, install Mroonga like the following:

$ mysql -u root -p < /usr/share/mroonga/install.sql

Destruct tables explicitly for this plugin:

$ mysql -u root -p ${database_name}
> DROP TABLE IF EXISTS fts_query_expansions;
> DROP TABLE IF EXISTS fts_targets;
> DROP TABLE IF EXISTS fts_tags;
> DROP TABLE IF EXISTS fts_tag_types;
> DROP TABLE IF EXISTS fts_types;

Recreate schema for this plugin:

$ cd redmine
$ RAILS_ENV=production bin/rails redmine:plugins:migrate NAME=full_text_search VERSION=0
$ RAILS_ENV=production bin/rails redmine:plugins:migrate NAME=full_text_search

If you're using Redmine plugin Delayed Job, you need to start workers:

$ sudo -H systemctl start [email protected]

Synchronize:

$ cd redmine
$ RAILS_ENV=production bin/rails full_text_search:synchronize UPSERT=later

How to develop

Preparation

Here are some useful tools to prepare:

  • dev/run-mysql.sh and dev/run-postgresql.sh: Run new RDBMS instance by Docker.
  • dev/initialize-redmine.sh: Initialize Redmine.
  • dev/run-test.sh: Run tests for the full text search plugin.

Clone source codes. This is required only once.

$ git clone https://github.com/redmine/redmine.git
$ cd redmine
$ git checkout 5.0-stable # or something
$ git clone [email protected]:${YOUR_FORK}/redmine_full_text_search.git plugins/full_text_search

You can add more plugins to plugins/.

Choose suitable database configuration:

$ ln -fs ../plugins/full_text_search/config/database.yml.example.${REDMINE_VERSION}.${RDBMS} config/database.yml

Here is an example to use Redmine 5.0 and MySQL:

$ ln -fs ../plugins/full_text_search/config/database.yml.example.5.0.mysql config/database.yml

Run RDBMS.

For MySQL:

$ plugins/full_text_search/dev/run-mysql.sh /tmp/mysql

For PostgreSQL:

$ plugins/full_text_search/dev/run-postgresql.sh /tmp/postgresql

Initialize Redmine:

$ plugins/full_text_search/dev/initialize-redmine.sh

Run tests:

$ plugins/full_text_search/dev/run-test.sh

How to add a new search target

You need to create mapper classes for each search target. See lib/full_text_search/*_mapper.rb for details.

You need to add require_dependency "full_text_search/XXX_mapper to init.rb to load these new mapper classes.

You can confirm your changes by usual Redmine development ways.

For example, here is a command line to run Redmine:

$ bin/rails server

You need to add tests to the following files:

  • test/unit/full_text_search/XXX_test.rb
  • test/functional/full_text_search/search_controller_test.rb

Here is a command line to run tests:

$ plugins/full_text_search/dev/run-test.sh

You can specify test options by TESTOPTS:

$ plugins/full_text_search/dev/run-test.sh TESTOPTS="-n/test_XXX/"

You can see all test options by TESTOPTS=--help:

$ plugins/full_text_search/dev/run-test.sh TESTOPTS=--help

Authors

License

The MIT License. See LICENSE for details.

Exceptions

  • asserts/stylesheets/fontawesome*/**/*
    • Author: @fontawesome
    • Fonts: SIL OFL 1.1 License
    • Codes: MIT License
    • See asserts/stylesheets/fontawesome*/LICENSE.txt for details

Contributing

  1. Fork it ( http://github.com/clear-code/redmine_full_text_search/fork )
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Add some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create new Pull Request

redmine_full_text_search's People

Contributors

a9zawa avatar abetomo avatar dependabot[bot] avatar hashidatks avatar iquiw avatar komainu8 avatar kou avatar okkez avatar otegami avatar ppyv avatar tkusukawa avatar tsuchm avatar two-pack avatar uhuntu avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

redmine_full_text_search's Issues

Pluggable search functionality for Redmine

ここでやる話でもないが、考えたことをメモしておかないと忘れてしまうので。

/search をプラグイン側で丸ごと(controllerとview)上書きすることができるので、力技で実現することはできる。

PoCレベルでいいので、↑の方針で一度実装してみるのがよいような気がする。
知見が足りなくて、考えていることが正しいのかどうかよくわからない。

今のところ結局のところページネーションとキャッシュを隠蔽している Redmine::Search の API をどうにかすればいいようには見えている。

Wrong reorder for Wiki page search

lib/full_text_search/mroonga.rbfetch_ranks_and_ids

joins("INNER JOIN fts_wiki_contents ON (wiki_contents.id = fts_wiki_contents.wiki_content_id)").
reorder("score1 DESC, score2 DESC").distinct.limit(limit).map do |record|
if order_target == "score"
  [record.score1 * 100 + record.score2, record.id]
else
  [record.order_target.to_i, record.id]
end

と書いているけど、order_target"score"じゃないときはreorder("order_target, id")にしないといけないと思う。

そうしないと、limit10で実際にヒットした件数が100件のときに違う結果が返ると思う。

Duplicate key error in searcher_records table.

When migrating full_text_search, the following error was displayed.

redmine              | Migrating plugins. Please be patient, this could take a while...
redmine              | rake aborted!
redmine              | StandardError: An error has occurred, all later migrations canceled:
redmine              |
redmine              | Mysql2::Error: Duplicate entry '1073-CustomValue' for key 'index_searcher_records_unique':     INSERT INTO searcher_records(original_id, original_type, project_id, project_name, original_created_on, original_updated_on, value, custom_field_id)
redmine              |     SELECT base.id, 'CustomValue', p.id, p.name, NULL, NULL, base.value, base.custom_field_id FROM custom_values AS base
redmine              |      JOIN custom_fields AS f ON (base.custom_field_id = f.id)
redmine              |     JOIN projects AS p ON (base.customized_id = p.id)
redmine              |  WHERE searchable = true;
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/mysql2-0.4.10/lib/mysql2/client.rb:120:in `_query'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/mysql2-0.4.10/lib/mysql2/client.rb:120:in `block in query'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/mysql2-0.4.10/lib/mysql2/client.rb:119:in `handle_interrupt'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/mysql2-0.4.10/lib/mysql2/client.rb:119:in `query'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:317:in `block in execute'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/connection_adapters/abstract_adapter.rb:484:in `block in log'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activesupport-4.2.11/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/connection_adapters/abstract_adapter.rb:478:in `log'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:317:in `execute'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/connection_adapters/mysql2_adapter.rb:217:in `execute'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:665:in `block in method_missing'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:634:in `block in say_with_time'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:634:in `say_with_time'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:654:in `method_missing'
redmine              | /home/redmine/redmine/plugins/full_text_search/db/migrate/20170630063757_copy_records_to_searcher_records.rb:124:in `load_custom_values'
redmine              | /home/redmine/redmine/plugins/full_text_search/db/migrate/20170630063757_copy_records_to_searcher_records.rb:33:in `block (2 levels) in change'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:511:in `up'
redmine              | /home/redmine/redmine/plugins/full_text_search/db/migrate/20170630063757_copy_records_to_searcher_records.rb:4:in `block in change'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:547:in `block in reversible'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:746:in `execute_block'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:547:in `reversible'
redmine              | /home/redmine/redmine/plugins/full_text_search/db/migrate/20170630063757_copy_records_to_searcher_records.rb:3:in `change'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:608:in `exec_migration'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:592:in `block (2 levels) in migrate'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:591:in `block in migrate'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/connection_adapters/abstract/connection_pool.rb:292:in `with_connection'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:590:in `migrate'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:768:in `migrate'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:1023:in `block in execute_migration_in_transaction'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:1071:in `ddl_transaction'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:1022:in `execute_migration_in_transaction'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:984:in `block in migrate'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:980:in `each'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:980:in `migrate'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:823:in `up'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:801:in `migrate'
redmine              | /home/redmine/redmine/lib/redmine/plugin.rb:481:in `migrate_plugin'
redmine              | /home/redmine/redmine/lib/redmine/plugin.rb:453:in `migrate'
redmine              | /home/redmine/redmine/lib/redmine/plugin.rb:467:in `block in migrate'
redmine              | /home/redmine/redmine/lib/redmine/plugin.rb:466:in `each'
redmine              | /home/redmine/redmine/lib/redmine/plugin.rb:466:in `migrate'
redmine              | /home/redmine/redmine/lib/tasks/redmine.rake:135:in `block (3 levels) in '
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/rake-12.3.2/exe/rake:27:in `'
redmine              |
redmine              | Caused by:
redmine              | ActiveRecord::RecordNotUnique: Mysql2::Error: Duplicate entry '1073-CustomValue' for key 'index_searcher_records_unique':     INSERT INTO searcher_records(original_id, original_type, project_id, project_name, original_created_on, original_updated_on, value, custom_field_id)
redmine              |     SELECT base.id, 'CustomValue', p.id, p.name, NULL, NULL, base.value, base.custom_field_id FROM custom_values AS base
redmine              |      JOIN custom_fields AS f ON (base.custom_field_id = f.id)
redmine              |     JOIN projects AS p ON (base.customized_id = p.id)
redmine              |  WHERE searchable = true;
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/mysql2-0.4.10/lib/mysql2/client.rb:120:in `_query'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/mysql2-0.4.10/lib/mysql2/client.rb:120:in `block in query'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/mysql2-0.4.10/lib/mysql2/client.rb:119:in `handle_interrupt'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/mysql2-0.4.10/lib/mysql2/client.rb:119:in `query'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:317:in `block in execute'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/connection_adapters/abstract_adapter.rb:484:in `block in log'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activesupport-4.2.11/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/connection_adapters/abstract_adapter.rb:478:in `log'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:317:in `execute'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/connection_adapters/mysql2_adapter.rb:217:in `execute'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:665:in `block in method_missing'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:634:in `block in say_with_time'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:634:in `say_with_time'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:654:in `method_missing'
redmine              | /home/redmine/redmine/plugins/full_text_search/db/migrate/20170630063757_copy_records_to_searcher_records.rb:124:in `load_custom_values'
redmine              | /home/redmine/redmine/plugins/full_text_search/db/migrate/20170630063757_copy_records_to_searcher_records.rb:33:in `block (2 levels) in change'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:511:in `up'
redmine              | /home/redmine/redmine/plugins/full_text_search/db/migrate/20170630063757_copy_records_to_searcher_records.rb:4:in `block in change'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:547:in `block in reversible'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:746:in `execute_block'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:547:in `reversible'
redmine              | /home/redmine/redmine/plugins/full_text_search/db/migrate/20170630063757_copy_records_to_searcher_records.rb:3:in `change'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:608:in `exec_migration'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:592:in `block (2 levels) in migrate'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:591:in `block in migrate'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/connection_adapters/abstract/connection_pool.rb:292:in `with_connection'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:590:in `migrate'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:768:in `migrate'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:1023:in `block in execute_migration_in_transaction'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:1071:in `ddl_transaction'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:1022:in `execute_migration_in_transaction'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:984:in `block in migrate'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:980:in `each'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:980:in `migrate'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:823:in `up'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:801:in `migrate'
redmine              | /home/redmine/redmine/lib/redmine/plugin.rb:481:in `migrate_plugin'
redmine              | /home/redmine/redmine/lib/redmine/plugin.rb:453:in `migrate'
redmine              | /home/redmine/redmine/lib/redmine/plugin.rb:467:in `block in migrate'
redmine              | /home/redmine/redmine/lib/redmine/plugin.rb:466:in `each'
redmine              | /home/redmine/redmine/lib/redmine/plugin.rb:466:in `migrate'
redmine              | /home/redmine/redmine/lib/tasks/redmine.rake:135:in `block (3 levels) in '
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/rake-12.3.2/exe/rake:27:in `'
redmine              |
redmine              | Caused by:
redmine              | Mysql2::Error: Duplicate entry '1073-CustomValue' for key 'index_searcher_records_unique'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/mysql2-0.4.10/lib/mysql2/client.rb:120:in `_query'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/mysql2-0.4.10/lib/mysql2/client.rb:120:in `block in query'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/mysql2-0.4.10/lib/mysql2/client.rb:119:in `handle_interrupt'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/mysql2-0.4.10/lib/mysql2/client.rb:119:in `query'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:317:in `block in execute'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/connection_adapters/abstract_adapter.rb:484:in `block in log'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activesupport-4.2.11/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/connection_adapters/abstract_adapter.rb:478:in `log'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:317:in `execute'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/connection_adapters/mysql2_adapter.rb:217:in `execute'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:665:in `block in method_missing'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:634:in `block in say_with_time'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:634:in `say_with_time'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:654:in `method_missing'
redmine              | /home/redmine/redmine/plugins/full_text_search/db/migrate/20170630063757_copy_records_to_searcher_records.rb:124:in `load_custom_values'
redmine              | /home/redmine/redmine/plugins/full_text_search/db/migrate/20170630063757_copy_records_to_searcher_records.rb:33:in `block (2 levels) in change'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:511:in `up'
redmine              | /home/redmine/redmine/plugins/full_text_search/db/migrate/20170630063757_copy_records_to_searcher_records.rb:4:in `block in change'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:547:in `block in reversible'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:746:in `execute_block'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:547:in `reversible'
redmine              | /home/redmine/redmine/plugins/full_text_search/db/migrate/20170630063757_copy_records_to_searcher_records.rb:3:in `change'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:608:in `exec_migration'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:592:in `block (2 levels) in migrate'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:591:in `block in migrate'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/connection_adapters/abstract/connection_pool.rb:292:in `with_connection'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:590:in `migrate'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:768:in `migrate'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:1023:in `block in execute_migration_in_transaction'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:1071:in `ddl_transaction'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:1022:in `execute_migration_in_transaction'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:984:in `block in migrate'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:980:in `each'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:980:in `migrate'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:823:in `up'
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.11/lib/active_record/migration.rb:801:in `migrate'
redmine              | /home/redmine/redmine/lib/redmine/plugin.rb:481:in `migrate_plugin'
redmine              | /home/redmine/redmine/lib/redmine/plugin.rb:453:in `migrate'
redmine              | /home/redmine/redmine/lib/redmine/plugin.rb:467:in `block in migrate'
redmine              | /home/redmine/redmine/lib/redmine/plugin.rb:466:in `each'
redmine              | /home/redmine/redmine/lib/redmine/plugin.rb:466:in `migrate'
redmine              | /home/redmine/redmine/lib/tasks/redmine.rake:135:in `block (3 levels) in '
redmine              | /home/redmine/redmine/vendor/bundle/ruby/2.3.0/gems/rake-12.3.2/exe/rake:27:in `'

The result of the subquery of the insert statement is as follows.

mysql>     SELECT base.id, 'CustomValue', p.id, p.name, NULL, NULL, base.value, base.custom_field_id FROM custom_values AS base
    ->      JOIN custom_fields AS f ON (base.custom_field_id = f.id)
    ->     JOIN projects AS p ON (base.customized_id = p.id)
    ->  WHERE searchable = true;
+------+-------------+----+--------------+------+------+-------+-----------------+
| id   | CustomValue | id | name         | NULL | NULL | value | custom_field_id |
+------+-------------+----+--------------+------+------+-------+-----------------+
| 1073 | CustomValue |  5 | XXXXXXXX     | NULL | NULL |       |               1 |
+------+-------------+----+--------------+------+------+-------+-----------------+
1 row in set (2.93 sec)

Is there a solution?

Internal server error occurs when full text search

何らかの原因で searcher_records に journals に存在しない レコードを指すレコードが残っていた場合、全文検索を実行するとサーバーエラーになることがある。

ログを確認すると mecab のエラーで issue_contents にレコードが作成できていない様子だった。
これは、あるチケットに追加するコメントが増えると発生する可能性がある。
手元で確認した感じでは約1.8MBで発生した。

しかし、チケットに長いコメントを追加してmecabのエラーを発生させても、searcher_recordsにゴミレコードは残らない。

Equal search index is missing in Mroonga

All indexes including for original_type are FULLTEXT INDEX.
FULLTEXT INDEX is meaningless for original_type =="Issue" query.

We need normal INDEX for equal search.

Conflict Redmine People plugin

以下のプラグインを使っている場合、コンフリクトします。
People - Plugins - Redmine

アバターアイコンを変更出来るのですが、変更を確定した際に以下の500エラーが出て、以下のログが出てました。

Started PUT "/redmine/people/157" for 10.5.72.123 at 2018-08-31 12:34:42 +0900
Processing by PeopleController#update as HTML
  Parameters: {"utf8"=>"✓", "authenticity_token"=>"tottemo+tottemo+ttotemo+himitu", "person"=>{"information_attributes"=>{"is_system"=>"0", "middlename"=>"", "gender"=>"0", "appearance_date"=>"", "birthday"=>"", "job_title"=>"", "address"=>"", "phone"=>"", "skype"=>"", "facebook"=>"", "twitter"=>"", "linkedin"=>"", "background"=>"", "id"=>"157"}, "firstname"=>"太郎", "lastname"=>"山田", "mail"=>"[email protected]"}, "person_avatar"=>{"file"=>#<ActionDispatch::Http::UploadedFile:0x00007f788ee4f238 @tempfile=#<Tempfile:/tmp/RackMultipart20180831-13308-1g115zw.png>, @original_filename="taro.png", @content_type="image/png", @headers="Content-Disposition: form-data; name=\"person_avatar[file]\"; filename=\"nagatomi.png\"\r\nContent-Type: image/png\r\n">}, "pwfmt_select_format"=>"textile", "pwfmt"=>{"formats"=>{"person_information_attributes_background"=>"textile"}}, "commit"=>"保存", "id"=>"157"}
  Current user: taro (id=157)
Saving attachment '/var/lib/redmine/files/2018/08/180831123442_taro.png' (560782 bytes)
Completed 500 Internal Server Error in 93ms (ActiveRecord: 36.2ms)

NoMethodError (undefined method `project_id' for #<User:0x00007f788e730600>
Did you mean?  project_ids
               project_ids=
               project
               projects
               project_ids_by_role
               object_id
               projects=):
  plugins/full_text_search/lib/full_text_search/model.rb:175:in `after_save'
  plugins/redmine_people/lib/acts_as_attachable_global/lib/acts_as_attachable_global.rb:96:in `block in attach_saved_attachments'
  plugins/redmine_people/lib/acts_as_attachable_global/lib/acts_as_attachable_global.rb:95:in `each'
  plugins/redmine_people/lib/acts_as_attachable_global/lib/acts_as_attachable_global.rb:95:in `attach_saved_attachments'
  app/models/attachment.rb:286:in `attach_files'
  plugins/redmine_people/app/controllers/people_controller.rb:277:in `attach_avatar'
  plugins/redmine_people/app/controllers/people_controller.rb:116:in `update'
  lib/redmine/sudo_mode.rb:63:in `sudo_mode'
  plugins/full_text_search/lib/full_text_search/model.rb:175:in `after_save'

とあったので確認したのですが、 redmine_full_text_search/model.rb at L175 の辺りに、

return if record.container.try(:project_id).blank?

を入れてみたところ、登録できるようになりました。

実行環境

Redmine 3.4.3
redmine_full_text_search v0.7.3
PGroonga 2.1.1
ruby 2.3.7p456
Rails 4.2.8
PostgreSQL 10.5
CentOS Linux release 7.5.1804 (Core)

PostgreSQLは公式のyumリポジトリを追加してインストール。
PGroongaもyumでインストール。

Redmineの状態

Environment:
  Redmine version                3.4.6.stable.17477
  Ruby version                   2.3.7-p456 (2018-03-28) [x86_64-linux]
  Rails version                  4.2.8
  Environment                    production
  Database adapter               PostgreSQL
SCM:
  Subversion                     1.7.14
  Git                            2.16.4
  Filesystem                     
Redmine plugins:
  clipboard_image_paste          1.12
  full_text_search               0.7.2
  periodictask                   3.2.1
  redmine_banner                 0.1.2
  redmine_checklists             3.1.11
  redmine_default_custom_query   1.3.0
  redmine_favorite_projects      2.0.3
  redmine_issue_badge            0.0.6
  redmine_issue_templates        0.2.0
  redmine_local_avatars          1.0.3
  redmine_logs                   0.1.1
  redmine_niko_cale              1.1.3-dev
  redmine_people                 1.3.2
  redmine_persist_wfmt           1.1.0
  redmine_pivot_table            0.0.6
  redmine_theme_changer          0.3.1
  redmine_wiki_extensions        0.8.1
  redmine_wiki_lists             0.0.9
  redmine_work_days              1.3.0
  redmine_work_time              0.3.4
  sidebar_hide                   0.0.8
  view_customize                 1.1.4

Can't uninstall full_text_search

環境情報

  • OS:CentOS7.6
  • Redmine: 4.0.4.stable.18325
  • redmine_full_text_search:commit 9f293f1
  • groonga:9.0.4
  • PGroonga:2.2.0
  • Ruby:2.5.5-p157 (2019-03-15) [x86_64-linux]
  • Rails:5.2.3
  • AppServer:unicorn 5.4.1
  • PostgreSQL:11.4
bundle exec rake redmine:plugins:migrate RAILS_ENV=production VERSION=0 NAME=full_text_search

の実行時に以下のエラーが発生します。

== 20170630075028 AddIndexToSearcherRecords: reverting ========================
rake aborted!
StandardError: An error has occurred, this and all later migrations canceled:

Index name 'index_searcher_records_pgroonga' on table 'searcher_records' does not exist
/opt/redmine/vendor/bundle/ruby/2.5.0/gems/activerecord-5.2.3/lib/active_record/migration/compatibility.rb:209:in `index_name_for_remove'
/opt/redmine/vendor/bundle/ruby/2.5.0/gems/activerecord-5.2.3/lib/active_record/migration/compatibility.rb:183:in `remove_index'

対処としては以下の通り、ダミーのインデックスを付加してアンインストールしました

sudo -u postgres -H psql -d db_redmine --command 'CREATE INDEX index_searcher_records_pgroonga on searcher_records(id) '

Uninstalling the plugin

Please suggest the correct way to uninstall the plugin.

I tried to remove it from plugins directory and run 'rake redmine:plugins:migrate RAILS_ENV=production'

It seemed to be ok at first, but then different problems emerged. I even can't backup my DB.
It shows: mysqldump: Got error: 1286: Unknown storage engine 'Mroonga' when using LOCK TABLES

Had to return the plugin back and managed to backup the redmine DB.

Migration error while removing plugin with MySQL

After #37, with MySQL, bundle exec rake redmine:plugins:migrate NAME=full_text_search VERSION=0 causes the following error.

== 20170630062526 ResetSchemaForMroonga: reverting ============================
-- create_table(:fts_custom_values, {:if_exists=>true})
   -> 0.1749s
-- create_table(:fts_wiki_contents, {:if_exists=>true})
   -> 0.3746s
-- create_table(:fts_wiki_pages, {:if_exists=>true})
   -> 0.3160s
-- create_table(:fts_changesets, {:if_exists=>true})
   -> 0.1998s
-- create_table(:fts_attachments, {:if_exists=>true})
   -> 0.4337s
-- create_table(:fts_messages, {:if_exists=>true})
   -> 0.3493s
-- create_table(:fts_documents, {:if_exists=>true})
   -> 0.2166s
-- create_table(:fts_journals, {:if_exists=>true})
   -> 0.3249s
-- create_table(:fts_issues, {:if_exists=>true})
   -> 0.2666s
-- create_table(:fts_news, {:if_exists=>true})
   -> 0.1748s
-- create_table(:fts_projects, {:if_exists=>true})
   -> 0.2502s
== 20170630062526 ResetSchemaForMroonga: reverted (3.0985s) ===================

== 20170630062504 ResetSchemaForPgroonga: reverting ===========================
== 20170630062504 ResetSchemaForPgroonga: reverted (0.0004s) ==================

== 20170227125753 ChangeColumnTypeToText: reverting ===========================
== 20170227125753 ChangeColumnTypeToText: reverted (0.0001s) ==================

== 20170223142427 StopUsingMultiColumnIndex: reverting ========================
-- remove_index(:fts_projects, :name)
rake aborted!
StandardError: An error has occurred, all later migrations canceled:

Index name 'index_fts_projects_on_name' on table 'fts_projects' does not exist
/usr/local/bundle/gems/activerecord-4.2.8/lib/active_record/connection_adapters/abstract/schema_statements.rb:1003:in `index_name_for_remove'
/usr/local/bundle/gems/activerecord-4.2.8/lib/active_record/connection_adapters/abstract/schema_statements.rb:593:in `remove_index'

The choices of the key value list are excluded from the full-text search.

Full_Text_Search pluginを利用させて頂きました。
気付いた点がありますので報告します。

■現象

キーバリューリストで選択した文字列が全文検索の対象外となる。
リスト項目は全文検索の対象となる。
#キーバリューリストはRedmine3.2にてカスタムフィールドに追加された種別

ユーザからは、選択項目がリストかキーバリューリストかは識別できないため、
一部項目が検索されない状態になる。

Redmine本体の検索機能でも同一状況ですが、対応して頂けると有難いです。

■原因

リストはテキスト、キーバリューリストは内部のインデックス番号でDB上に保存されているため。

■対応案

単純には、fts_custom_valuesの保存時にインデックスではなくテキストで保存して頂ければと考えましたが、
キーバリューリストの文字列定義内容を変更した場合に追随しませんね。

■補足

本件(Redmine本体)、下記に登録済です。

The choices of the key value list are excluded from the full-text search.
http://www.redmine.org/issues/25000

キーバリューリストの選択した文字列が全文検索の対象外となる。
https://redmine.tokyo/issues/299

Github-issueの書き込みは初めてなので、不適切な点がありましたら教えてください。

NameError (undefined local variable or method `comments' for FullTextSearch::Model::Callbacks:Class):

ご提供の検索エンジンが社内で大評判です!ありがとうございますm(_ _)m

さっそくですが、
管理 > 設定 > リポジトリ の「コミットを自動取得する」にチェックを入れてる状態で
プロジェクトのリポジトリに接続しますと、下記のエラーが発生します。

NameError (undefined local variable or method comments' for FullTextSearch::Model::Callbacks:Class): plugins/full_text_search/lib/full_text_search/model.rb:75:in after_save'
app/models/repository/subversion.rb:72:in block (2 levels) in fetch_changesets' app/models/repository/subversion.rb:71:in block in fetch_changesets'
app/models/repository/subversion.rb:70:in reverse_each' app/models/repository/subversion.rb:70:in fetch_changesets'
app/controllers/repositories_controller.rb:86:in show' lib/redmine/sudo_mode.rb:63:in sudo_mode'

Redmine情報は下記の通りです。
Environment:
Redmine version 3.4.2.stable
Ruby version 2.3.4-p301 (2017-03-30) [x86_64-linux-gnu]
Rails version 4.2.8
Environment production
Database adapter Mysql2
SCM:
Subversion 1.8.8
Darcs 2.8.4
Mercurial 2.8.2
Cvs 1.12.13
Bazaar 2.7.0
Git 2.13.0
Filesystem
Redmine plugins:
clipboard_image_paste 1.12
easy_gantt 1.9
full_text_search 0.5.0
redmine_banner 0.1.2
redmine_issue_templates 0.1.8
view_customize 1.1.4

では、よろしくお願いします。

Error on search with "Search titles only"

I get the following sql error

ActiveRecord::StatementInvalid (Mysql2::Error: Can't find FULLTEXT index matching the column list: SELECT DISTINCT issues.id, MATCH(fts_issues.subject) AGAINST ('_D+ some search term' IN BOOLEAN MODE) AS score FROM issues INNER JOIN projects [...] AND (issues.project_id IN (1742)) AND (MATCH(fts_issues.subject) AGAINST ('_D+ some search term' IN BOOLEAN MODE)) ORDER BY score DESC):

Require PGroonga 2.0.0 or later

過去のマイグレーションで使っているオペレータクラスも変更してPGroonga2.0.0以降を必須にする。

Duplicate entry error when creating searcher_records for custom value

When migrating Full Text Search, I got the following error.

-- execute("    INSERT INTO searcher_records(original_id, original_type, project_id, original_created_on, original_updated_on, comments)\n    SELECT base.id, 'Changeset', project_id, base.committed_on, NULL, base.comments FROM changesets AS base\n JOIN repositories AS r ON (base.repository_id = r.id);")rake aborted!
StandardError: An error has occurred, all later migrations canceled:
Mysql2::Error: Duplicate entry '3303-CustomValue' for key 'index_searcher_records_unique':     INSERT INTO searcher_records(original_id, original_type, project_id, status_id, is_private, original_created_on, original_updated_on, value, custom_field_id)
    SELECT base.id, 'CustomValue', r.project_id, status_id, is_private, NULL, NULL, base.value, base.custom_field_id FROM custom_values AS base
     JOIN issues AS i ON (base.customized_id = i.id)
    JOIN custom_fields AS f ON (base.custom_field_id = f.id)
    JOIN custom_fields_projects AS r ON (base.custom_field_id = r.custom_field_id)
 WHERE searchable = true;
/usr/local/bundle/gems/mysql2-0.4.9/lib/mysql2/client.rb:120:in `_query'
/usr/local/bundle/gems/mysql2-0.4.9/lib/mysql2/client.rb:120:in `block in query'
/usr/local/bundle/gems/mysql2-0.4.9/lib/mysql2/client.rb:119:in `handle_interrupt'
/usr/local/bundle/gems/mysql2-0.4.9/lib/mysql2/client.rb:119:in `query'
/usr/local/bundle/gems/activerecord-4.2.8/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:309:in `block in execute'
/usr/local/bundle/gems/activerecord-4.2.8/lib/active_record/connection_adapters/abstract_adapter.rb:484:in `block in log'
/usr/local/bundle/gems/activesupport-4.2.8/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
/usr/local/bundle/gems/activerecord-4.2.8/lib/active_record/connection_adapters/abstract_adapter.rb:478:in `log'
/usr/local/bundle/gems/activerecord-4.2.8/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:309:in `execute'
/usr/local/bundle/gems/activerecord-4.2.8/lib/active_record/connection_adapters/mysql2_adapter.rb:225:in `execute'
/usr/local/bundle/gems/activerecord-4.2.8/lib/active_record/migration.rb:665:in `block in method_missing'
/usr/local/bundle/gems/activerecord-4.2.8/lib/active_record/migration.rb:634:in `block in say_with_time'
/usr/local/bundle/gems/activerecord-4.2.8/lib/active_record/migration.rb:634:in `say_with_time'
/usr/local/bundle/gems/activerecord-4.2.8/lib/active_record/migration.rb:654:in `method_missing'
/usr/src/redmine/plugins/full_text_search/db/migrate/20170630063657_create_searcher_records.rb:219:in `load_custom_values'

The table contents are like this.

mysql> SELECT base.id, 'CustomValue', r.project_id, status_id, is_private, NULL, NULL, base.value, base.custom_field_id FROM custom_values AS base JOIN issues AS i ON (base.customized_id = i.id) JOIN custom_fields AS f ON (base.custom_field_id = f.id) JOIN custom_fields_projects AS r ON (base.custom_field_id = r.custom_field_id) WHERE searchable = true;
+------+-------------+------------+-----------+------------+------+------+-------+-----------------+
| id   | CustomValue | project_id | status_id | is_private | NULL | NULL | value | custom_field_id |
+------+-------------+------------+-----------+------------+------+------+-------+-----------------+
| 3303 | CustomValue |         16 |        14 |          0 | NULL | NULL | XXXX  |               5 |
| 3303 | CustomValue |         81 |        14 |          0 | NULL | NULL | XXXX  |               5 |
| 3303 | CustomValue |         89 |        14 |          0 | NULL | NULL | XXXX  |               5 |
| 3303 | CustomValue |         90 |        14 |          0 | NULL | NULL | XXXX  |               5 |
| 3303 | CustomValue |         92 |        14 |          0 | NULL | NULL | XXXX  |               5 |

Redmine3.4.3: Migration error

PGroongaをredmineデータベースに入れているのですが、redmine:plugins:migrateでコケてしまいます。

エラーログを見ると select pgroonga_command('plugin_register functions/vector') となっていてpgroonga_commandが無いとエラーが出ています。

こんな感じ↓

Migrating full_text_search (Full Text Search plugin)...
== 20170630063557 RegisterFunctionsVector: migrating ==========================
-- execute("select pgroonga_command('plugin_register functions/vector')")
rake aborted!
StandardError: An error has occurred, this and all later migrations canceled:

PG::UndefinedFunction: ERROR:  function pgroonga_command(unknown) does not exist
LINE 1: select pgroonga_command('plugin_register functions/vector')
               ^
HINT:  No function matches the given name and argument types. You might need to add explicit type casts.
: select pgroonga_command('plugin_register functions/vector')

また、GRANT USAGE ON SCHEME pgroonga | PGroongaを参考に、

PGroongaを入れたredmineスキーマにて、
SELECT pgroonga.snippet_html('PGroonga is fast', Array['fast']);
した結果は返ってくるのでGRANT USAGEは出来ているようです。

何か解決のヒントは無いでしょうか。

<環境情報>
Redmine 3.4.3
redmine_full_text_search v0.7.3
PGroonga 2.1.1
ruby 2.3.7p456
Rails 4.2.8
PostgreSQL 10.5

bitnami_redmine install full text search

Hi,
I installed the redmine for Bitnami_redmine(mysql+apache),I have just installed Mrooga, but the MySQL database is not found, how to configure support?
Thanks.

May return duplicated result

詳細はわかっていないんですが、「2017-04-07」で検索すると「2017-04-17 朝会」というタイトルの(同一でどちらも3610番の)チケットが2つリストされます。スコアは片方が1100でもう片方が100です。

チケットの重複判定がうまくいっていないのかもしれません。

@okkez さんもアクセスできるサーバーなので詳細はそこで確認できます。

Highlight doesn't support "OR" and "(...)"

GroongaはXXX OR YYYと書くとXXXまたはYYYのどちらかがヒットしたらマッチという挙動になるが、Redmineのハイライト処理はORのことをわからないのでORもハイライトしてしまう。

同様に、(XXX YYY) OR ZZZと書くと先にXXX YYYを処理してから(...) OR ZZZを評価するのだけれど、これもRedmineのハイライト処理が(...)のことをわからないので(XXXYYY)をハイライトしようとする。結果としてXXXYYYはハイライトされない。

Groongaが提供しているハイライト機能を使うといいのだけれど、難しいかしら。

Score in jounal is ignored

issueのコメントでのヒット数がスコアに換算されていない。

if !options[:titles_only] && searchable_options[:search_journals]
  s = ActiveRecord::Base.send(:sanitize_sql_array,
                              search_tokens_condition(columns, tokens, options[:all_words]))
  r |= fetch_ranks_and_ids(
    search_scope(user, projects, options).
    select(:id, "#{s} AS score, #{target_column_name} AS order_target").
    joins(fts_relation).
    joins(:journals).
    joins("INNER JOIN fts_journals ON (journals.id = fts_journals.journal_id)").
    where("#{::Journal.table_name}.private_notes = ? OR (#{::Project.allowed_to_condition(user, :view_private_notes)})", false).
    where(search_tokens_condition(["#{::Journal.table_name}.notes"], tokens, options[:all_words])),
    options[:limit],
    **kw
  )
  queries += 1
end

#{s} AS scoreとしているが、sはissueのMATCH AGAINSTであり、コメントのMATCH AGAINSTではない。

issueの説明の方がコメントよりも重要だと思うので、issueのスコアはコメントのスコアの10倍重要とかいう感じでスコアを足し合わせるのがよいと思う。

r |= fetch_ranks_and_idsだと、スコアが違う場合は同じissueが複数ヒットしたことになるし、そもそもスコアの足し合わせもできないので、このままだとよくない。個人的にはsearch_result_ranks_and_ids内の条件はORでつなげてSQL一発でいけないの?と思うけどどうなんだろう。

Logical NOT is ignored

Groongaは-で論理否定(その単語が含まれていないという条件)を指定できますが、検索ボックスで「Groonga - PostgreSQL」と指定しても無視され、「Groongaを含むがPostgreSQLは含まない」ではなく「GroongaもPostgreSQLも含む」と解釈されます。

たぶん、Redmineのレベルで1文字のトークンが無視されているから https://github.com/redmine/redmine/blob/master/lib/redmine/search.rb#L63 な気がします。ユーザーからの入力(このクラスの@question相当?)をそのままGroongaで解釈できるとうれしいなぁと思います。

なお、回避策として-の後に空白を入れずに「Groonga -PostgreSQL」とする方法があります。

Internal Server Error when I make new query with single-quote only

v0.7.2をRedmine v3.4.5-stableと組み合わせて利用しています。

チケットの検索を ' (シングルクオート)のみで実施すると、 Internal Server Error が発生します。
(意味のある検索かどうかはともかく……)

その際の production.log には、以下のように記録されていました。

I, [2018-06-07T14:45:01.023089 #1486]  INFO -- : Started GET "/search?utf8=%E2%9C%93&scope=&q=%27" for 127.0.0.1 at 2018-06-07 14:45:01 +0900
I, [2018-06-07T14:45:01.025122 #1486]  INFO -- : Processing by SearchController#index as HTML
I, [2018-06-07T14:45:01.025237 #1486]  INFO -- :   Parameters: {"utf8"=>"✓", "scope"=>"", "q"=>"'"}
I, [2018-06-07T14:45:01.038254 #1486]  INFO -- :   Current user: xxx (id=xxx)
W, [2018-06-07T14:45:01.107041 #1486]  WARN -- : Mysql2::Error: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '', '((original_type == "Issue" && is_private == false && in_values(project_id, 1' at line 19:         select mroonga_command(
                 'select',
                 'table', 'searcher_records',
                 'output_columns', '*,_score',
                 'columns[title_digest].stage', 'output',
'columns[title_digest].type', 'ShortText',
'columns[title_digest].flags', 'COLUMN_SCALAR',
'columns[title_digest].value', 'highlight_html(subject+title+filename+name+short_comments)',
'columns[description_digest].stage', 'output',
'columns[description_digest].type', 'ShortText',
'columns[description_digest].flags', 'COLUMN_VECTOR',
'columns[description_digest].value', 'snippet_html(content+text+notes+description+summary+value+long_comments) || vector_new()',
'columns[calculated_updated_on].stage', 'filtered',
'columns[calculated_updated_on].type', 'Time',
'columns[calculated_updated_on].flags', 'COLUMN_SCALAR',
'columns[calculated_updated_on].value', 'max(original_created_on,original_updated_on)',
                 'drilldown', 'original_type',
                 'match_columns', 'name*100||identifier*100||title*100||subject*100||filename*100||description||summary||comments||content||notes||text||value',
                 'query', ''',

本家
RedmineJP
ではどちらもエラーにはならないようでした。

Support similar issue search

このプラグインのスコープかどうかから検討する必要があると思いますが、今見ているissueと類似のissueを検索できる(issue表示画面に勝手に類似issueが表示される方がよさそう)と「この問題はどこかで見た気がするな。。。どのissueだろう。。。」というときにすぐに見つけられてうれしいです。

技術的には、Mroongaの場合は、1つテーブルを用意して、1issue(+全コメント)につき1レコードを作ってそこのカラムにテキストを格納し、そのカラムにTokenMecabな全文検索用インデックスを張って、そのカラムに対して検索対象のissue(+全コメント)のテキストを検索文字列として(IN BOOLEAN MODEじゃなくて)IN NATURAL LANGUAGE MODEで全文検索すると実現できます。

イメージ:

CREATE TABLE issue_texts (
  issue_id int NOT NULL,
  text longtext NOT NULL,
  FULLTEXT KEY (text)
) ENGINE=Mroonga
  DEFAULT CHAREST=utf8mb4;

INSERT INTO issue_texts
  SELECT CONCAT(subject, '\n', description),  -- コメントの内容も入れたい
         id
    FROM issues;

SELECT *
  FROM issues
  WHERE id IN (
    SELECT issue_id
      FROM issue_texts
      WHERE MATCH(text) AGAINST('見ているissueのタイトルと説明とコメント全部を連結したテキスト' IN NATURAL LANGUAGE MODE)
     ORDER BY MATCH(text) AGAINST(...↑と同じもの...) DESC
     LIMIT 3
  );

Support synonym

Groongaには同義語展開という機能があり、入力されたクエリーを拡張して検索できます。たとえば、「ドリルダウン」で検索したら自動で「ドリルダウン OR drilldown」で検索するみたいな感じです。

MroongaとPGroongaにはこれらの機能を作る機能はまだないんですが、MroongaとPGroongaで使えるようになったらこのプラグインでも使えるようになるとうれしいです。使えるようになる、というのは、登録されている同義語を使う機能があるということと同義語管理UIがあるということです。

同義語はRedmineに入っているデータから学習して半自動で整備できるようになるとウハウハだなぁと思っています。

Search result date is incorrect.

If you execute the synchronize task after applying this plugin, the following problems occur.
In the case of CustomValue, fixed value '1970-01-01 00:00:00' are entered in searcher_records.original_created_on and searcher_records.original_updated_on.
So '1970-01-01 00:00:00' is displayed on the search result screen.
Can not be the original issue datetime?

pgroonga: cannot display score for wiki

pgroonga で wiki を全文検索の対象にしたときにスコアがゼロになる。
たぶん pgroonga の制限。orをandに変えるとスコアはゼロにならない。
💭 何回も同じこと調べてる気がする

select 
  pgroonga.score(c) 
from
  wiki_pages as p 
  join
  wiki_contents as c
    on p.id = c.page_id 
where 
  text @@ '全文検索' 
  or
  title @@ '全文検索';

Migration is too slow with PGroonga

== 20170810031258 LoadIssueIdToSearcherRecord: migrated (857.4493s) ===========

で時間がかかっている。

class LoadIssueIdToSearcherRecord < ActiveRecord::Migration
  def change
    reversible do |d|
      d.up do
        sql = "UPDATE searcher_records SET issue_id = original_id WHERE original_type = 'Issue'"
        execute(sql)
        # Following query on PostgreSQL causes error, so I decided to update searcher_records using AR
        # sql = "UPDATE searcher_records AS s SET issue_id = j.journalized_id FROM journals AS j WHERE original_id = j.id AND original_type = 'Journal'"
        # execute(sql)
        journal_ids = Journal.all.pluck(:id, :journalized_id)
        n_records = Journal.count(:id)
        n_pages = n_records / 1000
        (0..n_pages).each do |offset|
          Journal.limit(1000).offset(offset * 1000).each do |record|
            FullTextSearch::SearcherRecord
              .where(original_id: record.id, original_type: "Journal")
              .update_all(issue_id: record.journalized_id)
          end
        end
      end
      d.down do
        # Do nothing
      end
    end
  end
end

こんな感じのコードになっていて、Journalのレコード数分ループを回してしまっている。
SQL一発で更新できれば速くなるはずだけど、コメントに書いたSQLだとエラーになってしまう。

Support weight

タイトル(issueならsubject)にキーワードが含まれていた場合は本文に含まれている場合よりも高いスコアーにして欲しい。理由はタイトルに含まれている方がより目的の項目である可能性が高いから。

pgroonga is slower than mroonga???

#30, #31 に対応してから明らかに性能差が出ている。
pgroonga の方でも何かすれば同等の速度になるはず。

mroongaで初回100ms以下のクエリがpgroongaでは250ms-500ms程度になってしまう。

Is limit unused?

MATCH AGAINSTで検索している条件にlimitが付いていない気がする、けど気のせいかしら。
Redmineから値が渡ってきていない?

先頭10件しか返さないときでもlimitなしで検索されるとムダに重いので表示件数に上限があるときはlimit付きで検索して欲しい。

Add score as data attribute to search results

全文検索のスコアをデータ属性として検索結果に追加したい。

Redmine::Seach::Fetcher#load_result_ids でスコアを捨ててしまっているので簡単にはできない。
プラグインだけで実現しようとすると上書きしなければならない箇所が多くなりすぎるので、本体の機能としてサポートしてもらえるように頑張るのがよさそう。

📝 Redmine::Search::ResultEntry を導入してその配列を Redmine::Search 側で扱うようにすれば見通しは良くなりそうだけど、どうだろう。

💭 3.4に間に合うかどうかは微妙。

Support sort by date

検索結果を日付でもソートできるようにして欲しい。

○○というキーワードを含んだ最新のチケットを見つける、というときに使いたいです。たとえば、「リリース」で検索して最新リリースに関するチケットを見つける、というときに使いたいです。

NameError (uninitialized constant Issue::FullTextSearch::Mroonga::FtsIssue)

お世話になっております。
下記の環境でplugin migrate、redmineのリブートは正常に終了するのですが、
検索ボックスにワードを入れて検索しようとすると500 internal errorとなります。

Environment:
  Redmine version                3.3.2.stable
  Ruby version                   2.1.9-p490 (2016-03-30) [x86_64-linux-gnu]
  Rails version                  4.2.7.1
  Environment                    production
  Database adapter               Mysql2
SCM:
  Subversion                     1.8.8
  Darcs                          2.8.4
  Mercurial                      2.8.2
  Cvs                            1.12.13
  Bazaar                         2.7.0
  Git                            2.11.0
  Filesystem                     
Redmine plugins:
  full_text_search               0.3.0
  redmine_backlogs               v1.0.6

production.log

Started GET "/search?utf8=%E2%9C%93&q=mroonga" for 127.0.0.1 at 2017-04-04 00:41:02 +0900
Processing by SearchController#index as HTML
  Parameters: {"utf8"=>"磬\~S", "q"=>"mroonga"}
  Current user: sho.suzuki (id=7)
Completed 500 Internal Server Error in 65ms (ActiveRecord: 10.1ms)

NameError (uninitialized constant Issue::FullTextSearch::Mroonga::FtsIssue):
  plugins/redmine_full_text_search/lib/full_text_search/mroonga.rb:51:in `fetch_ranks_and_ids'
  plugins/redmine_full_text_search/lib/full_text_search/mroonga.rb:119:in `search_result_ranks_and_ids'
  plugins/redmine_full_text_search/lib/full_text_search/fetcher.rb:24:in `block in load_result_ids'
  plugins/redmine_full_text_search/lib/full_text_search/fetcher.rb:22:in `each'
  plugins/redmine_full_text_search/lib/full_text_search/fetcher.rb:22:in `load_result_ids'
  lib/redmine/search.rb:115:in `block in load_result_ids_from_cache'
  lib/redmine/search.rb:114:in `load_result_ids_from_cache'
  lib/redmine/search.rb:99:in `result_ids'
  lib/redmine/search.rb:70:in `result_count'
  app/controllers/search_controller.rb:75:in `index'
  plugins/redmine_full_text_search/lib/full_text_search/hooks/controller_search_index.rb:9:in `index'
  lib/redmine/sudo_mode.rb:63:in `sudo_mode'

logを見るに

  • Parameters: {"utf8"=>"磬~S", "q"=>"mroonga"} のutf8のvalueがおかしい
  • Issue::FullTextSearch::Mroonga::FtsIssueが初期化されていない

ように見えます。対処方法についてご教授願えますでしょうか。

Search results have same item

Search results have same item when some fields match keyword of searching.
But I think to show only one item on search results.

How to reproduce

  1. Add custom fields for issue.
  2. Create new issue, and set test-keyword to Description and your custom fields.
  3. Search with test-keyword. Searching results has same item of your issue.

Example
I attach my case's screenshot.
My custom fields are cf1 and cf2 and searching keyword is Results.
3 items, issue 5, are on the list, but I think the list should have only a item.
screenshot_search_results

In Japanese,
説明とカスタムフィールドなど、検索される要素にそれぞれ検索キーワードが含まれる場合、同じチケットなどが複数列挙されます。検索結果としては、同じチケットが複数表示されない方が良いと思いますがいかがでしょうか?

If there is an inconsistency in the DB contents, the synchronization process may end abnormally

If there is an inconsistency in the DB contents, the process may end abnormally during the synchronization process.
I think that it is better not to process in the record where the reference destination does not exist.

This phenomenon occurs in Redmine, in which data inconsistency occurs in DB contents.
(Inconsistent DB content, but normal operation of Redmine can be done without problems)

Operating environment

Redmine3.4.6 (CentOS7 MariaDB 5.5.60)
fulltextsearch0.8.1


Example
The deleted issueID remains in the custom_value table.

During the process of synchronize
There is no issue that corresponds to customized_id of custom_value.
Abnormal termination without acquiring project of issue.

Error occurrence point
redmine_full_text_search/lib/full_text_search/custom_value_mapper.rb

    searcher_record.project_name = @record.customized.project.name

Other than the above, also in the following cases, it ended abnormally similarly.

  • repository referring to deleted projects, changeset using that repository
  • the issue has been deleted journal
  • customized_type is abnormal

After deleting the above abnormal data and performing synchronize, it worked normally.

I'm impressed by the super fast full text search. Thank you.


Note

It seems that this trouble does not occur with normal Redmine.
I introduced a full text search plug-in to two Redmines, and this problem only occurred with one.
(Both have over 50,000 tickets)

The server where this trouble occurred, performed server consolidation work of Redmine 5 years ago.
I did not delete customvalue etc. which became unnecessary at this time.
So far, no problems have occurred for 5 years.

Implementation details for server consolidation (reference)
https://www.slideshare.net/y503unavailable/redmine-42182169

I think it would be better if Redmine had a function to check data integrity.

Mroonga's method is called when using PostgreSQL

環境情報

  • OS:CentOS7.6
  • Redmine: 4.0.4.stable.18325
  • redmine_full_text_search:commit 9f293f1
  • groonga:9.0.4
  • PGroonga:2.2.0
  • Ruby:2.5.5-p157 (2019-03-15) [x86_64-linux]
  • Rails:5.2.3
  • AppServer:unicorn 5.4.1
  • PostgreSQL:11.4

Pgroongaの導入後、redmineの再起動時に以下のログが unicorn.stderr.log に出力されて起動できません。

bundler: failed to load command: unicorn_rails (/opt/redmine/vendor/bundle/ruby/2.5.0/bin/unicorn_rails)
NoMethodError: undefined method `mroonga_vector_load_is_supported?' for #<Class:0x0000000008329878>
  /opt/redmine/vendor/bundle/ruby/2.5.0/gems/activerecord-5.2.3/lib/active_record/dynamic_matchers.rb:22:in `method_missing'
  /opt/redmine/plugins/full_text_search/app/types/full_text_search/mroonga_integer_array_type.rb:29:in `<class:MroongaIntegerArrayType>'
  /opt/redmine/plugins/full_text_search/app/types/full_text_search/mroonga_integer_array_type.rb:3:in `<module:FullTextSearch>'
  /opt/redmine/plugins/full_text_search/app/types/full_text_search/mroonga_integer_array_type.rb:1:in `<top (required)>'

プラグインのインストール/アンインストールを何度か繰り返したのですが解決せず、ふと思いついて
plugins/full_text_search/app/types/full_text_search/mroonga_integer_array_type.rb
をリネームしたところ起動し、検索結果も表示されるようになりました。

何かこちらの環境に問題があるのでしょうか。

Support disabling Mroonga FTS dinamically

普通の人は必要のない機能なのでアレなんですが、このプラグインの開発をする人とこのプラグインの紹介をする人にとってうれしい機能を提案します。(私はredmine.tokyo第12回勉強会で紹介するのであるとうれしい。)

Redmineでプラグインを無効にしなくても、リクエスト毎にMroongaでの全文検索機能を無効にして従来の検索機能を使う機能を提案します。たとえば、リクエストにenable_mroonga=falseと指定すると従来の検索機能を使う機能です。

この機能があると次のことがしやすくなり、このプラグインの開発をする人と紹介する人にとってうれしいです。

  • このプラグインでどれくらい速くなっているかの確認
  • このプラグインでどれくらいよい検索結果を返すようになるかの確認

実装イメージです。( #23 がマージされた前提。)

--- a/lib/full_text_search/mroonga.rb
+++ b/lib/full_text_search/mroonga.rb
@@ -36,7 +36,7 @@ module FullTextSearch
 
     module ClassMethods
       # Overwrite ActsAsSearchable
-      def fetch_ranks_and_ids(scope, limit, attachments: false, order_target: "score", order_type: "desc")
+      def fetch_ranks_and_ids_mroonga(scope, limit, attachments: false, order_target: "score", order_type: "desc")
         if self == ::WikiPage && !attachments
           scope.
             joins("INNER JOIN fts_wiki_contents ON (wiki_contents.id = fts_wiki_contents.wiki_content_id)").
@@ -62,6 +62,7 @@ module FullTextSearch
         tokens = [] << tokens unless tokens.is_a?(Array)
         projects = [] << projects if projects.is_a?(::Project)
         params = options[:params]
+        return super if params[:enable_mroonga] == "false"
         target_column_name = "#{table_name}.#{order_column_name}"
         kw = {
           order_type: params[:order_type] || "desc",
@@ -85,7 +86,7 @@ module FullTextSearch
             c1 = search_tokens_condition_single(column1, tokens, options[:all_words])
             c2 = search_tokens_condition_single(column2, tokens, options[:all_words])
             c, t = c1.zip(c2).to_a
-            r = fetch_ranks_and_ids(
+            r = fetch_ranks_and_ids_mroonga(
               search_scope(user, projects, options).
               select(:id, "#{s1} * 100 + #{s2} AS score, #{target_column_name} AS order_target").
               joins(fts_relation).
@@ -116,7 +117,7 @@ module FullTextSearch
               c, t = conditions.first.zip(*conditions[1..-1]).to_a
               p [2, c, t]
             end
-            r = fetch_ranks_and_ids(
+            r = fetch_ranks_and_ids_mroonga(
               search_scope(user, projects, options).
               select(:id, "#{scores.join(" + ")} AS score, #{target_column_name} AS order_target").
               joins(fts_relation).
@@ -142,7 +143,7 @@ module FullTextSearch
               condition = search_tokens_condition_single("#{::CustomValue.table_name}.value", tokens, options[:all_words])
               score = ActiveRecord::Base.send(:sanitize_sql_array, condition)
 
-              r |= fetch_ranks_and_ids(
+              r |= fetch_ranks_and_ids_mroonga(
                 search_scope(user, projects, options).
                 select(:id, "#{score} AS score, #{target_column_name} AS order_target").
                 joins(fts_relation).
@@ -177,7 +178,7 @@ module FullTextSearch
             conditions << note_condition
             scores << note_score
             c, t = conditions.first.zip(*conditions[1..-1]).to_a
-            r |= fetch_ranks_and_ids(
+            r |= fetch_ranks_and_ids_mroonga(
               search_scope(user, projects, options).
               select(:id, "SUM(#{scores.join(" + ")}) AS score, #{target_column_name} AS order_target").
               joins(fts_relation).
@@ -201,7 +202,7 @@ module FullTextSearch
             ActiveRecord::Base.send(:sanitize_sql_array, _condition)
           end
           c, t = conditions.first.zip(*conditions[1..-1]).to_a
-          r |= fetch_ranks_and_ids(
+          r |= fetch_ranks_and_ids_mroonga(
             search_scope(user, projects, options).
             select(:id, "#{scores.join(" + ")} AS score, #{target_column_name} AS order_target").
             joins(fts_relation).

Ticket title search may be broken?

ちゃんと調べていないんですが、「2017-04-04 朝会」というタイトルのチケットを検索しようと「2017-04-04」で検索してもこのチケットがヒットしません。このチケットが存在することは確認済みです。

Mroongaのログレベルを↓でINFOにあげて/var/lib/mysql/groonga.logを確認すると、

set global mroonga_log_level = info;

See also: http://mroonga.org/ja/docs/reference/server_variables.html#mroonga-log-level

次のように「2017-04-04」で検索してなにかに5件ヒットしているように見えます。

2017-04-04 09:29:45.743570|i|9787f700|[object][search][index][key][exact] <fts_custom_values#index_fts_custom_values_on_value.index>
2017-04-04 09:29:45.750371|i|9787f700|grn_ii_sel > (2017-04-04)
2017-04-04 09:29:45.751185|i|9787f700|exact: 0
2017-04-04 09:29:45.751527|i|9787f700|unsplit: 0
2017-04-04 09:29:45.751618|i|9787f700|partial: 0
2017-04-04 09:29:45.751653|i|9787f700|hits=0
2017-04-04 09:29:45.759315|i|9787f700|[object][search][index][key][exact] <fts_journals#index_fts_journals_on_notes.index>
2017-04-04 09:29:45.765353|i|9787f700|grn_ii_sel > (2017-04-04)
2017-04-04 09:29:45.767195|i|9787f700|n=5 (2017-04-04)
2017-04-04 09:29:45.768382|i|9787f700|exact: 0
2017-04-04 09:29:45.768512|i|9787f700|unsplit: 0
2017-04-04 09:29:45.784721|i|9787f700|n=5 (2017-04-04)
2017-04-04 09:29:45.786091|i|9787f700|partial: 0
2017-04-04 09:29:45.786172|i|9787f700|hits=0

なので、どこかでおかしくなって検索結果として表示されていないような気がします。

Internal server error when custom value exists in search relusts

Completed 500 Internal Server Error in 716ms (ActiveRecord: 674.1ms)

ActionView::Template::Error (uninitialized constant Customvalue):
    45:     <h3><%= l(:label_result_plural) %> (<%= @search_result.count %>)</h3>
    46:     <dl id="search-results">
    47:       <% @search_result.each do |e| %>
    48:         <dt class="<%= e.event_type %> icon icon-<%= e.event_type %>">
    49:           <%= content_tag('span', e.project, :class => 'project') unless @project == e.project %>
    50:           <%= link_to(e.event_title_digest, e.event_url, :data => { :rank => e.rank }) %>
    51:         </dt>
  plugins/full_text_search/app/models/full_text_search/searcher_record.rb:220:in `original_record'
  plugins/full_text_search/app/models/full_text_search/searcher_record.rb:254:in `_type'
  lib/plugins/acts_as_event/lib/acts_as_event.rb:64:in `event_type'
  plugins/full_text_search/app/views/search/index.html.erb:48:in `block in _plugins_full_text_search_app_views_search_index_html_erb___1145967650517626388_70308553923880'
  plugins/full_text_search/lib/full_text_search/searcher.rb:116:in `block in each'
  plugins/full_text_search/lib/full_text_search/searcher.rb:115:in `each'
  plugins/full_text_search/lib/full_text_search/searcher.rb:115:in `each'
  plugins/full_text_search/app/views/search/index.html.erb:47:in `_plugins_full_text_search_app_views_search_index_html_erb___1145967650517626388_70308553923880'
  lib/redmine/sudo_mode.rb:63:in `sudo_mode'

ea09acb

0.7.1の変更により customvalue を capitilize した時に、恐らく CustomValue であるべきところが
Customvalue になっている(vが小文字)ために、検索結果にカスタムフィールドの値が含まれた時にエラーになってしまうと思われます。

500 error occurs when create ticket

チケット作成時に、500エラーが必ず発生してしまいます。

チケット作成ボタンを押下すると以下のエラーが出ていました。

Started POST "/redmine/projects/test-niko/issues" for 10.xx.xx.xx at 2018-08-27 12:39:33 +0900
Processing by IssuesController#create as HTML
  Parameters: {"utf8"=>"✓", "authenticity_token"=>"C6H7jdvLqyoLmQvveNoMCIm1Fs+9w44mwVx6ktWeJPBft5y3MkZyF22IKUoRjmTL376pCwEuVA62Z2xHt17nig==", "form_update_triggered_by"=>"", "issue"=>{"is_private"=>"0", "tracker_id"=>"6", "subject"=>"チケット作成テスト", "description"=>"", "status_id"=>"1", "priority_id"=>"4", "assigned_to_id"=>"157", "parent_issue_id"=>"", "start_date"=>"2018-08-27", "due_date"=>"", "estimated_hours"=>"", "done_ratio"=>"0", "watcher_user_ids"=>[""]}, "issue_template"=>"", "template_search_filter"=>"", "pwfmt_select_format"=>"textile", "pwfmt"=>{"formats"=>{"issue_description"=>"textile"}}, "was_default_status"=>"1", "commit"=>"作成", "project_id"=>"test-niko"}
  Current user: test (id=157)
  Rendered mailer/_issue.text.erb (9.4ms)
  Rendered mailer/issue_add.text.erb within layouts/mailer (16.4ms)
  Rendered mailer/_issue.html.erb (3.0ms)
  Rendered mailer/issue_add.html.erb within layouts/mailer (9.5ms)
Redirected to http://10.xx.xx.xx/redmine/issues/248957
Completed 302 Found in 612ms (ActiveRecord: 171.0ms)
Started GET "/redmine/issues/248957" for 10.xx.xx.xx at 2018-08-27 12:39:34 +0900
Processing by IssuesController#show as HTML
  Parameters: {"id"=>"248957"}
  Current user: test (id=157)
  Rendered issues/_action_menu.html.erb (11.0ms)
  Rendered plugins/full_text_search/app/views/issues/full_text_search/_view_issues_show_description_bottom.html.erb (258.9ms)
  Rendered issues/show.html.erb within layouts/base (336.2ms)
Completed 500 Internal Server Error in 439ms (ActiveRecord: 45.3ms)

ActionView::Template::Error (undefined method `display_score?' for #<#<Class:0x00007f1456e075d8>:0x00007f144c572dc8>
Did you mean?  display_comments_tree):
    1: <% if display_score? %>
    2:   <%
    3:   duration = nil
    4:   ActiveSupport::Notifications.subscribe("groonga.similar.search") do |*args|
  plugins/full_text_search/app/views/issues/full_text_search/_view_issues_show_description_bottom.html.erb:1:in `_plugins_full_text_search_app_views_issues_full_text_search__view_issues_show_description_bottom_html_erb__336950841572077697_69862578822200'
  lib/redmine/hook/view_listener.rb:59:in `block (2 levels) in render_on'
  lib/redmine/hook/view_listener.rb:57:in `map'
  lib/redmine/hook/view_listener.rb:57:in `block in render_on'
  lib/redmine/hook.rb:61:in `block (2 levels) in call_hook'
  lib/redmine/hook.rb:61:in `each'
  lib/redmine/hook.rb:61:in `block in call_hook'
  lib/redmine/hook.rb:58:in `tap'
  lib/redmine/hook.rb:58:in `call_hook'
  lib/redmine/hook.rb:96:in `call_hook'
  app/views/issues/show.html.erb:97:in `_app_views_issues_show_html_erb___2971280503261266164_69862578453100'
  app/controllers/issues_controller.rb:106:in `block (2 levels) in show'
  app/controllers/issues_controller.rb:99:in `show'
  lib/redmine/sudo_mode.rb:63:in `sudo_mode'

Backで戻った後にチケット一覧を見ると、チケットは作成出来ているようです。
ただし、そのチケットを参照しようとすると以下の500エラーが出て、表示できません。

Started GET "/redmine/issues/248957" for 10.xx.xx.xx at 2018-08-27 13:14:17 +0900
Processing by IssuesController#show as HTML
  Parameters: {"id"=>"248957"}
  Current user: test (id=157)
  Rendered issues/_action_menu.html.erb (7.7ms)
  Rendered plugins/full_text_search/app/views/issues/full_text_search/_view_issues_show_description_bottom.html.erb (288.8ms)
  Rendered issues/show.html.erb within layouts/base (331.0ms)
Completed 500 Internal Server Error in 384ms (ActiveRecord: 37.1ms)

ActionView::Template::Error (undefined method `display_score?' for #<#<Class:0x00007f1456e075d8>:0x00007f144e3ef080>
Did you mean?  display_comments_tree):
    1: <% if display_score? %>
    2:   <%
    3:   duration = nil
    4:   ActiveSupport::Notifications.subscribe("groonga.similar.search") do |*args|
  plugins/full_text_search/app/views/issues/full_text_search/_view_issues_show_description_bottom.html.erb:1:in `_plugins_full_text_search_app_views_issues_full_text_search__view_issues_show_description_bottom_html_erb__336950841572077697_69862578822200'
  lib/redmine/hook/view_listener.rb:59:in `block (2 levels) in render_on'
  lib/redmine/hook/view_listener.rb:57:in `map'
  lib/redmine/hook/view_listener.rb:57:in `block in render_on'
  lib/redmine/hook.rb:61:in `block (2 levels) in call_hook'
  lib/redmine/hook.rb:61:in `each'
  lib/redmine/hook.rb:61:in `block in call_hook'
  lib/redmine/hook.rb:58:in `tap'
  lib/redmine/hook.rb:58:in `call_hook'
  lib/redmine/hook.rb:96:in `call_hook'
  app/views/issues/show.html.erb:97:in `_app_views_issues_show_html_erb___2971280503261266164_69862578453100'
  app/controllers/issues_controller.rb:106:in `block (2 levels) in show'
  app/controllers/issues_controller.rb:99:in `show'
  lib/redmine/sudo_mode.rb:63:in `sudo_mode'

実行環境はこちらです。
Redmine 3.4.3
redmine_full_text_search v0.7.3
PGroonga 2.1.1
ruby 2.3.7p456
Rails 4.2.8
PostgreSQL 10.5
CentOS Linux release 7.5.1804 (Core)

PostgreSQLは公式のyumリポジトリを追加してインストール。
PGroongaもyumでインストール。

また、Redmineの状態は以下です。

Environment:
  Redmine version                3.4.6.stable.17473
  Ruby version                   2.3.7-p456 (2018-03-28) [x86_64-linux]
  Rails version                  4.2.8
  Environment                    production
  Database adapter               PostgreSQL
SCM:
  Subversion                     1.7.14
  Git                            2.16.4
  Filesystem                     
Redmine plugins:
  clipboard_image_paste          1.12
  full_text_search               0.7.2
  periodictask                   3.2.1
  redmine_banner                 0.1.2
  redmine_checklists             3.1.11
  redmine_default_custom_query   1.3.0
  redmine_favorite_projects      2.0.3
  redmine_issue_badge            0.0.6
  redmine_issue_templates        0.2.0
  redmine_local_avatars          1.0.3
  redmine_logs                   0.1.1
  redmine_niko_cale              1.1.3-dev
  redmine_people                 1.3.2
  redmine_persist_wfmt           1.1.0
  redmine_pivot_table            0.0.6
  redmine_theme_changer          0.3.1
  redmine_wiki_extensions        0.8.1
  redmine_wiki_lists             0.0.9
  redmine_work_time              0.3.4
  sidebar_hide                   0.0.8
  view_customize                 1.1.4

また、プラグインの設定で「スコアを表示」はOFFの状態です。

他のプラグインで display_score を使用しているものがあるのか確認したところ、
以下の結果となり、他プラグインには存在しません。

# find plugins -type f | xargs grep display_score
バイナリファイル plugins/full_text_search/.git/objects/pack/pack-80b3d37bafd17610dc6c6306f653c0132b46ab69.pack に一致しました
plugins/full_text_search/app/views/issues/full_text_search/_view_issues_show_description_bottom.html.erb:<% if display_score? %>
plugins/full_text_search/app/views/issues/full_text_search/_view_issues_show_description_bottom.html.erb:    <% if display_score? %>
plugins/full_text_search/app/views/search/index.html.erb:    <% if display_score? %>
plugins/full_text_search/app/views/search/index.html.erb:<%- if display_score? %>
plugins/full_text_search/app/views/settings/_full_text_search.html.erb:  <%= label_tag("settings[display_score]", l(:label_full_text_search_display_score)) %>
plugins/full_text_search/app/views/settings/_full_text_search.html.erb:  <%= check_box_tag("settings[display_score]", "1", settings.present? && settings["display_score"] == "1") %>
plugins/full_text_search/config/locales/en.yml:  label_full_text_search_display_score: Display score
plugins/full_text_search/config/locales/ja.yml:  label_full_text_search_display_score: スコアを表示
plugins/full_text_search/init.rb:  settings default: { display_score: "0" }, partial: "settings/full_text_search"
plugins/full_text_search/lib/full_text_search/hooks/settings_helper.rb:      def display_score?
plugins/full_text_search/lib/full_text_search/hooks/settings_helper.rb:        setting["display_score"] == "1"

Use higher score for project name match

プロジェクト名にマッチしたら高めのスコアがつくとうれしいです。

「打ち合わせ」で検索して、いろんなプロジェクトでの打ち合わせissueがヒットしたとします。このときにキーワードに「プロジェクト名」を追加して絞り込むと目的のプロジェクトの打ち合わせissueが上の方にくるとうれしいです。

The default sort order isn't "desc"

ページ上部の検索ボックスに入力して検索したとき、「検索結果の並び順」フォームでは「降順」となっていますが、実際は「昇順」で並び替えている。

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.