Giter VIP home page Giter VIP logo

textacular's Introduction

textacular

Gem Version Build Status

DESCRIPTION:

Textacular exposes full text search capabilities from PostgreSQL, extending ActiveRecord with scopes making search easy and fun!

FEATURES/PROBLEMS:

  • Only works with PostgreSQL
  • Anything that mucks with the SELECT statement (notably pluck), is likely to cause problems.

SYNOPSIS:

Quick Start

In the project's Gemfile add

gem 'textacular', '~> 5.0'

Rails 3, Rails 4

In the project's Gemfile add

gem 'textacular', '~> 4.0'

ActiveRecord outside of Rails

require 'textacular'

ActiveRecord::Base.extend(Textacular)

Usage

Your models now have access to search methods:

The #basic_search method is what you might expect: it looks literally for what you send to it, doing nothing fancy with the input:

Game.basic_search('Sonic') # will search through the model's :string columns
Game.basic_search(title: 'Mario', system: 'Nintendo')

The #advanced_search method lets you use Postgres's search syntax like '|', '&' and '!' ('or', 'and', and 'not') as well as some other craziness. The ideal use for advanced_search is to take a search DSL you make up for your users and translate it to PG's syntax. If for some reason you want to put user input directly into an advanced search, you should be sure to catch exceptions from syntax errors. Check [the Postgres docs] (http://www.postgresql.org/docs/9.2/static/datatype-textsearch.html) for more:

Game.advanced_search(title: 'Street|Fantasy')
Game.advanced_search(system: '!PS2')

The #web_search method lets you use Postgres' 11+ websearch_to_tsquery function supporting websearch like syntax:

  • unquoted text: text not inside quote marks will be converted to terms separated by & operators, as if processed by plainto_tsquery.
  • "quoted text": text inside quote marks will be converted to terms separated by <-> operators, as if processed by phraseto_tsquery.
  • OR: logical or will be converted to the | operator.
  • -: the logical not operator, converted to the the ! operator.
Game.web_search(title: '"Street Fantasy"')
Game.web_search(title: 'Street OR Fantasy')
Game.web_search(system: '-PS2')

Finally, the #fuzzy_search method lets you use Postgres's trigram search functionality.

In order to use this, you'll need to make sure your database has the pg_trgm module installed. Create and run a migration to install the module:

rake textacular:create_trigram_migration
rake db:migrate

Once that's installed, you can use it like this:

Comic.fuzzy_search(title: 'Questio') # matches Questionable Content

Note that fuzzy searches are subject to a similarity threshold imposed by the pg_trgm module. The default is 0.3, meaning that at least 30% of the total string must match your search content. For example:

Comic.fuzzy_search(title: 'Pearls') # matches Pearls Before Swine
Comic.fuzzy_search(title: 'Pear') # does not match Pearls Before Swine

The similarity threshold is hardcoded in PostgreSQL and can be modified on a per-connection basis, for example:

ActiveRecord::Base.connection.execute("SELECT set_limit(0.9);")

For more info, view the pg_trgm documentation, specifically F.35.2. Functions and Operators.

Searches are also chainable:

Game.fuzzy_search(title: 'tree').basic_search(system: 'SNES')

If you want to search on two or more fields with the OR operator use a hash for the conditions and pass false as the second parameter:

Game.basic_search({name: 'Mario', nickname: 'Mario'}, false)

Setting Language

To set proper searching dictionary just override class method on your model:

def self.searchable_language
  'russian'
end

And all your queries would go right! And don`t forget to change the migration for indexes, like shown below.

Setting Searchable Columns

To change the default behavior of searching all text and string columns, override the searchable_columns class method on your model:

def self.searchable_columns
  [:column1, :column2]
end

Creating Indexes for Super Speed

You can have Postgresql use an index for the full-text search. To declare a full-text index, in a migration add code like the following:

For basic_search

add_index :email_logs, %{to_tsvector('english', subject)}, using: :gin
add_index :email_logs, %{to_tsvector('english', email_address)}, using: :gin

For fuzzy_search

add_index :email_logs, :subject, using: :gist, opclass: :gist_trgm_ops
add_index :email_logs, :email_address, using: :gist, opclass: :gist_trgm_ops

In the above example, the table email_logs has two text columns that we search against, subject and email_address. You will need to add an index for every text/string column you query against, or else Postgresql will revert to a full table scan instead of using the indexes.

REQUIREMENTS:

  • ActiveRecord
  • Ruby 1.9.2

INSTALL:

$ gem install textacular

Contributing

If you'd like to contribute, please see the contribution guidelines.

Releasing

Maintainers: Please make sure to follow the release steps when it's time to cut a new release.

LICENSE:

(The MIT License)

Copyright (c) 2011 Aaron Patterson

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the 'Software'), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

textacular's People

Contributors

adamsanderson avatar andrewvos avatar benhamill avatar davidwessman avatar dotnofoolin avatar ecin avatar flavorjones avatar gdiggs avatar gregmolnar avatar hlidotbe avatar jeremyruppel avatar karthikc avatar kookster avatar kronn avatar michaelkoper avatar michaelrkn avatar mikem836 avatar muxamilian avatar nathanvda avatar ngauthier avatar olivierlacan avatar petergoldstein avatar pixeltrix avatar simi avatar sj26 avatar sjaq avatar sorentwo avatar supremebeing7 avatar tenderlove avatar yesezra avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

textacular's Issues

ORing textacular scopes with other queries

I've got a scope for basic_search:

scope :text_search, ->(query) {
  basic_search(query)
}

I need this to also search the ID of the record which is of course an Integer and not searchable using textacular.

Problem is I have no idea how to OR the where(id: query) with the scope above.

Is there any textacular way of doing this?

Is there a way set up synonyms

Hey there,

Is there a way to set up synonyms/aliases so if a user searches for "attorney", it will return the results for "lawyer"?

Any way to use unaccent ?

Hi there,

Is there anyway to use unaccent package with textacular? I know PG_Search is more "configurable", but Textacular looks cleaner to me and that's mostly the only feature that's missing for me (because from what I see, we can use advanced_search to do most of the things like prefix and such).

@benhamill I'm also willing to do a PR implementing this, just need some hints before 👍

Thanks!

Negation operator "!" escaping issue using advanced_search method

Right out of the box I am not having issues with using using the AND (&) or the OR (|) operator with the advanced search method. The following works great:

Keyword.advanced_search("foo|bar") 
or 
Keyword.advanced_search("foo & bar")

Negation by itself also works great:

Keyword.advanced_search("!baz")

However when combining negation with any other type of query param returns the following pg syntax error:

Keyword.advanced_search("foo !bar")

PG::SyntaxError: ERROR: syntax error in tsquery: "foo\ !bar"
: SELECT "keywords".*, COALESCE(ts_rank(to_tsvector('english', "keywords"."keyword"::text), to_tsquery('english', 'foo\ !bar'::text)), 0) AS "rank89557751754006092" FROM "keywords" WHERE (to_tsvector('english', "keywords"."keyword"::text) @@ to_tsquery('english', 'foo\ !bar'::text)) ORDER BY "rank89557751754006092" DESC

As stated in the ReadMe, I have checked the postgresql docs, and I notice that the query generated by the textacular gem adds a \ ad the at the end of each word within the query, where as the postgres documentation gives examples of this type of query without the . Do I simply need to escape the type of query I am trying to preform? I have checked the documentation for the this gem and have not found any reference to it, and how would I go about escaping it properly? Thanks in advance.

Stack level too deep error when used on jruby/torquebox

I'm getting a:
SystemStackError
stack level too deep

when using this gem on jruby/torquebox

it seems the error may be occurring due to the respond_to? override on:
textacular (3.2.0) lib/textacular.rb:63:in `respond_to?'

Here is the stacktrace:

textacular (3.2.0) lib/textacular.rb:63:in `respond_to?'
torquebox-transactions (3.1.0) lib/torquebox/active_record_adapters.rb:148:in `prepare'
activerecord-jdbc-adapter (1.3.10) lib/arjdbc/jdbc/adapter.rb:79:in `initialize'
activerecord-jdbc-adapter (1.3.10) lib/arjdbc/postgresql/adapter.rb:1320:in `initialize'
activerecord-jdbc-adapter (1.3.10) lib/arjdbc/jdbc/connection_methods.rb:12:in `jdbc_connection'
activerecord-jdbc-adapter (1.3.10) lib/arjdbc/postgresql/connection_methods.rb:45:in `postgresql_connection'
activerecord (4.0.6) lib/active_record/connection_adapters/abstract/connection_pool.rb:440:in `new_connection'
activerecord (4.0.6) lib/active_record/connection_adapters/abstract/connection_pool.rb:450:in `checkout_new_connection'
activerecord (4.0.6) lib/active_record/connection_adapters/abstract/connection_pool.rb:421:in `acquire_connection'
activerecord (4.0.6) lib/active_record/connection_adapters/abstract/connection_pool.rb:356:in `checkout'
/home/user/torquebox-3.1.1/jruby/lib/ruby/1.9/monitor.rb:211:in `mon_synchronize'
/home/user/torquebox-3.1.1/jruby/lib/ruby/1.9/monitor.rb:210:in `mon_synchronize'
activerecord (4.0.6) lib/active_record/connection_adapters/abstract/connection_pool.rb:355:in `checkout'
activerecord (4.0.6) lib/active_record/connection_adapters/abstract/connection_pool.rb:265:in `connection'
/home/user/torquebox-3.1.1/jruby/lib/ruby/1.9/monitor.rb:211:in `mon_synchronize'
/home/user/torquebox-3.1.1/jruby/lib/ruby/1.9/monitor.rb:210:in `mon_synchronize'
activerecord (4.0.6) lib/active_record/connection_adapters/abstract/connection_pool.rb:264:in `connection'
activerecord (4.0.6) lib/active_record/connection_adapters/abstract/connection_pool.rb:546:in `retrieve_connection'
activerecord (4.0.6) lib/active_record/connection_handling.rb:79:in `retrieve_connection'
activerecord (4.0.6) lib/active_record/connection_handling.rb:53:in `connection'
activerecord (4.0.6) lib/active_record/model_schema.rb:208:in `columns'
textacular (3.2.0) lib/textacular.rb:63:in `respond_to?'

It seems the override is done for dynamic matchers (it geves me the feeling of rails 2 dynamic matchers though I could be wrong). If it is in place for older rails versions, it might be better to set it to check for the rails version instead of overriding permanently.

column "rank0.23510353309199694" does not exist

Hi,

I found this issue from @benhamill which looks like to my issue.

Ho do you use this method to solve my problem?

ActiveRecord::SpawnMethod#except

The complete message error:

ActiveRecord::StatementInvalid: PG::Error: ERROR:  column "rank0.23510353309199694" does not exist
LINE 1: ...query('english_unaccent', 'test'::text)) ORDER BY "rank0.235...

This problem happens when I include a relation, for example:

Account.includes(:property).basic_search('hello')

Thanks.

Indexing and sql schema

It seems not good for me to change schema format to sql. That ruby one is much more clear and intuitive. Is there some way to keep on going with it? Maybe manually add that execute ... code into the schema where all other indexes are?

Thanks in advance.

Latest rails RuntimeError: unsupported: String

In the project I'm working on, we have been getting a new set of errors since we upgraded to the latest ruby/rails versions.

We can no longer run

BloggerGroup.basic_search(name: query).where_values.reduce(:and).to_sql

This now renders a "RuntimeError: unsupported: String" error.
After some debugging, it seems to happen when there are hardcoded sql strings in a where clause (such as in the basic_condition_string method in lib/textacular.rb).

I also found this issue on arel that mentions the same issue

I believe the issue might get fixed by using Arel::Nodes::NamedFunction.new('to_tsquery'... instead of a hardcoded string.

Would this be a viable solution?

Support Rails 4.2

Haven't had a chance to dig deep into the internals, but any chance of relaxing the rails dependency to support 4.2 now it has been released?

Thanks

Search against associations (model relations)

Do search against associated model attributes work? Given this example of models:

class User < ActiveRecord::Base
    has_many :projects
end

class Project < ActiveRecord::Base
    belongs_to :user
    has_one :calendar
end

class Calendar < ActiveRecord::Base
    belongs_to :project
end

How can I perform the following search:

search = 'Test'
User.basic_search(name: search, projects: { name: search })
Project.basic_search(name: search, user: { name: search }, calendar: { title: search })

Full text search (tsearch)

How can I search on my model with full text? I search like this:

Paper.basic_search(shortcut: 'F')

# Database entries
# FIB <= should be in result
# FAR <= should be in result
# FAA <= should be in result
# RIT
# SER
# ...

Paper.basic_search(shortcut: 'FA')

# Results should be
# FAR
# FAA

but I get no results!

Colons mess up advanced search

I have a Lesson AR model. If I run Lesson.advanced_search(":"), I get PG::SyntaxError: ERROR: syntax error in tsquery: ":".

Cannot get count of search results

This query runs fine when getting results back, but when we tack a count onto it, it fails with invalid SQL. Feels like a bug.

> ItemDetail.basic_search(title: 'Small').count
   (1.6ms)  SELECT COUNT("item_details".*, COALESCE(ts_rank(to_tsvector('english', "item_details"."title"::text), plainto_tsquery('english', 'Small'::text)), 0) AS "rank97053981154986493") FROM "item_details" WHERE "item_details"."deleted_at" IS NULL AND (to_tsvector('english', "item_details"."title"::text) @@ plainto_tsquery('english', 'Small'::text))
PG::SyntaxError: ERROR:  syntax error at or near "AS"
LINE 1: ...), plainto_tsquery('english', 'Small'::text)), 0) AS "rank97...
                                                             ^
: SELECT COUNT("item_details".*, COALESCE(ts_rank(to_tsvector('english', "item_details"."title"::text), plainto_tsquery('english', 'Small'::text)), 0) AS "rank97053981154986493") FROM "item_details" WHERE "item_details"."deleted_at" IS NULL AND (to_tsvector('english', "item_details"."title"::text) @@ plainto_tsquery('english', 'Small'::text))
ActiveRecord::StatementInvalid: PG::SyntaxError: ERROR:  syntax error at or near "AS"
LINE 1: ...), plainto_tsquery('english', 'Small'::text)), 0) AS "rank97...
                                                             ^
: SELECT COUNT("item_details".*, COALESCE(ts_rank(to_tsvector('english', "item_details"."title"::text), plainto_tsquery('english', 'Small'::text)), 0) AS "rank97053981154986493") FROM "item_details" WHERE "item_details"."deleted_at" IS NULL AND (to_tsvector('english', "item_details"."title"::text) @@ plainto_tsquery('english', 'Small'::text))
from /Users/lance/.rvm/gems/ruby-2.1.5/gems/activerecord-4.2.0/lib/active_record/connection_adapters/postgresql_adapter.rb:592:in `exec'

Could you stop using "rank~" alias in assemble_query method?

Currently, def assemble_query is using alias rank name by AS.

select("#{quoted_table_name + '.*,' if select_values.empty?} #{similarities.join(" + ")} AS #{rank}").

Also ActiveRecord change query to alias name using such as "AS alias_0" if use include method for eagar loading.

Post.includes(:comments).search(body: "hoge")
SELECT  DISTINCT "posts"."id", "posts"."rank67198880134037959" AS alias_0 FROM "posts" LEFT OUTER JOIN ...

This query causes syntax error because "rank~" column doesn't exist to overwrite query by ActiveRecord if use eagar loading.

Could you please stop using "rank" alias?

For example, the following code is passed even if use eager loading.

def assemble_query(similarities, conditions, exclusive)
    select("#{quoted_table_name + '.*,' if select_values.empty?} #{similarities.join(" + ")}").
      where(conditions.join(exclusive ? " AND " : " OR ")).
      order("#{similarities.join(" + ")} DESC")
end

Trigram installation fails on Heroku

This is more of a Heroku issue than Textacular, but running

heroku run rake textacular:install_trigram

gives

ls: cannot access /usr/share/postgresql/9.2/extension/pg_trgm.control: No such file or directory

The culprit seems to be that $ pg_config --sharedir returns /usr/share/postgresql/9.2, but that directory doesn't exist.

Nevertheless, simply running

$ ActiveRecord::Base.connection.execute("CREATE EXTENSION pg_trgm;")

successfully installs the extension. I can open a ticket with Heroku, but I thought I'd check here and see if it makes sense to update Textacular in some way.

Can we have a new version?

Lots of good pulls, fixes and improvements lately. None of the remaining issues appear to be breaking bugs

3.0.0 was 5 months ago, any objection to putting out a 3.1 or 3.0.1 release?

PG::SyntaxError at or near 'AS' with basic_search

I am using rails 4.1.5 with textacular 3.2.1, and have an App model. When I call @apps = App.basic_search('book'), I get the error below when I call @apps.count:

ActionView::Template::Error (PG::SyntaxError: ERROR:  syntax error at or near "AS"
LINE 1: ...t), plainto_tsquery('english', 'book'::text)), 0) AS "rank31...
                                                             ^
: SELECT COUNT("apps".*, COALESCE(ts_rank(to_tsvector('english', "apps"."name"::text), plainto_tsquery('english', 'book'::text)), 0) + COALESCE(ts_rank(to_tsvector('english', "apps"."description"::text), plainto_tsquery('english', 'book'::text)), 0) + COALESCE(ts_rank(to_tsvector('english', "apps"."creator_type"::text), plainto_tsquery('english', 'book'::text)), 0) AS "rank31756262127284230") FROM "apps"  WHERE (to_tsvector('english', "apps"."name"::text) @@ plainto_tsquery('english', 'book'::text) OR to_tsvector('english', "apps"."description"::text) @@ plainto_tsquery('english', 'book'::text) OR to_tsvector('english', "apps"."creator_type"::text) @@ plainto_tsquery('english', 'book'::text))):

Related to #59, #47 and related rails issue 15138.
A workaround (for smaller data sets) is to convert the returned results into an array and then call count on it.

Question: Getting IDs

Is it possible to just get the ID's of the result set?

search = Question.advanced_search(title: 'how do i derp')
ids = search.select(:id)
     ActiveRecord::StatementInvalid:
       PG::SyntaxError: ERROR:  subquery has too many columns

Using .pluck isn't good either.

Upgrading to rails and activerecord 5.0.0.1

Allow for activerecord version above 5.0

bundle update for rails 5.0.0.1

Bundler could not find compatible versions for gem "activerecord":
In Gemfile:
rails (= 5.0.0.1) ruby depends on
activerecord (= 5.0.0.1) ruby

textacular (~> 4.0.0) ruby depends on
  activerecord (<= 5.0, >= 3.0) ruby

Allow for activerecord version above 5.0

Error when basic_search combined with count

When trying to combine basic_search with count this happens:

[46] pry(main)> Entity.basic_search('something').count

ActiveRecord::StatementInvalid: PG::SyntaxError: ERROR:  syntax error at or near "AS"
LINE 1: ...lainto_tsquery('english', 'something'::text)), 0) AS "rank32...

The full snippet

Migration fails when textacular is in the gemfile

==  AddPaymentTypeAndPaymentIdToPayments: migrating ===========================
-- add_column(:payments, :payment_object_type, :string)
rake aborted!
An error has occurred, this and all later migrations canceled:

PG::Error: ERROR:  current transaction is aborted, commands ignored until end of transaction block
: ALTER TABLE "payments" ADD COLUMN "payment_object_type" character varying(255)

This is what I have in my migration:

class AddPaymentTypeAndPaymentIdToPayments < ActiveRecord::Migration
  class Payment < ActiveRecord::Base
  end

  def change
    add_column :payments, :payment_object_type, :string
    ...
  end
end

Migration passes if I remove textacular from Gemfile.

Using textacular with joins

Hi,

I stumbled across textacular yesterday. Is there is a way of performing a search against all the attributes of a join model.

I can search against a single attribute like so

Instructor.joins(:profile).advanced_search(profiles: { first_name: search_term } )

But let's say Profiles has a first_name, last_name, and nickname that I want to search. Is there a way to search against all the string columns of the table.

I was thinking of something along the lines of.

Instructor.joins(:profile).advanced_search(profiles: search_term )

Bug report : specific string issue with rank field & includes

With any regular string it work (ex. rabiole bebe) :

bundle exec rails c
Loading development environment (Rails 3.2.16)
2.1.0p0 :001 > Product.search('rabiole bebe').includes(:category)
  Product Load (91.6ms)  SELECT "products".*, COALESCE(ts_rank(to_tsvector('french', "products"."name"::text), plainto_tsquery('french', 'rabiole\ bebe'::text)), 0) + COALESCE(ts_rank(to_tsvector('french', "products"."brand"::text), plainto_tsquery('french', 'rabiole\ bebe'::text)), 0) + COALESCE(ts_rank(to_tsvector('french', "products"."format"::text), plainto_tsquery('french', 'rabiole\ bebe'::text)), 0) + COALESCE(ts_rank(to_tsvector('french', "products"."code"::text), plainto_tsquery('french', 'rabiole\ bebe'::text)), 0) + COALESCE(ts_rank(to_tsvector('french', "products"."category_name"::text), plainto_tsquery('french', 'rabiole\ bebe'::text)), 0) AS "rank47332022578049868" FROM "products" WHERE (to_tsvector('french', "products"."name"::text) @@ plainto_tsquery('french', 'rabiole\ bebe'::text) OR to_tsvector('french', "products"."brand"::text) @@ plainto_tsquery('french', 'rabiole\ bebe'::text) OR to_tsvector('french', "products"."format"::text) @@ plainto_tsquery('french', 'rabiole\ bebe'::text) OR to_tsvector('french', "products"."code"::text) @@ plainto_tsquery('french', 'rabiole\ bebe'::text) OR to_tsvector('french', "products"."category_name"::text) @@ plainto_tsquery('french', 'rabiole\ bebe'::text)) ORDER BY "rank47332022578049868" DESC
  Category Load (0.8ms)  SELECT "categories".* FROM "categories" WHERE "categories"."id" IN (32)
[
  [0] #<Product:0x000001088600e8> {
    :brand            => "",
    :category_id      => 32,
    :category_name    => "Fruits & Légumes              ",
    :code             => "",
    :created_at       => Thu, 30 Jan 2014 13:47:35 EST -05:00,
    :flb_category0    => 1,
    :flb_category1    => 45,
    :format           => "2 DZ",
    :id               => 3050002,
    :name             => "Rabiole bébé tige",
    :sale_category_id => nil,
    :updated_at       => Thu, 30 Jan 2014 13:47:35 EST -05:00
  }
]
2.1.0p0 :002 >

But with THIS string rabiole. bebe, it doesn't work and the generated query field list seems incorrect :

Product.search('rabiole. bebe').includes(:category)
  SQL (1.0ms)  SELECT "products"."id" AS t0_r0, "products"."name" AS t0_r1, "products"."brand" AS t0_r2, "products"."format" AS t0_r3, "products"."code" AS t0_r4, "products"."flb_category0" AS t0_r5, "products"."flb_category1" AS t0_r6, "products"."category_id" AS t0_r7, "products"."created_at" AS t0_r8, "products"."updated_at" AS t0_r9, "products"."category_name" AS t0_r10, "products"."sale_category_id" AS t0_r11, "categories"."id" AS t1_r0, "categories"."name" AS t1_r1, "categories"."slug" AS t1_r2, "categories"."flb_category0" AS t1_r3, "categories"."flb_category1" AS t1_r4, "categories"."flb_category2" AS t1_r5, "categories"."parent_id" AS t1_r6, "categories"."created_at" AS t1_r7, "categories"."updated_at" AS t1_r8 FROM "products" LEFT OUTER JOIN "categories" ON "categories"."id" = "products"."category_id" WHERE (to_tsvector('french', "products"."name"::text) @@ plainto_tsquery('french', 'rabiole.\ bebe'::text) OR to_tsvector('french', "products"."brand"::text) @@ plainto_tsquery('french', 'rabiole.\ bebe'::text) OR to_tsvector('french', "products"."format"::text) @@ plainto_tsquery('french', 'rabiole.\ bebe'::text) OR to_tsvector('french', "products"."code"::text) @@ plainto_tsquery('french', 'rabiole.\ bebe'::text) OR to_tsvector('french', "products"."category_name"::text) @@ plainto_tsquery('french', 'rabiole.\ bebe'::text)) ORDER BY "rank11759567071545294" DESC
ActiveRecord::StatementInvalid: PG::UndefinedColumn: ERROR:  column "rank11759567071545294" does not exist
LINE 1: ...query('french', 'rabiole.\ bebe'::text)) ORDER BY "rank11759...
                                                             ^

Same error with english or french set on searchable_language.

Without the includes(:category), it work properly. Weird issue, I'll simply remove my includes for now.

Returns empty result unless full title is given at search

Im having this problem where I have 3 entries of Track model in the database, they have title: "testar". when I then try to search to get these 3 record

Track.seach "tes" # witch gives me

Track Load (1.0ms) SELECT "tracks".*, ts_rank(to_tsvector('english', "tracks"."title"::text), plainto_tsquery('english', 'tes'::text)) + ts_rank(to_tsvector('english', "tracks"."territory"::text), plainto_tsquery('english', 'tes'::text)) + ts_rank(to_tsvector('english', "tracks"."country"::text), plainto_tsquery('english', 'tes'::text)) + ts_rank(to_tsvector('english', "tracks"."audio_file"::text), plainto_tsquery('english', 'tes'::text)) AS "rank0.6132058516241189" FROM "tracks" WHERE (to_tsvector('english', "tracks"."title"::text) @@ plainto_tsquery('english', 'tes'::text) OR to_tsvector('english', "tracks"."territory"::text) @@ plainto_tsquery('english', 'tes'::text) OR to_tsvector('english', "tracks"."country"::text) @@ plainto_tsquery('english', 'tes'::text) OR to_tsvector('english', "tracks"."audio_file"::text) @@ plainto_tsquery('english', 'tes'::text)) ORDER BY "rank0.6132058516241189" DESC
=> []

but Track.search "testar" gives me

Track Load (1.2ms) SELECT "tracks".*, ts_rank(to_tsvector('english', "tracks"."title"::text), plainto_tsquery('english', 'testar'::text)) + ts_rank(to_tsvector('english', "tracks"."territory"::text), plainto_tsquery('english', 'testar'::text)) + ts_rank(to_tsvector('english', "tracks"."country"::text), plainto_tsquery('english', 'testar'::text)) + ts_rank(to_tsvector('english', "tracks"."audio_file"::text), plainto_tsquery('english', 'testar'::text)) AS "rank0.6562876665202837" FROM "tracks" WHERE (to_tsvector('english', "tracks"."title"::text) @@ plainto_tsquery('english', 'testar'::text) OR to_tsvector('english', "tracks"."territory"::text) @@ plainto_tsquery('english', 'testar'::text) OR to_tsvector('english', "tracks"."country"::text) @@ plainto_tsquery('english', 'testar'::text) OR to_tsvector('english', "tracks"."audio_file"::text) @@ plainto_tsquery('english', 'testar'::text)) ORDER BY "rank0.6562876665202837" DESC
=> [#<Track id: 1, title: "testar", cataloge_id: 318854401, record_date: "2013-06-03", duration: 182, territory: "AL", country: "AL", quality: 5, category_id: nil, release_year: nil, audio_file: "Bondax_-Gold__Snakehips_Remix.mp3", created_at: "2013-03-14 01:53:01", updated_at: "2013-03-14 01:53:26">, #<Track id: 3, title: "testar", cataloge_id: 318854401, record_date: "2013-06-03", duration: 297, territory: "AL", country: "AL", quality: 5, category_id: nil, release_year: nil, audio_file: "Of_Monsters___Men_-Little_Talks__The_Knocks_Remix...", created_at: "2013-03-14 01:53:01", updated_at: "2013-03-14 01:53:26">]

what could cause it not to find them in the first example ? any help would be appreciated

Next release

Here is the list of tasks I think we could do before the next release:

  • #60 Remove the deprecated methods. This only applies if we do a major release
  • #73 This would be a neat new feature but I am not sure whether it can be done or not. We need to do some research
  • #39 Another nice new possible addition

I think these are the open issues we should close and we could do a major release, and maybe drop Rails 3 support. What do you think @benhamill @ecin ?
We also fixed a few bugs on master so we could also do a patch release from current master.
//cc @simi

It does not search non-string columns

I am implementing general and multi-option search features for my web app. In either case, I cannot search for integers using the fuzzy_search method.

In a general search, it finds no matches, because, for instance, the string '2002' isn't anywhere in string columns.

In a multi-option search, searching the year field with something like '2002' throws an error, giving:

PG::DatatypeMismatch: ERROR: argument of WHERE must be type boolean, not type integer LINE 1: SELECT COUNT(*) FROM "bicycles" WHERE (("bicycles"."year" %... ^ : SELECT COUNT(*) FROM "bicycles" WHERE (("bicycles"."year" % '2002'))

Presumably, this is because the gem is explicitly a text finder, and does not support number matching. Is there support for matching other datatypes, or is this beyond the scope of this gem?

Thanks,

Braden

Uninitialized constant Textacular::PostgresModuleInstaller

I've found that in order to run the install_trigram rake task to run, I need to do the following in my Gemfile:

gem 'textacular', :git => "git://github.com/textacular/textacular.git", 
  require:   ['textacular/postgres_module_installer', 'textacular/rails']

Without the above, I get:

rake textacular:install_trigram
rake aborted!
uninitialized constant Textacular::PostgresModuleInstaller

Is there an easier way of doing this?

Filtering results of view-based search

Hi,

I'm coming from Elasticsearch and, in an effort to reduce dependencies, I'm looking into replacing it with Postgres full-text search. One of my main requirements is to have all searches return Location objects, while looking up the search term in multiple models, including associations two levels deep.

A Location belongs to an Organization, has many Services, and Services have and belong to many Categories. I should be able to do a generic keyword search that will look up the search term within all of those models.

Then, if the search includes another parameter, like location (street address or ZIP code), or category for example, I want to be able to return Locations that are near that address AND who have Services whose Category name matches the value of the category parameter.

I tried the basic_search with multiple parameters, and it works great when querying a single model. I also tried setting up a basic searches view in the DB with just the Location name, description, and Organization name, and it works great. But now I'm having a hard time figuring out how to combine them. Whereas basic_search returns an ActiveRecord::Relation, the Search class returns an Array.

With Textacular, is it possible to search across multiple models and filter at the same time?

Thanks for any insight you can provide.

Update generated migration

For Rails 5 we should specify migration version. So it shoule be like this:

class InstallTrigram < ActiveRecord::Migration[5.0]
  def change
    enable_extension 'pg_trgm'
  end
end

Deprecation warning running search with .includes

Hey guys,

programs.includes(:address).search({addresses: {city: term}}, false) 

causes the following deprecation warning.

DEPRECATION WARNING: It looks like you are eager loading table(s) (one of: programs, addresses) that are referenced in a string SQL snippet. For example: 

    Post.includes(:comments).where("comments.title = 'foo'")

Currently, Active Record recognizes the table in the string, and knows to JOIN the comments table to the query, rather than loading comments in a separate query. However, doing this without writing a full-blown SQL parser is inherently flawed. Since we don't want to write an SQL parser, we are removing this functionality. From now on, you must explicitly tell Active Record when you are referencing a table from a string:

    Post.includes(:comments).where("comments.title = 'foo'").references(:comments)

It works fine by the way, as does using join

textacular:create_trigram_migration stuffs entire migration in filename (rails 4.1.5)

PG::Error: ERROR: function similarity(character varying, unknown) does not exist

Running bundle exec rake textacular:install_trigram RAILS_ENV=test installs the trigram module in my DB, but my cucumber features fail with the following message:

    PG::Error: ERROR:  function similarity(character varying, unknown) does not exist
    LINE 1: SELECT "projects".*, similarity("projects"."name", '24') AS ...

    HINT:  No function matches the given name and argument types. You might need to add explicit type casts.

I've got trigram installed, and everything works fine in development. My tests used to pass up until yesterday. The thing that triggered this appears to be me running my rspec specs, and then switching to cucumber.

    Gemfile:
    gem 'textacular', :git => "git://github.com/textacular/textacular.git"

I have the most recent changes from master. I've run the rake task and it is successful.

Search in association throws missing FROM-clause entry

This:

Place.joins(:address).advanced_search(name: 'moraes', address: { route: 'moraes' }).explain

Gives me this:

  Place Load (2.2ms)  SELECT "places".*, COALESCE(ts_rank(to_tsvector('portuguese', "places"."name"::text), to_tsquery('portuguese', 'moraes'::text)), 0) + COALESCE(ts_rank(to_tsvector('portuguese', "address"."route"::text), to_tsquery('portuguese', 'moraes'::text)), 0) AS "rank40213073888883937" FROM "places" INNER JOIN "addresses" ON "addresses"."id" = "places"."address_id" WHERE (to_tsvector('portuguese', "places"."name"::text) @@ to_tsquery('portuguese', 'moraes'::text) AND to_tsvector('portuguese', "address"."route"::text) @@ to_tsquery('portuguese', 'moraes'::text))  ORDER BY "rank40213073888883937" DESC
ActiveRecord::StatementInvalid: PG::UndefinedTable: ERROR:  missing FROM-clause entry for table "address"
LINE 1: ..., 0) + COALESCE(ts_rank(to_tsvector('portuguese', "address"....
                                                             ^
: SELECT "places".*, COALESCE(ts_rank(to_tsvector('portuguese', "places"."name"::text), to_tsquery('portuguese', 'moraes'::text)), 0) + COALESCE(ts_rank(to_tsvector('portuguese', "address"."route"::text), to_tsquery('portuguese', 'moraes'::text)), 0) AS "rank40213073888883937" FROM "places" INNER JOIN "addresses" ON "addresses"."id" = "places"."address_id" WHERE (to_tsvector('portuguese', "places"."name"::text) @@ to_tsquery('portuguese', 'moraes'::text) AND to_tsvector('portuguese', "address"."route"::text) @@ to_tsquery('portuguese', 'moraes'::text))  ORDER BY "rank40213073888883937" DESC
    from /usr/local/bundle/gems/rack-mini-profiler-0.10.1/lib/patches/db/pg.rb:90:in `exec'
    from /usr/local/bundle/gems/rack-mini-profiler-0.10.1/lib/patches/db/pg.rb:90:in `async_exec'
    from /usr/local/bundle/gems/activerecord-4.2.7/lib/active_record/connection_adapters/postgresql_adapter.rb:592:in `block in exec_no_cache'
    from /usr/local/bundle/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract_adapter.rb:484:in `block in log'
    from /usr/local/bundle/gems/activesupport-4.2.7/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
    from /usr/local/bundle/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract_adapter.rb:478:in `log'
    from /usr/local/bundle/gems/activerecord-4.2.7/lib/active_record/connection_adapters/postgresql_adapter.rb:592:in `exec_no_cache'
    from /usr/local/bundle/gems/activerecord-4.2.7/lib/active_record/connection_adapters/postgresql_adapter.rb:584:in `execute_and_clear'
    from /usr/local/bundle/gems/activerecord-4.2.7/lib/active_record/connection_adapters/postgresql/database_statements.rb:160:in `exec_query'
    from /usr/local/bundle/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/database_statements.rb:356:in `select'
    from /usr/local/bundle/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/database_statements.rb:32:in `select_all'
    from /usr/local/bundle/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/query_cache.rb:70:in `select_all'
    from /usr/local/bundle/gems/activerecord-4.2.7/lib/active_record/querying.rb:39:in `find_by_sql'
    from /usr/local/bundle/gems/activerecord-4.2.7/lib/active_record/relation.rb:639:in `exec_queries'
    from /usr/local/bundle/gems/activerecord-4.2.7/lib/active_record/relation.rb:238:in `block in explain'
    from /usr/local/bundle/gems/activerecord-4.2.7/lib/active_record/explain.rb:10:in `collecting_queries_for_explain'
... 3 levels...
    from /usr/local/bundle/gems/railties-4.2.7/lib/rails/commands/console.rb:9:in `start'
    from /usr/local/bundle/gems/railties-4.2.7/lib/rails/commands/commands_tasks.rb:68:in `console'
    from /usr/local/bundle/gems/railties-4.2.7/lib/rails/commands/commands_tasks.rb:39:in `run_command!'
    from /usr/local/bundle/gems/railties-4.2.7/lib/rails/commands.rb:17:in `<top (required)>'
    from /usr/local/bundle/gems/activesupport-4.2.7/lib/active_support/dependencies.rb:274:in `require'
    from /usr/local/bundle/gems/activesupport-4.2.7/lib/active_support/dependencies.rb:274:in `block in require'
    from /usr/local/bundle/gems/activesupport-4.2.7/lib/active_support/dependencies.rb:240:in `load_dependency'
    from /usr/local/bundle/gems/activesupport-4.2.7/lib/active_support/dependencies.rb:274:in `require'
    from /shared/bin/rails:9:in `<top (required)>'
    from /usr/local/bundle/gems/activesupport-4.2.7/lib/active_support/dependencies.rb:268:in `load'
    from /usr/local/bundle/gems/activesupport-4.2.7/lib/active_support/dependencies.rb:268:in `block in load'
    from /usr/local/bundle/gems/activesupport-4.2.7/lib/active_support/dependencies.rb:240:in `load_dependency'
    from /usr/local/bundle/gems/activesupport-4.2.7/lib/active_support/dependencies.rb:268:in `load'
    from /usr/local/lib/ruby/site_ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require'
    from /usr/local/lib/ruby/site_ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require'
    from -e:1:in `<main>'

I'm using Rails 4.2.7 and PostgreSQL 9.5.3.

Migration fails and more problems due to method_missing

We had a several problems using textacular, i.e. migrations would fail. It turned out to be the implementation of method_missing in textacular.

I forked the project and simply removed method_missing which fixes the issues for us:
https://github.com/DrTom/textacular

It is up to you how to fix the problem but I kindly suggest not to overwrite method_missing as the troubles that come with it are often not worth the benefits.

Travis

I see someone added the repo to travis but there is no configuration for it yet. If nobody gets there earlier than me I will create the config file this week.

Is this compatible with serialized attributes?

Hi!

I'd like to use textacular in a project where my Rails models have serialized attributes, and I need to search through them. Is this compatible?

I've been looking at the code but could not find any reference to it :)

Thanks!

Trigram should be installed as a migration

The rake task for installing trigram is great. But it should really be a migration:

  • so that other developers on my team can seamlessly install it without special instructions;
  • so that it's added to my schema and loaded onto my test database;
  • so that when I deploy, I can simply run my migrations for my app to work.

I'd suggest updating the README to recommend this approach, rather than (or as a recommended alternative to) the rake task.

It might also be nice to create a Rails generator that creates the migration automatically.

If this seems like a reasonable suggestion, I'm happy to open a pull request.

Includes

Hi

I want to use an includes and i get an error am i doing this right:

#works
@ccgs = @ccgs.advanced_search({ code: search, name: search }, false)

error

@ccgs = @ccgs.includes(:local_area_team).advanced_search({ code: search, name: search, local_area_teams: { name: search } }, false)

ActiveRecord::StatementInvalid (PG::UndefinedTable: ERROR: missing FROM-clause entry for table "local_area_team"
LINE 1: ...'''wandsworth'''::text) OR to_tsvector('english', "local_are...

I am obviously doing something stupid, but really not sure what.

Many thanks

Index migration generator is broken

$ rake textacular:create_index_migration[Lesson]
rake aborted!
undefined method `indexable_columns' for #<Class:0x007fb0928cf830>

Looking through full_text_indexer.rb, I don't see indexable_columns defined anywhere.

Also, once this is fixed, it would be helpful to document this rake task in the README.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.