flowpack / flowpack.simplesearch Goto Github PK

A simple php/sqlite search engine for generic data.

License: MIT License

PHP 100.00%

flowpack.simplesearch's Introduction

Flowpack.SimpleSearch

A simple php search engine based on SQLite or MySQL. Performance is acceptable but decreases quickly with the amount of entries. Depending on the queries you want to perform a sane upper limit is somewhere around 50000 entries (for SQLite).

This package has no hard dependencies on anything so could be used in any project.

Code and API are still pretty rough. I just implemented the minimum version. A higher level query API will follow (see the ContentRepositoryAdaptor for the direction).

If you look at the code the sqlite storage of properties looks pretty strange but with SQlite3 the actual storage type is determined per row, so a column can contain different data types in each row. That should make all those empty rows more or less acceptable. We are trying to mimic a document database here after all.

Using MySQL

To use MySQL, switch the implementation for the interfaces in your Objects.yaml and configure the DB connection as needed:

Flowpack\SimpleSearch\Domain\Service\IndexInterface:
  className: 'Flowpack\SimpleSearch\Domain\Service\MysqlIndex'

Neos\ContentRepository\Search\Search\QueryBuilderInterface:
  className: 'Flowpack\SimpleSearch\ContentRepositoryAdaptor\Search\MysqlQueryBuilder'

Flowpack\SimpleSearch\Domain\Service\MysqlIndex:
  arguments:
    1:
      value: 'Neos_CR'
    2:
      value: 'mysql:host=%env:DATABASE_HOST%;dbname=%env:DATABASE_NAME%;charset=utf8mb4'
  properties:
    username:
      value: '%env:DATABASE_USERNAME%'
    password:
      value: '%env:DATABASE_PASSWORD%'

The arguments are the index identifier (can be chosen freely) and the DSN.

flowpack.simplesearch's People

Contributors

Stargazers

Watchers

Forkers

piseth daniellienert sebobo kuborgh-mspindelhirn beheist ger4003 gerhard-boden dimitriadisg gesagtgetan kdambekalns mberhorst rolandschuetz mindscreen dlubitz

flowpack.simplesearch's Issues

Support Neos 5

If you try to install this package in a Neos 5 enviroment a fairly old version gets installed. Therefore it's unusable because the package tries to use the very old TYPO3\Flow, etc classes.

Invalid MySQL syntax for date comparison

The date functions used in the MysqlQueryBuilder are copied from the SqliteQuerybuilder but don't work with mysql:

- sprintf("datetime(`%s`) %s strftime('%s', '%s')", $propertyName, $comparator, '%Y-%m-%d %H:%M:%S', $propertyValue->format('Y-m-d H:i:s'));
+ sprintf("`%s` %s str_to_date('%s', '%s')", $propertyName, $comparator, $propertyValue->format('Y-m-d H:i:s'), '%Y-%m-%d %H:%i:%S');

str_to_date takes the format as the second argument and minutes are %i.
Unfortunately I don't see a good solution for the first datetime call, as the format depends on the indexing configuration on the property. In my tests I got the impression that there is some implicit type conversion happening, but I'm not sure if that's really what's going on.

Usage instructions

Hi. I have a sqlite PHP site and I would like to use this class. Can you give me some instructions on how to perform the searches with it?

500 Internal Server Error (Unable to prepare statement)

description
I got an internal server error after installing the flowpack/simplesearch-contentrepositoryadaptor and trying to add the Search Form into the Page. I wanted to open this issue in flowpack/simplesearch-contentrepositoryadaptor at first but the error message points to the Flowpack.SimpleSearch package.

reproduce

php70 composer.phar create-project neos/neos-base-distribution Neos

install neos with demo package

cd Neos
php70 ../composer.phar require flowpack/simplesearch-contentrepositoryadaptor --no-update 
php70 ../composer.phar update --no-dev
php70 flow flow:cache:flush

login to the neos backend
add "Search" NodeType (Flowpack.SimpleSearch.ContentRepositoryAdaptor:Search) into the main content collection.

error appears:

Server communication error: 500 Internal Server Error
Warning: SQLite3::query(): Unable to prepare statement: 1, no such column: __identifier in /usr/www/users/c0000053/Neos/Data/Temporary/Development/Cache/Code/Flow_Object_Classes/Flowpack_SimpleSearch_Domain_Service_SqLiteIndex.php line 193

Exception Code1Exception TypeNeos\Flow\Error\ExceptionLog Reference20170719103616095e07Thrown in FilePackages/Framework/Neos.Flow/Classes/Error/ErrorHandler.phpLine81

my setup

Neos.Neos 3.1.3
Neos.Flow 4.1.3
Flowpack.SimpleSearch 2.0.0
Flowpack.SimpleSearch.ContentRepositoryAdaptor 2.0.1

PostgreSQL Support

Hey,

we use PostgreSQL as our main storage for Neos and know we want to integrate a site search which should ideally use PostgreSQL.

PostgreSQL already had a good implementation through "tsvector"

Would you accept a PR for this?

Error in findOneByIdentifier function of SqLiteIndex class

When query execution fails for some reason the function causes an error.

Failed execution $statement->execute() returns false if the query execution fails. And this seems to cause the error.

HTTP ERROR 500

Hello,
I've a login script which uses prepared statements to login.
On the XAMPP it works, but when i upload it to the server, it doesn't work anymore.

(HTTP ERROR 500, UNABLE TO HANDLE THIS REQUEST)

``            $sql= "SELECT * FROM employee WHERE UName=?";
            $stmt = mysqli_stmt_init($conn);
            if(!mysqli_stmt_prepare($stmt, $sql)) {
                header('Location: login.php?MYSQLI_ERROR');
            }
           else {
               mysqli_stmt_bind_param($stmt, "s", $userName);
               mysqli_stmt_execute($stmt);
               $result = mysqli_stmt_get_result($stmt); 
           }
            if(mysqli_fetch_assoc($result))
            {
                $_SESSION['User'] =$_POST['UName'];
                $_SESSION['email'] =$_POST['email'];
                $_SESSION['name'] =$_POST['name'];
                $_SESSION['group'] =$_POST['groep'];
                header("location:wellcome.php?uid=".$_POST['UName']);
            }
is there something wrong with my statement or is it because of my server?

with kind regards,

Missing required constructor argument $storageFolder in class Flowpack\SimpleSearch\Domain\Service\SqLiteIndex.

While using Flow 3.1.0 and Neos 2.1.3 the following error occures:

Note that constructor injection is only support for objects of scope singleton (and this is not a singleton) – for other scopes you must pass each required argument to the constructor yourself.

Type: TYPO3\Flow\Object\Exception\UnresolvedDependenciesException
Code: 1296143788
File: Data/Temporary/Development/Cache/Code/Flow_Object_Classes/Flowpack_SimpleSearch_Domain_Service_SqLiteIndex.php
Line: 322

Exception #1296143788 in line 48 of Data/Temporary/Production/Cache/Code/Flow_Object_Classes/Flowpack_SimpleSearch_Factory_IndexFactory.php: Missing required constructor argument $storageFolder in class Flowpack\SimpleSearch\Domain\Service\SqLiteIndex.

Search causes 500 server error when search phrase contains one " character

When using Flowpack.SimpleSearch.ContentRepositoryAdaptor the search crashes if the search phrase contains one " character.

Some examples of search phrases causing the error:

"test
test"

SQLite match statement gets malformed, most likely because SQLite uses " character in a similar way as MySQL uses ` to identify field name.

Indexing nodes with custom defined roles

After updating to the newest version i got a problem with indexing nodes with a custom defined role. In version 1.3 it works well.

Version: 1.3.2
Neos: 2.1.3
Flowpack.SimpleSearch: 1.3.2

I configured a custom role called SpanishEditor, which has only access to edit content in the spanish dimension. You find the policy file attached: Policy.txt

If any user tries to pusblish or save some changes, the following exception occurs:

Exception: Access denied for method
Method: TYPO3\TYPO3CR\Domain\Service\Context::validateWorkspace()
Exception file: exception.txt

Did i miss something in the role configuration or is it a bug in the package? Would be nice to know.

Thanks for a hint!

`./flow nodeindex:build` exits with error

Problem

./flow nodeindex:build runs into an error. It doesn't finish, therefore.

Error logs

Terminal:

./flow nodeindex:build
Workspace "live" and dimensions "{"language":["en"]}" done. (Indexed 33 nodes)
str_replace(): Argument #3 ($subject) must be of type array|string, null given

  Type: TypeError
  File: Data/Temporary/Development/Cache/Code/Flow_Object_Classes/Neos_ContentRepository_Search_Eel_IndexingHelper.php
  Line: 161

System log

21-11-07 13:38:34 20747      CRITICAL                       Exception in line 161 of /var/www/neos/Data/Temporary/Development/Cache/Code/Flow_Object_Classes/Neos_ContentRepository_Search_Eel_IndexingHelper.php: str_replace(): Argument #3 ($subject) must be of type array|string, null given - See also: 2021110713382687ac3e.txt

Exception log

Exception in line 161 of /var/www/neos/Data/Temporary/Development/Cache/Code/Flow_Object_Classes/Neos_ContentRepository_Search_Eel_IndexingHelper.php: str_replace(): Argument #3 ($subject) must be of type array|string, null given

35 str_replace()
34 Neos\ContentRepository\Search\Eel\IndexingHelper_Original::extractHtmlTags()
33 call_user_func_array()
32 Neos\Eel\Context::call()
31 Neos\Eel\ProtectedContext::call()
30 Neos\Eel\Context::callAndWrap()
29 Neos\Eel\CompilingEvaluator_Original::{closure}()
28 Neos\Eel\CompilingEvaluator_Original::evaluateAndUnwrap()
27 Neos\Eel\CompilingEvaluator_Original::evaluate()
26 Neos\Eel\Utility_Original::evaluateEelExpression()
25 Neos\ContentRepository\Search\Indexer\AbstractNodeIndexer::evaluateEelExpression()
24 Neos\ContentRepository\Search\Indexer\AbstractNodeIndexer::extractFulltext()
23 Neos\ContentRepository\Search\Indexer\AbstractNodeIndexer::extractPropertiesAndFulltext()
22 Flowpack\SimpleSearch\ContentRepositoryAdaptor\Indexer\NodeIndexer_Original::indexNode()
21 Flowpack\SimpleSearch\ContentRepositoryAdaptor\Command\NodeIndexCommandController_Original::traverseNodes()
20 Flowpack\SimpleSearch\ContentRepositoryAdaptor\Command\NodeIndexCommandController_Original::traverseNodes()
19 Flowpack\SimpleSearch\ContentRepositoryAdaptor\Command\NodeIndexCommandController_Original::traverseNodes()
18 Flowpack\SimpleSearch\ContentRepositoryAdaptor\Command\NodeIndexCommandController_Original::traverseNodes()
17 Flowpack\SimpleSearch\ContentRepositoryAdaptor\Command\NodeIndexCommandController_Original::traverseNodes()
16 Flowpack\SimpleSearch\ContentRepositoryAdaptor\Command\NodeIndexCommandController_Original::traverseNodes()
15 Flowpack\SimpleSearch\ContentRepositoryAdaptor\Command\NodeIndexCommandController_Original::traverseNodes()
14 Flowpack\SimpleSearch\ContentRepositoryAdaptor\Command\NodeIndexCommandController_Original::traverseNodes()
13 Flowpack\SimpleSearch\ContentRepositoryAdaptor\Command\NodeIndexCommandController_Original::traverseNodes()
12 Flowpack\SimpleSearch\ContentRepositoryAdaptor\Command\NodeIndexCommandController_Original::traverseNodes()
11 Flowpack\SimpleSearch\ContentRepositoryAdaptor\Command\NodeIndexCommandController_Original::indexWorkspace()
10 Flowpack\SimpleSearch\ContentRepositoryAdaptor\Command\NodeIndexCommandController_Original::buildCommand()
9 Neos\Flow\Cli\CommandController_Original::callCommandMethod()
8 Neos\Flow\Cli\CommandController_Original::processRequest()
7 Neos\Flow\Cli\Dispatcher_Original::dispatch()
6 Neos\Flow\Cli\CommandRequestHandler::Neos\Flow\Cli\{closure}()
5 Closure::__invoke()
4 Neos\Flow\Security\Context_Original::withoutAuthorizationChecks()
3 Neos\Flow\Cli\CommandRequestHandler::handleRequest()
2 Neos\Flow\Core\Bootstrap::run()
1 require("/var/www/neos/Packages/Framework/Neos.Flow/Scripts/flow.php")

Packages

- Neos.Neos                                      7.2.0 
- Neos.Flow                                      7.2.1
- Neos.ContentRepository                         7.2.0 
- Neos.ContentRepository.Search                  4.1.0
Flowpack.SimpleSearch                            5.0.0 
- Flowpack.SimpleSearch.ContentRepositoryAdaptor 4.0.1