blevesearch / bleve-explorer Goto Github PK

View Code? Open in Web Editor NEW

119.0 4.0 29.0 3.39 MB

An example app providing an HTTP/REST/JSON front-end to bleve

Go 10.52% CSS 12.28% HTML 14.78% JavaScript 62.43%

bleve-explorer's Introduction

bleve-explorer

An example app providing an HTTP/REST/JSON front-end to bleve. It provides a REST API and an HTML interface to:

create/list/delete indexes
view index document count
view index mapping
index/delete documents
query indexes
monitor system performance

Watch a video introduction to bleve-explorer.

Building

go build -tags full

The -tags full is optional, but includes all the optional components of bleve.

Running

mkdir data
./bleve-explorer

This will use the default "data" dir for storing indexes. Once started you can access the web UI at http://localhost:8095/

REST API

Bleve explorer uses the handlers provided by the bleve.http package. The handlers are attached to the following URLs:

PUT /api/{indexName} - create new index
GET /api/{indexName} - get index details
DELETE /api/{indexName} - delete index
GET /api - list indexes
PUT /api/{indexName}/{docID} - index document
GET /api/{indexName}/_count - count documents in index
GET /api/{indexName}/{docID} - return stored fields of document
DELETE /api/{indexName}/{docID} - delete document
POST /api/{indexName}/_search - search index
GET /api/{indexName}/_fields - list fields used by documents in index
GET /api/{indexName}/{docID}/_debug - return rows in index related to document

Script to load documents

In the video introduction to bleve-explorer I ran a script to load a directory of JSON documents. Here is that script:

#!/bin/bash

for JsonFile in  *.json
do
    curl -X PUT http://localhost:8095/api/beer-search/$JsonFile -d @$JsonFile
done

Screenshots

Tabs showing operations available on an index

The monitoring capabilities

bleve-explorer's People

Contributors

Stargazers

Watchers

bleve-explorer's Issues

Issues building using dep

go version go1.10.1 darwin/amd64

$ dep init
$ dep ensure
$ go build 
# github.com/blevesearch/bleve-explorer/vendor/github.com/couchbase/moss
vendor/github.com/couchbase/moss/api.go:173:15: undefined: ghistogram.Histograms
vendor/github.com/couchbase/moss/collection.go:98:13: undefined: ghistogram.Histograms
# github.com/blevesearch/bleve-explorer/vendor/github.com/blevesearch/bleve-mapping-ui
vendor/github.com/blevesearch/bleve-mapping-ui/mapping.go:105:13: undefined: bleve.IndexMapping

toml has


[[constraint]]
  name = "github.com/blevesearch/bleve"
  version = "0.7.0"

[[constraint]]
  name = "github.com/blevesearch/bleve-mapping-ui"
  version = "0.1.0"

[[constraint]]
  name = "github.com/elazarl/go-bindata-assetfs"
  version = "1.0.0"

[[constraint]]
  name = "github.com/gorilla/mux"
  version = "1.6.1"

[prune]
  go-tests = true
  unused-packages = true

Get all the documents in an Index

Is there any Rest API to load all the Documents Present in the Index file.

Currently, i am indexing the Document (in all 25 document) and on search i am able to get the Corresponding Documents in the Search text input.

How to load the entire Document (all 25 document) irrespective of any text search.

can't delete a document structure child field

In the index mapping UI editor, after creating a sub-field in the Document Structure tree panel, a user can edit the properties of that sub-field but cannot delete that sub-field.

How to choose storage or get access to bleve command?

How to pass a storage paramter when creating index

How do I do it manually using bleve command parameter --s in bleve-explorer project?

[bug]date is not automatically recognized and converted correctly in putdata

date is not automatically recognized and converted correctly in put data
This causes date to not be able to use date's
date:>"2022-10-01"
make an inquiry

cannot find package "golang.org/x/text/unicode/norm"

$ go build

cannot find package "golang.org/x/text/unicode/norm" in any of:
/Users/.../projects/go-projects/src/github.com/blevesearch/bleve/vendor/golang.org/x/text/
unicode/norm (vendor tree)
/usr/local/go/src/golang.org/x/text/unicode/norm (from $GOROOT)
/Users/.../projects/go-projects/src/golang.org/x/text/unicode/norm (from $GOPATH)

Documentation / question : explanation on the structure & routing of the project

Would be nice to have a little introduction to this project.
For example the routing in main.go for /API is very clear
And they are pointing to handlers there : bleveHttp "github.com/blevesearch/bleve/http"

But when getting to the route /indexes/ it gets confusing of where this is ...
Is it somewhere within angular ?
http://127.0.0.1:8095/indexes/_new

invalid memory address

In my linux debian appear this error when you try to navitage
//trace

2016/01/04 23:16:13 http: panic serving [::1]:36988: runtime error: invalid memory address or nil pointer dereference
goroutine 9 [running]:
net/http.(_conn).serve.func1(0xc820144000, 0x7f94cd544d88, 0xc820030260)
/usr/local/go/src/net/http/server.go:1287 +0xb5
github.com/elazarl/go-bindata-assetfs.(_AssetFS).Open(0xc8200d3740, 0xc82014c360, 0x11, 0x0, 0x0, 0x0, 0x0)
/home/albertog/go_path/src/github.com/elazarl/go-bindata-assetfs/assetfs.go:148 +0x22e
net/http.serveFile(0x7f94cc4fd058, 0xc82015a000, 0xc8200e8d20, 0x7f94cd5438a8, 0xc8200d3740, 0xa37008, 0x1, 0x756ea1346ab401)
/usr/local/go/src/net/http/fs.go:394 +0x5b1
net/http.(_fileHandler).ServeHTTP(0xc8200dc110, 0x7f94cc4fd058, 0xc82015a000, 0xc8200e8d20)
/usr/local/go/src/net/http/fs.go:483 +0x19c
main.myFileHandler.ServeHTTP(0x7f94cd5438d0, 0xc8200dc110, 0x7f94cc4fd058, 0xc82015a000, 0xc8200e8d20)
/home/albertog/Downloads/bleve-explorer-master/http_util.go:59 +0xb5
main.(_myFileHandler).ServeHTTP(0xc8200dc1e0, 0x7f94cc4fd058, 0xc82015a000, 0xc8200e8d20)
:7 +0xb6
net/http.StripPrefix.func1(0x7f94cc4fd058, 0xc82015a000, 0xc8200e8d20)
/usr/local/go/src/net/http/server.go:1455 +0x1a5
net/http.HandlerFunc.ServeHTTP(0xc8200d3800, 0x7f94cc4fd058, 0xc82015a000, 0xc8200e8d20)
/usr/local/go/src/net/http/server.go:1422 +0x3a
github.com/gorilla/mux.(_Router).ServeHTTP(0xc820015a90, 0x7f94cc4fd058, 0xc82015a000, 0xc8200e8d20)
/home/albertog/go_path/src/github.com/gorilla/mux/mux.go:103 +0x270
net/http.(_ServeMux).ServeHTTP(0xc820010a20, 0x7f94cc4fd058, 0xc82015a000, 0xc8200e8d20)
/usr/local/go/src/net/http/server.go:1699 +0x17d
net/http.serverHandler.ServeHTTP(0xc82012c660, 0x7f94cc4fd058, 0xc82015a000, 0xc8200e8d20)
/usr/local/go/src/net/http/server.go:1862 +0x19e
net/http.(_conn).serve(0xc820144000)
/usr/local/go/src/net/http/server.go:1361 +0xbee
created by net/http.(_Server).Serve
/usr/local/go/src/net/http/server.go:1910 +0x3f6
2016/01/04 23:16:13 http: panic serving [::1]:36989: runtime error: invalid memory address or nil pointer dereference
goroutine 10 [running]:
net/http.(_conn).serve.func1(0xc8201440b0, 0x7f94cd544d88, 0xc820030268)
/usr/local/go/src/net/http/server.go:1287 +0xb5
github.com/elazarl/go-bindata-assetfs.(_AssetFS).Open(0xc8200d3740, 0xc82014c700, 0x11, 0x0, 0x0, 0x0, 0x0)
/home/albertog/go_path/src/github.com/elazarl/go-bindata-assetfs/assetfs.go:148 +0x22e
net/http.serveFile(0x7f94cc4fd058, 0xc82015a2c0, 0xc8200e8e00, 0x7f94cd5438a8, 0xc8200d3740, 0xa37008, 0x1, 0x756ea102ab1001)
/usr/local/go/src/net/http/fs.go:394 +0x5b1
net/http.(_fileHandler).ServeHTTP(0xc8200dc110, 0x7f94cc4fd058, 0xc82015a2c0, 0xc8200e8e00)
/usr/local/go/src/net/http/fs.go:483 +0x19c
main.myFileHandler.ServeHTTP(0x7f94cd5438d0, 0xc8200dc110, 0x7f94cc4fd058, 0xc82015a2c0, 0xc8200e8e00)
/home/albertog/Downloads/bleve-explorer-master/http_util.go:59 +0xb5
main.(_myFileHandler).ServeHTTP(0xc8200dc1e0, 0x7f94cc4fd058, 0xc82015a2c0, 0xc8200e8e00)
:7 +0xb6
net/http.StripPrefix.func1(0x7f94cc4fd058, 0xc82015a2c0, 0xc8200e8e00)
/usr/local/go/src/net/http/server.go:1455 +0x1a5
net/http.HandlerFunc.ServeHTTP(0xc8200d3800, 0x7f94cc4fd058, 0xc82015a2c0, 0xc8200e8e00)
/usr/local/go/src/net/http/server.go:1422 +0x3a
github.com/gorilla/mux.(_Router).ServeHTTP(0xc820015a90, 0x7f94cc4fd058, 0xc82015a2c0, 0xc8200e8e00)
/home/albertog/go_path/src/github.com/gorilla/mux/mux.go:103 +0x270
net/http.(_ServeMux).ServeHTTP(0xc820010a20, 0x7f94cc4fd058, 0xc82015a2c0, 0xc8200e8e00)
/usr/local/go/src/net/http/server.go:1699 +0x17d
net/http.serverHandler.ServeHTTP(0xc82012c660, 0x7f94cc4fd058, 0xc82015a2c0, 0xc8200e8e00)
/usr/local/go/src/net/http/server.go:1862 +0x19e
net/http.(_conn).serve(0xc8201440b0)
/usr/local/go/src/net/http/server.go:1361 +0xbee
created by net/http.(_Server).Serve
/usr/local/go/src/net/http/server.go:1910 +0x3f6
2016/01/04 23:16:13 http: panic serving [::1]:36990: runtime error: invalid memory address or nil pointer dereference
goroutine 18 [running]:
net/http.(_conn).serve.func1(0xc82015a420, 0x7f94cd544d88, 0xc820174010)
/usr/local/go/src/net/http/server.go:1287 +0xb5
github.com/elazarl/go-bindata-assetfs.(_AssetFS).Open(0xc8200d3740, 0xc820138540, 0x11, 0x0, 0x0, 0x0, 0x0)
/home/albertog/go_path/src/github.com/elazarl/go-bindata-assetfs/assetfs.go:148 +0x22e
net/http.serveFile(0x7f94cc4fd058, 0xc820144160, 0xc8201cc000, 0x7f94cd5438a8, 0xc8200d3740, 0xa37008, 0x1, 0x756ea17084c301)
/usr/local/go/src/net/http/fs.go:394 +0x5b1
net/http.(_fileHandler).ServeHTTP(0xc8200dc110, 0x7f94cc4fd058, 0xc820144160, 0xc8201cc000)
/usr/local/go/src/net/http/fs.go:483 +0x19c
main.myFileHandler.ServeHTTP(0x7f94cd5438d0, 0xc8200dc110, 0x7f94cc4fd058, 0xc820144160, 0xc8201cc000)
/home/albertog/Downloads/bleve-explorer-master/http_util.go:59 +0xb5
main.(_myFileHandler).ServeHTTP(0xc8200dc1e0, 0x7f94cc4fd058, 0xc820144160, 0xc8201cc000)
:7 +0xb6
net/http.StripPrefix.func1(0x7f94cc4fd058, 0xc820144160, 0xc8201cc000)
/usr/local/go/src/net/http/server.go:1455 +0x1a5
net/http.HandlerFunc.ServeHTTP(0xc8200d3800, 0x7f94cc4fd058, 0xc820144160, 0xc8201cc000)
/usr/local/go/src/net/http/server.go:1422 +0x3a
github.com/gorilla/mux.(_Router).ServeHTTP(0xc820015a90, 0x7f94cc4fd058, 0xc820144160, 0xc8201cc000)
/home/albertog/go_path/src/github.com/gorilla/mux/mux.go:103 +0x270
net/http.(_ServeMux).ServeHTTP(0xc820010a20, 0x7f94cc4fd058, 0xc820144160, 0xc8201cc000)
/usr/local/go/src/net/http/server.go:1699 +0x17d
net/http.serverHandler.ServeHTTP(0xc82012c660, 0x7f94cc4fd058, 0xc820144160, 0xc8201cc000)
/usr/local/go/src/net/http/server.go:1862 +0x19e
net/http.(_conn).serve(0xc82015a420)
/usr/local/go/src/net/http/server.go:1361 +0xbee
created by net/http.(_Server).Serve
/usr/local/go/src/net/http/server.go:1910 +0x3f6

Does not build on osx :(

::: ~ $ git clone [email protected]:blevesearch/bleve-explorer.git
Cloning into 'bleve-explorer'...
remote: Enumerating objects: 347, done.
remote: Total 347 (delta 0), reused 0 (delta 0), pack-reused 347
Receiving objects: 100% (347/347), 3.39 MiB | 13.65 MiB/s, done.
Resolving deltas: 100% (157/157), done.
::: ~ $ cd bleve-explorer 
bleve-explorer $ git:(master) go mod init github.com/blevesearch/bleve-explorer
go: creating new go.mod: module github.com/blevesearch/bleve-explorer
::: bleve-explorer $ go build -tags full
go: finding module for package github.com/blevesearch/bleve/config
go: finding module for package github.com/gorilla/mux
go: finding module for package github.com/blevesearch/bleve-mapping-ui
go: finding module for package github.com/elazarl/go-bindata-assetfs
go: finding module for package github.com/blevesearch/bleve
go: finding module for package github.com/blevesearch/bleve/http
go: found github.com/blevesearch/bleve in github.com/blevesearch/bleve v1.0.14
go: found github.com/blevesearch/bleve-mapping-ui in github.com/blevesearch/bleve-mapping-ui v0.4.0
go: found github.com/blevesearch/bleve/config in github.com/blevesearch/bleve v1.0.14
go: found github.com/blevesearch/bleve/http in github.com/blevesearch/bleve v1.0.14
go: found github.com/elazarl/go-bindata-assetfs in github.com/elazarl/go-bindata-assetfs v1.0.1
go: found github.com/gorilla/mux in github.com/gorilla/mux v1.8.0
# github.com/jmhodges/levigo
../../go/pkg/mod/github.com/jmhodges/[email protected]/batch.go:4:11: fatal error: 'leveldb/c.h' file not found
 #include "leveldb/c.h"
          ^~~~~~~~~~~~~
1 error generated.
# github.com/blevesearch/blevex/icu
../../go/pkg/mod/github.com/blevesearch/[email protected]/icu/boundary.go:15:11: fatal error: 'unicode/utypes.h' file not found
 #include "unicode/utypes.h"
          ^~~~~~~~~~~~~~~~~~
1 error generated.
# github.com/tecbot/gorocksdb
../../go/pkg/mod/github.com/tecbot/[email protected]/array.go:4:11: fatal error: 'rocksdb/c.h' file not found
 #include "rocksdb/c.h"
          ^~~~~~~~~~~~~
1 error generated.
::: bleve-explorer $

build broken

bleve-explorer git:(master) ✗ go build .
# github.com/blevesearch/bleve-explorer
./bindata_assetfs.go:105:6: bindataRead redeclared in this block
        previous declaration at ./bindata.go:104:53
./bindata_assetfs.go:125:6: asset redeclared in this block
        previous declaration at ./bindata.go:124:6
./bindata_assetfs.go:130:6: bindataFileInfo redeclared in this block
        previous declaration at ./bindata.go:129:6
./bindata_assetfs.go:156:5: _staticCssAppCss redeclared in this block
        previous declaration at ./bindata.go:166:5
./bindata_assetfs.go:158:6: staticCssAppCssBytes redeclared in this block
        previous declaration at ./bindata.go:168:38
./bindata_assetfs.go:165:6: staticCssAppCss redeclared in this block
        previous declaration at ./bindata.go:175:33
./bindata_assetfs.go:176:5: _staticCssBleveCss redeclared in this block
        previous declaration at ./bindata.go:186:5
./bindata_assetfs.go:178:6: staticCssBleveCssBytes redeclared in this block
        previous declaration at ./bindata.go:188:40
./bindata_assetfs.go:185:6: staticCssBleveCss redeclared in this block
        previous declaration at ./bindata.go:195:35
./bindata_assetfs.go:196:5: _staticCssBootstrapThemeCss redeclared in this block
        previous declaration at ./bindata.go:206:5
./bindata_assetfs.go:196:5: too many errors

Can't build the package

build fails on 2 dependencies

# github.com/couchbase/moss
../../../../pkg/mod/github.com/couchbase/[email protected]/api.go:176:15: undefined: ghistogram.Histograms
../../../../pkg/mod/github.com/couchbase/[email protected]/collection.go:98:13: undefined: ghistogram.Histograms
# github.com/blevesearch/bleve-mapping-ui
../../../../pkg/mod/github.com/blevesearch/[email protected]/mapping.go:105:13: undefined: bleve.IndexMapping

Files indexing.

Hello!
i need to index files in bleve (not the content but the file and search the file name using the query)
How can i do this?

Also could you add an raw export link at every result? So i can simply export a document by clicking?

Undefined func / struct http.IndexStats

Hi,

When running go build in '~/go/src/github.com/blevesearch/bleve-explorer', the following error is generated:

./main.go:53: undefined: "github.com/blevesearch/bleve/http".IndexStats

This is the line in blevesearch/bleve-explorer/main.go that the error is referring to:

expvars.Set("indexes", bleveHttp.IndexStats())

However, as far as I am able to tell there is no function IndexStats() contained in blevesearch/bleve/http, but there is a struct IndexStat contained in blevesearch/bleve/index_stats.go.

Your thoughts / advice on how to resolve this issue will be greatly appreciated.

Quick start / Building Readme

If you follow the current readme, you will have to install pachage that are complex to install and requires C.
Which is not what most people will want to jump start with explorer.
It brings simmilar issues as : #8
Better starting with a simple "go build "

Where as the current readme advise :
Building
go build -tags full
Will give you this :
"

github.com/blevesearch/blevex/icu

../blevex/icu/boundary.go:15:11: fatal error: unicode/utypes.h: No such file or directory
// #include "unicode/utypes.h"
^~~~~~~~~~~~~~~~~~
compilation terminated.

github.com/jmhodges/levigo

../../jmhodges/levigo/batch.go:4:11: fatal error: leveldb/c.h: No such file or directory
// #include "leveldb/c.h"
^~~~~~~~~~~~~
compilation terminated.

github.com/tecbot/gorocksdb

../../tecbot/gorocksdb/array.go:4:11: fatal error: rocksdb/c.h: No such file or directory
// #include "rocksdb/c.h"
^~~~~~~~~~~~~
compilation terminated.
"
Apart from this the bleve-explorer example works like a charm ;)

API for search query strings

Hello, the general API instructions seems to be missing a section on how search queries can be made through REST API.

Are there some general (or detailed) instructions I could follow to make API calls to the bleve server/ index and have the results sent back as json?

Hope this is not too much trouble.

getting it running...

Just trying to get this going to play around.
Pulled the code...
Then from read me ran the go build -tags full and boom:::

go get -u github.com/blevesearch/bleve-explorer
x-MacBook-Pro:blevesearch apple$ ls
bleve           bleve-explorer      bleve-mapping-ui    go-porterstemmer    segment
x-MacBook-Pro:blevesearch apple$ cd bleve-explorer/
x-MacBook-Pro:bleve-explorer apple$ ls
README.md       bleve-explorer      docs            main.go
bindata_assetfs.go  data            http_util.go        static
x-MacBook-Pro:bleve-explorer apple$ go build -tags full
../bleve/config/config_cznicb.go:15:2: cannot find package "github.com/blevesearch/blevex/cznicb" in any of:
    /usr/local/go/src/github.com/blevesearch/blevex/cznicb (from $GOROOT)
    /Users/apple/workspace/go/src/github.com/blevesearch/blevex/cznicb (from $GOPATH)
../bleve/config/config_cld2.go:15:2: cannot find package "github.com/blevesearch/blevex/detect_lang" in any of:
    /usr/local/go/src/github.com/blevesearch/blevex/detect_lang (from $GOROOT)
    /Users/apple/workspace/go/src/github.com/blevesearch/blevex/detect_lang (from $GOPATH