Giter VIP home page Giter VIP logo

kibana-xlsx-import's Introduction

XLSX import

Kibana plugin for import XLSX/CSV file to ElasticSearch


Installation

Use this example or manually pick a release

/opt/kibana/bin/kibana-plugin install https://github.com/Orange-OpenSource/kibana-xlsx-import/releases/download/6.5.4/kibana-xlsx-import-6.5.4-latest.zip

How to use

1 - Select your file and sheet

2 - Configure

- If you want to define your own mapping, you have to use an index name which doesn't already exist.
- Index name must be all lower case.

Use your own mapping

For every field you can choose the type and apply more options with the advanced JSON.
The list of parameters can be found here, https://www.elastic.co/guide/en/elasticsearch/reference/6.2/mapping-params.html

Use a custom Kibana ID

To create a custom Kibana ID with your field values you can use a template with placeholder {fieldname}.
In our example, if I use the following template {ID}-{Nom} this will lead us to the following ID for the first document :

  • 1-Toto

You're done !

Advanced settings

In the kibana advanced setting you can find those parameters
Warning : All those options might cause crash or slowing the plugin.

  • kibana-xlsx-import:bulk_package_size
    Allow you to define the number of json item you want to send per bulk package

  • kibana-xlsx-import:displayed_rows
    Define the number of rows which will be displayed in the preview


development

See the kibana contributing guide for instructions setting up your development environment. Once you have completed that, use the following npm tasks.

  • npm start

    Start kibana and have it include this plugin

  • npm start -- --config kibana.yml

    You can pass any argument that you would normally send to bin/kibana by putting them after -- when running npm start

  • npm run build

    Build a distributable archive

  • npm run test:browser

    Run the browser tests in a real web browser

  • npm run test:server

    Run the server tests using mocha

For more information about any of these commands run npm run ${task} -- --help.

kibana-xlsx-import's People

Contributors

bjacomy avatar dependabot[bot] avatar gwintzer avatar jmcwork avatar kyushy avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

kibana-xlsx-import's Issues

Strange behavior when loading data from a column with "space" in column header

Loading a 3 column xlsx file,
column 1 : "date" (date)
column 2 : "downloads" (integer)
column 3 : 'daily downloads' (integer)

I chose date for c1 mapping, double (or float in an other test) for c2 et c3.
Data is loaded:

  {
    "_index": "myindex",
    "_type": "doc",
    "_id": "0eg3K2MBlI2vUD8Ex95q",
    "_score": 1,
    "_source": {
      "date": "2015-01-11",
      "downloads": "503",
      "daily downloads": "53"
    }
  },

But when I create the kibana index I get the following fields:

daily downloads | string
daily downloads.keyword | string
daily_downloads | number

Then in the vizualusations, I can only use "daily_downloads" as numerical field but this field is null...
By removing the " " in the column header, everything is fine.

Support for 5.6.2

Hi,

First of all thanks for this plugin.
Now I was able to install the nice plugin on my Kibana 5.6.2 using the 5.6.5 build by changing the package.json. Installation was successful and restarted the Kibana with the XLSX plugin on my menu.
However i am not able to load any CSV/xlsx files and getting an error.
Uncaught ReferenceError:XLSX is not defined(../bundles/xlsx-import.bundle.js?v=15543:27)

I suspect this is cause of the version mismatch, Is there anyway i could run this for the v5.6.2?

Appreciate a needful response.

Problem with big volume data and timeout

Hello,
I have 2 problem ;

Firstly, I tried to import 1.800.000 document with 30 field then I got this error: "Page not responding" (I think It s a kind of timeout issue)

Secondly, I got a mapping error like that: "Something wrong happened, check your advanced JSON", but I did not do any change in advanced JSON part.(just click next and import) My data is:

Capture

Thank you.

data format

Is there a fault date format,which the program will detect automatically the fields ,because by default it is considered as text . on setting custom mappings , as date , it throws the following error.
failed to parse [ts] Invalid format: "2015-Jan-01 01:00:00" is malformed at "-Jan-01 01:00:00".
Advanced JSON also failed .Could anyone suggest the correct format of Advanced JSON.

Thanks for the effort team.This can turn out really valuable

Add mapping options (fielddata, multifield) on text field

Hi,

In the mapping tab, when i choose a text-typed field, i'd like have options (like checkbox or radio button) to specify if the field :

  • must be aggregatable (fielddata: true)
  • must have a keyword multi-field ({fields: {"keyword": {"type": "keyword"}}})

version update required

This doesnot work with latest version, have tried it on both 6.2.4 and 6.3.1 no luck. Think so version upgrade is required..

Nested object support

Is there any way to import a field containing nested object, for example a cell containing {"field1":"value1","field2":"value2"} imported to a nested object type???

Cannot resolve react-dom/server (kibana 6.6.0 compat)

I was checking if this repo is compatible with kibana 6.6.0. So, i forked it, made few adjustments (wzulfikar@1f726bd), tried to upload csv, and everything works.

The thing is, to make it works, I had to install react-dom. Without installing react-dom, kibana-plugin install will fail:

Module not found: Error: Can't resolve 'react-dom/server'...

The error was triggered because below ReactDOMServer isn't resolvable:

import ReactDOMServer from 'react-dom/server';

Upon checking package.json, I realized that react-dom is not installed. I went further to see if react-dom was installed transitively and it turns out that it's listed in elastic/eui, but as devDependencies (elastic/eui#L145).

My question is, is this error expected (due to lack of react-dom in package.json), or kibana-plugin is supposed to install devDependencies (so that react-dom is available via elastic/eui)?

Thanks for the awesome plugin, btw :)

  • edit 1 (Feb 27, 2019): add screenshot

Here's additional info to add more context.

And here's images on how it looks like in kibana 6.6.0.

importing csv:
screenshot from 2019-02-27 02-37-21

csv imported:
screenshot from 2019-02-27 02-38-10

Compatibility with 6.2.4

Hello,

I'm not able to install the current version with version 6.2.4. Is there a new version soon ?

Best regards

Typo, Presentation... : proposals

PLugin title (XLSX Import) is not really a title and not aligned with any Kibana title (lot of different titles in Kibana btw)
In French replace "convertir" by sthg more accurate
(?) following "ID dans elasticsearch" is too big compare to space left between lines
Put some space betwwen what you can change eg mapping and mandatory fields

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.