opendefinition / tuoda Goto Github PK

Turn off the feature to create database in the ArangoDB connector

Find a way to determine data type of each log field prior to insertion. This will make querying easier.

Issues seen in Zeek files:
Fields may have multiple datatypes. For example integrr if it has values, a dash if not set.

During insertion, print status messages.

After insertion, write stats on how many records inserted and how many failed.

Colorization should be implemented:

• green for success
• red for error
• orange for warnings

JSON is cumbersome to manually edit. YAML is proposed as a better alternative.

When launching the Tuoda app for the first time, and the application configuration is not present, auto create it.

• If configuration folder not present -> create it
• If configuration JSON file not present -> create it

Due to the implementation using SHA256 as document ID, records may have the same hash and this causes key constraint violation.

Tuoda should instead use a guid and stull calculate the hash. The hash will then be used to find duplicates.

When importing other CSV's than Zeek, Tuoda tosses Nil pointer exceptions.

Tuoda must have a feature to pre analyze log files to determine parser configuration files

In addition to ArabgoDB, Tuoda should support Neo4j

For column names, replace . with _ to avoid issues with AQL.

This also applies to user provided header column names

Add documentation:

• CLI arguments
• Application configuration