Giter VIP home page Giter VIP logo

labyrinth's Introduction

Labyrinth Tests

Labyrinth Logo

Labyrinth

The beautiful network analyzer, mapper, and monitor.

Labyrinth Screenshot

Install

  1. sudo bash install.sh - this will walk you through the setup needed for Auth0 information.
  2. If you are running docker as non-root, then remove the top section from install.sh and re-run.

Redis notes

Redis is also going to be used a write cache for incoming metrics. This way, the load on the metric database server will be greatly reduced. We can tune the time to write the metrics as well.

Database Notes

Attempt 2

I'm converting the Mongo databsae to a timeseries. This involved the following Python code:

db.create_collection(collection, timeseries={ 'timeField': 'timestamp', "metaField" : "metadata"})

I will also resume sending metrics directly to the second table. I'll have to set up TTL at a later date (through the Mongo Shell NOT python):

use labyrinth

db.runCommand({
   collMod: "metrics",
   expireAfterSeconds: 2419201
})

Indexes

I did a compound index of tags.labyrinth_name, tags.host, tags.ip, tags.mac - seemed to work very well.

I also added a timestamp, -1 index to help with the graphs. NOTE: This didn't end up doing anything.

Attempt 1

Labyrinth is depending on a Mongo Database trigger to move metrics from metrics-latest to metrics collection.

Here is the trigger's javascript code (I'm personally running it at 15 minute intervals):

exports = async function() {
  /*
    A Scheduled Trigger will always call a function without arguments.
    Documentation on Triggers: https://www.mongodb.com/docs/atlas/app-services/triggers/overview/

    Functions run by Triggers are run as System users and have full access to Services, Functions, and MongoDB Data.

    Access a mongodb service:
    const collection = context.services.get(<SERVICE_NAME>).db("db_name").collection("coll_name");
    const doc = collection.findOne({ name: "mongodb" });

    Note: In Atlas Triggers, the service name is defaulted to the cluster name.

    Call other named functions if they are defined in your application:
    const result = context.functions.execute("function_name", arg1, arg2);

    Access the default http client and execute a GET request:
    const response = context.http.get({ url: <URL> })

    Learn more about http client here: https://www.mongodb.com/docs/atlas/app-services/functions/context/#std-label-context-http
  */
  const mongodb = context.services.get("altamont");
  const metrics_current  = mongodb.db("labyrinth").collection("metrics-latest");
  
  
  return await metrics_current.aggregate([{"$project" : { "_id" : 0 }}, {"$merge" : "metrics"}]).toArray()
};


FAQ

1. Aren't you reinventing the wheel?

Sure - to some extent. Labyrinth is built upon very solid projects: NMap, Ansible, and Telegraf.

However, Labyrinth does some things better than other popular projects:

  • Labyrinth looks good. Yes, you can use Grafana to make pretty good dashboards, but I want something that's naturally nice looking and simple. Grafana dashboards are endlessly customizable, and by that virtue - never completed. Grafana also struggles a little with multi-host displays - I wasn't able to make it look quite like I wanted.

  • Labyrinth has better autodiscovery: port scanning. Projects like Prometheus have auto discovery, but they are very cloud-centric. Labyrinth fits best in a on-prem or hybrid situation, with many different kinds of clients joining and leaving. TCP/UDP are some of the fundamental protocols: if you can't communicate over them, something's probably wrong. Furthermore, Labyrinth wants to know if something unexpected has happened: a port being openend or closed, a new client that wasn't supposed to be in that subnet, etc.

  • Labyrinth has simple management. By using Ansible and Telegraf, it's very easy to provision from the web interface. Don't want to do that? No problem - the ports based nature of Labyrinth can give a good idea of network status without an agent.

  • Labyrinth is easy. Start it up as a docker, use the web interface, done - up and running in minutes. If you want to get into Telegraf configuration files, you're able to do that.

2. What does Labyrinth NOT do well?

Labyrinth is meant for hybrid, dynamic, check based network management. For homogenous services or full cloud offerings, there are tons of better projects: Prometheus, ELK stack, Sensu Go, etc.

Labyrinth also isn't built with metric analysis or time-series in mind - you can do them, but there are tons of better tools out there: Graylog, ELK, etc.

Labyrinth is built for smaller to midsize networks - I simply don't know how it works on large networks, since I'm building to solve my problem.

3. Who is Labyrinth made for?

Labyrinth is for whatever poor sysadmin has a small to midsize network they can't keep up with - and they just want something easy and pretty to occasionally look at. Whether that's a homelab admin or a one-man devops band, Labyrinth is here to help.

4. Can you use it with Kubernetes?

You probably can use Labyrinth for K8, but there are plenty of better, specialized tools that you should probably use instead.

Development

Start a development docker-compose stack with the following commands:

  • docker-compose -f docker-compose-development.yml up --build -d
  • Port 8100 will be the Vue frontend server. Go there to start up the development server.
  • Once the Vue frontend server has been started, navigate to :8101 to see the live frontend.
  • Certificates: you may need to point your browser to :7200 to accept the self-signed certificate. If you navigate to the frontend without doing that, you will receive "Network Error" messages.

TODO

  • Documentation on setting up Auth0 for the system. Also notes on how to disable using auth (can just have it as an ENV variable in the docker compose)

Other Wonderful Projects

  • https://github.com/SabyasachiRana/WebMap - although this project is pretty quiet currently, this had lots of good ideas. I just wish it was more of a network management tool than just scanning.
  • Prometheus
  • Sensu Go
  • etc.

labyrinth's People

Contributors

amunchet avatar

Stargazers

 avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

labyrinth's Issues

Deepcode 9

No catch method for promise [:125] . This may result in an unhandled promise rejection.

CreateEditSubnet.vue - Line 125
authService.js - Line 161

Deepcode 2

Unsanitized input from the HTTP request body [:117] flows [:117, :120, ... :139] into the return value of upload [:139] , where it is used to render an HTML page returned to the user. This may result in a Cross-Site Scripting attack (XSS).

https://owasp.org/www-community/attacks/xss/

serve.py

  • Line 139
  • Line 149
  • Line 193
  • Line 256
  • Line 326
  • Line 447
  • Line 506
  • Line 516
  • Line 584
  • Line 596
  • Line 610
  • Line 626
  • Line 637
  • Line 853

Deepcode 12

test_03_ansible.py

Line 70

Use os.makedirs instead of os.mkdir [:70] because the given path may require creating the parent directories.

Deepcode 6

Use os.makedirs instead of os.mkdir [:128] because the given path may require creating the parent directories.

serve.py

  • Line 128

ansible_helper.py

  • Line 125
  • Line 129

Deepcode 7

Missing close for concurrent.futures.ThreadPoolExecutor [:34] , add close or use a with block.

serve.py
Line 34
Line 525
Line 533
Line 558
Line 576
Line 593

watcher.py
Line 38

Deepcode 4

The global event listener for event resize [:213] on window [:213] , added in addEventListener [:213] in lifecycle method created [:212] is never removed. Remove it in the vue lifecycle method beforeDestroy.

Line 213

Dashboard.vue

Deepcode 3

map [:97] does not modify this array [:97] , it returns a new object. Ignoring the return value of map [:97] is not the intended use of this api. Did you mean to modify this array [:97] ?

CreateEditHost.vue

Line 97

Deepcode 10

Use the is operator instead of == [:20] to compare to None [:20] , because == [:20] calls eq which is slower and error-prone.

common_auth.py

  • Line 20
  • Line 25

Deepcode 5

The function defined here [:309] takes 0 parameter(s), but is called here [:274] with 1 argument(s). The extra arguments will be ignored.
Line 274

Deploy.vue

Deepcode 1

Unsanitized input from an HTTP parameter [:110] flows [:110, :114, :133] into open, where it is used as a path. This may result in a Path Traversal vulnerability and allow an attacker to read arbitrary files.

https://owasp.org/www-community/attacks/Path_Traversal

serve.py

  • Line 133
  • Line 134
  • Line 376
  • Line 496
  • Line 516
  • Line 636
  • Line 655
  • Line 685

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.