Giter VIP home page Giter VIP logo

cloudical-io / ancientt Goto Github PK

View Code? Open in Web Editor NEW
13.0 3.0 1.0 1.28 MB

A tool to automate network testing tools, like iperf3, in dynamic environments such as Kubernetes and more to come dynamic environments.

Home Page: https://cloudical-io.github.io/ancientt/

License: Apache License 2.0

Go 98.05% Makefile 1.45% Dockerfile 0.50%
kubernetes network iperf3 testing hacktoberfest golang

ancientt's Introduction

ancientt

A tool to automate network testing tools, like iperf3, in dynamic environments such as Kubernetes and more to come dynamic environments.

Container Image available from:

Container Image Tags:

  • main - Latest build of the main branch.
  • vx.y.z - Tagged build of the application.

Features

TL;DR A network test tool, like iperf3 can be run in, e.g., Kubernetes, cluster from all-to-all Nodes.

  • Run network tests with the following projects:
  • Tests can be run through the following "runners":
    • Ansible (an inventory file is needed)
    • Kubernetes (a kubeconfig connected to a cluster)
  • Results of the network tests can be output in different formats:
    • CSV
    • Dump (uses pp.Sprint() (GitHub k0kubun/pp, pretty print library))
    • Excel files (Excelize)
    • go-chart Charts (WIP)
    • MySQL
    • SQLite

Usage

Either build (go get), download the Ancientt executable from the GitHub release page or use the Container image.

A config file containing test definitions must be given by flag --testdefinition (or short flag -c) or named testdefinition.yaml in the current directory.

Below command will try loading your-testdefinitions.yaml as the test definitions config:

$ ancientt --testdefinition your-testdefinitions.yaml
# You can also use the short flag `-c` instead of `--testdefinition`
# and also with `-y` run the tests immediately
$ ancientt -c your-testdefinitions.yaml -y

Demos

See Demos.

Goals of this Project

  • A bit like Prometheus blackbox exporter which contains "definitions" for probes. The "tests" would be pluggable through a Golang interface.
  • "Runner" interface, e.g., for Kubernetes, Ansible, etc. The "runner" abstracts the "how it is run", e.g., for Kubernetes creates a Job, Ansible (download and) trigger a playbook to run the test.
  • Store result data in different formats, e.g., CSV, excel, MySQL
    • Up for discussion: graph database (Dgraph) and / or TSDB support
  • "Visualization" for humans, e.g., possibility to automatically draw "shiny" graphs from the results.

Development

Golang version: v1.17 or higher (tested with v1.17.6 on linux/amd64)

Creating Release

  1. Add new entry for release to CHANGELOG.md.
  2. Update VERSION with new version number.
  3. git commit and git push both changes (e.g., version: update to VERSION_HERE).
  4. Now create the git tag and push the tag git tag VERSION_HERE followed by git push --tags.

Dependencies

go mod is used to manage the dependencies.

Building

Quickest way to just get ancientt built is to run the following command:

go get -u github.com/cloudical-io/ancientt/cmd/ancientt

Licensing

Ancientt is licensed under the Apache 2.0 License.

ancientt's People

Contributors

galexrt avatar yvnash avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

Forkers

huongbn

ancientt's Issues

outputs: SQLite and MySQL

An output option to SQLite and MySQL would be good, as being able to run SELECTs over the data is "faster" than, e.g., doing excel foo depending on which excel belt you have.

For MySQL it would just take db information to connect to a database and then even create the necessary table(s) for it.

testers: More network testers tools

Must come testers are:

  • iperf - iperf3 works fine.
  • smokeping
  • siege and bombardment (use siege2csv for bombardment parsing)
  • qperf

Optional testers to come:

  • traceroute (multiple runs in one go)

Several issues with Ansible runner

  • Allow parallel Ansible setup module calls (configurable)
  • "Cache" Ansible host info (e.g., host ABC used as a server and client has the facts setup command run twice)
  • Output of which servers were okay and which failed is broken/ doesn't look good as it is weirdly broken up.
  • Ansible host info seems to take "forever" on Ancientt's side to get the response from the Ansible command.

runners: Ansible

Add an Ansible runner.

The Ansible runner would need to take care of installing the test tool during runners.Runner.Prepare() func call.
The get hosts func would return info based on the inventory (ansible-inventory command can probably be used).

Transformation Gbit

As of now the Data is shown bit_per_second only.
It would be really helpful if there is an additional column with "gbits_per_second".

testers: Siege

Add siege as a tester tool.

Old ticket info:

siege and bombardment (use siege2csv for bombardment parsing)

outputs: Prometheus

The test results should be made available to Prometheus in some to be defined way.

TODO:

  • Discuss potential implementation ways for this.
  • Future question: How can ancientt be changed that it can run in a daemon / exporter like mode / way to expose Prometheus metrics?

Update error reporting logic

Needs to be so that one error, unless "fatal" doesn't cause the whole program to hang in the end, especially when the user gives the ignoreErrors run option for a task.

This currently causes unwanted test abortions.

In that round of code change(s), it would be good to have a Status column as a "Standard" added to the results per server_host -> client_host task, e.g., wasn't able to start the server Pod will have a Status Error or so.
Problem with implementing this would mean a even higher size of the results file (especially for, e.g., CSV, Excel and others..).

outputs: go-chart

Pkg: github.com/wcharczuk/go-chart

Allow creating charts for various variation "types" (e.g., per server, summary, per data column) through this output.

Improve Release process

Following things need improvement:

  • Multi arch builds
  • Tar releases together with LICENSE, NOTICE and example config file
  • (Mostly) automatic changelog generation

*: Rework pkg structure

Currently all outputs are in the outputs/ directory of the repository.
To put it bluntly "it is getting crowded in there" with all constant and variable names.

Would be good to look into splitting it into sub pkg per output or so.

Edit: This will probably be needed for parsers, runners and testers as well..

config: better handling of defaults

It is tedious to have all those if config.ABC == nil and so on in each outputs, parsers, runners and testers.

Would be good to improve that and and add validation in the same change.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.