MT-bulk is a toolset to help manage multiple Mikrotik/RouterOS devices by sending in parallel predefined or custom commands to multiple devices at once using Mikrotik SSL API, SSH and SFTP.
Version 2.x introduces a major breaking changes, please read Version 2 breaking changes section of this readme before upgrading.
MT-bulk toolset contains two tools:
CLI tool that process devices list and commands provided by command line arguments, loaded from file or external SQL database. Commands are distributed to several internal workers and processed parallel.
Simple REST API server that process HTTPS POST requests with specified pair of commands and hosts to asynchronously execute on.
- Command's options
- Download
- List of possible operations/modes
- Configuration description
- Quick Overview / Tutorial
- Troubleshooting
MT-bulk.
Usage:
mt-bulk gen-api-certs [options]
mt-bulk gen-ssh-keys [options]
mt-bulk init-secure-api [options] [<hosts>...]
mt-bulk init-publickey-ssh [options] [<hosts>...]
mt-bulk change-password (--new=<newpass>) [--user=<login>] [options] [<hosts>...]
mt-bulk system-backup (--name=<name>) (--backup-store=<backups>) [options] [<hosts>...]
mt-bulk security-audit [options] [<hosts>...]
mt-bulk sftp <source> <target> [options] [<hosts>...]
mt-bulk custom-api [--commands-file=<commands>] [options] [<hosts>...]
mt-bulk custom-ssh [--commands-file=<commands>] [options] [<hosts>...]
mt-bulk -h | --help
mt-bulk --version
Options:
-C <config-file> Use configuration file, e.g. keys/certs locations, ports, commands sequences, custom commands, etc...
--source-db Load hosts using database configured by -C <config-file>
--source-file=<file-in> Load hosts from file <file-in>
<hosts>... List of space separated hosts in format IP[:PORT]
MT-bulk REST API gateway.
Usage:
mt-bulk-rest-gw [options]
mt-bulk-rest-gw gen-https-certs [options]
mt-bulk-rest-gw -h | --help
mt-bulk-rest-gw --version
Options:
-C <config-file> Use configuration file
Current and historical releases of mt-bulk
and mt-bulk-rest-api
at https://github.com/migotom/mt-bulk/releases
List of possible operations to execute by CLI and REST API:
- Generate Mikrotik API SSL certificate
- Generate SSH RSA Private/Public keys
- Initialize device to use Mikrotik SSL API
- Initialize device to use Public key SSH authentication
- Change user's password
- System backup
- SFTP
- Scan for CVEs and security audit
- Execute sequence of custom commands
MT-bulk supports two formats of configuration files:
- TOML format (https://github.com/toml-lang/toml)
- YAML format (https://yaml.org/spec/)
Default configuration format since version 2.x is YAML.
- Application defaults
- System (
/etc/mt-bulk/config.yml
,/Library/Application Support/MT-bulk/config.yml
) - Home (
~/.mt-bulk.yml
,~/Library/Application Support/MT-bulk/config.yml
) - Command line
-C
option
Host can be specified using:
-
simple text format:
ip
ip:port
foo.bar.com:port
-
using YAML:
host:
- ip: "192.168.1.1"
password: "secret"
- ip: "192.168.1.2:22"
user: "john"
password: "secret"
Detailed examples and sample configurations at examples/ directory of project.
mt-bulk gen-api-certs -C examples/configurations/mt-bulk.example.yml
Create new CA, host and device certificates.
mt-bulk init-secure-api -C examples/configurations/mt-bulk.example.yml 192.168.1.2 192.168.1.3:222 192.168.1.4:6654
Initialize 192.168.1.2, 192.168.1.3 and 192.168.1.4 (in this example each device has running SSH on different port) with user, SSL API and certificates pointed by mtbulk.cfg (section [service.clients.mikrotik_api.keys_store]).
mt-bulk change-password -w 16 -C examples/configurations/mt-bulk.example.yml --new=supersecret --user=admin --source-db
Change admin's password to supersecret for admin user on devices selected by SQL query using 16 workers. Connection details and query pointed by mtbulk.cfg (section [db])
mt-bulk-rest-api gen-https-certs -C examples/configurations/mt-bulk-rest-api.example.yml
Generates self signed SSL certificates and starts REST API daemon.
mt-bulk-rest-api -C examples/configurations/mt-bulk-rest-api.example.yml
Request authentication token:
$ curl -k -H "Content-Type: application/json" -d '{"key":"abc"}' -X POST https://localhost:8080/authenticate
{"token":"yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy.zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz.xxxxxxxxxxxxxxxxxxx"}
Request execute custom command:
$ curl -k -H "Authorization: some.token.value" -H "Content-Type: application/json" -d '{"host":{"ip":"10.0.0.1","user":"admin","password":"secret"},"kind":"CustomSSH","commands":[{"body":"/user print","expect":"LAST"}]}}' -X POST https://localhost:8080/job
{"results":[{"body":"/\u003cmt-bulk\u003eestablish connection","responses":["/\u003cmt-bulk\u003eestablish connection"," --\u003e attempt #0, password #0, job #bmnb10sllhcjk1tifn2g"]},{"body":"/user print","responses":["/user print\nFlags: X - disabled \n # NAME GROUP ADDRESS LAST-LOGGED-IN \n 0 ;;; system default user\n admin full jul/19/2019 07:38:54\n"]}]}
- POST https://localhost:8080/authenticate
Authenticate and obtain auth token."key"
is one of access keys defined in configuration [authenticate.key
], each"key"
can have list of regexp rules defining list of allowed device IP addresses to use in requests.
{
"key": "abc"
}
- POST https://localhost:8080/job
MT-bulk API request. Run and execute specified job with optional additional commands on specified host. Each request must have valid token asAuthorization
header field. List of possible operations
{
"host": {
"ip": "10.0.0.1",
"user": "admin",
"password": "secret"
},
"kind": "CustomSSH",
"commands": [{ "body": "/user print", "expect": "LAST-LOGGED-IN" }]
}
-
GET https://localhost:8080/{root_directory}/{path_to_file}
Download file (eg. uploaded earlier bySystemBackup
operation to MT-bulkroot_directory
defined in configuration). Each request must have valid token asAuthorization
header field. -
POST https://localhost:8080/upload
Upload a file as multipart/form-data request with fieldfile
. Uploaded file is accessible inroot_directory
to operations likeSFTP
orSystemBackup
. Each request must have valid token asAuthorization
header field.
- Verify host with running MT-bulk have access to Mikrotik device
- Verify username, password, host and port are valid, double check using eg. OpenSSH (MT-bulk doesn't require any additional tools or libraries, uses builtin in runtime SSH implementation)
- Some older RouterOS allows old and insecure ciphers, SSH implementation builtin MT-bulk will not establish connection using such ciphers, please upgrade your Mikrotik/RouterOS device
- Use strong-crypto by setting
/ip ssh set strong-crypto=yes
on RouterOS - If nothing helps please provide log of establishing connection using ssh command
ssh -vvv <user>@<ip>:<port>
CLI in version 2.x is simplified, all switches and configuring options are moved into configuration file, CLI is used to specify operation mode, configuration file and source of hosts to parse.
Configuration structure is rewritten and divided into few sections. Some of options changed as well (eg. verify_check_sleep
into verify_check_sleep_ms
). Please compare your current configuration file with attached mt-bulk.example.cfg
.
To let know to MT-bulk that configuration file is compatible with version 2.x new entry in config file was added: version = 2
.
Application was developed by Tomasz Kolaj and is licensed under Apache License Version 2.0. Please reports bugs at https://github.com/migotom/mt-bulk/issues.