Giter VIP home page Giter VIP logo

mikrotik-hosts-parser's Introduction

Logo

MikroTik hosts parser

Release version Build Status Coverage Image size License

This application provides HTTP server, that can generate script for RouterOS-based routers for blocking "AD" hosts using 3rd party host-lists (available by HTTP):

## Limit: 5000
## Cache lifetime: 30m0s
## Format: routeros
## Redirect to: 127.0.0.1
## Sources list:
##  - <https://adaway.org/hosts.txt>
##  - <https://www.malwaredomainlist.com/hostslist/hosts.txt>
## Excluded hosts:
##  - broadcasthost
##  - ip6-allhosts
##  - ip6-allnodes
##  - ip6-allrouters
## Cache HIT for <https://adaway.org/hosts.txt> (expires after 25m55s)
## Cache miss for <https://www.malwaredomainlist.com/hostslist/hosts.txt>

/ip dns static
add address=127.0.0.1 comment="ADBlock" disabled=no name="1-1ads.com"
add address=127.0.0.1 comment="ADBlock" disabled=no name="101com.com"
add address=127.0.0.1 comment="ADBlock" disabled=no name="101order.com"
add address=127.0.0.1 comment="ADBlock" disabled=no name="123freeavatars.com"

# ...

Hosts file format (example):

# Any comments
127.0.0.1   1-1ads.com
127.0.0.1   101com.com 101order.com
0.0.0.0     123freeavatars.com

All what you need is:

  • Start current application HTTP server
  • Make an HTTP request to the script generator endpoint /script/source?sources_urls=... with all required parameters (like records limit, hosts file URLs, exclusion list and others)
  • Generated script source execute on your RouterOS-based hardware

More information can be found here.

Previous version (PHP) can be found in php-version branch.

Installing

Download latest binary file for your os/arch from releases page or use our docker image (ghcr.io). Also you may need in configuration file ./configs/config.yml and ./web directory content for web UI access.

Usage

This application supports next sub-commands:

Sub-command Description
serve Start HTTP server
healthcheck Health checker for the HTTP server (use case - docker healthcheck) (hidden in CLI help)
version Display application version

And global flags:

Flag Description
--verbose, -v Verbose output
--debug Debug output
--log-json Logs in JSON format

HTTP server starting

serve sub-command allows to use next flags:

Flag Description Default value Environment variable
--listen, -l IP address to listen on 0.0.0.0 (all interfaces) LISTEN_ADDR
--port, -p TCP port number 8080 LISTEN_PORT
--resources-dir, -r Path to the directory with public assets ./web RESOURCES_DIR
--config, -c Config file path ./configs/config.yml CONFIG_PATH
--caching-engine Caching engine (memory or redis) memory CACHING_ENGINE
--cache-ttl Cached entries lifetime (examples: 50s, 1h30m) 30m CACHE_TTL
--redis-dsn Redis server DSN, required only if redis caching engine is enabled redis://127.0.0.1:6379/0 REDIS_DSN

Environment variables have higher priority then flag values.

Server starting command example:

$ ./mikrotik-hosts-parser serve \
    --config ./configs/config.yml \
    --listen 0.0.0.0 \
    --port 8080 \
    --resources-dir ./web

This command will start HTTP server using configuration from ./configs/config.yml on TCP port 8080 and use directory ./web for serving static files. Configuration file well-documented, so, feel free to change any settings on your choice!

Configuration file allows you to use environment variables with default values (used library)!

After that you can navigate your browser to http://127.0.0.1:8080/ and you will see something like that:

screenshot

Special endpoint /script/source?sources_urls=... generates RouterOS-based script using passed http-get parameters (watch examples on index page).

Using docker

image stats

All supported image tags can be found here and here.

Just execute in your terminal:

$ docker run --rm -p 8080:8080/tcp tarampampam/mikrotik-hosts-parser:X.X.X

Where X.X.X is image tag (application version).

Demo

I can't guarantee that this links will available forever, but you can use this application by the following links:

Testing

For application testing and building we use built-in golang testing feature and docker-ce + docker-compose as develop environment. So, just write into your terminal after repository cloning:

$ make test

Or build the binary file:

$ make build

Releasing

New versions publishing is very simple - just make required changes in this repository, update changelog file and "publish" new release using repo releases page.

Binary files and docker images will be build and published automatically.

New release will overwrite the latest docker image tag in both registers.

Changes log

Release date Commits since latest release

Changes log can be found here.

Support

Issues Issues

If you will find any package errors, please, make an issue in current repository.

License

This is open-sourced software licensed under the MIT License.

mikrotik-hosts-parser's People

Contributors

dependabot[bot] avatar tarampampam avatar github-actions[bot] avatar renovate[bot] avatar dependabot-preview[bot] avatar reallife avatar zeckson avatar timhok avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.