Giter VIP home page Giter VIP logo

cidr-merger's Introduction

cidr-merger

A simple utility to merge ip/ip cidr/ip range, support IPv4/IPv6

$ cidr-merger --help
Usage: cidr-merger [OPTION]... [FILE]...
Write sorted result to standard output.

Options:
     --batch               batch mode (default if input files supplied or stdin
                           is not a tty), read file content into memory, then
                           write to the specified file
     --cidr                print as ip/cidr (default if not console mode)
 -c, --console             console mode(default if no input files supplied and
                           stdin is a tty), all input output files are ignored,
                           write to stdout immediately
     --empty-policy=value  indicate how to process empty input file
                             ignore(default): process as if it is not empty
                             skip: don't create output file
                             error: raise an error and exit
 -e, --error-if-empty      same as --empty-policy=error
 -h, --help                show this help menu
     --ignore-empty        same as --empty-policy=ignore
 -k, --skip-empty          same as --empty-policy=skip
     --merge               sort and merge input values (default)
     --original-order      output as the order of input, without merging
 -o, --output=file         output values to <file>, if multiple output files
                           specified, the count should be same as input files,
                           and will be processed respectively
 -r, --range               print as ip ranges
     --simple              output as single ip as possible (default)
                             ie. 192.168.1.2/32 -> 192.168.1.2
                                 192.168.1.2-192.168.1.2 -> 192.168.1.2
 -s, --standard            don't output as single ip
 -v, --version             show version info

Sample Usage:

$ echo '1.0.0.1-223.255.255.254' | cidr-merger
> 1.0.0.1
  1.0.0.2/31
  1.0.0.4/30
  1.0.0.8/29
  ......
  1.128.0.0/9
  2.0.0.0/7
  4.0.0.0/6
  8.0.0.0/5
  16.0.0.0/4
  32.0.0.0/3
  64.0.0.0/2
  128.0.0.0/2
  192.0.0.0/4
  208.0.0.0/5
  216.0.0.0/6
  220.0.0.0/7
  222.0.0.0/8
  223.0.0.0/9
  ......
  223.255.255.240/29
  223.255.255.248/30
  223.255.255.252/31
  223.255.255.254
$ echo '1.1.1.0' > a; \
    echo '1.1.1.1' > b; \
    echo '1.1.1.2/31' > c; \
    echo '1.1.1.3-1.1.1.7' > d; \
    cidr-merger -o merge a b c d; \
    cat merge; \
    rm a b c d merge
> 1.1.1.0/29
$ wget -O- "https://ftp.apnic.net/stats/apnic/`TZ=UTC date +%Y`/delegated-apnic-`TZ=UTC+24 date +%Y%m%d`.gz" | \
    gzip -d | awk -F\| '!/^\s*(#.*)?$/&&/CN\|ipv4/{print $4 "/" 32-log($5)/log(2)}' | \
    cidr-merger -eo/etc/chinadns_chnroute.txt # update ip on router
$ #              ^ e: means error if input is empty
$ echo 'fe80::/10' | cidr-merger -r
> fe80::-febf:ffff:ffff:ffff:ffff:ffff:ffff:ffff
$ echo '1.1.1.0' > a; echo '1.1.1.1' | cidr-merger - a; rm a
$ #                                                ^ -: means standard input
> 1.1.1.0/31

Difference between standard and simple(default)

$ echo '1.1.1.1/32' | cidr-merger
> 1.1.1.1
$ echo '1.1.1.1/32' | cidr-merger -s
> 1.1.1.1/32
$ echo '1.1.1.1/32' | cidr-merger -r
> 1.1.1.1
$ echo '1.1.1.1/32' | cidr-merger -rs
> 1.1.1.1-1.1.1.1

Difference about empty policy

$ cidr-merger -o txt /dev/null # an empty file named `txt` is created.
$ cidr-merger -ko txt /dev/null # no file is created, and this program exit with code zero
$ #            ^ same as `cat /dev/null | cidr-merger --skip-empty --output txt`
$ cidr-merger -eo txt /dev/null # no file is created, and this program exit with code non zero
$ #            ^ same as `cat /dev/null | cidr-merger --error-if-empty --output txt`
$ # option `-e` might be useful when download file from internet and then write to a file

$ # There is no difference if you redirect output to a file such as following
$ cat /dev/null | cidr-merger -e > txt
  # file `txt` is created, but this program exit with code non zero

cidr-merger's People

Contributors

zhanhb avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

cidr-merger's Issues

general question about merging ip ranges

Over some time I have accumulated blacklists of /24 ip ranges. What I have noticed is that these blacklisted /24 are often part of bigger ranges maybe /16[1]

To keep latency down I want to have of course as less as possible entries in the blacklist. It would be nice if your tool could assist in this matter. I have been searching a bit on the internet but cannot find something useful, or the usage is not clear to me.

If you look at the list below, I could decide to keep 200 entries with 156.202 or exchange that for one 156/202.

How would I use your tool to accumulate these 200 entries[2] to only a few ranges?

[2]
35.161.1.0
35.161.10.0
35.161.100.0
35.161.101.0
35.161.102.0
35.161.103.0
35.161.105.0
35.161.106.0
35.161.107.0
35.161.108.0
35.161.109.0
35.161.11.0
35.161.110.0
35.161.111.0
35.161.112.0
35.161.113.0
35.161.114.0
35.161.115.0
35.161.117.0
35.161.118.0
35.161.119.0
35.161.12.0
35.161.120.0
35.161.121.0
35.161.122.0
35.161.123.0
35.161.124.0
35.161.125.0
35.161.126.0
35.161.127.0
35.161.128.0
35.161.129.0
35.161.13.0
35.161.131.0
35.161.132.0
35.161.133.0
35.161.135.0
35.161.137.0
35.161.139.0
35.161.14.0
35.161.140.0
..
..
35.161.76.0
35.161.77.0
35.161.80.0
35.161.81.0
35.161.82.0
35.161.83.0
35.161.84.0
35.161.85.0
35.161.86.0
35.161.87.0
35.161.88.0
35.161.89.0
35.161.9.0
35.161.90.0
35.161.91.0
35.161.92.0
35.161.93.0
35.161.95.0
35.161.96.0
35.161.97.0
35.161.98.0

[1]

[@]# grep blacklistweb ipset | cut -d' ' -f3|sort | cut -d'.' -f1,2| uniq -c | sort -n | tail -n 10
    196 34.241
    196 35.160
    197 54.70
    198 34.227
    200 156.197
    200 156.202
    203 34.242
    204 52.36
    206 34.240
    208 35.161

Create release with binary files

Could you please create a release and upload binary files, such as: cidr-merger_linux_amd64.

Then we can get it with Nami

$ nami install github.com/zhanhb/cidr-merger

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.