vel21ripn / ndpi Goto Github PK
View Code? Open in Web Editor NEWThis project forked from ntop/ndpi
Open Source Deep Packet Inspection Software Toolkit
Home Page: http://www.ntop.org
License: GNU Lesser General Public License v3.0
ndpi's People
Contributors
Stargazers
Watchers
Forkers
zhaoxx063 sniperpr elico paglysalva arcanez dogfight360 d3adme4t exploitfate halvors natashar4in pasha49 stroboscope bhyvex loupiautte indigos33k3r applesky6 tonylu00 e-mailky clebig itsnayabsd liuxc0116 unveiltech drwatson32 virtua1 lunatickochiya fatihusta yurgen1975 jpizzribeiro mashraf09 ndmsystems lantis1008 jason416 roltd hqs-cyber xbeowulfx extart ttys3 lailitty ttpartizan whosea metercai hcg191132cug filali-ai someonebw rusbomber ewildgoose techteamcr vigilans dhimantthanki richardawesome oom-killer-design houmie omikhaylov aldaihanabdullah3 brainslayerndpi's Issues
example/ndpiReader returns immediately for netflow sources
this runs for 5 seconds, as expected, and then returns the proper data
./example/ndpiReader -i eth0 -s 5
Using nDPI (1.7.0-netfilter-215-4752575) [1 thread(s)]
Capturing live traffic from device eth0...
Capturing traffic up to 5 seconds
Running thread 0...
this runs and returns immediately instead of running for the expected 5 seconds. (it does get data for 1 second)
./example/ndpiReader -i nflog:42 -s 5
Using nDPI (1.7.0-netfilter-215-4752575) [1 thread(s)]
Capturing live traffic from device nflog:42...
Capturing traffic up to 5 seconds
Running thread 0...
Latest nDPI version support
Are you integrate the latest nDPI ?
seems some new interesting added protocols :
Especially QUIC used by default by Chrome browsers
- QUIC
- WhatsApp Voice
- Stracraft
- Teredo
- Snapchat
- Simet
- OpenSignal
- 99Taxi
- GloboTV
- Deezer
- Microsoft cloud services
- Twitch
- KakaoTalk Voice and Chat
- HotspotShield VPN
CentOS 7 build instructions.
I tried to build the module on CentOS 7 and succedded but the iptables rules doesn't match.
Anyone knows how to build the module on CentOS?
CentOS 7 elrepo compilation error
I used the same build node of the CentOS 7 which works but I removed the stock kernel and install the elrepo(ml) one.
Then when I try to compile I get the next error:
[root@02e740e242ca nDPI]# ./autogen.sh
autoreconf: Entering directory `.'
autoreconf: configure.ac: not using Gettext
autoreconf: running: aclocal --force -I m4
autoreconf: configure.ac: tracing
autoreconf: running: libtoolize --copy --force
libtoolize: putting auxiliary files in `.'.
libtoolize: copying file `./ltmain.sh'
libtoolize: putting macros in AC_CONFIG_MACRO_DIR, `m4'.
libtoolize: copying file `m4/libtool.m4'
libtoolize: copying file `m4/ltoptions.m4'
libtoolize: copying file `m4/ltsugar.m4'
libtoolize: copying file `m4/ltversion.m4'
libtoolize: copying file `m4/lt~obsolete.m4'
autoreconf: running: /usr/bin/autoconf --force
autoreconf: running: /usr/bin/autoheader --force
autoreconf: running: automake --add-missing --copy --force-missing
configure.ac:7: installing './config.guess'
configure.ac:7: installing './config.sub'
configure.ac:5: installing './install-sh'
configure.ac:5: installing './missing'
example/Makefile.am: installing './depcomp'
autoreconf: Leaving directory `.'
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /usr/bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking how to print strings... printf
checking for style of include used by make... GNU
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking dependency style of gcc... gcc3
checking for a sed that does not truncate output... /usr/bin/sed
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for fgrep... /usr/bin/grep -F
checking for ld used by gcc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 1572864
checking whether the shell understands some XSI constructs... yes
checking whether the shell understands "+="... yes
checking how to convert x86_64-unknown-linux-gnu file names to x86_64-unknown-linux-gnu format... func_convert_file_noop
checking how to convert x86_64-unknown-linux-gnu file names to toolchain format... func_convert_file_noop
checking for /usr/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for ar... ar
checking for archiver @FILE support... @
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from gcc object... ok
checking for sysroot... no
checking for mt... no
checking if : is a manifest tool... no
checking how to run the C preprocessor... gcc -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for dlfcn.h... yes
checking for objdir... .libs
checking if gcc supports -fno-rtti -fno-exceptions... no
checking for gcc option to produce PIC... -fPIC -DPIC
checking if gcc PIC flag -fPIC -DPIC works... yes
checking if gcc static flag -static works... no
checking if gcc supports -c -o file.o... yes
checking if gcc supports -c -o file.o... (cached) yes
checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking whether -lc should be explicitly linked in... no
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... yes
checking for gcc... (cached) gcc
checking whether we are using the GNU C compiler... (cached) yes
checking whether gcc accepts -g... (cached) yes
checking for gcc option to accept ISO C89... (cached) none needed
checking dependency style of gcc... (cached) gcc3
checking if compiler needs -Werror to reject unknown flags... no
checking for the pthreads library -lpthreads... no
checking whether pthreads work without any flags... no
checking whether pthreads work with -Kthread... no
checking whether pthreads work with -kthread... no
checking for the pthreads library -llthread... no
checking whether pthreads work with -pthread... yes
checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE
checking if more special flags are required for pthreads... no
checking for PTHREAD_PRIO_INHERIT... yes
checking netinet/in.h usability... yes
checking netinet/in.h presence... yes
checking for netinet/in.h... yes
checking for stdint.h... (cached) yes
checking for stdlib.h... (cached) yes
checking for string.h... (cached) yes
checking for unistd.h... (cached) yes
checking for pcap_open_live in -lpcap... yes
Package json-c was not found in the pkg-config search path.
Perhaps you should add the directory containing `json-c.pc'
to the PKG_CONFIG_PATH environment variable
No package 'json-c' found
Package json-c was not found in the pkg-config search path.
Perhaps you should add the directory containing `json-c.pc'
to the PKG_CONFIG_PATH environment variable
No package 'json-c' found
checking for NT_Init in -lntapi... no
checking for json_object_new_object in -ljson-c... no
checking for pthread_setaffinity_np in -lpthread... yes
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating Makefile
config.status: creating src/lib/Makefile
config.status: creating example/Makefile
config.status: creating libndpi.pc
config.status: creating config.h
config.status: executing depfiles commands
config.status: executing libtool commands
[root@02e740e242ca nDPI]# cd ndpi-netfilter/
[root@02e740e242ca ndpi-netfilter]# MODULES_DIR=/lib/modules/`ls /lib/modules/` KERNEL_DIR=$MODULES_DIR/build/ make
make -C ipt
make[1]: Entering directory `/build/nDPI/ndpi-netfilter/ipt'
make[1]: Nothing to be done for `all'.
make[1]: Leaving directory `/build/nDPI/ndpi-netfilter/ipt'
make -C src
make[1]: Entering directory `/build/nDPI/ndpi-netfilter/src'
make -j 4 -C /lib/modules/4.13.0-1.el7.elrepo.x86_64/build/ M=$PWD modules -w;
make[2]: Entering directory `/usr/src/kernels/4.13.0-1.el7.elrepo.x86_64'
CC [M] /build/nDPI/ndpi-netfilter/src/main.o
CC [M] /build/nDPI/ndpi-netfilter/src/../lib/protocols/../third_party/src/node.o
CC [M] /build/nDPI/ndpi-netfilter/src/../lib/protocols/../third_party/src/ahocorasick.o
CC [M] /build/nDPI/ndpi-netfilter/src/../lib/ndpi_main.o
/build/nDPI/ndpi-netfilter/src/main.c: In function \u2018ndpi_net_exit\u2019:
/build/nDPI/ndpi-netfilter/src/main.c:2502:2: error: implicit declaration of function \u2018nf_ct_iterate_cleanup\u2019 [-Werror=implicit-function-declaration]
nf_ct_iterate_cleanup(net, __ndpi_free_flow, n, 0 ,0);
^
cc1: some warnings being treated as errors
make[3]: *** [/build/nDPI/ndpi-netfilter/src/main.o] Error 1
make[3]: *** Waiting for unfinished jobs....
make[2]: *** [_module_/build/nDPI/ndpi-netfilter/src] Error 2
make[2]: Leaving directory `/usr/src/kernels/4.13.0-1.el7.elrepo.x86_64'
make[1]: *** [modules] Error 2
make[1]: Leaving directory `/build/nDPI/ndpi-netfilter/src'
make: *** [all] Error 2
So I removed this section from the main.c file:
#if LINUX_VERSION_CODE < KERNEL_VERSION(3, 12, 1)
nf_ct_iterate_cleanup(net, __ndpi_free_flow, n);
#else
nf_ct_iterate_cleanup(net, __ndpi_free_flow, n, 0 ,0);
#endif
And the module works.
I am adding a DROP rule to the OUTPUT filter table and it blocks.
I don't know what this specific line might lead to so I cannot say what it might do.
Maybe a memory leak? maybe another thing?
This is the details of the kernel and the module relevant stats(dns):
# uname -srv
Linux 4.13.0-1.el7.elrepo.x86_64 #1 SMP Sun Sep 3 19:07:24 EDT 2017
# lsmod |grep ndpi
xt_ndpi 491520 0
nf_conntrack 135168 7 xt_ndpi,nf_conntrack_ipv6,nf_conntrack_ipv4,nf_nat_ipv6,xt_conntrack,nf_nat_ipv4,nf_nat
# cat /proc/net/xt_ndpi/proto|grep dns
05 5/000000ff dns # 893
08 8/000000ff mdns # 0
AT_INGRESS undeclared in 4.4.47-amd64
root@vyos:/home/vyos/vel21/nDPI/ndpi-netfilter# make
make -C ipt
make[1]: Entering directory '/home/vyos/vel21/nDPI/ndpi-netfilter/ipt'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/home/vyos/vel21/nDPI/ndpi-netfilter/ipt'
make -C src
make[1]: Entering directory '/home/vyos/vel21/nDPI/ndpi-netfilter/src'
make -j 4 -C /lib/modules/4.4.47-amd64-vyos/build M=$PWD modules -w;
make[2]: Entering directory '/usr/src/linux-headers-4.4.47-amd64-vyos'
CC [M] /home/vyos/vel21/nDPI/ndpi-netfilter/src/main.o
CC [M] /home/vyos/vel21/nDPI/ndpi-netfilter/src/../lib/protocols/../third_party/src/ahocorasick.o
CC [M] /home/vyos/vel21/nDPI/ndpi-netfilter/src/../lib/ndpi_main.o
CC [M] /home/vyos/vel21/nDPI/ndpi-netfilter/src/../lib/protocols/../third_party/src/node.o
In file included from include/linux/filter.h:16:0,
from include/net/sock.h:62,
from include/linux/tcp.h:22,
from /home/vyos/vel21/nDPI/ndpi-netfilter/src/../include/ndpi_unix.h:48,
from /home/vyos/vel21/nDPI/ndpi-netfilter/src/../include/ndpi_main.h:67,
from /home/vyos/vel21/nDPI/ndpi-netfilter/src/../include/ndpi_api.h:29,
from /home/vyos/vel21/nDPI/ndpi-netfilter/src/../lib/protocols/../third_party/src/node.c:27:
include/net/sch_generic.h: In function ‘skb_at_tc_ingress’:
In file included from include/linux/filter.h:16:0,
from include/net/sock.h:62,
from include/linux/tcp.h:22,
from /home/vyos/vel21/nDPI/ndpi-netfilter/src/../include/ndpi_unix.h:48,
from /home/vyos/vel21/nDPI/ndpi-netfilter/src/../include/ndpi_main.h:67,
from /home/vyos/vel21/nDPI/ndpi-netfilter/src/../include/ndpi_api.h:29,
from /home/vyos/vel21/nDPI/ndpi-netfilter/src/../lib/ndpi_main.c:35:
include/net/sch_generic.h: In function ‘skb_at_tc_ingress’:
include/net/sch_generic.h:414:2: error: implicit declaration of function ‘G_TC_AT’ [-Werror=implicit-function-declaration]
return G_TC_AT(skb->tc_verd) & AT_INGRESS;
^
include/net/sch_generic.h:414:33: error: ‘AT_INGRESS’ undeclared (first use in this function)
return G_TC_AT(skb->tc_verd) & AT_INGRESS;
^
include/net/sch_generic.h:414:33: note: each undeclared identifier is reported only once for each function it appears in
In file included from include/linux/filter.h:16:0,
from include/net/sock.h:62,
from include/linux/tcp.h:22,
from /home/vyos/vel21/nDPI/ndpi-netfilter/src/../include/ndpi_unix.h:48,
from /home/vyos/vel21/nDPI/ndpi-netfilter/src/../include/ndpi_main.h:67,
from /home/vyos/vel21/nDPI/ndpi-netfilter/src/../include/ndpi_api.h:29,
from /home/vyos/vel21/nDPI/ndpi-netfilter/src/../lib/protocols/../third_party/src/ahocorasick.c:28:
include/net/sch_generic.h: In function ‘skb_at_tc_ingress’:
include/net/sch_generic.h:414:2: error: implicit declaration of function ‘G_TC_AT’ [-Werror=implicit-function-declaration]
return G_TC_AT(skb->tc_verd) & AT_INGRESS;
^
include/net/sch_generic.h:414:33: error: ‘AT_INGRESS’ undeclared (first use in this function)
return G_TC_AT(skb->tc_verd) & AT_INGRESS;
^
include/net/sch_generic.h:414:33: note: each undeclared identifier is reported only once for each function it appears in
include/net/sch_generic.h:414:2: error: implicit declaration of function ‘G_TC_AT’ [-Werror=implicit-function-declaration]
return G_TC_AT(skb->tc_verd) & AT_INGRESS;
^
include/net/sch_generic.h:414:33: error: ‘AT_INGRESS’ undeclared (first use in this function)
return G_TC_AT(skb->tc_verd) & AT_INGRESS;
^
include/net/sch_generic.h:414:33: note: each undeclared identifier is reported only once for each function it appears in
In file included from include/linux/filter.h:16:0,
from include/net/sock.h:62,
from include/linux/tcp.h:22,
from include/linux/ipv6.h:73,
from /home/vyos/vel21/nDPI/ndpi-netfilter/src/main.c:30:
include/net/sch_generic.h: In function ‘skb_at_tc_ingress’:
include/net/sch_generic.h:414:2: error: implicit declaration of function ‘G_TC_AT’ [-Werror=implicit-function-declaration]
return G_TC_AT(skb->tc_verd) & AT_INGRESS;
^
include/net/sch_generic.h:414:33: error: ‘AT_INGRESS’ undeclared (first use in this function)
return G_TC_AT(skb->tc_verd) & AT_INGRESS;
^
include/net/sch_generic.h:414:33: note: each undeclared identifier is reported only once for each function it appears in
/home/vyos/vel21/nDPI/ndpi-netfilter/src/../lib/ndpi_main.c: At top level:
/home/vyos/vel21/nDPI/ndpi-netfilter/src/../lib/ndpi_main.c:54:35: fatal error: ndpi_network_list.c.inc: No such file or directory
#include "ndpi_network_list.c.inc"
^
cc1: some warnings being treated as errors
compilation terminated.
cc1: some warnings being treated as errors
scripts/Makefile.build:258: recipe for target '/home/vyos/vel21/nDPI/ndpi-netfilter/src/../lib/protocols/../third_party/src/node.o' failed
make[3]: *** [/home/vyos/vel21/nDPI/ndpi-netfilter/src/../lib/protocols/../third_party/src/node.o] Error 1
make[3]: *** Waiting for unfinished jobs....
cc1: some warnings being treated as errors
scripts/Makefile.build:258: recipe for target '/home/vyos/vel21/nDPI/ndpi-netfilter/src/../lib/protocols/../third_party/src/ahocorasick.o' failed
make[3]: *** [/home/vyos/vel21/nDPI/ndpi-netfilter/src/../lib/protocols/../third_party/src/ahocorasick.o] Error 1
cc1: some warnings being treated as errors
scripts/Makefile.build:258: recipe for target '/home/vyos/vel21/nDPI/ndpi-netfilter/src/main.o' failed
make[3]: *** [/home/vyos/vel21/nDPI/ndpi-netfilter/src/main.o] Error 1
scripts/Makefile.build:258: recipe for target '/home/vyos/vel21/nDPI/ndpi-netfilter/src/../lib/ndpi_main.o' failed
make[3]: *** [/home/vyos/vel21/nDPI/ndpi-netfilter/src/../lib/ndpi_main.o] Error 1
Makefile:1402: recipe for target 'module/home/vyos/vel21/nDPI/ndpi-netfilter/src' failed
make[2]: *** [module/home/vyos/vel21/nDPI/ndpi-netfilter/src] Error 2
make[2]: Leaving directory '/usr/src/linux-headers-4.4.47-amd64-vyos'
Makefile:165: recipe for target 'modules' failed
make[1]: *** [modules] Error 2
make[1]: Leaving directory '/home/vyos/vel21/nDPI/ndpi-netfilter/src'
Makefile:2: recipe for target 'all' failed
make: *** [all] Error 2
Could you help to check where cause the issue how to fix it ?
ndpi-netfilter2.2 branch missing folder ndpi-netfilter
Measuring Latency
HI I have a Query. Does anyone can tell how can we measure the Latency involved if we use ndpi-netfilter? Any tools or method?
Linux 4.13: call to ‘__read_overflow2’ declared with attribute error: detected read beyond size of object passed as 2nd parameter
AFAIK this occurs when kernel builded with enabled FORTIFY_SOURCE.
[k0ste@WorkStation tmp]$ git clone [email protected]:vel21ripn/nDPI.git ipt_ndpi
Cloning into 'ipt_ndpi'...
remote: Counting objects: 8231, done.
remote: Total 8231 (delta 0), reused 0 (delta 0), pack-reused 8231
Receiving objects: 100% (8231/8231), 23.55 MiB | 480.00 KiB/s, done.
Resolving deltas: 100% (5484/5484), done.
[k0ste@WorkStation tmp]$ cd ipt_ndpi/
[k0ste@WorkStation ipt_ndpi]$ ./autogen.sh
autoreconf: Entering directory `.'
autoreconf: configure.ac: not using Gettext
autoreconf: running: aclocal --force -I m4
autoreconf: configure.ac: tracing
autoreconf: running: libtoolize --copy --force
libtoolize: putting auxiliary files in '.'.
libtoolize: copying file './ltmain.sh'
libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'.
libtoolize: copying file 'm4/libtool.m4'
libtoolize: copying file 'm4/ltoptions.m4'
libtoolize: copying file 'm4/ltsugar.m4'
libtoolize: copying file 'm4/ltversion.m4'
libtoolize: copying file 'm4/lt~obsolete.m4'
autoreconf: running: /usr/bin/autoconf --force
autoreconf: running: /usr/bin/autoheader --force
autoreconf: running: automake --add-missing --copy --force-missing
configure.ac:7: installing './compile'
configure.ac:7: installing './config.guess'
configure.ac:7: installing './config.sub'
configure.ac:5: installing './install-sh'
configure.ac:5: installing './missing'
example/Makefile.am: installing './depcomp'
autoreconf: Leaving directory `.'
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /usr/bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
checking how to print strings... printf
checking for style of include used by make... GNU
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking whether gcc understands -c and -o together... yes
checking dependency style of gcc... gcc3
checking for a sed that does not truncate output... /usr/bin/sed
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for fgrep... /usr/bin/grep -F
checking for ld used by gcc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 1572864
checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop
checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop
checking for /usr/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for ar... ar
checking for archiver @FILE support... @
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from gcc object... ok
checking for sysroot... no
checking for a working dd... /usr/bin/dd
checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1
checking for mt... no
checking if : is a manifest tool... no
checking how to run the C preprocessor... gcc -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for dlfcn.h... yes
checking for objdir... .libs
checking if gcc supports -fno-rtti -fno-exceptions... no
checking for gcc option to produce PIC... -fPIC -DPIC
checking if gcc PIC flag -fPIC -DPIC works... yes
checking if gcc static flag -static works... yes
checking if gcc supports -c -o file.o... yes
checking if gcc supports -c -o file.o... (cached) yes
checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking whether -lc should be explicitly linked in... no
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... yes
checking for gcc... (cached) gcc
checking whether we are using the GNU C compiler... (cached) yes
checking whether gcc accepts -g... (cached) yes
checking for gcc option to accept ISO C89... (cached) none needed
checking whether gcc understands -c and -o together... (cached) yes
checking dependency style of gcc... (cached) gcc3
checking if compiler needs -Werror to reject unknown flags... no
checking for the pthreads library -lpthreads... no
checking whether pthreads work without any flags... no
checking whether pthreads work with -Kthread... no
checking whether pthreads work with -kthread... no
checking for the pthreads library -llthread... no
checking whether pthreads work with -pthread... yes
checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE
checking if more special flags are required for pthreads... no
checking for PTHREAD_PRIO_INHERIT... yes
checking netinet/in.h usability... yes
checking netinet/in.h presence... yes
checking for netinet/in.h... yes
checking for stdint.h... (cached) yes
checking for stdlib.h... (cached) yes
checking for string.h... (cached) yes
checking for unistd.h... (cached) yes
checking for pcap_open_live in -lpcap... yes
Package json-c was not found in the pkg-config search path.
Perhaps you should add the directory containing `json-c.pc'
to the PKG_CONFIG_PATH environment variable
No package 'json-c' found
Package json-c was not found in the pkg-config search path.
Perhaps you should add the directory containing `json-c.pc'
to the PKG_CONFIG_PATH environment variable
No package 'json-c' found
checking for NT_Init in -lntapi... no
checking for json_object_new_object in -ljson-c... no
checking for pthread_setaffinity_np in -lpthread... yes
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating Makefile
config.status: creating src/lib/Makefile
config.status: creating example/Makefile
config.status: creating libndpi.pc
config.status: creating config.h
config.status: executing depfiles commands
config.status: executing libtool commands
[k0ste@WorkStation ipt_ndpi]$ cd ndpi-netfilter/
[k0ste@WorkStation ndpi-netfilter]$ makemake -C ipt
make[1]: Entering directory '/tmp/ipt_ndpi/ndpi-netfilter/ipt'
cc -fPIC -I../include -I../lib -I../src -DOPENDPI_NETFILTER_MODULE -O2 -Wall -D_INIT=libxt_ndpi_init -c -o libxt_ndpi.o libxt_ndpi.c;
cc -shared -o libxt_ndpi.so libxt_ndpi.o;
rm libxt_ndpi.o
make[1]: Leaving directory '/tmp/ipt_ndpi/ndpi-netfilter/ipt'
make -C src
make[1]: Entering directory '/tmp/ipt_ndpi/ndpi-netfilter/src'
make -j 4 -C /lib/modules/4.13.9-1-ARCH/build M=$PWD modules -w;
make[2]: Entering directory '/usr/lib/modules/4.13.9-1-ARCH/build'
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/main.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/../third_party/src/node.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/../third_party/src/ahocorasick.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/ndpi_main.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/afp.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/aimini.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/applejuice.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/armagetron.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/ayiya.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/battlefield.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/bgp.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/bittorrent.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/citrix.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/ciscovpn.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/collectd.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/corba.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/crossfire.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/csgo.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/dcerpc.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/dhcp.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/dhcpv6.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/directconnect.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/directdownloadlink.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/dns.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/dofus.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/dropbox.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/eaq.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/edonkey.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/fasttrack.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/fiesta.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/filetopia.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/florensia.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/ftp_control.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/ftp_data.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/gnutella.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/gtp.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/guildwars.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/h323.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/halflife2_and_mods.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/http.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/http_activesync.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/iax.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/icecast.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/imesh.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/ipp.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/irc.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/jabber.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/kakaotalk_voice.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/kerberos.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/kontiki.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/ldap.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/lotus_notes.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/mail_imap.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/mail_pop.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/mail_smtp.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/maplestory.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/mdns.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/meebo.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/mgcp.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/mms.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/mpegts.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/msn.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/mssql.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/mysql.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/netbios.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/netflow.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/nfs.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/noe.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/non_tcp_udp.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/ntp.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/openft.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/openvpn.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/oracle.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/oscar.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/pando.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/megaco.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/pcanywhere.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/postgres.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/pplive.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/ppstream.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/pptp.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/qq.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/quake.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/quic.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/radius.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/rdp.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/redis_net.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/rsync.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/rtp.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/rtcp.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/rtmp.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/rtsp.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/sflow.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/shoutcast.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/sip.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/skinny.o
CC [M] /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/skype.o
In file included from ./include/linux/bitmap.h:8:0,
from ./include/linux/cpumask.h:11,
from ./include/linux/mm_types_task.h:13,
from ./include/linux/mm_types.h:4,
from ./include/linux/kmemcheck.h:4,
from ./include/linux/skbuff.h:18,
from ./include/linux/ip.h:20,
from /tmp/ipt_ndpi/ndpi-netfilter/src/../include/ndpi_unix.h:47,
from /tmp/ipt_ndpi/ndpi-netfilter/src/../include/ndpi_main.h:66,
from /tmp/ipt_ndpi/ndpi-netfilter/src/../include/ndpi_api.h:29,
from /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/sip.c:26:
In function ‘memcmp’,
inlined from ‘ndpi_search_sip_handshake’ at /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/sip.c:124:10,
inlined from ‘ndpi_search_sip.part.0’ at /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/sip.c:189:7,
inlined from ‘ndpi_search_sip’:
./include/linux/string.h:346:4: error: call to ‘__read_overflow2’ declared with attribute error: detected read beyond size of object passed as 2nd parameter
__read_overflow2();
^~~~~~~~~~~~~~~~~~
make[3]: *** [scripts/Makefile.build:302: /tmp/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/sip.o] Error 1
make[3]: *** Waiting for unfinished jobs....
make[2]: *** [Makefile:1512: _module_/tmp/ipt_ndpi/ndpi-netfilter/src] Error 2
make[2]: Leaving directory '/usr/lib/modules/4.13.9-1-ARCH/build'
make[1]: *** [Makefile:165: modules] Error 2
make[1]: Leaving directory '/tmp/ipt_ndpi/ndpi-netfilter/src'
make: *** [Makefile:3: all] Error 2Kernel 4.15
Is there some one ready to make patch for new kernel 4.15 ?
xt_ndpi: kernel module version missmatch.
After installing xt_ndpi module via DKMS I can't create rules for iptables and get error: iptables v1.6.0: xt_ndpi: kernel module version missmatch.
OS: Ubuntu 16.04 Xenial 64bit
Kernel: Linux server 4.4.0-64-generic #85-Ubuntu SMP Mon Feb 20 11:50:30 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
[ 17.772352] xt_ndpi: Unknown symbol __aeabi_uldivmod (err 0)
produced in: mvebu arm 32bit
Seems it is introduced in commit:
b0cf474
Fixed wrong tick_timestamp calculation.
can not match Youtube traffic
Hi All
I tried to control YouTube traffic...
But I added the following commands:
/sbin/iptables -w -t mangle -A PREROUTING -s 192.168.1.0/24 -m ndpi --YouTube -j MARK --set-mark 0x3ea
/sbin/iptables -w -t mangle -A PREROUTING -s 192.168.1.0/24 -m ndpi --YouTubeUpload -j MARK --set-mark 0x3ea
No traffic can be matched. Is there any possibility to cause the problem?
My kernel is 3.18.0 and I patched the kernel according to this project's document.
I also tried the project: https://github.com/betolj/ndpi-netfilter with same command(Only -m ndpi --YouTube), Youtube traffic can be matched via IE/Forefox/Edge,but Chrome can not work.
Can't apply kernel patch on Linux 4.8
patching file include/net/netfilter/nf_conntrack_extend.h
Hunk #1 succeeded at 27 (offset -3 lines).
Hunk #2 FAILED at 101.
Hunk #3 succeeded at 112 (offset -7 lines).
1 out of 3 hunks FAILED -- saving rejects to file include/net/netfilter/nf_conntrack_extend.h.rej
patching file net/netfilter/Kconfig
Hunk #1 succeeded at 81 (offset 8 lines).
patching file net/netfilter/nf_conntrack_acct.c
Hunk #1 succeeded at 56 (offset 2 lines).
patching file net/netfilter/nf_conntrack_extend.c
Hunk #2 succeeded at 169 (offset -11 lines).
Hunk #3 succeeded at 210 (offset -11 lines).
patching file net/netfilter/nf_conntrack_standalone.c
Hunk #1 succeeded at 242 (offset 37 lines).
Hunk #2 succeeded at 253 with fuzz 2 (offset 39 lines).
xt_ndpi: Unknown symbol nf_ct_destroy (err 0) [solved]
Hi,
I build the netfilter modules for my firewall distribution. Basically it is compiling the kernel modules. I am using Kernel 4.9.13 at the moment without the ndpi kernel patches. When I try to load the module, I get a
"xt_ndpi: Unknown symbol nf_ct_destroy (err 0)"
error in dmesg.
Building and loading with these sources is working: https://github.com/betolj/ndpi-netfilter
But I had a kernel crahs, did not investigate any further on this.
Any idea why the nf_ct_destroy error occures, as far as I can see, it is part of the netfilter core.
BR
Ben
ndpi-netfilter (netfilter-2.6 branch) doesn't recognize YouTube, GMail, sometimes WhatsApp
Hi,
I cannot get ndpi-netfilter (netfilter-2.6 branch) to recognize GMail, YouTube and WhatsApp.
I guess there are other protocols that are not recognized but at least I'm sure of those ones.
It's likely a libndpi related problem because example/ndpiReader cannot detect them neither.
This is true at least since 0a0ff0d.
Clément
Ubuntu Xenail 16.04 docker container build
For debian 8 compilation I composed the next Docker:
https://github.com/elico/debian8-dev-ndpi-vel
I will try to compose also one for xenial(it should be around here on my server already.. so couple days and it's here)
Openwrt Compilation
Hi @vel21ripn
I have been using ndpi-netfilter from betolj's repository. I would like to try yours to see if i can get more stability and more active development.
Are you able to assist in modifying this Makefile to suit your repository? I have made some of my own attempts (here) but am currently unsuccessful.
ndpi-netfilter 2.0-stable
The netfilter2 branch is available for testing.
Do I need to patch the kernel of openSUSE leap?
I installed in openSUSE leap the required packages+patterns:
- Base Development (devel_basis)
- C/C++ Development (devel_C_C++)
- Linux Kernel Development (devel_kernel)
- ibxtables-devel
- libnetfilter_conntrack-devel
- xtables-addons
- libjson-c-devel
Then I ran:
# cat /boot/config-$(uname -r)|egrep "NETFILTER_XT_MATCH_CONNLABEL|NF_CONNTRACK_LABELS|NF_CONNTRACK="
CONFIG_NF_CONNTRACK=m
CONFIG_NF_CONNTRACK_LABELS=y
CONFIG_NETFILTER_XT_MATCH_CONNLABEL=m
# cd /tmp/ && git clone -b netfilter https://github.com/vel21ripn/nDPI.git
# cd nDPI
# ./autogen.sh
# ./configure
# cd ndpi-netfilter/ && PKG_CONFIG=/usr/bin/pkg-config make && make install && make modules_install && echo $? && modprobe xt_ndpi && lsmod|grep ndpi
> ...
> make[1]: Leaving directory '/opt/src/nDPI/ndpi-netfilter/src'
0
xt_ndpi 524288 5
nf_conntrack 114688 1 xt_ndpi
x_tables 36864 5 xt_mark,xt_ndpi,ip_tables,iptable_filter,iptable_mangle
Now I have tried to add two rules as a "counter" to verify it works(I am not sure if it should) using the commands:
# iptables -O INPUT -m ndpi --proto http -j ACCEPT
# iptables -I OUTPUT -m ndpi --proto http -j ACCEPT
# curl http://www.google.com/
...
# iptables -L -nv
Chain INPUT (policy ACCEPT 2581 packets, 348K bytes)
pkts bytes target prot opt in out source destination
0 0 ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 ndpi protocol http
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
Chain OUTPUT (policy ACCEPT 2518 packets, 391K bytes)
pkts bytes target prot opt in out source destination
0 0 ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 ndpi protocol http
#
Which means that the module probably not working as I expect it.
From the docs\article at: http://a7lanov.blogspot.co.il/2015/01/centos-ndpi-imq.html
I understood that I might need to patch the kernel source but it's in Russian and google translate doesn't give me enough details to verify it.
Should I do something specific? Am I doing something wrong?
Kernel 4.13.0 nf_ct_iterate_cleanup function
Hi!
It seems that in new kernel version the function nf_ct_iterate_cleanup was renamed and the module no longer copiles against it.
Other issue that I'd noticed, with module loaded against v4.12 on an armhf and arm64 platform I'm getting kernel hangs, no matter what module or traffic is going on, the result is the same... Anyone experiencing this?
Thanks!
update nDpi
is there is a way to get the latest ndpi signatures , protocols ,... and update the one in this repository ?
i tried to replace the files but with no success .
any idea ?
regards.
After some changes ndpi stuck my system.
When I was debugging for #41 I'll found my kernels stuck by unknown reason.
Debugging deeper I was found that is not involved by kernel, just ndpi updates. I checked this with last commit of netfilter-2.6 branch (1.2_2.6.0.1423.dfc2fc1), then I revert ndpi to stable version 1.2_2.3.0.1262.97099ff and bug is not reproducible. I use this version with kernel 4.17.13 from "Tue 07 Aug 2018 02:16:35 PM UTC" and this pretty stable for me.
I think only way out is test all commits from 97099ff and find when it's start.
ndpi-netfilter and netfilter2 freezing
Hi!
Since kernel 4.12 we've noticed some nf_conntrack functions changing names and, don't know if this is an coincidence, but from that version to this very moment, every time I load xt_ndpi module with iptables rules, I try to use any kind of SSL traffic, the linux box hangs and restarts.
Still couldn't get any syslog or dmesg output... The machine freezes and I can't do anything till it reboots itself.
This happens in x86_64, armhf and arm64 as well... Kernels I've tested 4.12.0, 4.13.0, 4.13.12, 4.14.0. Version 4.11 downwards aren't affected this way.
Any thoughts?
Isn't working on ubuntu 16.04 aws version
ubuntu/images/hvm-ssd/ubuntu-xenial-16.04-amd64-server-20170721 (ami-09d2fb69)
I have tried as root
cd /root
apt-get update
apt-get install libtool autoconf pkg-config iptables-dev libpcap-dev autogen
git clone https://github.com/vel21ripn/nDPI.git
cd nDPI
./autogen.sh
cd ndpi-netfilter
make
make modules_install
make install
modprobe xt_ndpi
I have tried various iptables rule sets and none of them seem to work.
I have tried both with and without mangle
iptables -t mangle -A PREROUTING -m ndpi --all
iptables -A INPUT -m ndpi --http -j DROP
to test I just did
apt-get install nginx
service nginx start
Then I just went to its ip and the site still loaded.
iptables -vL
Chain INPUT (policy ACCEPT 57 packets, 4116 bytes)
pkts bytes target prot opt in out source destination
0 0 DROP all -- any any anywhere anywhere ndpi protocol http
iptables -vL -t mangle
Chain PREROUTING (policy ACCEPT 280 packets, 19404 bytes)
pkts bytes target prot opt in out source destination
0 0 all -- any any anywhere anywhere ndpi all protocols
Linux 4.12 patch
[k0ste@WorkStation linux-4.12]$ patch -p1 -i /tmp/linux-4.8-ndpi.diff
patching file include/net/netfilter/nf_conntrack_extend.h
Hunk #2 FAILED at 113.
1 out of 2 hunks FAILED -- saving rejects to file include/net/netfilter/nf_conntrack_extend.h.rej
patching file net/netfilter/Kconfig
Hunk #1 succeeded at 85 (offset 4 lines).
patching file net/netfilter/nf_conntrack_extend.c
Hunk #1 succeeded at 108 with fuzz 2 (offset -37 lines).
Hunk #2 FAILED at 190.
1 out of 2 hunks FAILED -- saving rejects to file net/netfilter/nf_conntrack_extend.c.rej
All traffic marked as "error"
Hi,
I compiled ndpi-netfilter against kernel 4.4.X given instructions in INSTALL file.
When testing with the following script (checking out http, all, unknown and error), iptables counters show that all traffic is classified as "error" by ndpi-netfilter!
script:
iptables -t mangle -A PREROUTING -m ndpi --http -j LOG
iptables -t mangle -A PREROUTING -m ndpi --all -j LOG
iptables -t mangle -A PREROUTING -m ndpi --unknown -j LOG
iptables -t mangle -A PREROUTING -m ndpi --error -j LOG
I tried loading module with options log_debug=100 and log_trace=100, but it only gives me lots of initialization messages, nothing showing what the errors could be.
What could I be doing wrong? Is there a way to debug?
I tested with and without the kernel patch (with/without CONNLABELS), and I get the same error.
[Help] Documentation or resources needed for using xt_ndpi.ko
I have cross compiled ndpi-netfilter for ARM and able to load the xt_ndpi module successfully.
I am able to see iptables -m ndpi --help giving all the options related to ndpi and the supported protocols.
Would you please provide any document or resources on how to monitor the traffic and classify the information?
kernel 4.15.x
Hello is there someone that use new stable line 4.15.3? how to make it work with ndpi-netfilter ?
ndpi-netfilter 2.2 ( 2.3-dev )
The Netfilter-2.2 branch is available for testing.
Known issues: Bittorrent detects significantly less traffic than netfilter-1.7
Does nDPI modules work with ip6tables?
I see that lot's of dissector are capable of recognizing ipv6 traffic but can ipv6 traffic be mangeled with iptables or do we need ip6tables and some sort of xt_nDPI_v6? I am asking because I am not able to block icmpv6 with nDPI ip_icmpv6 and iptables...
example.zip
and also thank you for your work:)
Building modules, stage 2. MODPOST 1 modules WARNING: "__aeabi_uldivmod" [/home/xw/workspace/github/openwrt-master/build_dir/target-arm_cortex-a9+vfpv3_glibc_eabi/linux-mvebu_cortexa9/ndpi-netfilter-a360566-2.6/ndpi-netfilter/src/xt_ndpi.ko] undefined!
32-bit CPU.
Sorry, I can't figure out which file cause this issue.
Usages of --match-proto --match-master and --have-master
Dear Vitaly,
I've seen some new options (--match-proto --match-master and --have-master) in your libxt_ndpi code and I would like to know how to take benefit from them.
Could you give us an example of an usage of these options with an application relying on SSL or HTTP as master protocol ? In which situation are those option useful ?
Thank you,
Clément
Linux 4.19.2: nf_ct_l3proto_try_module_get undefined
[root@linux01 tmp]# git clone https://github.com/vel21ripn/nDPI.git
Cloning into 'nDPI'...
remote: Enumerating objects: 1, done.
remote: Counting objects: 100% (1/1), done.
remote: Total 10515 (delta 0), reused 0 (delta 0), pack-reused 10514
Receiving objects: 100% (10515/10515), 25.26 MiB | 4.76 MiB/s, done.
Resolving deltas: 100% (7113/7113), done.
[root@linux01 tmp]# cd nDPI/
[root@linux01 nDPI]# git checkout netfilter-2.2
Already on 'netfilter-2.2'
Your branch is up to date with 'origin/netfilter-2.2'.
[root@linux01 nDPI]# ./autogen.sh
autoreconf: Entering directory `.'
autoreconf: configure.ac: not using Gettext
autoreconf: running: aclocal --force -I m4
autoreconf: configure.ac: tracing
autoreconf: running: libtoolize --copy --force
libtoolize: putting auxiliary files in '.'.
libtoolize: copying file './ltmain.sh'
libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'.
libtoolize: copying file 'm4/libtool.m4'
libtoolize: copying file 'm4/ltoptions.m4'
libtoolize: copying file 'm4/ltsugar.m4'
libtoolize: copying file 'm4/ltversion.m4'
libtoolize: copying file 'm4/lt~obsolete.m4'
autoreconf: running: /usr/bin/autoconf --force
autoreconf: running: /usr/bin/autoheader --force
autoreconf: running: automake --add-missing --copy --force-missing
configure.ac:7: installing './compile'
configure.ac:7: installing './config.guess'
configure.ac:7: installing './config.sub'
configure.ac:5: installing './install-sh'
configure.ac:5: installing './missing'
example/Makefile.am: installing './depcomp'
parallel-tests: installing './test-driver'
autoreconf: Leaving directory `.'
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /usr/bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
checking how to print strings... printf
checking whether make supports the include directive... yes (GNU style)
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking whether gcc understands -c and -o together... yes
checking dependency style of gcc... gcc3
checking for a sed that does not truncate output... /usr/bin/sed
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for fgrep... /usr/bin/grep -F
checking for ld used by gcc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 1572864
checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop
checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop
checking for /usr/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for ar... ar
checking for archiver @FILE support... @
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from gcc object... ok
checking for sysroot... no
checking for a working dd... /usr/bin/dd
checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1
checking for mt... no
checking if : is a manifest tool... no
checking how to run the C preprocessor... gcc -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for dlfcn.h... yes
checking for objdir... .libs
checking if gcc supports -fno-rtti -fno-exceptions... no
checking for gcc option to produce PIC... -fPIC -DPIC
checking if gcc PIC flag -fPIC -DPIC works... yes
checking if gcc static flag -static works... yes
checking if gcc supports -c -o file.o... yes
checking if gcc supports -c -o file.o... (cached) yes
checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking whether -lc should be explicitly linked in... no
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... yes
checking for gcc... (cached) gcc
checking whether we are using the GNU C compiler... (cached) yes
checking whether gcc accepts -g... (cached) yes
checking for gcc option to accept ISO C89... (cached) none needed
checking whether gcc understands -c and -o together... (cached) yes
checking dependency style of gcc... (cached) gcc3
checking whether gcc is Clang... no
checking whether pthreads work with -pthread... yes
checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE
checking whether more special flags are required for pthreads... no
checking for PTHREAD_PRIO_INHERIT... yes
checking netinet/in.h usability... yes
checking netinet/in.h presence... yes
checking for netinet/in.h... yes
checking for stdint.h... (cached) yes
checking for stdlib.h... (cached) yes
checking for string.h... (cached) yes
checking for unistd.h... (cached) yes
checking for numa_available in -lnuma... yes
which: no clang in (/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl)
checking for pcap_open_live in -lpcap... yes
checking for json_object_new_object in -ljson-c... yes
checking for pthread_setaffinity_np in -lpthread... yes
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating Makefile
config.status: creating example/Makefile
config.status: creating tests/Makefile
config.status: creating libndpi.pc
config.status: creating src/include/ndpi_define.h
config.status: creating src/lib/Makefile
config.status: creating src/include/ndpi_config.h
config.status: executing depfiles commands
config.status: executing libtool commands
[root@linux01 nDPI]# cd src/lib/
[root@linux01 lib]# make ndpi_network_list.c.inc
depbase=`echo ndpi_network_list_compile.o | sed 's|[^/]*$|.deps/&|;s|\.o$||'`;\
gcc -DHAVE_CONFIG_H -I. -I../../src/include -I../../src/include/ -I../../src/lib/third_party/include/ -g -O2 -I/usr/include/json-c -fPIC -DPIC -DNDPI_LIB_COMPILATION -MT ndpi_network_list_compile.o -MD -MP -MF $depbase.Tpo -c -o ndpi_network_list_compile.o ndpi_network_list_compile.c &&\
mv -f $depbase.Tpo $depbase.Po
ndpi_network_list_compile.c: In function ‘main’:
ndpi_network_list_compile.c:324:5: warning: ‘strncat’ specified bound 512 equals destination size [-Wstringop-overflow=]
strncat(pnl->comments,wordarg,sizeof(pnl->comments));
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ndpi_network_list_compile.c:385:4: warning: ‘strncat’ specified bound 512 equals destination size [-Wstringop-overflow=]
strncat(pnl->comments,word,sizeof(pnl->comments));
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/bin/sh ../../libtool --tag=CC --mode=link gcc -g -O2 -I/usr/include/json-c -fPIC -DPIC -DNDPI_LIB_COMPILATION -ljson-c -o ndpi_network_list_compile ndpi_network_list_compile.o
libtool: link: gcc -g -O2 -I/usr/include/json-c -fPIC -DPIC -DNDPI_LIB_COMPILATION -o ndpi_network_list_compile ndpi_network_list_compile.o -ljson-c
./ndpi_network_list_compile -o ndpi_network_list.c.inc ndpi_network_list_tor.yaml ndpi_network_list_std.yaml
Warning: line 78: '52.82.0.0/14' is not network
Warning: line 423: '141.101.88.0/19' is not network
Warning: line 609: '209.148.214.135/21' is not network
Warning: line 717: '208.64.201.0/22' is not network
Warning: line 723: '159.122.189.48/21' is not network
Warning: line 795: '1.44.48.64/23' is not network
Warning: line 796: '1.44.69.80/25' is not network
Warning: line 797: '2.46.138.52/28' is not network
Warning: line 798: '2.47.103.32/23' is not network
Warning: line 852: '169.55.60.170/27' is not network
Warning: line 854: '169.63.76.17/25' is not network
Warning: line 855: '169.63.73.64/25' is not network
Warning: line 860: '185.60.218.53/27' is not network
[root@linux01 lib]# cd ../../ndpi-netfilter/
[root@linux01 ndpi-netfilter]# make
make -C ipt
make[1]: Entering directory '/tmp/nDPI/ndpi-netfilter/ipt'
cc -fPIC -I../.. -I../../src/include -I../../src/lib -I../src -I../libre -DOPENDPI_NETFILTER_MODULE -O2 -Wall -D_INIT=libxt_ndpi_init -c -o libxt_ndpi.o libxt_ndpi.c;
cc -shared -o libxt_ndpi.so libxt_ndpi.o;
rm libxt_ndpi.o
make[1]: Leaving directory '/tmp/nDPI/ndpi-netfilter/ipt'
make -C src
make[1]: Entering directory '/tmp/nDPI/ndpi-netfilter/src'
make -C /lib/modules/4.19.2-arch1-1-nfcustom/build M=$PWD modules -w;
make[2]: Entering directory '/usr/lib/modules/4.19.2-arch1-1-nfcustom/build'
CC [M] /tmp/nDPI/ndpi-netfilter/src/main.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/third_party/src/node.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/third_party/src/ahocorasick.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/third_party/src/libcache.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/third_party/src/hash.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/third_party/src/ndpi_patricia.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/ndpi_main.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/afp.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/ajp.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/aimini.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/amqp.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/applejuice.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/apple_push.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/armagetron.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/ayiya.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/battlefield.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/bgp.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/bittorrent.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/bjnp.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/checkmk.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/citrix.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/ciscovpn.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/coap.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/collectd.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/corba.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/crossfire.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/csgo.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/dcerpc.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/diameter.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/dhcp.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/dhcpv6.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/directconnect.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/directdownloadlink.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/dns.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/dofus.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/drda.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/dropbox.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/eaq.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/edonkey.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/fasttrack.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/fbzero.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/fiesta.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/fix.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/florensia.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/ftp_control.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/ftp_data.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/git.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/gnutella.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/gtp.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/guildwars.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/h323.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/halflife2_and_mods.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/hangout.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/hep.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/http.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/http_activesync.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/iax.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/icecast.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/ipp.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/irc.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/jabber.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/kakaotalk_voice.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/kerberos.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/kontiki.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/ldap.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/lisp.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/lotus_notes.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/mail_imap.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/mail_pop.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/mail_smtp.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/maplestory.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/memcached.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/mdns.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/mgcp.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/mpegts.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/mqtt.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/msn.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/mssql_tds.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/mysql.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/netbios.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/netflow.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/nfs.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/nintendo.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/noe.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/non_tcp_udp.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/ntp.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/openft.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/openvpn.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/oracle.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/oscar.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/pando.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/megaco.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/pcanywhere.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/postgres.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/pplive.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/ppstream.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/pptp.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/qq.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/quic.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/radius.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/rdp.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/redis_net.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/rsync.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/rtp.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/rtcp.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/rtmp.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/rtsp.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/rx.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/sflow.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/shoutcast.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/sip.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/skinny.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/skype.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/smb.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/smpp.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/snmp.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/socks45.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/someip.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/sopcast.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/soulseek.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/spotify.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/ssdp.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/ssh.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/ssl.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/starcraft.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/stealthnet.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/steam.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/stun.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/syslog.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/teamviewer.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/teamspeak.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/telnet.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/telegram.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/teredo.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/tftp.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/thunder.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/tinc.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/tor.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/tcp_udp.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/tvants.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/tvuplayer.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/ubntac2.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/usenet.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/vhua.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/viber.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/vmware.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/vnc.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/whatsapp.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/warcraft3.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/whoisdas.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/world_of_kung_fu.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/world_of_warcraft.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/xbox.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/xdmcp.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/yahoo.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/zattoo.o
CC [M] /tmp/nDPI/ndpi-netfilter/src/../../src/lib/protocols/zeromq.o
LD [M] /tmp/nDPI/ndpi-netfilter/src/xt_ndpi.o
Building modules, stage 2.
MODPOST 1 modules
WARNING: "nf_ct_l3proto_try_module_get" [/tmp/nDPI/ndpi-netfilter/src/xt_ndpi.ko] undefined!
WARNING: "nf_ct_l3proto_module_put" [/tmp/nDPI/ndpi-netfilter/src/xt_ndpi.ko] undefined!
CC /tmp/nDPI/ndpi-netfilter/src/xt_ndpi.mod.o
LD [M] /tmp/nDPI/ndpi-netfilter/src/xt_ndpi.ko
make[2]: Leaving directory '/usr/lib/modules/4.19.2-arch1-1-nfcustom/build'
make[1]: Leaving directory '/tmp/nDPI/ndpi-netfilter/src'see at:
WARNING: "nf_ct_l3proto_try_module_get" [/tmp/nDPI/ndpi-netfilter/src/xt_ndpi.ko] undefined!
WARNING: "nf_ct_l3proto_module_put" [/tmp/nDPI/ndpi-netfilter/src/xt_ndpi.ko] undefined!
Module loading:
[root@linux01 ndpi-netfilter]# insmod src/xt_ndpi.ko
insmod: ERROR: could not insert module src/xt_ndpi.ko: Unknown symbol in module
[root@linux01 ndpi-netfilter]# dmesg -TL | grep -i ndpi
[Sun Nov 18 07:37:13 2018] xt_ndpi: loading out-of-tree module taints kernel.
[Sun Nov 18 07:37:13 2018] xt_ndpi: module verification failed: signature and/or required key missing - tainting kernel
[Sun Nov 18 07:37:13 2018] xt_ndpi: Unknown symbol nf_ct_ext_add (err -2)
[Sun Nov 18 07:37:13 2018] xt_ndpi: Unknown symbol nf_ct_netns_get (err -2)
[Sun Nov 18 07:37:13 2018] xt_ndpi: Unknown symbol nf_ct_extend_unregister (err -2)
[Sun Nov 18 07:37:13 2018] xt_ndpi: Unknown symbol nf_ct_l3proto_module_put (err -2)
[Sun Nov 18 07:37:13 2018] xt_ndpi: Unknown symbol nf_ct_iterate_cleanup_net (err -2)
[Sun Nov 18 07:37:13 2018] xt_ndpi: Unknown symbol nf_ct_extend_custom_register (err -2)
[Sun Nov 18 07:37:13 2018] xt_ndpi: Unknown symbol nf_ct_l3proto_try_module_get (err -2)
[Sun Nov 18 07:37:13 2018] xt_ndpi: Unknown symbol nf_ct_netns_put (err -2)
[root@linux01 ndpi-netfilter]# make problem on debian jessie 4.12.14
When I try to make. It just stops compilating.
make -C ipt
make[1]: Entering directory '/usr/src/nDPI-netfilter/ndpi-netfilter/ipt'
cc -fPIC -I../include -I../lib -I../src -DOPENDPI_NETFILTER_MODULE -O2 -Wall -c -o libxt_ndpi.o libxt_ndpi.c
cc -shared -o libxt_ndpi.so libxt_ndpi.o;
rm libxt_ndpi.o
make[1]: Leaving directory '/usr/src/nDPI-netfilter/ndpi-netfilter/ipt'
make -C src
make[1]: Entering directory '/usr/src/nDPI-netfilter/ndpi-netfilter/src'
make -j 4 -C /lib/modules/4.12.14-041214-generic/build M=$PWD modules -w;
make[2]: Entering directory '/usr/src/linux-headers-4.12.14-041214-generic'
CC [M] /usr/src/nDPI-netfilter/ndpi-netfilter/src/main.o
CC [M] /usr/src/nDPI-netfilter/ndpi-netfilter/src/../lib/protocols/../third_party/src/node.o
CC [M] /usr/src/nDPI-netfilter/ndpi-netfilter/src/../lib/protocols/../third_party/src/ahocorasick.o
CC [M] /usr/src/nDPI-netfilter/ndpi-netfilter/src/../lib/ndpi_main.o
/usr/src/nDPI-netfilter/ndpi-netfilter/src/../lib/ndpi_main.c:54:35: fatal error: ndpi_network_list.c.inc: No such file or directory
#include "ndpi_network_list.c.inc"
^
compilation terminated.
CC [M] /usr/src/nDPI-netfilter/ndpi-netfilter/src/../lib/protocols/aimini.o
CC [M] /usr/src/nDPI-netfilter/ndpi-netfilter/src/../lib/protocols/afp.o
/usr/src/nDPI-netfilter/ndpi-netfilter/src/main.c: In function ‘debug_printf’:
/usr/src/nDPI-netfilter/ndpi-netfilter/src/main.c:360:1: warning: the frame size of 1056 bytes is larger than 1024 bytes [-Wframe-larger-than=]
}
^
/usr/src/nDPI-netfilter/ndpi-netfilter/src/main.c: In function ‘n_ipdef_proc_read’:
/usr/src/nDPI-netfilter/ndpi-netfilter/src/main.c:2261:1: warning: the frame size of 1688 bytes is larger than 1024 bytes [-Wframe-larger-than=]
}
^
/usr/src/nDPI-netfilter/ndpi-netfilter/src/main.c: In function ‘nproto_proc_write’:
/usr/src/nDPI-netfilter/ndpi-netfilter/src/main.c:2486:1: warning: the frame size of 1160 bytes is larger than 1024 bytes [-Wframe-larger-than=]
}
^
CC [M] /usr/src/nDPI-netfilter/ndpi-netfilter/src/../lib/protocols/applejuice.o
CC [M] /usr/src/nDPI-netfilter/ndpi-netfilter/src/../lib/protocols/armagetron.o
/usr/src/nDPI-netfilter/ndpi-netfilter/src/main.c: In function ‘n_ipdef_proc_write’:
/usr/src/nDPI-netfilter/ndpi-netfilter/src/main.c:2298:1: warning: the frame size of 1608 bytes is larger than 1024 bytes [-Wframe-larger-than=]
}
^
CC [M] /usr/src/nDPI-netfilter/ndpi-netfilter/src/../lib/protocols/ayiya.o
scripts/Makefile.build:302: recipe for target '/usr/src/nDPI-netfilter/ndpi-netfilter/src/../lib/ndpi_main.o' failed
make[3]: *** [/usr/src/nDPI-netfilter/ndpi-netfilter/src/../lib/ndpi_main.o] Error 1
make[3]: *** Waiting for unfinished jobs....
Makefile:1515: recipe for target 'module/usr/src/nDPI-netfilter/ndpi-netfilter/src' failed
make[2]: *** [module/usr/src/nDPI-netfilter/ndpi-netfilter/src] Error 2
make[2]: Leaving directory '/usr/src/linux-headers-4.12.14-041214-generic'
Makefile:165: recipe for target 'modules' failed
make[1]: *** [modules] Error 2
make[1]: Leaving directory '/usr/src/nDPI-netfilter/ndpi-netfilter/src'
Makefile:2: recipe for target 'all' failed
make: *** [all] Error 2
linux-headers-4.4.0
/usr/src/nDPI/ndpi-netfilter/src/../../src/lib/ndpi_main.c:41:35: fatal error: ndpi_network_list.c.inc: No such file or directory
#include "ndpi_network_list.c.inc"
^
compilation terminated.
HTTP dynamic domain matching module?
I have seen this nice project:
https://github.com/Lochnair/xt_tls
Which actually is based on this project code.
I was wondering if it would be possible to either create or extend the current http/1.x module to be able to match more then just it but also a dynamic domain such as "rule --domain x.y.net -j DROP".
Kernel 4.11 - error: passing argument 2 of ‘atomic_sub_and_test’ from incompatible pointer type
==> Starting build()...
make -C ipt
make[1]: Entering directory '/srv/raid/filez/builder/PACKAGES/4.11.0/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/ipt'
cc -fPIC -I../include -I../lib -I../src -DOPENDPI_NETFILTER_MODULE -O2 -Wall -D_INIT=libxt_ndpi_init -c -o libxt_ndpi.o libxt_ndpi.c;
cc -shared -o libxt_ndpi.so libxt_ndpi.o;
rm libxt_ndpi.o
make[1]: Leaving directory '/srv/raid/filez/builder/PACKAGES/4.11.0/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/ipt'
make -C src
make[1]: Entering directory '/srv/raid/filez/builder/PACKAGES/4.11.0/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/src'
make -j 4 -C /usr/lib/modules/4.11.0-1-ARCH/build M=$PWD modules -w;
make[2]: Entering directory '/usr/lib/modules/4.11.0-1-ARCH/build'
CC [M] /srv/raid/filez/builder/PACKAGES/4.11.0/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/src/main.o
CC [M] /srv/raid/filez/builder/PACKAGES/4.11.0/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/../third_party/src/node.o
CC [M] /srv/raid/filez/builder/PACKAGES/4.11.0/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/../third_party/src/ahocorasick.o
CC [M] /srv/raid/filez/builder/PACKAGES/4.11.0/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/src/../lib/ndpi_main.o
/srv/raid/filez/builder/PACKAGES/4.11.0/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/src/main.c: In function ‘ndpi_free_id’:
/srv/raid/filez/builder/PACKAGES/4.11.0/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/src/main.c:585:35: error: passing argument 2 of ‘atomic_sub_and_test’ from incompatible pointer type [-Werror=incompatible-pointer-types]
if (atomic_sub_and_test((int) 1, &id->refcnt.refcount)) {
^
In file included from ./arch/x86/include/asm/msr.h:66:0,
from ./arch/x86/include/asm/processor.h:20,
from ./arch/x86/include/asm/cpufeature.h:4,
from ./arch/x86/include/asm/thread_info.h:52,
from ./include/linux/thread_info.h:25,
from ./arch/x86/include/asm/preempt.h:6,
from ./include/linux/preempt.h:80,
from ./include/linux/spinlock.h:50,
from ./include/linux/seqlock.h:35,
from ./include/linux/time.h:5,
from ./include/linux/stat.h:18,
from ./include/linux/module.h:10,
from /srv/raid/filez/builder/PACKAGES/4.11.0/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/src/main.c:25:
./arch/x86/include/asm/atomic.h:78:29: note: expected ‘atomic_t * {aka struct <anonymous> *}’ but argument is of type ‘refcount_t * {aka struct refcount_struct *}’
static __always_inline bool atomic_sub_and_test(int i, atomic_t *v)
^~~~~~~~~~~~~~~~~~~
CC [M] /srv/raid/filez/builder/PACKAGES/4.11.0/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/afp.o
CC [M] /srv/raid/filez/builder/PACKAGES/4.11.0/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/aimini.o
CC [M] /srv/raid/filez/builder/PACKAGES/4.11.0/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/applejuice.o
CC [M] /srv/raid/filez/builder/PACKAGES/4.11.0/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/src/../lib/protocols/armagetron.o
cc1: some warnings being treated as errors
make[3]: *** [scripts/Makefile.build:294: /srv/raid/filez/builder/PACKAGES/4.11.0/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/src/main.o] Error 1
make[3]: *** Waiting for unfinished jobs....
make[2]: *** [Makefile:1492: _module_/srv/raid/filez/builder/PACKAGES/4.11.0/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/src] Error 2
make[2]: Leaving directory '/usr/lib/modules/4.11.0-1-ARCH/build'
make[1]: *** [Makefile:165: modules] Error 2
make[1]: Leaving directory '/srv/raid/filez/builder/PACKAGES/4.11.0/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/src'
make: *** [Makefile:3: all] Error 2
==> ERROR: A failure occurred in build().
Aborting...vmap allocation for size 430080 failed: use vmalloc=<size> to increase size
[ 13.998495] vmap allocation for size 430080 failed: use vmalloc= to increase size
[ 14.006613] kmodloader: vmalloc: allocation failure: 425984 bytes, mode:0x14000c0(GFP_KERNEL), nodemask=(null)
[ 14.016701] CPU: 0 PID: 1134 Comm: kmodloader Not tainted 4.14.107 #0
[ 14.023173] Hardware name: Marvell Armada 380/385 (Device Tree)
[ 14.029138] [] (unwind_backtrace) from [] (show_stack+0x10/0x14)
[ 14.036929] [] (show_stack) from [] (dump_stack+0x88/0x9c)
[ 14.044196] [] (dump_stack) from [] (warn_alloc+0xbc/0x168)
[ 14.051550] [] (warn_alloc) from [] (__vmalloc_node_range+0x228/0x234)
[ 14.059866] [] (__vmalloc_node_range) from [] (module_alloc+0x4c/0x54)
[ 14.068176] [] (module_alloc) from [] (load_module+0x9e8/0x2100)
[ 14.075961] [] (load_module) from [] (SyS_init_module+0x12c/0x164)
[ 14.083922] [] (SyS_init_module) from [] (ret_fast_syscall+0x0/0x54)
[ 14.092064] Mem-Info:
[ 14.094356] active_anon:1335 inactive_anon:0 isolated_anon:0
[ 14.094356] active_file:4023 inactive_file:1316 isolated_file:0
[ 14.094356] unevictable:0 dirty:31 writeback:0 unstable:0
[ 14.094356] slab_reclaimable:832 slab_unreclaimable:3042
[ 14.094356] mapped:678 shmem:4 pagetables:50 bounce:0
[ 14.094356] free:110284 free_pcp:159 free_cma:0
[ 14.126756] Node 0 active_anon:5340kB inactive_anon:0kB active_file:16092kB inactive_file:5264kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:2712kB dirty:124kB writeback:0kB shmem:16kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no
[ 14.149386] Normal free:440820kB min:2856kB low:3568kB high:4280kB active_anon:5340kB inactive_anon:0kB active_file:16092kB inactive_file:5264kB unevictable:0kB writepending:124kB present:524288kB managed:510928kB mlocked:0kB kernel_stack:1520kB pagetables:124kB bounce:0kB free_pcp:688kB local_pcp:136kB free_cma:0kB
[ 14.177498] lowmem_reserve[]: 0 0 0
[ 14.181008] Normal: 14kB (U) 18kB (M) 316kB (UME) 032kB 264kB (UE) 1128kB (U) 0256kB 2512kB (UE) 11024kB (M) 02048kB 107*4096kB (M) = 440636kB
[ 14.194755] 5348 total pagecache pages
[ 14.198524] 0 pages in swap cache
[ 14.201872] Swap cache stats: add 0, delete 0, find 0/0
[ 14.207122] Free swap = 0kB
[ 14.210023] Total swap = 0kB
[ 14.212932] 131072 pages RAM
[ 14.215826] 0 pages HighMem/MovableOnly
[ 14.219680] 3340 pages reserved
[ 14.229348] xt_time: kernel timezone is -0000
[ 14.241452] PPP generic driver version 2.4.2
[ 14.246925] PPP MPPE Compression module registered
[ 14.252801] NET: Registered protocol family 24
[ 14.258097] PPTP driver version 0.8.5
[ 14.266912] vmap allocation for size 430080 failed: use vmalloc= to increase size
[ 14.274995] kmodloader: vmalloc: allocation failure: 425984 bytes, mode:0x14000c0(GFP_KERNEL), nodemask=(null)
[ 14.285079] CPU: 0 PID: 1134 Comm: kmodloader Not tainted 4.14.107 #0
[ 14.291550] Hardware name: Marvell Armada 380/385 (Device Tree)
[ 14.297523] [] (unwind_backtrace) from [] (show_stack+0x10/0x14)
[ 14.305314] [] (show_stack) from [] (dump_stack+0x88/0x9c)
[ 14.312577] [] (dump_stack) from [] (warn_alloc+0xbc/0x168)
[ 14.319932] [] (warn_alloc) from [] (__vmalloc_node_range+0x228/0x234)
[ 14.328251] [] (__vmalloc_node_range) from [] (module_alloc+0x4c/0x54)
[ 14.336561] [] (module_alloc) from [] (load_module+0x9e8/0x2100)
[ 14.344356] [] (load_module) from [] (SyS_init_module+0x12c/0x164)
[ 14.352325] [] (SyS_init_module) from [] (ret_fast_syscall+0x0/0x54)
[ 14.360768] kmodloader: failed to load xt_ndpi
[ 14.366718] kmodloader: 1 module could not be probed
[ 14.371892] kmodloader: - xt_ndpi - 0
root@OpenWrt:~# cat /proc/meminfo
MemTotal: 510928 kB
MemFree: 348972 kB
MemAvailable: 382276 kB
Buffers: 18996 kB
Cached: 60896 kB
SwapCached: 0 kB
Active: 66396 kB
Inactive: 35180 kB
Active(anon): 22844 kB
Inactive(anon): 4348 kB
Active(file): 43552 kB
Inactive(file): 30832 kB
Unevictable: 0 kB
Mlocked: 0 kB
HighTotal: 0 kB
HighFree: 0 kB
LowTotal: 510928 kB
LowFree: 348972 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 0 kB
Writeback: 0 kB
AnonPages: 21716 kB
Mapped: 22668 kB
Shmem: 5508 kB
Slab: 23592 kB
SReclaimable: 8452 kB
SUnreclaim: 15140 kB
KernelStack: 1072 kB
PageTables: 1540 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 255464 kB
Committed_AS: 137268 kB
VmallocTotal: 507904 kB
VmallocUsed: 0 kB
VmallocChunk: 0 kB
Using SLUB
При make modules_install ошибка
Ошибка вида
At main.c:222:
- SSL error:02001002:system library:fopen:No such file or directory: bss_file.c:169
- SSL error:2006D080:BIO routines:BIO_new_file:no such file: bss_file.c:172
sign-file: certs/signing_key.pem: No such file or directory
Lost of packet filtering on bittorrent
Hi! Sorry for using the issue tracker to ask question here.
Recently im trying to filter bittorrent on my Ubuntu 16.04 server (which is a strongswan vpn server). After a few days effort of solution searching, i've found your repository, which is nice to use with and base the modern DPI library nDPI (compared with those ipp2p and l7-filter).
However, the performance of bittorrent filtering is not that effective as expected. Some bittorrent packet is being dropped, but still many of them are passed as they are "unknown" packet.
The photo attached is the snapshot when the user have tried to bt through the vpn for a few seconds.
As in the photo, you will there are lots of packet going to the "unknown" rules. If I blocked the unknown packets, there will be a mess. (as you mention in the installation instruction). However, if i remove it, the bittorrent will able to be continued as normal.
Any idea/solution can help? Thanks!
1.7 and 1.8
the ndpi 1.7 is working fine on my system but i am guessing if we can use the ndpi 1.8
BUILD_BUG_ON(NF_CT_EXT_NUM > 9) - On kernel 4.14.97
Hi,
wanted to report a problem with the v4.14.0.diff kernel patch. Am using an IPFire appliance with a 4.14.97 kernel . The problem log portion looks like this:
In file included from ./include/linux/init.h:5:0, from ./include/linux/netfilter.h:5, from net/netfilter/nf_conntrack_core.c:18: In function 'total_extension_size', inlined from 'nf_conntrack_init_start' at net/netfilter/nf_conntrack_core.c:2011:2: ./include/linux/compiler.h:328:38: error: call to '__compiletime_assert_1977' declared with attribute error: BUILD_BUG_ON failed: NF_CT_EXT_NUM > 9 _compiletime_assert(condition, msg, __compiletime_assert_, __LINE__) ^ ./include/linux/compiler.h:308:4: note: in definition of macro '__compiletime_assert' prefix ## suffix(); \ ^~~~~~ ./include/linux/compiler.h:328:2: note: in expansion of macro '_compiletime_assert' _compiletime_assert(condition, msg, __compiletime_assert_, __LINE__) ^~~~~~~~~~~~~~~~~~~ ./include/linux/build_bug.h:47:37: note: in expansion of macro 'compiletime_assert' #define BUILD_BUG_ON_MSG(cond, msg) compiletime_assert(!(cond), msg) ^~~~~~~~~~~~~~~~~~ ./include/linux/build_bug.h:71:2: note: in expansion of macro 'BUILD_BUG_ON_MSG' BUILD_BUG_ON_MSG(condition, "BUILD_BUG_ON failed: " #condition) ^~~~~~~~~~~~~~~~ net/netfilter/nf_conntrack_core.c:1977:2: note: in expansion of macro 'BUILD_BUG_ON' BUILD_BUG_ON(NF_CT_EXT_NUM > 9); ^~~~~~~~~~~~ make[3]: *** [scripts/Makefile.build:326: net/netfilter/nf_conntrack_core.o] Error 1 make[2]: *** [scripts/Makefile.build:585: net/netfilter] Error 2
you can also find the full build log in here --> https://people.ipfire.org/~ummeegge/logs/kernel_log_KCFG-ERROR-nDPI-netfilter .
If you need some more information, just let it me know.
Best,
Erik
Linux 5.0 build issues
make -C ipt
make[1]: Entering directory '/home/someman/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/ipt'
cc -fPIC -I../.. -I../../src/include -I../../src/lib -I../src -I../libre -DOPENDPI_NETFILTER_MODULE -O2 -Wall -D_INIT=libxt_ndpi_init -c -o libxt_ndpi.o libxt_ndpi.c;
cc -shared -o libxt_ndpi.so libxt_ndpi.o;
rm libxt_ndpi.o
make[1]: Leaving directory '/home/someman/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/ipt'
make -C src
make[1]: Entering directory '/home/someman/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/src'
$xt_ndpi-y is [main.o ndpi_strcol.o ndpi_proc_parsers.o ndpi_proc_generic.o ../../src/lib/third_party/src/node.o ../../src/lib/third_party/src/ahocorasick.o ../../src/lib/third_party/src/libcache.o ../../src/lib/third_party/src/ndpi_patricia.o ../../src/lib/third_party/src/ht_hash.o ../../src/lib/third_party/src/lruc.o ../../src/lib/third_party/src/btlib.o ../../src/lib/ndpi_main.o ../../src/lib/protocols/*.o]
\
make -C /usr/lib/modules/5.0.0-arch1-1-ARCH/build M=$PWD modules -w;
make[2]: Entering directory '/usr/lib/modules/5.0.0-arch1-1-ARCH/build'
CC [M] /home/someman/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/src/main.o
/home/someman/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/src/main.c: In function ‘_ninfo_proc_read’:
/home/someman/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/src/main.c:1536:39: error: macro "access_ok" passed 3 arguments, but takes just 2
if (!(access_ok(VERIFY_WRITE, buf, l) &&
^
/home/someman/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/src/main.c:1536:9: error: ‘access_ok’ undeclared (first use in this function)
if (!(access_ok(VERIFY_WRITE, buf, l) &&
^~~~~~~~~
/home/someman/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/src/main.c:1536:9: note: each undeclared identifier is reported only once for each function it appears in
/home/someman/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/src/main.c:1561:39: error: macro "access_ok" passed 3 arguments, but takes just 2
if (!(access_ok(VERIFY_WRITE, buf, l) &&
^
/home/someman/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/src/main.c:1576:42: error: macro "access_ok" passed 3 arguments, but takes just 2
if (!(access_ok(VERIFY_WRITE, buf+p, l) && !__copy_to_user(buf+p, lbuf, l)))
^
/home/someman/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/src/main.c:1583:41: error: macro "access_ok" passed 3 arguments, but takes just 2
if (!(access_ok(VERIFY_WRITE, buf+p, l) &&
^
/home/someman/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/src/main.c:1595:39: error: macro "access_ok" passed 3 arguments, but takes just 2
if (!(access_ok(VERIFY_WRITE, buf, l) &&
^
/home/someman/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/src/main.c:1613:42: error: macro "access_ok" passed 3 arguments, but takes just 2
if (!(access_ok(VERIFY_WRITE, buf+p, l) &&
^
/home/someman/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/src/main.c: In function ‘ninfo_proc_write’:
/home/someman/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/src/main.c:1652:46: error: macro "access_ok" passed 3 arguments, but takes just 2
if (!(access_ok(VERIFY_READ, buffer, length) &&
^
/home/someman/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/src/main.c:1652:9: error: ‘access_ok’ undeclared (first use in this function)
if (!(access_ok(VERIFY_READ, buffer, length) &&
^~~~~~~~~
/home/someman/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/src/main.c: In function ‘nann_proc_read’:
/home/someman/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/src/main.c:1687:41: error: macro "access_ok" passed 3 arguments, but takes just 2
if (!(access_ok(VERIFY_WRITE, buf+p, l) &&
^
/home/someman/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/src/main.c:1687:9: error: ‘access_ok’ undeclared (first use in this function)
if (!(access_ok(VERIFY_WRITE, buf+p, l) &&
^~~~~~~~~
/home/someman/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/src/main.c: In function ‘n_ipdef_proc_read’:
/home/someman/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/src/main.c:1765:41: error: macro "access_ok" passed 3 arguments, but takes just 2
if (!(access_ok(VERIFY_WRITE, buf+p, l) &&
^
/home/someman/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/src/main.c:1765:9: error: ‘access_ok’ undeclared (first use in this function)
if (!(access_ok(VERIFY_WRITE, buf+p, l) &&
^~~~~~~~~
/home/someman/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/src/main.c: In function ‘nproto_proc_read’:
/home/someman/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/src/main.c:1825:41: error: macro "access_ok" passed 3 arguments, but takes just 2
if (!(access_ok(VERIFY_WRITE, buf+p, l) &&
^
/home/someman/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/src/main.c:1825:9: error: ‘access_ok’ undeclared (first use in this function)
if (!(access_ok(VERIFY_WRITE, buf+p, l) &&
^~~~~~~~~
/home/someman/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/src/main.c: In function ‘n_hostdef_proc_read’:
/home/someman/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/src/main.c:2003:46: error: macro "access_ok" passed 3 arguments, but takes just 2
if (!(access_ok(VERIFY_WRITE, buf+bpos, l) &&
^
/home/someman/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/src/main.c:2003:11: error: ‘access_ok’ undeclared (first use in this function)
if (!(access_ok(VERIFY_WRITE, buf+bpos, l) &&
^~~~~~~~~
make[3]: *** [scripts/Makefile.build:277: /home/someman/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/src/main.o] Error 1
make[2]: *** [Makefile:1553: _module_/home/someman/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/src] Error 2
make[2]: Leaving directory '/usr/lib/modules/5.0.0-arch1-1-ARCH/build'
make[1]: *** [Makefile:33: modules] Error 2
make[1]: Leaving directory '/home/someman/ipt_ndpi/src/ipt_ndpi/ndpi-netfilter/src'
make: *** [Makefile:3: all] Error 2xt_ndpi: Warning! Hostdef 'mmg-fna.whatsapp.net' missmatch! Skipping.
Dear all,
After having compiled the branch netfilter-2.6 and loaded the xt_ndpi module, I get the following in my dmesg :
`
[ 486.886228] [NDPI] Missmatch 'ggpht.com' proto YouTube origin PlayStore
[ 486.888417] xt_ndpi: Warning! Hostdef 'mmg-fna.whatsapp.net' missmatch! Skipping.
[ 486.888537] xt_ndpi: Warning! Hostdef 'ggpht.com' missmatch! Skipping.
[ 486.889990] xt_ndpi v1.2 ndpi 2.4.0-1320-97f4744 IPv6=YES debug_message=no
BT: hash_size 0k, hash_expiation 0 sec, log_size 128kb
sizeof hash_ip4p_node=44 id_struct=296 PATRICIA_MAXBITS=128
flow_struct=2024 packet_struct=1400
flow_tcp_struct=41 flow_udp_struct=27 int_one_line_struct=16
ndpi_ip_addr_t=16 ndpi_protocol=8 nf_ct_ext_ndpi=60
spinlock_t=4
NF_EXT_ID 9
[ 486.889991] xt_ndpi MAX_PROTOCOLS 320 LAST_PROTOCOL 243`
I'm wondering the meaning of the 3 messages containing "Missmatch" and if it makes the corresponding dissectors unusable.
Then, I see xt_ndpi v1.2 ndpi 2.4.0-1320-97f4744 IPv6=YES debug_message=no. I compiled the netfilter-2.6 branch, so what is the actual version of ndpi currently used by my xt_ndpi module ?
Thank you.
Clément
ndpiReader ends up with segmentation fault
Hi,
In the current state of the netfilter-2.6 branch, ndpiReader crashes with a segmentation fault.
However, strace shows that is crashes after the analysis. Redirecting strace stdout to a file enable me to see the test results.
Find attached a &> redirection of strace ndpiReader -i test.pcap
Compiling on AlpineLinux.
I have been working with AlpineLinux for quite some time and it seems that it's a pretty nice routing system but for now I was unable to compile nDPI on it.
I have seen this bug report on AlpineLinux: https://bugs.alpinelinux.org/issues/3526
But the issue is that there is no example of the build node.
CentOS 7 and Fedora 26 kmod testers are needed.
I took the work of @gsanchietti and @filippocarletti at:
https://github.com/NethServer/xt_ndpi-kmod
And converted it into a docker build container which I have tested to on both CentOS 7 and Fedora 26.
More details are at the wiki and we need testers that want to help and verify if these modules run as expected on other systems then my test lab.
For CentOS 7 there is a repository and the RPM can be downloaded directly from:
http://ngtech.co.il/repo/centos/7/x86_64/kmod-xt_ndpi-2.0.1-2.el7.centos.x86_64.rpm
Setup nDPI-netfilter inline mode
i install nDPI-netfilter module to my ubuntu 16.04 LTC and i need to setup my machine to work inline-mode between the gateway and my network.
i have 3 network interface on my machine, one for control the machine and two for forward and control the traffic between the internet and the network.
how i can setup ndpi-netfiler iptables for controlling the traffic without NAT rules (without setting ip addresses for eth0 and eth1)?
what iptables rules i need to set for forward and control the traffic with iptables and tc ?
netfilter2 compile error !!
i try to compile netfilter2 on my ubuntu 14.04.5 64bit i do the following:
git clone https://github.com/vel21ripn/nDPI/
git checkout netfilter2
./autogen.sh
make
and i get this error:
depbase=`echo ndpiReader.o | sed 's|[^/]*$|.deps/&|;s|\.o$||'`;\
gcc -DHAVE_CONFIG_H -I. -I.. -I../src/include -pthread -g -O2 -MT ndpiReader.o -MD -MP -MF $depbase.Tpo -c -o ndpiReader.o ndpiReader.c &&\
mv -f $depbase.Tpo $depbase.Po
depbase=`echo ndpi_util.o | sed 's|[^/]*$|.deps/&|;s|\.o$||'`;\
gcc -DHAVE_CONFIG_H -I. -I.. -I../src/include -pthread -g -O2 -MT ndpi_util.o -MD -MP -MF $depbase.Tpo -c -o ndpi_util.o ndpi_util.c &&\
mv -f $depbase.Tpo $depbase.Po
ndpi_util.c:35:24: fatal error: pcap/nflog.h: No such file or directory
#include <pcap/nflog.h>
^
compilation terminated.
make[2]: *** [ndpi_util.o] Error 1
Plans on new releases?
Hi!
Any plans on a new release? I would be great!
Thanks!
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.