majek / dump Goto Github PK

Hey,

when trying to run your script like this

stap -v histogram-kernel.stp 'kernel.function("net_rx_action")'

I've got a metric ton of errors. Do I need to change it for my kernel version? Can it be made version independent?

Pass 1: parsed user script and 95 library script(s) using 86584virt/30540res/5116shr/26196data kb, in 70usr/70sys/156real ms.
Pass 2: analyzed script: 4 probe(s), 3 function(s), 3 embed(s), 3 global(s) using 538896virt/80340res/7000shr/73864data kb, in 1260usr/1700sys/4879real ms.
Pass 3: translated to C into "/tmp/stap8EwY4x/stap_e9940a8a3353a2d196e72ff059fa60c3_3482_src.c" using 536376virt/84028res/10820shr/73864data kb, in 0usr/0sys/14real ms.
In file included from /usr/share/systemtap/runtime/linux/runtime.h:192:0,
from /usr/share/systemtap/runtime/runtime.h:24,
from /tmp/stap8EwY4x/stap_e9940a8a3353a2d196e72ff059fa60c3_3482_src.c:24:
/usr/share/systemtap/runtime/linux/alloc.c: In function '_stp_kmalloc':
/usr/share/systemtap/runtime/linux/alloc.c:19:8: error: '__GFP_WAIT' undeclared (first use in this function)
& ~__GFP_WAIT)
^
/usr/share/systemtap/runtime/linux/alloc.c:275:32: note: in expansion of macro 'STP_ALLOC_FLAGS'
return _stp_kmalloc_gfp(size, STP_ALLOC_FLAGS);
^
/usr/share/systemtap/runtime/linux/alloc.c:19:8: note: each undeclared identifier is reported only once for each function it appears in
& ~__GFP_WAIT)
^
/usr/share/systemtap/runtime/linux/alloc.c:275:32: note: in expansion of macro 'STP_ALLOC_FLAGS'
return _stp_kmalloc_gfp(size, STP_ALLOC_FLAGS);
^
/usr/share/systemtap/runtime/linux/alloc.c: In function '_stp_kzalloc':
/usr/share/systemtap/runtime/linux/alloc.c:19:8: error: '__GFP_WAIT' undeclared (first use in this function)
& ~__GFP_WAIT)
^
/usr/share/systemtap/runtime/linux/alloc.c:331:33: note: in expansion of macro 'STP_ALLOC_FLAGS'
return _stp_kzalloc_gfp(size, STP_ALLOC_FLAGS);
^
/usr/share/systemtap/runtime/linux/alloc.c: In function '_stp_kmalloc_node':
/usr/share/systemtap/runtime/linux/alloc.c:19:8: error: '__GFP_WAIT' undeclared (first use in this function)
& ~__GFP_WAIT)
^
/usr/share/systemtap/runtime/linux/alloc.c:428:43: note: in expansion of macro 'STP_ALLOC_FLAGS'
return _stp_kmalloc_node_gfp(size, node, STP_ALLOC_FLAGS);
^
/usr/share/systemtap/runtime/linux/alloc.c: In function '_stp_kzalloc_node':
/usr/share/systemtap/runtime/linux/alloc.c:19:8: error: '__GFP_WAIT' undeclared (first use in this function)
& ~__GFP_WAIT)
^
/usr/share/systemtap/runtime/linux/alloc.c:432:43: note: in expansion of macro 'STP_ALLOC_FLAGS'
return _stp_kzalloc_node_gfp(size, node, STP_ALLOC_FLAGS);
^
In file included from /usr/share/systemtap/runtime/linux/task_finder2.c:4:0,
from /usr/share/systemtap/runtime/linux/task_finder.c:17,
from /usr/share/systemtap/runtime/linux/runtime.h:202,
from /usr/share/systemtap/runtime/runtime.h:24,
from /tmp/stap8EwY4x/stap_e9940a8a3353a2d196e72ff059fa60c3_3482_src.c:24:
/usr/share/systemtap/runtime/stp_utrace.c: In function 'utrace_task_alloc':
/usr/share/systemtap/runtime/stp_utrace.c:409:59: error: 'GFP_IOFS' undeclared (first use in this function)
struct utrace _utrace = kmem_cache_zalloc(utrace_cachep, GFP_IOFS);
^
/usr/share/systemtap/runtime/stp_utrace.c: In function 'utrace_attach_task':
/usr/share/systemtap/runtime/stp_utrace.c:668:50: error: 'GFP_IOFS' undeclared (first use in this function)
engine = kmem_cache_alloc(utrace_engine_cachep, GFP_IOFS);
^
In file included from /usr/share/systemtap/runtime/linux/runtime.h:192:0,
from /usr/share/systemtap/runtime/runtime.h:24,
from /tmp/stap8EwY4x/stap_e9940a8a3353a2d196e72ff059fa60c3_3482_src.c:24:
/usr/share/systemtap/runtime/task_finder_vma.c: In function '__stp_tf_vma_new_entry':
/usr/share/systemtap/runtime/linux/alloc.c:19:8: error: '__GFP_WAIT' undeclared (first use in this function)
& ~__GFP_WAIT)
^
/usr/share/systemtap/runtime/task_finder_vma.c:58:64: note: in expansion of macro 'STP_ALLOC_FLAGS'
STP_ALLOC_FLAGS);
^
In file included from /usr/share/systemtap/runtime/print.c:17:0,
from /usr/share/systemtap/runtime/runtime_context.h:22,
from /tmp/stap8EwY4x/stap_e9940a8a3353a2d196e72ff059fa60c3_3482_src.c:174:
/usr/share/systemtap/runtime/linux/print.c: In function '_stp_print_kernel_info':
/usr/share/systemtap/runtime/linux/print.c:242:20: error: 'struct module' has no member named 'module_core'
THIS_MODULE->module_core,
^
/usr/share/systemtap/runtime/linux/print.c:243:37: error: 'struct module' has no member named 'core_size'
(unsigned long) (THIS_MODULE->core_size - THIS_MODULE->core_text_size)/1024,
^
/usr/share/systemtap/runtime/linux/print.c:243:62: error: 'struct module' has no member named 'core_text_size'
(unsigned long) (THIS_MODULE->core_size - THIS_MODULE->core_text_size)/1024,
^
/usr/share/systemtap/runtime/linux/print.c:244:44: error: 'struct module' has no member named 'core_text_size'
(unsigned long) (THIS_MODULE->core_text_size)/1024,
^
In file included from /tmp/stap8EwY4x/stap_e9940a8a3353a2d196e72ff059fa60c3_3482_src.c:311:0:
/usr/share/systemtap/runtime/time.c: In function '__stp_time_cpufreq_callback':
/usr/share/systemtap/runtime/time.c:178:14: error: 'CPUFREQ_RESUMECHANGE' undeclared (first use in this function)
case CPUFREQ_RESUMECHANGE:
^
In file included from /usr/share/systemtap/runtime/timer.c:17:0,
from /tmp/stap8EwY4x/stap_e9940a8a3353a2d196e72ff059fa60c3_3482_src.c:1292:
/usr/share/systemtap/runtime/linux/timer.c: In function '_stp_hrtimer_init':
/usr/share/systemtap/runtime/linux/timer.c:49:2: error: implicit declaration of function 'hrtimer_get_res' [-Werror=implicit-function-declaration]
hrtimer_get_res (CLOCK_MONOTONIC, &res);
^
In file included from /usr/share/systemtap/runtime/linux/runtime.h:192:0,
from /usr/share/systemtap/runtime/runtime.h:24,
from /tmp/stap8EwY4x/stap_e9940a8a3353a2d196e72ff059fa60c3_3482_src.c:24:
/usr/share/systemtap/runtime/linux/alloc.c: In function 'stp_kmalloc':
/usr/share/systemtap/runtime/linux/alloc.c:276:1: error: control reaches end of non-void function [-Werror=return-type]
}
^
cc1: all warnings being treated as errors
make[1]: ** [/tmp/stap8EwY4x/stap_e9940a8a3353a2d196e72ff059fa60c3_3482_src.o] Error 1
make: *** [module/tmp/stap8EwY4x] Error 2
WARNING: kbuild exited with status: 2
Pass 4: compiled C into "stap_e9940a8a3353a2d196e72ff059fa60c3_3482.ko" in 5010usr/4410sys/11814real ms.
Pass 4: compilation failed. [man error::pass4]
Tip: /usr/share/doc/systemtap/README.Debian should help you get started.

Hello,

The following patch enables user to specify payload size to the 'udpsender' program (part of 'how-to-receive-a-million-packets' project). This may be useful, if someone wants to generate packets of different sizes.

--- a/udpsender.c   2015-09-21 12:35:30.341486515 +0530
+++ b/udpsender.c   2015-09-21 17:07:00.730645321 +0530
@@ -8,6 +8,10 @@

 #include "common.h"

+/* Minimum payload: 64 - 42 (frame overhead)
+ * Maximum payload: 1518 - 42 (frame overhead) */
+#define MIN_PAYLOAD 22
+#define MAX_PAYLOAD 1476

 struct state {
    struct net_addr *target_addr;
@@ -64,13 +68,18 @@
 int main(int argc, const char *argv[])
 {
    int packets_in_buf = 1024;
-   const char *payload = (const char[32]){0};
-   int payload_sz = 32;
+   const char *payload = (const char[MAX_PAYLOAD]){0};
+   int payload_sz = MIN_PAYLOAD;

-   if (argc == 1) {
-       FATAL("Usage: %s [target ip:port] [target ...]", argv[0]);
+   if (argc <= 2) {
+       FATAL("Usage: %s [payload size] [target ip:port] [target ...]", argv[0]);
    }

+    payload_sz = atoi(argv[1]);
+    payload_sz = (payload_sz >= MIN_PAYLOAD && payload_sz <= MAX_PAYLOAD) ? payload_sz : MIN_PAYLOAD;
+    argv++;
+    argc--;
+
    struct net_addr *target_addrs = calloc(argc-1, sizeof(struct net_addr));
    int thread_num = argc - 1;

Please how can i compile udpsender.c and udpreceiver1.c on Linux CentOS 6.7

hi
it does not seem to show the per second in the total . is tha deliberate ?

Every 2.0s: ./softnet_stat.sh Thu Sep 7 12:56:43 2017

cpu total dropped squeezed collision rps flow_limit
0 620315792 0 11 0 0 0
1 692783857 0 22 0 0 0
2 633627628 0 16 0 0 0
3 705611806 0 20 0 0 0
4 708270938 0 16 0 0 0
5 750169201 0 23 0 0 0
6 681272174 0 6 0 0 0
7 735968643 0 19 0 0 0
8 742507143 0 20 0 0 0
9 755944537 0 9 0 0 0
10 725148038 0 18 0 0 0
11 713036160 0 12 0 0 0
12 616773747 0 21 0 0 0
13 666581627 0 26 0 0 0
14 686718967 0 32 0 0 0
15 720161220 0 29 0 0 0
16 684179694 0 43 0 0 0
17 686634382 0 29 0 0 0
18 715089244 0 28 0 0 0
19 714415250 0 24 0 0 0
20 716458895 0 30 0 0 0
21 730670464 0 36 0 0 0
22 703662730 0 32 0 0 0
23 828731206 0 79 0 0 0

Hey, I wanted to use your spinlock code, but I noticed that there wasn't a license file or license included in the file.

Could you add a LICENSE file to the repo, preferably BSD, MIT, WTFPL, etc? Thanks!

Hello,

While trying to compile the 'how-to-receive-a-million-packets' code, I got the following error.

$ ./build.sh 
/tmp/udpreceiver1-4ee1c6.o: In function `main':
udpreceiver1.c:143: undefined reference to `__atomic_load_8'
udpreceiver1.c:144: undefined reference to `__atomic_load_8'
/tmp/udpreceiver1-4ee1c6.o: In function `thread_loop':
udpreceiver1.c:67: undefined reference to `__atomic_fetch_add_8'
udpreceiver1.c:68: undefined reference to `__atomic_fetch_add_8'
clang: error: linker command failed with exit code 1 (use -v to see invocation)

A quick google search revealed that the possible cause is incompatibility with 32 bit OS. So, I did the following changes and it worked fine (however, I am not very sure about any side effects).

--- a/udpreceiver1.c    2015-09-21 12:35:54.002816478 +0530
+++ b/udpreceiver1.c    2015-09-21 17:04:36.236634027 +0530
@@ -18,12 +18,12 @@

 struct state {
    int fd;
-   volatile uint64_t bps;
-   volatile uint64_t pps;
+   volatile uint32_t bps;
+   volatile uint32_t pps;
    struct mmsghdr messages[MAX_MSG];
    char buffers[MAX_MSG][MTU_SIZE];
    struct iovec iovecs[MAX_MSG];
-} __attribute__ ((aligned (64)));
+} __attribute__ ((aligned (32)));

 struct state *state_init(struct state *s) {
    int i;
@@ -121,8 +121,8 @@
        thread_spawn(thread_loop, state);
    }

-   uint64_t last_pps = 0;
-   uint64_t last_bps = 0;
+   uint32_t last_pps = 0;
+   uint32_t last_bps = 0;

    while (1) {
        struct timeval timeout =
@@ -137,7 +137,7 @@
            }
        }

-       uint64_t now_pps = 0, now_bps = 0;
+       uint32_t now_pps = 0, now_bps = 0;
        for (t = 0; t < thread_num; t++) {
            struct state *state = &array_of_states[t];
            now_pps += __atomic_load_n(&state->pps, 0);

even if it's been a long time ago , you botguard dump is just insane to me , i m a young french teen of 16 barely knew in the world of cryptographie and decrypting this "identifier","!zc6lzu9CH9... are just a big pain for me , you released a python version but i know a bit C and C# and i really don't know how to use it or if it is up to date ? any issue for me , I seriously really admire the work you've done thanks axel ;)

~/dump/how-to-receive-a-packet$ ./build.sh
udpclient.c:31:6: error: conflicting types for 'gettid'
long gettid() { return syscall(SYS_gettid); }
     ^
/usr/include/x86_64-linux-gnu/bits/unistd_ext.h:34:16: note: previous declaration is here
extern __pid_t gettid (void) __THROW;
               ^
udpclient.c:73:16: warning: format specifies type 'long' but the argument has type '__pid_t' (aka 'int') [-Wformat]
                                 getpid(), gettid(), packet_no);
                                           ^~~~~~~~
/usr/include/x86_64-linux-gnu/bits/stdio2.h:61:31: note: expanded from macro 'snprintf'
                            __glibc_objsize (str), __VA_ARGS__)
                                                   ^~~~~~~~~~~
1 warning and 1 error generated.

Here was a error, and no binary file was.
I'm ask how to fix it in Copilot, and I found worked decision.

After that almost all is fine:

./build.sh
udpclient.c:73:16: warning: format specifies type 'long' but the argument has type '__pid_t' (aka 'int') [-Wformat]
                                 getpid(), gettid(), packet_no);
                                           ^~~~~~~~
/usr/include/x86_64-linux-gnu/bits/stdio2.h:61:31: note: expanded from macro 'snprintf'
                            __glibc_objsize (str), __VA_ARGS__)
                                                   ^~~~~~~~~~~
1 warning generated.

https://sl.bing.net/jEm33FtTJv2

Hi, I don't know the purpose of this "dump" repository, but I noticed some issues. With a casual glance, you probably can spot them immediate when looking at the code.

In the file "dump/arduino/ard-04/lilypod.ino", the while-loop between Wire.requestFrom() and the Wire.read()s is not needed. There is not need to wait for something.

In the file "dump/arduino/ard-06/hmc5883l.cpp", in the function "Read()", the Wire.requestFrom() should not have the Wire.beginTransmission() and Wire.endTransmission(). A serious bug is the pointer that is returned, because the buffer is out of scope as soon as the function returns. A quick and dirty fix is to make the buffer static. It's dirty, I know.
There is no original for the loveelectronics.co.uk code on github. I also mentioned the issue here: landis/arduino#2

Your msqueue files look rather promising, although for general usage, they might need a couple of modifications. Would you prefer:

1. I copy them to my project's dependencies folder (with a link back to this repo), and then change them (which should be alright, since you said this is all MIT'd and my project is also MIT'd).
2. I send you a PR with the changes and then copy the changed version back to my project.
3. Alternative option?

Hello,

I would really like to use some of the code in the dump repository for free software projects, particularly the code in "how-to-receive-a-packet".

Closed issue #1 has relevant information, as well as pull request 16. In PR 16 you state, "Code that are reused from somwhere are... well under some license. Code authored by me are either "public domain" or MIT. When in doubt use MIT as suggested in issue 1" I think that this is difficult for users to find. It also does not propagate to forks of the repository.

I can't tell whether code in a particular directory is written by you or not, as I don't see statements in the files crediting the author of the code.

As an example of why an obvious license statement is important, user unixlab has forked the code in "how-to-receive-a-packet", but unixlab's code has no license statement. unixlab probably didn't know what license to apply to it. I was about to use the code, checked the license and found myself unable to use it without knowing what license terms to announce and comply with for my fork of the code.

Thanks for your contributions to free software and the computing industry.

When i run stap -v histogram-kernel.stp 'kernel.function("tcp_collapse")' 5.
report error

Hi,

when running the PMTU test on icmpcheck.popcount.org from a DNS64/NAT64 enabled network it seems to fail whether there is a PMTU problem or not. In such a network traffic towards IPv4 services will pass through a v4-to-v6 translation service (SIIT/RFC6145). The specific problem seems to be that when translating a v4 ICMP package-too-big error to v6 the reported MTU is increased to at least 1280, instead of 905 sent by the icmpcheck service in my test. This is as per RFC6145 Section 6:

1. In the IPv4-to-IPv6 direction: if the MTU value of ICMPv4 Packet
   Too Big (PTB) messages is less than 1280, change it to 1280.
   This is intended to cause the IPv6 host and IPv6 firewall to
   process the ICMP PTB message and generate subsequent packets to
   this destination with an IPv6 Fragment Header.

In order to ensure communication with lower than 1280 MTU links can still work the RFC mandates setting DF=0 in the return direction (when the packet len is <1280). I've confirmed my translation service correctly implements this but the icmpcheck service just keeps responding with packet-too-big. So my guess is DF=0 is not respected in icmpcheck.

I'm happy to provide pcaps privately on request if that helps.

To test this on your end there are a number of public DNS64/NAT64 services out there, see https://nat64.net/public-providers. Just configure their DNS server on an IPv6 capable test host and you should be good to go.

Thanks,
--Daniel

PS: I hope I found the right issue tracker, it's non-obvious where the source code for the icmpcheck.popcount.org service is at. A link might be nice :]

Just FYI you're hurting your performance numbers if you use SO_REUSEPORT on the sender side. An average NIC will include SRC Port and IP in the hash and this will ensure that all packets from a multi-threaded udpsender process get delivered to the same queue.