shuveb / io_uring-by-example Goto Github PK

The current copy_file() does not wait for all submitted I/O's to complete before returning to main() which the latter closes the files and calls io_uring_queue_exit(). Any I/Os which are still in progress will be lost. This manifests as the output file having fewer bytes than the input file. This is tested and is repeatable in Ubuntu 20 running in a VM under VMWare Workstation, using liburing2_2.1-2build1_amd64.deb.

One way to fix this is to add the following codes before the return statement at the end of copy_file() to wait for all I/Os to complete before returning:

int num_pending = reads + writes; // # of I/O requests still pending.
for (int i=0; i<num_pending; i++) { // Wait for them to complete or else some may be discarded upon premature io_uring exit.
    io_uring_wait_cqe(ring, &cqe);
    io_uring_cqe_get_data(cqe);
    io_uring_cqe_seen(ring, cqe);
}

Cheers,
Jack

Reading the comment above the function copy_file_contents() it says the following.

/*
 * Once a static file is identified to be served, this function is used to read the file
 * and write it over the client socket using Linux's sendfile() system call. This saves us
 * the hassle of transferring file buffers from kernel to user space and back.
 * */

But the function itself loads the entire file at once into userspace memory.
The function does not use sendfile().

please see my debug screen.

it goes into below error path, cqe.res value is strange. looks like no one assign it.

please guide me.

intel@wcp116:/wayne/miniobin/distributedminioBWprofile$ uname -a
Linux wcp116 5.4.0-42-generic #46-Ubuntu SMP Fri Jul 10 00:24:02 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
intel@wcp116:/wayne/miniobin/distributedminioBWprofile$

intel@wcp116:~/wayne/io_uring-by-example$ sudo ./webserver_liburing
hello from Wayne debug here!
2
Async request failed: Invalid argument for event: 0

Hi,
it's possible edit the web server to have file access async with io_uring? (open/read/close)
Cheers,

tried install liburing with
https://hnakamur.github.io/blog/2020/04/07/tried_io_uring_example/
but got error below

can you provide instructions to test?

./webserver_liburing 83123
Async request failed: Invalid argument for event: 0

Nice work! How do u show the remote ip address of the client?

Hey there, thanks for putting these examples together. I'm practicing io_uring and you're resources have been invaluable. Your site is currently down though :( I had to use archive.org.

Anyways, I'm having trouble running example 3 on files that are larger than 8530944 bytes. Calculated by sum total of posix_memalign successes before crash which was 8331 * BLOCK_SZ on my machine.

I'm using WSL2 and Ubuntu 20.10 with Linux Kernel 5.4.81-microsoft-standard-WSL2+

My C is really rusty and it's late which is why I'm reaching out.

I ran the build cmd
clang -luring -g -fsanitize=address -fno-omit-frame-pointer main.c

followed by
./a.out random_chars_largefile.txt

this was the output

Async readv failed.

=================================================================
==23776==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 33337 byte(s) in 1 object(s) allocated from:
    #0 0x496aad in malloc /home/brian/src/final/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:145:3
    #1 0x4c6d16 in submit_read_request /tmp/cat_libring/main.c:102:28
    #2 0x4c72ea in main /tmp/cat_libring/main.c:156:19
    #3 0x7f804fa5bcb1 in __libc_start_main csu/../csu/libc-start.c:314:16

Indirect leak of 8531968 byte(s) in 2083 object(s) allocated from:
    #0 0x497547 in posix_memalign /home/brian/src/final/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:226:3
    #1 0x4c6de2 in submit_read_request /tmp/cat_libring/main.c:120:13
    #2 0x4c72ea in main /tmp/cat_libring/main.c:156:19
    #3 0x7f804fa5bcb1 in __libc_start_main csu/../csu/libc-start.c:314:16

SUMMARY: AddressSanitizer: 8565305 byte(s) leaked in 2084 allocation(s).

It looks like the offending code is the following

struct file_info *fi = malloc(sizeof(*fi) +
                                            (sizeof(struct iovec) * blocks));

Which causes an indirect leak with

if( posix_memalign(&buf, BLOCK_SZ, BLOCK_SZ)) {

I couldn't really wrap my mind around how to determine what the value for the align parameter should be. I know it had to be a multiple of 2 and larger than sizeof(void *) but how did you derive BLOCK_SZ should be the value? A comment in the code would really help noobs like me out :(

Very nice work here, would be nice to have some examples also when using polled IO

It seems to only register a read request when a connection is closed for whatever reason.

This output only seems to occur when I close the tab or otherwise stop the page from loading.
If I don't, it just carries on loading indefinitely, and the page doesn't send.
I'm on Ubuntu 20.04, my kernel version is 5.9.10-050910-generic, I installed liburing by cloning https://github.com/axboe/liburing, and copying the include and library files to /usr/local/include and /usr/local/lib respectively.
Any idea what the issue is?

fi->iovecs->iov_base looks wrong to me. It's equivalent to fi->iovecs[0].iov_base, but shouldn't it be fi->iovecs[i].iov_base? Won't this corrupt any file longer than 4KB?

gcc -o webserver_liburing 05_webserver_liburing/main.c -luring
./webserver_liburing
Async request failed: Invalid argument for event: 0