shuveb / io_uring-by-example Goto Github PK
View Code? Open in Web Editor NEWA companion repository for the io_uring by Example article series
License: MIT License
A companion repository for the io_uring by Example article series
License: MIT License
please see my debug screen.
it goes into below error path, cqe.res value is strange. looks like no one assign it.
please guide me.
intel@wcp116:/wayne/miniobin/distributedminioBWprofile$ uname -a/wayne/miniobin/distributedminioBWprofile$
Linux wcp116 5.4.0-42-generic #46-Ubuntu SMP Fri Jul 10 00:24:02 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
intel@wcp116:
intel@wcp116:~/wayne/io_uring-by-example$ sudo ./webserver_liburing
hello from Wayne debug here!
2
Async request failed: Invalid argument for event: 0
The current copy_file() does not wait for all submitted I/O's to complete before returning to main() which the latter closes the files and calls io_uring_queue_exit(). Any I/Os which are still in progress will be lost. This manifests as the output file having fewer bytes than the input file. This is tested and is repeatable in Ubuntu 20 running in a VM under VMWare Workstation, using liburing2_2.1-2build1_amd64.deb.
One way to fix this is to add the following codes before the return statement at the end of copy_file() to wait for all I/Os to complete before returning:
int num_pending = reads + writes; // # of I/O requests still pending.
for (int i=0; i<num_pending; i++) { // Wait for them to complete or else some may be discarded upon premature io_uring exit.
io_uring_wait_cqe(ring, &cqe);
io_uring_cqe_get_data(cqe);
io_uring_cqe_seen(ring, cqe);
}
Cheers,
Jack
Reading the comment above the function copy_file_contents()
it says the following.
/*
* Once a static file is identified to be served, this function is used to read the file
* and write it over the client socket using Linux's sendfile() system call. This saves us
* the hassle of transferring file buffers from kernel to user space and back.
* */
But the function itself loads the entire file at once into userspace memory.
The function does not use sendfile()
.
It seems to only register a read request when a connection is closed for whatever reason.
This output only seems to occur when I close the tab or otherwise stop the page from loading.
If I don't, it just carries on loading indefinitely, and the page doesn't send.
I'm on Ubuntu 20.04, my kernel version is 5.9.10-050910-generic, I installed liburing by cloning https://github.com/axboe/liburing, and copying the include and library files to /usr/local/include and /usr/local/lib respectively.
Any idea what the issue is?
Hey there, thanks for putting these examples together. I'm practicing io_uring and you're resources have been invaluable. Your site is currently down though :( I had to use archive.org.
Anyways, I'm having trouble running example 3 on files that are larger than 8530944 bytes. Calculated by sum total of posix_memalign
successes before crash which was 8331 * BLOCK_SZ on my machine.
I'm using WSL2
and Ubuntu 20.10
with Linux Kernel 5.4.81-microsoft-standard-WSL2+
My C is really rusty and it's late which is why I'm reaching out.
I ran the build cmd
clang -luring -g -fsanitize=address -fno-omit-frame-pointer main.c
followed by
./a.out random_chars_largefile.txt
this was the output
Async readv failed.
=================================================================
==23776==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 33337 byte(s) in 1 object(s) allocated from:
#0 0x496aad in malloc /home/brian/src/final/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:145:3
#1 0x4c6d16 in submit_read_request /tmp/cat_libring/main.c:102:28
#2 0x4c72ea in main /tmp/cat_libring/main.c:156:19
#3 0x7f804fa5bcb1 in __libc_start_main csu/../csu/libc-start.c:314:16
Indirect leak of 8531968 byte(s) in 2083 object(s) allocated from:
#0 0x497547 in posix_memalign /home/brian/src/final/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:226:3
#1 0x4c6de2 in submit_read_request /tmp/cat_libring/main.c:120:13
#2 0x4c72ea in main /tmp/cat_libring/main.c:156:19
#3 0x7f804fa5bcb1 in __libc_start_main csu/../csu/libc-start.c:314:16
SUMMARY: AddressSanitizer: 8565305 byte(s) leaked in 2084 allocation(s).
It looks like the offending code is the following
struct file_info *fi = malloc(sizeof(*fi) +
(sizeof(struct iovec) * blocks));
Which causes an indirect leak with
if( posix_memalign(&buf, BLOCK_SZ, BLOCK_SZ)) {
I couldn't really wrap my mind around how to determine what the value for the align
parameter should be. I know it had to be a multiple of 2 and larger than sizeof(void *)
but how did you derive BLOCK_SZ
should be the value? A comment in the code would really help noobs like me out :(
io_uring-by-example/02_cat_uring/main.c
Line 222 in 8d069f0
fi->iovecs->iov_base
looks wrong to me. It's equivalent to fi->iovecs[0].iov_base
, but shouldn't it be fi->iovecs[i].iov_base
? Won't this corrupt any file longer than 4KB?
tried install liburing with
https://hnakamur.github.io/blog/2020/04/07/tried_io_uring_example/
but got error below
can you provide instructions to test?
./webserver_liburing 83123
Async request failed: Invalid argument for event: 0
Nice work! How do u show the remote ip address of the client?
Hi,
it's possible edit the web server to have file access async with io_uring? (open/read/close)
Cheers,
gcc -o webserver_liburing 05_webserver_liburing/main.c -luring
./webserver_liburing
Async request failed: Invalid argument for event: 0
Very nice work here, would be nice to have some examples also when using polled IO
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.