Hello,
Thank you for your great library and I'm looking forward to new Cython version.
I'm interested in io_uring, and now I've been writing a asyncio wrapper for this library and file operation API.
https://github.com/tree4096/uring_file (forked from https://github.com/qweeze/uring_file)
Can I contribute my code on your library or community of io_uring developers?
After implemeting some optimization, I achieved relatively better performance than other async file I/O libs such as aiofiles, aiofile, and anyio.
The results of performance test as below.
Test code
https://github.com/tree4096/uring_file/blob/master/performance_test.py
Writing Test
Library |
Time [s] |
Process Time [s] |
anyio |
10.69 |
15.62 |
aiofiles |
4.47 |
6.19 |
uring_file ⭐ |
1.92 |
1.88 |
aiofile* |
7.51 |
15.08 |
Reading Test
Library |
Time [s] |
Process Time [s] |
anyio |
0.482 |
0.816 |
aiofiles |
0.349 |
0.623 |
uring_file ⭐ |
0.118 |
0.118 |
aiofile* |
0.473 |
0.828 |
*aiofile has Linux AIO mode, but does not work on Ubuntu due to the bug. So it run on C threading mode.
Environment
OS: Ubuntu 20.04.4 LTS on WSL2
Kernel ver.: 5.10.102.1-microsoft-standard-WSL2
CPU: AMD Ryzen 7 5800U
SSD: MTFDHBA1T0TDV-1AZ1AABHA
Python ver.: 3.10.5
My library provides simple and efficient APIs to access the io_uring like a database session and the file I/O APIs are written over them.
Here is the sample code.
# Create new Uring
uring = uring_file.Uring(sq_size=8, cq_size=64, session_sq_size=4)
# Get SQE and submit (Open file)
async with uring.session() as session:
sqe = session.get_sqe()
sqe.prep_openat(liburing.AT_FDCWD, b"hello.txt", os.O_RDONLY, 0o644)
# Get result of CQE
fd = sqe.result()
print("FD:", fd)
# Close file
async with uring.session() as session:
# Raw SQE object can be accessed from the _sqe property
liburing.io_uring_prep_close(session.get_sqe()._sqe, fd)
# Same expression as session.get_sqe().prep_close(fd)
# Get default Uring
default_uring = uring_file.get_default_uring()
The SQEs of the sessions are automatically merged and submitted together under high frequency access, and I call it "auto bulk submit".
The auto bulk submit is realized by controlling the task scheduling of asyncio in a little hacky way and it makes large performance improvement.
And the overflow protection for the CQ is also implemented by using Semaphore.
I'm not sure because I'm just a beginner but I think these APIs may be useful for developing the io_uring applications on Python.
If you are interested in my library, could you give me some comments or advice?