FileChannel Implementation

[manticore-projects]

Greetings!

The idea of JUring is fantastic but I feel broader adoption will benefit from an implementation of the FileChannel interface.
For example, the Java H2 database allows plugin in any FileChannel implementation directly and certainly would benefit from JUring on Linux.

So I have started some work: https://github.com/manticore-projects/JUring/tree/filechannel

• Gradle support for better JMH integration (gradle jmh is just simple)
• JUringFileChannel
• Unit tests
• JMH Benchmark

Its very early POC and basically I have no idea what I am doing, but at least some basic tests work.

I am facing one big challenge though: FileChannel.transferTo() is much slower for the JUring backend compared to the standard NIO -- and I do not understand why.

Benchmark                                Mode  Cnt   Score   Error   Units
FileTransferBenchmark.juringTransferTo  thrpt    3   7.208 ± 3.178  ops/ms
FileTransferBenchmark.nioTransferTo     thrpt    3  27.936 ± 6.665  ops/ms

Certainly I am doing things wrong but I would kindly ask for some pointers please.

[davidtos]

Wow! This is really awesome! I Fully agree that it would benefit from a Filechannel implementation. I didn't have time yet to check out the code and give it a try myself (will do later this week).

There is quite a gap between those it seems. My first guess would be the sequential reads, JUring doesn't perform those as good as Random reads. The MAX_IN_FLIGHT might also cause some performance degradation but I would have to try it to be sure.

Another path (maybe) worth exploring is that if the source and target FilleChannels are both from JUring it could just pass the pointer of the buffer to the write operation. This would save a lot object creations and maybe give a little boost. Will also check out the current Filechannel implementation to see what that is doing to get that score.

[manticore-projects]

Thanks for the quick roundturn @davidtos, appreciated.
From my understanding, NIO Filechannel on Linux uses Zero-Copy approach and thus avoids userspace while my first sketch is all userspace of course.

So two things are needed indeed:

1. if JUring provides any Zero-copy facility, we could use it when both source and target are JUring backed
2. in the meantime for a more realistic benchmark we need forcefully avoid Zero-Copy by setting a non NIO target

As I said, I am just trying to understand all this stuff and I am not to concerned on transferFrom() and transferTo() since H2 database won't use it. I saw it more as a low hanging fruit for benchmarking.

[manticore-projects]

We have just started!
I will keep trying and keep you posted on any progress, even when small.

Cheers and best

[manticore-projects]

I made some progress with reserving and reusing buffers via:

        // int io_uring_register_buffers(struct io_uring *ring, const struct iovec *iovecs, unsigned nr_iovecs);
        IO_URING_REGISTER_BUFFERS = FunctionDescriptor.of(
                JAVA_INT,  // Return type: int
                ADDRESS,   // io_uring*
                ADDRESS,   // iovec*
                JAVA_INT   // nr_iovecs
        );

        registerBuffers = linker.downcallHandle(
                liburing.find("io_uring_register_buffers").orElseThrow(),
                IO_URING_REGISTER_BUFFERS
        );

        // int io_uring_unregister_buffers(struct io_uring *ring);
        IO_URING_UNREGISTER_BUFFERS = FunctionDescriptor.of(
                JAVA_INT,
                ADDRESS // io_uring*
        );

        unregisterBuffers = linker.downcallHandle(
                liburing.find("io_uring_unregister_buffers").orElseThrow(),
                IO_URING_UNREGISTER_BUFFERS
        );

        IO_URING_PREP_READ_FIXED =
                FunctionDescriptor.ofVoid(
                        ADDRESS,  // sqe
                        JAVA_INT, // fd
                        ADDRESS,  // void *buf
                        JAVA_INT, // len
                        JAVA_LONG,// offset
                        JAVA_INT  // buf_index
                );

        io_uring_prep_read_fixed = linker.downcallHandle(
                liburing.find("io_uring_prep_read_fixed").orElseThrow(),
                IO_URING_PREP_READ_FIXED
        );

Though I wonder why your code does never call:

MemorySegment dataPtr = (MemorySegment) io_uring_cqe_get_data.invoke(nativeCqe);

But instead uses:

var nativeCqe = cqePtr.getAtIndex(ADDRESS, 0).reinterpret(io_uring_cqe_layout.byteSize());
long userData = nativeCqe.get(ValueLayout.JAVA_LONG, 0);
int res = nativeCqe.get(ValueLayout.JAVA_INT, 8);

Can you please help me with some explanation on this?

[manticore-projects]

I got a simple file-transfer working with reading into pre-allocated, reusable buffers. Please see https://github.com/manticore-projects/JUring/blob/d4604464d367757d2b89bfec158bf3612d7d2285/src/main/java/com/davidvlijmincx/lio/api/JUringFileChannel.java#L284

So far I have 2 major questions:
a) why not calling "io_uring_cqe_get_data"
b) why not freeing those buffers at the end

I have also added the equivalent C code for guidance.

My next step would be to use those pre-allocated buffers for writing too. Stay tuned.

[davidtos]

That is some really cool progress!

A) If I remember correctly using those two lines was faster than calling io_uring_cqe_get_data during benchmarks. It is worth revisiting as that one line looks nicer.
B) When unregistering the buffers? or the userData buffers? The userData gets freed in getResultFromCqe. I think freeing the buffers when unregistering them is fine for now.

I hope this answers your question, if not please let me know

In an effort to get JUring more performant I am trying to switch to a pool of buffers. Looking at your code I think using registered buffers is a very good idea! And something that should be used at more places depending on the performance. To support the idea I will add a pool of registered buffers making working with them a bit easier.

[manticore-projects]

In an effort to get JUring more performant I am trying to switch to a pool of buffers.

Also, the FileDescriptor(s) should be registered to avoid overhead. I am working on this too.

[manticore-projects]

Greetings!

I made quite some progress and have developed a better understanding of libUring now:

• I have mapped a lot of additional functions and constants around Pipe and Fixed File Descriptors
• I have written some plain C code as reference for ZeroCopy file transfer (w/ benchmarks)
• I have successfully mapped that ZeroCopy file transfer into JUring with tests (successful) and benchmarks (failing, no idea why)

I would like to ask to have a look at my code at https://github.com/manticore-projects/JUring/tree/filechannel and kindly give some feedback. I am confident that some of this stuff can be merged and I am more than willing to iron out any obstacles before sending a PR.

Thanks and cheers!

[davidtos]

Thanks for doing all of this! I will take a look at it today or tomorrow run some benchmarks and get back to you as soon as possible! From a quick glance it already looks really good.

[manticore-projects]

Please lets agree especially on the approach of where/how to manage the C code examples. At least to me this is useful documentation and proof of concept and I found that Java wrapping much easier after looking at working C code.

[davidtos]

I looked at the filechannel implementation last weekend. The code looks good, but I couldn't get it to outperform Java's filechannel yet. Still trying to figure out why. For the C code examples, a dedicated top-level directory would be my first guess.
Still looking into why the benchmarks are failing too.

[manticore-projects]

Greetings!

Thanks for feedback, I did some small tweaks:

• JUring needs to called with a rather large queueDepth, e.g. JUring(1000) instead of JUring(10)
• for any reason, we can't loop our tests too often. Copying a file 1000x works, copying it more often will fail. I assume OS cache/resource allocation plays foul.
• so for any JMH benchmark, we need to limit the number of executions and skip warmup

In the result, for a large 170MB file I achieve numbers that make sense:

Benchmark               Mode  Cnt  Score   Error   Units
jUringCopy             thrpt    3  0.017 ± 0.004  ops/ms
jUringCopyFixedFiles   thrpt    3  0.016 ± 0.001  ops/ms
jUringTransferTo       thrpt    3  0.007 ± 0.004  ops/ms
nioTransferTo          thrpt    3  0.007 ± 0.003  ops/ms
nioTransferToZeroCopy  thrpt    3  0.017 ± 0.001  ops/ms

JUring Zero-Copy is on par with NIO Zero Copy Transfer.
And even my first stupid block by block transfer (not optimised yet) is on par with NIO Non-Zero-Copy.

What baffles me is the fact that "Fixed Files" are a bit slower than dynamic File Descriptors in Java, but much faster in C.
Though the overhead of the Java Wrapper may outweigh is actual benefit.

[manticore-projects]

Further tweaking the JMH parameters and things start looking good:

Benchmark	Mode	Threads	Samples	Score	Score Error (99.9%)	Unit
jUringCopy	ss	1	200	68.406348	1.405607	ms/op
jUringCopyFixedFiles	ss	1	200	66.854152	0.860164	ms/op
jUringTransferTo	ss	1	200	143.726446	3.302417	ms/op
nioTransferTo	ss	1	200	154.855567	37.892569	ms/op
nioTransferToZeroCopy	ss	1	200	67.54023	0.875534	ms/op

Although it appears to me that JMH is not the best tool for testing any FFM methods.
We may need to fall back to simple loop based implementations.