]> git.baikalelectronics.ru Git - kernel.git/commit
io_uring: optimise io_cqring_wait() hot path
authorPavel Begunkov <asml.silence@gmail.com>
Mon, 9 Aug 2021 15:07:32 +0000 (09:07 -0600)
committerJens Axboe <axboe@kernel.dk>
Mon, 23 Aug 2021 19:07:59 +0000 (13:07 -0600)
commitb2e489b56d32202349810616829e35268bbbfcce
tree0fd19c7dbce6daf6796c08b4c466b637625ff168
parent2edbac769bc9498e0860f50845da65dca578f54f
io_uring: optimise io_cqring_wait() hot path

Turns out we always init struct io_wait_queue in io_cqring_wait(), even
if it's not used after, i.e. there are already enough of CQEs. And often
it's exactly what happens, for instance, requests may have been
completed inline, or in case of io_uring_enter(submit=N, wait=1).

It shows up in my profiler, so optimise it by delaying the struct init.

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/6f1b81c60b947d165583dc333947869c3d85d037.1628471125.git.asml.silence@gmail.com
[axboe: fixed up for new cqring wait]
Signed-off-by: Jens Axboe <axboe@kernel.dk>
fs/io_uring.c