]> git.baikalelectronics.ru Git - kernel.git/commit
io_uring: optimise io_cqring_wait() hot path
authorPavel Begunkov <asml.silence@gmail.com>
Mon, 9 Aug 2021 15:07:32 +0000 (09:07 -0600)
committerJens Axboe <axboe@kernel.dk>
Mon, 23 Aug 2021 19:07:59 +0000 (13:07 -0600)
commit0912b61a70f7c9208fece090949d64cd629cd1c8
tree0fd19c7dbce6daf6796c08b4c466b637625ff168
parent160ca03536e70e9fec74a5bcd8909e72446a60c0
io_uring: optimise io_cqring_wait() hot path

Turns out we always init struct io_wait_queue in io_cqring_wait(), even
if it's not used after, i.e. there are already enough of CQEs. And often
it's exactly what happens, for instance, requests may have been
completed inline, or in case of io_uring_enter(submit=N, wait=1).

It shows up in my profiler, so optimise it by delaying the struct init.

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/6f1b81c60b947d165583dc333947869c3d85d037.1628471125.git.asml.silence@gmail.com
[axboe: fixed up for new cqring wait]
Signed-off-by: Jens Axboe <axboe@kernel.dk>
fs/io_uring.c