io_uring: add fast path for io_run_local_work()

We'll grab uring_lock and call __io_run_local_work() with several
atomics inside even if there are no task works. Skip it if ->work_llist
is empty.

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Reviewed-by: Dylan Yudaken <dylany@fb.com>
Link: https://lore.kernel.org/r/f6a885f372bad2d77d9cd87341b0a86a4000c0ff.1662652536.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>

diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c
index 12e8acd30096759f37cb43ad35cd87a2886aa58f..433466455a5f666917830eed2aca9070f8edfa69 100644 (file)
--- a/io_uring/io_uring.c
+++ b/io_uring/io_uring.c
@@ -1208,6 +1208,9 @@ int io_run_local_work(struct io_ring_ctx *ctx)
        bool locked;
        int ret;
 
+       if (llist_empty(&ctx->work_llist))
+               return 0;
+
        locked = mutex_trylock(&ctx->uring_lock);
        ret = __io_run_local_work(ctx, locked);
        if (locked)