]> git.baikalelectronics.ru Git - kernel.git/commit
io_uring: move ctx->flags from SQ cacheline
authorPavel Begunkov <asml.silence@gmail.com>
Mon, 14 Jun 2021 22:37:21 +0000 (23:37 +0100)
committerJens Axboe <axboe@kernel.dk>
Tue, 15 Jun 2021 21:38:39 +0000 (15:38 -0600)
commitbf595693c5069e906021c86189326f65d5cd5f37
tree38f2cd1973044f51ed261d0f81452e46d7b40cdb
parentbf4e6c71f059acfc7053f080cbea18e365befb92
io_uring: move ctx->flags from SQ cacheline

ctx->flags are heavily used by both, completion and submission sides, so
move it out from the ctx fields related to submissions. Instead, place
it together with ctx->refs, because it's already cacheline-aligned and
so pads lots of space, and both almost never change. Also, in most
occasions they are accessed together as refs are taken at submission
time and put back during completion.

Do same with ctx->rings, where the pointer itself is never modified
apart from ring init/free.

Note: in percpu mode, struct percpu_ref doesn't modify the struct itself
but takes indirection with ref->percpu_count_ptr.

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/4c48c173e63d35591383ba2b87e8b8e8dfdbd23d.1623709150.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
fs/io_uring.c