]> git.baikalelectronics.ru Git - kernel.git/commit
bio: skip atomic inc/dec of ->bi_remaining for non-chains
authorJens Axboe <axboe@fb.com>
Fri, 17 Apr 2015 22:15:18 +0000 (16:15 -0600)
committerJens Axboe <axboe@fb.com>
Tue, 5 May 2015 19:32:47 +0000 (13:32 -0600)
commit7ac97e74b4c00714e1645337e49acd0c180d28f1
treec3ff2d0d2db5f7392e8651b4f0d2958895faf8b2
parent366d0505237bef8f867a0835d2b066fe7490bd91
bio: skip atomic inc/dec of ->bi_remaining for non-chains

Struct bio has an atomic ref count for chained bio's, and we use this
to know when to end IO on the bio. However, most bio's are not chained,
so we don't need to always introduce this atomic operation as part of
ending IO.

Add a helper to elevate the bi_remaining count, and flag the bio as
now actually needing the decrement at end_io time. Rename the field
to __bi_remaining to catch any current users of this doing the
incrementing manually.

For high IOPS workloads, this reduces the overhead of bio_endio()
substantially.

Tested-by: Robert Elliott <elliott@hp.com>
Acked-by: Kent Overstreet <kent.overstreet@gmail.com>
Reviewed-by: Jan Kara <jack@suse.cz>
Signed-off-by: Jens Axboe <axboe@fb.com>
block/bio.c
drivers/md/dm-cache-target.c
drivers/md/dm-raid1.c
drivers/md/dm-snap.c
drivers/md/dm-thin.c
include/linux/bio.h
include/linux/blk_types.h