]> git.baikalelectronics.ru Git - kernel.git/commit
nvme-pci: handle completions outside of the queue lock
authorJens Axboe <axboe@kernel.dk>
Thu, 17 May 2018 16:31:50 +0000 (18:31 +0200)
committerKeith Busch <keith.busch@intel.com>
Fri, 18 May 2018 20:41:36 +0000 (14:41 -0600)
commit0199587278ee94557d9b1feea6a4501701998ff4
tree0005cf154368aa9f1b5c655ca903403b49b0bb1c
parent8bb01b40f6db77beb58406779ea933f1ebb04e51
nvme-pci: handle completions outside of the queue lock

Split the completion of events into a two part process:

1) Reap the events inside the queue lock
2) Complete the events outside the queue lock

Since we never wrap the queue, we can access it locklessly after we've
updated the completion queue head. This patch started off with batching
events on the stack, but with this trick we don't have to. Keith Busch
<keith.busch@intel.com> came up with that idea.

Note that this kills the ->cqe_seen as well. I haven't been able to
trigger any ill effects of this. If we do race with polling every so
often, it should be rare enough NOT to trigger any issues.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
Signed-off-by: Keith Busch <keith.busch@intel.com>
[hch: refactored, restored poll early exit optimization]
Signed-off-by: Christoph Hellwig <hch@lst.de>
drivers/nvme/host/pci.c