]> git.baikalelectronics.ru Git - kernel.git/commit
arm64: atomics: avoid out-of-line ll/sc atomics
authorAndrew Murray <andrew.murray@arm.com>
Wed, 28 Aug 2019 17:50:07 +0000 (18:50 +0100)
committerWill Deacon <will@kernel.org>
Thu, 29 Aug 2019 14:53:42 +0000 (15:53 +0100)
commitb81ef975a065a5efaa8890f5f6470e560607c0ed
treee3bc9622b3b208a628d081e37be257ab5ad99a84
parenta4cd33331be6aac0379613aef37bfe084e8b9308
arm64: atomics: avoid out-of-line ll/sc atomics

When building for LSE atomics (CONFIG_ARM64_LSE_ATOMICS), if the hardware
or toolchain doesn't support it the existing code will fallback to ll/sc
atomics. It achieves this by branching from inline assembly to a function
that is built with special compile flags. Further this results in the
clobbering of registers even when the fallback isn't used increasing
register pressure.

Improve this by providing inline implementations of both LSE and
ll/sc and use a static key to select between them, which allows for the
compiler to generate better atomics code. Put the LL/SC fallback atomics
in their own subsection to improve icache performance.

Signed-off-by: Andrew Murray <andrew.murray@arm.com>
Signed-off-by: Will Deacon <will@kernel.org>
arch/arm64/include/asm/atomic.h
arch/arm64/include/asm/atomic_arch.h [new file with mode: 0644]
arch/arm64/include/asm/atomic_ll_sc.h
arch/arm64/include/asm/atomic_lse.h
arch/arm64/include/asm/cmpxchg.h
arch/arm64/include/asm/lse.h