]> git.baikalelectronics.ru Git - kernel.git/commit
sched/core: Use try_cmpxchg in set_nr_{and_not,if}_polling
authorUros Bizjak <ubizjak@gmail.com>
Wed, 29 Jun 2022 15:15:52 +0000 (17:15 +0200)
committerPeter Zijlstra <peterz@infradead.org>
Mon, 4 Jul 2022 07:23:08 +0000 (09:23 +0200)
commitf2134a9f9a53499da79e6a86aadf1e6e48be9ac4
tree6264e7eb7c29978562f5e24f296523e776d9c1dd
parentfaa15b1608344c58889907ee7b1326d73c92a1d6
sched/core: Use try_cmpxchg in set_nr_{and_not,if}_polling

Use try_cmpxchg instead of cmpxchg (*ptr, old, new) != old in
set_nr_{and_not,if}_polling. x86 cmpxchg returns success in ZF flag,
so this change saves a compare after cmpxchg.

The definition of cmpxchg based fetch_or was changed in the
same way as atomic_fetch_##op definitions were changed
in 0bbeb7d6bfe7001fbe7829ca61cd3beb9e3a7479.

Also declare these two functions as inline to ensure inlining. In the
case of set_nr_and_not_polling, the compiler (gcc) tries to outsmart
itself by constructing the boolean return value with logic operations
on the fetched value, and these extra operations enlarge the function
over the inlining threshold value.

Signed-off-by: Uros Bizjak <ubizjak@gmail.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/20220629151552.6015-1-ubizjak@gmail.com
kernel/sched/core.c