fsverity: Remove WQ_UNBOUND from fsverity read workqueue

commit f959325e6ac3f499450088b8d9c626d1177be160 upstream.

WQ_UNBOUND causes significant scheduler latency on ARM64/Android.  This
is problematic for latency sensitive workloads, like I/O
post-processing.

Removing WQ_UNBOUND gives a 96% reduction in fsverity workqueue related
scheduler latency and improves app cold startup times by ~30ms.
WQ_UNBOUND was also removed from the dm-verity workqueue for the same
reason [1].

This code was tested by running Android app startup benchmarks and
measuring how long the fsverity workqueue spent in the runnable state.

Before
Total workqueue scheduler latency: 553800us
After
Total workqueue scheduler latency: 18962us

[1]: https://lore.kernel.org/all/20230202012348.885402-1-nhuck@google.com/

Signed-off-by: Nathan Huckleberry <nhuck@google.com>
Fixes: 38486d6012ca ("fs-verity: add data verification hooks for ->readpages()")
Cc: stable@vger.kernel.org
Link: https://lore.kernel.org/r/20230310193325.620493-1-nhuck@google.com
Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

diff --git a/fs/verity/verify.c b/fs/verity/verify.c
index 3e8f2de44667f7b77e40161d600a545a51e26792..a7e24f949e3d4cc639c947541bf030427381c6a7 100644 (file)
--- a/fs/verity/verify.c
+++ b/fs/verity/verify.c
@@ -259,15 +259,15 @@ EXPORT_SYMBOL_GPL(fsverity_enqueue_verify_work);
 int __init fsverity_init_workqueue(void)
 {
        /*
-        * Use an unbound workqueue to allow bios to be verified in parallel
-        * even when they happen to complete on the same CPU.  This sacrifices
-        * locality, but it's worthwhile since hashing is CPU-intensive.
+        * Use a high-priority workqueue to prioritize verification work, which
+        * blocks reads from completing, over regular application tasks.
         *
-        * Also use a high-priority workqueue to prioritize verification work,
-        * which blocks reads from completing, over regular application tasks.
+        * For performance reasons, don't use an unbound workqueue.  Using an
+        * unbound workqueue for crypto operations causes excessive scheduler
+        * latency on ARM64.
         */
        fsverity_read_workqueue = alloc_workqueue("fsverity_read_queue",
-                                                 WQ_UNBOUND | WQ_HIGHPRI,
+                                                 WQ_HIGHPRI,
                                                  num_online_cpus());
        if (!fsverity_read_workqueue)
                return -ENOMEM;