mm: zero-seek shrinkers

The page cache and most shrinkable slab caches hold data that has been
read from disk, but there are some caches that only cache CPU work, such
as the dentry and inode caches of procfs and sysfs, as well as the subset
of radix tree nodes that track non-resident page cache.

Currently, all these are shrunk at the same rate: using DEFAULT_SEEKS for
the shrinker's seeks setting tells the reclaim algorithm that for every
two page cache pages scanned it should scan one slab object.

This is a bogus setting.  A virtual inode that required no IO to create is
not twice as valuable as a page cache page; shadow cache entries with
eviction distances beyond the size of memory aren't either.

In most cases, the behavior in practice is still fine.  Such virtual
caches don't tend to grow and assert themselves aggressively, and usually
get picked up before they cause problems.  But there are scenarios where
that's not true.

Our database workloads suffer from two of those.  For one, their file
workingset is several times bigger than available memory, which has the
kernel aggressively create shadow page cache entries for the non-resident
parts of it.  The workingset code does tell the VM that most of these are
expendable, but the VM ends up balancing them 2:1 to cache pages as per
the seeks setting.  This is a huge waste of memory.

These workloads also deal with tens of thousands of open files and use
/proc for introspection, which ends up growing the proc_inode_cache to
absurdly large sizes - again at the cost of valuable cache space, which
isn't a reasonable trade-off, given that proc inodes can be re-created
without involving the disk.

This patch implements a "zero-seek" setting for shrinkers that results in
a target ratio of 0:1 between their objects and IO-backed caches.  This
allows such virtual caches to grow when memory is available (they do
cache/avoid CPU work after all), but effectively disables them as soon as
IO-backed objects are under pressure.

It then switches the shrinkers for procfs and sysfs metadata, as well as
excess page cache shadow nodes, to the new zero-seek setting.

Link: http://lkml.kernel.org/r/20181009184732.762-5-hannes@cmpxchg.org
Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>
Reported-by: Domas Mituzas <dmituzas@fb.com>
Reviewed-by: Andrew Morton <akpm@linux-foundation.org>
Reviewed-by: Rik van Riel <riel@surriel.com>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

diff --git a/fs/kernfs/mount.c b/fs/kernfs/mount.c
index ff2716f9322e673d7a3e701c764c0f859ed84051..fdf527b6d79c8b06f9fb29ed6287ecc96d5b0e80 100644 (file)
--- a/fs/kernfs/mount.c
+++ b/fs/kernfs/mount.c
@@ -236,6 +236,9 @@ static int kernfs_fill_super(struct super_block *sb, unsigned long magic)
                sb->s_export_op = &kernfs_export_ops;
        sb->s_time_gran = 1;
 
+       /* sysfs dentries and inodes don't require IO to create */
+       sb->s_shrink.seeks = 0;
+
        /* get root inode, initialize and unlock it */
        mutex_lock(&kernfs_mutex);
        inode = kernfs_get_inode(sb, info->root->kn);

diff --git a/fs/proc/inode.c b/fs/proc/inode.c
index fc5306a31a1d8b489dc16483d79cf83b6445f387..5792f9e39466ef4420720c737b3a010da17a986f 100644 (file)
--- a/fs/proc/inode.c
+++ b/fs/proc/inode.c
@@ -516,6 +516,9 @@ int proc_fill_super(struct super_block *s, void *data, int silent)
         */
        s->s_stack_depth = FILESYSTEM_MAX_STACK_DEPTH;
        
+       /* procfs dentries and inodes don't require IO to create */
+       s->s_shrink.seeks = 0;
+
        pde_get(&proc_root);
        root_inode = proc_get_inode(s, &proc_root);
        if (!root_inode) {

diff --git a/mm/vmscan.c b/mm/vmscan.c
index 8ea87586925e7c0672ca87352427c95d3bd5faf4..28c9ae5633b9899be8b4f2dd05bb9070238007e4 100644 (file)
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -474,9 +474,18 @@ static unsigned long do_shrink_slab(struct shrink_control *shrinkctl,
        nr = atomic_long_xchg(&shrinker->nr_deferred[nid], 0);
 
        total_scan = nr;
-       delta = freeable >> priority;
-       delta *= 4;
-       do_div(delta, shrinker->seeks);
+       if (shrinker->seeks) {
+               delta = freeable >> priority;
+               delta *= 4;
+               do_div(delta, shrinker->seeks);
+       } else {
+               /*
+                * These objects don't require any IO to create. Trim
+                * them aggressively under memory pressure to keep
+                * them from causing refetches in the IO caches.
+                */
+               delta = freeable / 2;
+       }
 
        /*
         * Make sure we apply some minimal pressure on default priority

diff --git a/mm/workingset.c b/mm/workingset.c
index 7e6ef312cea598c3308a7ad10b26861a40e2af28..cbc13d4dfa795d66931c4300fb36fa770e4827d1 100644 (file)
--- a/mm/workingset.c
+++ b/mm/workingset.c
@@ -534,7 +534,7 @@ static unsigned long scan_shadow_nodes(struct shrinker *shrinker,
 static struct shrinker workingset_shadow_shrinker = {
        .count_objects = count_shadow_nodes,
        .scan_objects = scan_shadow_nodes,
-       .seeks = DEFAULT_SEEKS,
+       .seeks = 0, /* ->count reports only fully expendable nodes */
        .flags = SHRINKER_NUMA_AWARE | SHRINKER_MEMCG_AWARE,
 };