mm: mmap: register suitable readonly file vmas for khugepaged

The readonly FS THP relies on khugepaged to collapse THP for suitable
vmas.  But the behavior is inconsistent for "always" mode
(https://lore.kernel.org/linux-mm/00f195d4-d039-3cf2-d3a1-a2c88de397a0@suse.cz/).

The "always" mode means THP allocation should be tried all the time and
khugepaged should try to collapse THP all the time.  Of course the
allocation and collapse may fail due to other factors and conditions.

Currently file THP may not be collapsed by khugepaged even though all the
conditions are met.  That does break the semantics of "always" mode.

So make sure readonly FS vmas are registered to khugepaged to fix the
break.

Register suitable vmas in common mmap path, that could cover both readonly
FS vmas and shmem vmas, so remove the khugepaged calls in shmem.c.

Still need to keep the khugepaged call in vma_merge() since vma_merge() is
called in a lot of places, for example, madvise, mprotect, etc.

Link: https://lkml.kernel.org/r/20220510203222.24246-9-shy828301@gmail.com
Signed-off-by: Yang Shi <shy828301@gmail.com>
Reported-by: Vlastimil Babka <vbabka@suse.cz>
Acked-by: Vlastmil Babka <vbabka@suse.cz>
Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: Miaohe Lin <linmiaohe@huawei.com>
Cc: Song Liu <songliubraving@fb.com>
Cc: Rik van Riel <riel@surriel.com>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Zi Yan <ziy@nvidia.com>
Cc: Theodore Ts'o <tytso@mit.edu>
Cc: Song Liu <song@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

diff --git a/mm/mmap.c b/mm/mmap.c
index 4456bf81e11eb3673df36b0da9caa9623f540e65..2b9305ed0dda6df1f9b42c92572cd2dd0a82e0e1 100644 (file)
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -1847,6 +1847,13 @@ unsigned long mmap_region(struct file *file, unsigned long addr,
        }
 
        vma_link(mm, vma, prev, rb_link, rb_parent);
+
+       /*
+        * vma_merge() calls khugepaged_enter_vma() either, the below
+        * call covers the non-merge case.
+        */
+       khugepaged_enter_vma(vma, vma->vm_flags);
+
        /* Once vma denies write, undo our temporary denial count */
 unmap_writable:
        if (file && vm_flags & VM_SHARED)

diff --git a/mm/shmem.c b/mm/shmem.c
index 89f6f4fec3f9a0a472e1db3269ee2cd6d0c53181..67a3f3b05fb2e43dd1791a7659183225bb58014b 100644 (file)
--- a/mm/shmem.c
+++ b/mm/shmem.c
@@ -34,7 +34,6 @@
 #include <linux/export.h>
 #include <linux/swap.h>
 #include <linux/uio.h>
-#include <linux/khugepaged.h>
 #include <linux/hugetlb.h>
 #include <linux/fs_parser.h>
 #include <linux/swapfile.h>
@@ -2232,7 +2231,6 @@ static int shmem_mmap(struct file *file, struct vm_area_struct *vma)
 
        file_accessed(file);
        vma->vm_ops = &shmem_vm_ops;
-       khugepaged_enter_vma(vma, vma->vm_flags);
        return 0;
 }
 
@@ -4133,8 +4131,6 @@ int shmem_zero_setup(struct vm_area_struct *vma)
        vma->vm_file = file;
        vma->vm_ops = &shmem_vm_ops;
 
-       khugepaged_enter_vma(vma, vma->vm_flags);
-
        return 0;
 }