KVM: x86/mmu: Formalize TDP MMU's (unintended?) deferred TLB flush logic

Explicitly ignore the result of zap_gfn_range() when putting the last
reference to a TDP MMU root, and add a pile of comments to formalize the
TDP MMU's behavior of deferring TLB flushes to alloc/reuse.  Note, this
only affects the !shared case, as zap_gfn_range() subtly never returns
true for "flush" as the flush is handled by tdp_mmu_zap_spte_atomic().

Putting the root without a flush is ok because even if there are stale
references to the root in the TLB, they are unreachable because KVM will
not run the guest with the same ASID without first flushing (where ASID
in this context refers to both SVM's explicit ASID and Intel's implicit
ASID that is constructed from VPID+PCID+EPT4A+etc...).

Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-Id: <20220226001546.360188-5-seanjc@google.com>
Reviewed-by: Mingwei Zhang <mizhang@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
index edc852ca3fb9b8709fa569c78c0e2e93c1aaff10..43f92b111d7b66cce3684edbf00990006e99c001 100644 (file)
--- a/arch/x86/kvm/mmu/mmu.c
+++ b/arch/x86/kvm/mmu/mmu.c
@@ -5094,6 +5094,14 @@ int kvm_mmu_load(struct kvm_vcpu *vcpu)
        kvm_mmu_sync_roots(vcpu);
 
        kvm_mmu_load_pgd(vcpu);
+
+       /*
+        * Flush any TLB entries for the new root, the provenance of the root
+        * is unknown.  Even if KVM ensures there are no stale TLB entries
+        * for a freed root, in theory another hypervisor could have left
+        * stale entries.  Flushing on alloc also allows KVM to skip the TLB
+        * flush when freeing a root (see kvm_tdp_mmu_put_root()).
+        */
        static_call(kvm_x86_flush_tlb_current)(vcpu);
 out:
        return r;

diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c
index b97a4125feac70fd28dfed5ad5667cbc0f165e04..921fa386df9994a7690a7e36e4947113324f1979 100644 (file)
--- a/arch/x86/kvm/mmu/tdp_mmu.c
+++ b/arch/x86/kvm/mmu/tdp_mmu.c
@@ -93,7 +93,15 @@ void kvm_tdp_mmu_put_root(struct kvm *kvm, struct kvm_mmu_page *root,
        list_del_rcu(&root->link);
        spin_unlock(&kvm->arch.tdp_mmu_pages_lock);
 
-       zap_gfn_range(kvm, root, 0, -1ull, false, false, shared);
+       /*
+        * A TLB flush is not necessary as KVM performs a local TLB flush when
+        * allocating a new root (see kvm_mmu_load()), and when migrating vCPU
+        * to a different pCPU.  Note, the local TLB flush on reuse also
+        * invalidates any paging-structure-cache entries, i.e. TLB entries for
+        * intermediate paging structures, that may be zapped, as such entries
+        * are associated with the ASID on both VMX and SVM.
+        */
+       (void)zap_gfn_range(kvm, root, 0, -1ull, false, false, shared);
 
        call_rcu(&root->rcu_head, tdp_mmu_free_sp_rcu_callback);
 }