From: Sean Christopherson <seanjc@google.com> Date: Sat, 26 Feb 2022 00:15:42 +0000 (+0000) Subject: KVM: x86/mmu: WARN on any attempt to atomically update REMOVED SPTE X-Git-Tag: baikal/aarch64/sdk6.1~4409^2~16 X-Git-Url: https://git.baikalelectronics.ru/sdk/?a=commitdiff_plain;h=4399a375ff53347160f60045e7fb3c6a34a70479;p=kernel.git KVM: x86/mmu: WARN on any attempt to atomically update REMOVED SPTE Disallow calling tdp_mmu_set_spte_atomic() with a REMOVED "old" SPTE. This solves a conundrum introduced by commit acf149c82ae9 ("KVM: x86/mmu: Automatically update iter->old_spte if cmpxchg fails"); if the helper doesn't update old_spte in the REMOVED case, then theoretically the caller could get stuck in an infinite loop as it will fail indefinitely on the REMOVED SPTE. E.g. until recently, clear_dirty_gfn_range() didn't check for a present SPTE and would have spun until getting rescheduled. In practice, only the page fault path should "create" a new SPTE, all other paths should only operate on existing, a.k.a. shadow present, SPTEs. Now that the page fault path pre-checks for a REMOVED SPTE in all cases, require all other paths to indirectly pre-check by verifying the target SPTE is a shadow-present SPTE. Note, this does not guarantee the actual SPTE isn't REMOVED, nor is that scenario disallowed. The invariant is only that the caller mustn't invoke tdp_mmu_set_spte_atomic() if the SPTE was REMOVED when last observed by the caller. Cc: David Matlack <dmatlack@google.com> Signed-off-by: Sean Christopherson <seanjc@google.com> Message-Id: <20220226001546.360188-25-seanjc@google.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> --- diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index cd6ceca5f2ba8..af60922906ef0 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -612,16 +612,15 @@ static inline int tdp_mmu_set_spte_atomic(struct kvm *kvm, u64 *sptep = rcu_dereference(iter->sptep); u64 old_spte; - WARN_ON_ONCE(iter->yielded); - - lockdep_assert_held_read(&kvm->mmu_lock); - /* - * Do not change removed SPTEs. Only the thread that froze the SPTE - * may modify it. + * The caller is responsible for ensuring the old SPTE is not a REMOVED + * SPTE. KVM should never attempt to zap or manipulate a REMOVED SPTE, + * and pre-checking before inserting a new SPTE is advantageous as it + * avoids unnecessary work. */ - if (is_removed_spte(iter->old_spte)) - return -EBUSY; + WARN_ON_ONCE(iter->yielded || is_removed_spte(iter->old_spte)); + + lockdep_assert_held_read(&kvm->mmu_lock); /* * Note, fast_pf_fix_direct_spte() can also modify TDP MMU SPTEs and