]> git.baikalelectronics.ru Git - kernel.git/commit
bpf: Fix fexit trampoline.
authorAlexei Starovoitov <ast@kernel.org>
Tue, 16 Mar 2021 21:00:07 +0000 (14:00 -0700)
committerDaniel Borkmann <daniel@iogearbox.net>
Wed, 17 Mar 2021 23:22:51 +0000 (00:22 +0100)
commite21aa341785c679dd409c8cb71f864c00fe6c463
tree36b48928355e2c8b7fe659f2cd83e49c9ba7eadd
parent0a13e3537ea67452d549a6a80da3776d6b7dedb3
bpf: Fix fexit trampoline.

The fexit/fmod_ret programs can be attached to kernel functions that can sleep.
The synchronize_rcu_tasks() will not wait for such tasks to complete.
In such case the trampoline image will be freed and when the task
wakes up the return IP will point to freed memory causing the crash.
Solve this by adding percpu_ref_get/put for the duration of trampoline
and separate trampoline vs its image life times.
The "half page" optimization has to be removed, since
first_half->second_half->first_half transition cannot be guaranteed to
complete in deterministic time. Every trampoline update becomes a new image.
The image with fmod_ret or fexit progs will be freed via percpu_ref_kill and
call_rcu_tasks. Together they will wait for the original function and
trampoline asm to complete. The trampoline is patched from nop to jmp to skip
fexit progs. They are freed independently from the trampoline. The image with
fentry progs only will be freed via call_rcu_tasks_trace+call_rcu_tasks which
will wait for both sleepable and non-sleepable progs to complete.

Fixes: fec56f5890d9 ("bpf: Introduce BPF trampoline")
Reported-by: Andrii Nakryiko <andrii@kernel.org>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Paul E. McKenney <paulmck@kernel.org> # for RCU
Link: https://lore.kernel.org/bpf/20210316210007.38949-1-alexei.starovoitov@gmail.com
arch/x86/net/bpf_jit_comp.c
include/linux/bpf.h
kernel/bpf/bpf_struct_ops.c
kernel/bpf/core.c
kernel/bpf/trampoline.c