Yonghong Song [Mon, 23 May 2022 15:20:44 +0000 (08:20 -0700)]
selftests/bpf: fix btf_dump/btf_dump due to recent clang change
Latest llvm-project upstream had a change of behavior
related to qualifiers on function return type ([1]).
This caused selftests btf_dump/btf_dump failure.
The following example shows what changed.
$ cat t.c
typedef const char * const (* const (* const fn_ptr_arr2_t[5])())(char * (*)(int));
struct t {
int a;
fn_ptr_arr2_t l;
};
int foo(struct t *arg) {
return arg->a;
}
To adapt the selftest to both old and new llvm, this patch removed
the intermediate const qualifier in const-to-ptr types, to make the
test succeed again.
Unprivileged BPF disabled (kernel.unprivileged_bpf_disabled >= 1)
is the default in most cases now; when set, the BPF system call is
blocked for users without CAP_BPF/CAP_SYS_ADMIN. In some cases
however, it makes sense to split activities between capability-requiring
ones - such as program load/attach - and those that might not require
capabilities such as reading perf/ringbuf events, reading or
updating BPF map configuration etc. One example of this sort of
approach is a service that loads a BPF program, and a user-space
program that interacts with it.
Here - rather than blocking all BPF syscall commands - unprivileged
BPF disabled blocks the key object-creating commands (prog load,
map load). Discussion has alluded to this idea in the past [1],
and Alexei mentioned it was also discussed at LSF/MM/BPF this year.
Changes since v3 [2]:
- added acks to patch 1
- CI was failing on Ubuntu; I suspect the issue was an old capability.h
file which specified CAP_LAST_CAP as < CAP_BPF, leading to the logic
disabling all caps not disabling CAP_BPF. Use CAP_BPF as basis for
"all caps" bitmap instead as we explicitly define it in cap_helpers.h
if not already found in capabilities.h
- made global variables arguments to subtests instead (Andrii, patch 2)
Changes since v2 [3]:
- added acks from Yonghong
- clang compilation issue in selftest with bpf_prog_query()
(Alexei, patch 2)
- disable all capabilities for test (Yonghong, patch 2)
- add assertions that size of perf/ringbuf data matches expectations
(Yonghong, patch 2)
- add map array size definition, remove unneeded whitespace (Yonghong, patch 2)
Changes since RFC [4]:
- widened scope of commands unprivileged BPF disabled allows
(Alexei, patch 1)
- removed restrictions on map types for lookup, update, delete
(Alexei, patch 1)
- removed kernel CONFIG parameter controlling unprivileged bpf disabled
change (Alexei, patch 1)
- widened test scope to cover most BPF syscall commands, with positive
and negative subtests
With unprivileged BPF disabled, all cmds associated with the BPF syscall
are blocked to users without CAP_BPF/CAP_SYS_ADMIN. However there are
use cases where we may wish to allow interactions with BPF programs
without being able to load and attach them. So for example, a process
with required capabilities loads/attaches a BPF program, and a process
with less capabilities interacts with it; retrieving perf/ring buffer
events, modifying map-specified config etc. With all BPF syscall
commands blocked as a result of unprivileged BPF being disabled,
this mode of interaction becomes impossible for processes without
CAP_BPF.
As Alexei notes
"The bpf ACL model is the same as traditional file's ACL.
The creds and ACLs are checked at open(). Then during file's write/read
additional checks might be performed. BPF has such functionality already.
Different map_creates have capability checks while map_lookup has:
map_get_sys_perms(map, f) & FMODE_CAN_READ.
In other words it's enough to gate FD-receiving parts of bpf
with unprivileged_bpf_disabled sysctl.
The rest is handled by availability of FD and access to files in bpffs."
So key fd creation syscall commands BPF_PROG_LOAD and BPF_MAP_CREATE
are blocked with unprivileged BPF disabled and no CAP_BPF.
And as Alexei notes, map creation with unprivileged BPF disabled off
blocks creation of maps aside from array, hash and ringbuf maps.
Programs responsible for loading and attaching the BPF program
can still control access to its pinned representation by restricting
permissions on the pin path, as with normal files.
Tracing and syscall BPF program types are very convenient to add BPF
capabilities to subsystem otherwise not BPF capable.
When we add kfuncs capabilities to those program types, we can add
BPF features to subsystems without having to touch BPF core.
Yuntao Wang [Thu, 19 May 2022 15:06:10 +0000 (23:06 +0800)]
selftests/bpf: Add missing trampoline program type to trampoline_count test
Currently the trampoline_count test doesn't include any fmod_ret bpf
programs, fix it to make the test cover all possible trampoline program
types.
Since fmod_ret bpf programs can't be attached to __set_task_comm function,
as it's neither whitelisted for error injection nor a security hook, change
it to bpf_modify_return_test.
This patch also does some other cleanups such as removing duplicate code,
dropping inconsistent comments, etc.
Andrii Nakryiko [Fri, 20 May 2022 22:29:01 +0000 (15:29 -0700)]
Merge branch 'bpf: mptcp: Support for mptcp_sock'
Mat Martineau says:
====================
This patch set adds BPF access to mptcp_sock structures, along with
associated self tests. You may recognize some of the code from earlier
(https://lore.kernel.org/bpf/20200918121046.190240-6-nicolas.rybowski@tessares.net/)
but it has been reworked quite a bit.
v1 -> v2: Emit BTF type, add func_id checks in verifier.c and bpf_trace.c,
remove build check for CONFIG_BPF_JIT, add selftest check for CONFIG_MPTCP,
and add a patch to include CONFIG_IKCONFIG/CONFIG_IKCONFIG_PROC for the
BPF self tests.
v2 -> v3: Access sysctl through the filesystem to work around CI use of
the more limited busybox sysctl command.
v3 -> v4: Dropped special case kernel code for tcp_sock is_mptcp, use
existing bpf_tcp_helpers.h, and add check for 'ip mptcp monitor' support.
v4 -> v5: Use BPF test skeleton, more consistent use of ASSERT macros,
drop some unnecessary parameters / checks, and use tracing to acquire
MPTCP token.
Geliang Tang (6):
bpf: add bpf_skc_to_mptcp_sock_proto
selftests/bpf: Enable CONFIG_IKCONFIG_PROC in config
selftests/bpf: test bpf_skc_to_mptcp_sock
selftests/bpf: verify token of struct mptcp_sock
selftests/bpf: verify ca_name of struct mptcp_sock
selftests/bpf: verify first of struct mptcp_sock
====================
Geliang Tang [Thu, 19 May 2022 23:30:16 +0000 (16:30 -0700)]
selftests/bpf: Verify first of struct mptcp_sock
This patch verifies the 'first' struct member of struct mptcp_sock, which
points to the first subflow of msk. Save 'sk' in mptcp_storage, and verify
it with 'first' in verify_msk().
v5:
- Use ASSERT_EQ() instead of a manual comparison + log (Andrii).
Geliang Tang [Thu, 19 May 2022 23:30:15 +0000 (16:30 -0700)]
selftests/bpf: Verify ca_name of struct mptcp_sock
This patch verifies another member of struct mptcp_sock, ca_name. Add a
new function get_msk_ca_name() to read the sysctl tcp_congestion_control
and verify it in verify_msk().
v3: Access the sysctl through the filesystem to avoid compatibility
issues with the busybox sysctl command.
v4: use ASSERT_* instead of CHECK_FAIL (Andrii)
v5: use ASSERT_STRNEQ() instead of strncmp() (Andrii)
Geliang Tang [Thu, 19 May 2022 23:30:14 +0000 (16:30 -0700)]
selftests/bpf: Verify token of struct mptcp_sock
This patch verifies the struct member token of struct mptcp_sock. Add a
new member token in struct mptcp_storage to store the token value of the
msk socket got by bpf_skc_to_mptcp_sock(). Trace the kernel function
mptcp_pm_new_connection() by using bpf fentry prog to obtain the msk token
and save it in a global bpf variable. Pass the variable to verify_msk() to
verify it with the token saved in socket_storage_map.
v4:
- use ASSERT_* instead of CHECK_FAIL (Andrii)
- skip the test if 'ip mptcp monitor' is not supported (Mat)
v5:
- Drop 'ip mptcp monitor', trace mptcp_pm_new_connection instead (Martin)
- Use ASSERT_EQ (Andrii)
Geliang Tang [Thu, 19 May 2022 23:30:13 +0000 (16:30 -0700)]
selftests/bpf: Test bpf_skc_to_mptcp_sock
This patch extends the MPTCP test base, to test the new helper
bpf_skc_to_mptcp_sock().
Define struct mptcp_sock in bpf_tcp_helpers.h, use bpf_skc_to_mptcp_sock
to get the msk socket in progs/mptcp_sock.c and store the infos in
socket_storage_map.
Get the infos from socket_storage_map in prog_tests/mptcp.c. Add a new
function verify_msk() to verify the infos of MPTCP socket, and rename
verify_sk() to verify_tsk() to verify TCP socket only.
v2: Add CONFIG_MPTCP check for clearer error messages
v4:
- use ASSERT_* instead of CHECK_FAIL (Andrii)
- drop bpf_mptcp_helpers.h (Andrii)
v5:
- some 'ASSERT_*' were replaced in the next commit by mistake.
- Drop CONFIG_MPTCP (Martin)
- Use ASSERT_EQ (Andrii)
Nicolas Rybowski [Thu, 19 May 2022 23:30:12 +0000 (16:30 -0700)]
selftests/bpf: Add MPTCP test base
This patch adds a base for MPTCP specific tests.
It is currently limited to the is_mptcp field in case of plain TCP
connection because there is no easy way to get the subflow sk from a msk
in userspace. This implies that we cannot lookup the sk_storage attached
to the subflow sk in the sockops program.
v4:
- add copyright 2022 (Andrii)
- use ASSERT_* instead of CHECK_FAIL (Andrii)
- drop SEC("version") (Andrii)
- use is_mptcp in tcp_sock, instead of bpf_tcp_sock (Martin & Andrii)
v5:
- Drop connect_to_mptcp_fd (Martin)
- Use BPF test skeleton (Andrii)
- Use ASSERT_EQ (Andrii)
- Drop the 'msg' parameter of verify_sk
Geliang Tang [Thu, 19 May 2022 23:30:10 +0000 (16:30 -0700)]
bpf: Add bpf_skc_to_mptcp_sock_proto
This patch implements a new struct bpf_func_proto, named
bpf_skc_to_mptcp_sock_proto. Define a new bpf_id BTF_SOCK_TYPE_MPTCP,
and a new helper bpf_skc_to_mptcp_sock(), which invokes another new
helper bpf_mptcp_sock_from_subflow() in net/mptcp/bpf.c to get struct
mptcp_sock from a given subflow socket.
v2: Emit BTF type, add func_id checks in verifier.c and bpf_trace.c,
remove build check for CONFIG_BPF_JIT
v5: Drop EXPORT_SYMBOL (Martin)
Co-developed-by: Nicolas Rybowski <nicolas.rybowski@tessares.net> Co-developed-by: Matthieu Baerts <matthieu.baerts@tessares.net> Signed-off-by: Nicolas Rybowski <nicolas.rybowski@tessares.net> Signed-off-by: Matthieu Baerts <matthieu.baerts@tessares.net> Signed-off-by: Geliang Tang <geliang.tang@suse.com> Signed-off-by: Mat Martineau <mathew.j.martineau@linux.intel.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20220519233016.105670-2-mathew.j.martineau@linux.intel.com
Feng Zhou [Wed, 18 May 2022 02:50:53 +0000 (10:50 +0800)]
selftests/bpf: Fix some bugs in map_lookup_percpu_elem testcase
comments from Andrii Nakryiko, details in here:
https://lore.kernel.org/lkml/20220511093854.411-1-zhoufeng.zf@bytedance.com/T/
use /* */ instead of //
use libbpf_num_possible_cpus() instead of sysconf(_SC_NPROCESSORS_ONLN)
use 8 bytes for value size
fix memory leak
use ASSERT_EQ instead of ASSERT_OK
add bpf_loop to fetch values on each possible CPU
Fixes: 566c02346f345c9ae27b10e7a6124c5a989718b4 ("selftests/bpf: add test case for bpf_map_lookup_percpu_elem") Signed-off-by: Feng Zhou <zhoufeng.zf@bytedance.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Acked-by: Yonghong Song <yhs@fb.com> Link: https://lore.kernel.org/bpf/20220518025053.20492-1-zhoufeng.zf@bytedance.com
Andrii Nakryiko [Wed, 18 May 2022 18:59:15 +0000 (11:59 -0700)]
libbpf: remove bpf_create_map*() APIs
To test API removal, get rid of bpf_create_map*() APIs. Perf defines
__weak implementation of bpf_map_create() that redirects to old
bpf_create_map() and that seems to compile and run fine.
Andrii Nakryiko [Wed, 18 May 2022 18:59:14 +0000 (11:59 -0700)]
libbpf: start 1.0 development cycle
Start libbpf 1.0 development cycle by adding LIBBPF_1.0.0 section to
libbpf.map file and marking all current symbols as local. As we remove
all the deprecated APIs we'll populate global list before the final 1.0
release.
Larysa Zaremba [Fri, 13 May 2022 12:17:43 +0000 (14:17 +0200)]
bpftool: Use sysfs vmlinux when dumping BTF by ID
Currently, dumping almost all BTFs specified by id requires
using the -B option to pass the base BTF. For kernel module
BTFs the vmlinux BTF sysfs path should work.
This patch simplifies dumping by ID usage by loading
vmlinux BTF from sysfs as base, if base BTF was not specified
and the ID corresponds to a kernel module BTF.
Joanne Koong [Mon, 9 May 2022 22:42:52 +0000 (15:42 -0700)]
bpf: Add MEM_UNINIT as a bpf_type_flag
Instead of having uninitialized versions of arguments as separate
bpf_arg_types (eg ARG_PTR_TO_UNINIT_MEM as the uninitialized version
of ARG_PTR_TO_MEM), we can instead use MEM_UNINIT as a bpf_type_flag
modifier to denote that the argument is uninitialized.
Doing so cleans up some of the logic in the verifier. We no longer
need to do two checks against an argument type (eg "if
(base_type(arg_type) == ARG_PTR_TO_MEM || base_type(arg_type) ==
ARG_PTR_TO_UNINIT_MEM)"), since uninitialized and initialized
versions of the same argument type will now share the same base type.
In the near future, MEM_UNINIT will be used by dynptr helper functions
as well.
Andrii Nakryiko [Fri, 13 May 2022 17:37:03 +0000 (10:37 -0700)]
selftests/bpf: Fix usdt_400 test case
usdt_400 test case relies on compiler using the same arg spec for
usdt_400 USDT. This assumption breaks with Clang (Clang generates
different arg specs with varying offsets relative to %rbp), so simplify
this further and hard-code the constant which will guarantee that arg
spec is the same across all 400 inlinings.
Andrii Nakryiko [Thu, 12 May 2022 22:07:12 +0000 (15:07 -0700)]
libbpf: Add safer high-level wrappers for map operations
Add high-level API wrappers for most common and typical BPF map
operations that works directly on instances of struct bpf_map * (so
you don't have to call bpf_map__fd()) and validate key/value size
expectations.
These helpers require users to specify key (and value, where
appropriate) sizes when performing lookup/update/delete/etc. This forces
user to actually think and validate (for themselves) those. This is
a good thing as user is expected by kernel to implicitly provide correct
key/value buffer sizes and kernel will just read/write necessary amount
of data. If it so happens that user doesn't set up buffers correctly
(which bit people for per-CPU maps especially) kernel either randomly
overwrites stack data or return -EFAULT, depending on user's luck and
circumstances. These high-level APIs are meant to prevent such
unpleasant and hard to debug bugs.
This patch also adds bpf_map_delete_elem_flags() low-level API and
requires passing flags to bpf_map__delete_elem() API for consistency
across all similar APIs, even though currently kernel doesn't expect
any extra flags for BPF_MAP_DELETE_ELEM operation.
List of map operations that get these high-level APIs:
bpf: Fix combination of jit blinding and pointers to bpf subprogs.
The combination of jit blinding and pointers to bpf subprogs causes:
[ 36.989548] BUG: unable to handle page fault for address: 0000000100000001
[ 36.990342] #PF: supervisor instruction fetch in kernel mode
[ 36.990968] #PF: error_code(0x0010) - not-present page
[ 36.994859] RIP: 0010:0x100000001
[ 36.995209] Code: Unable to access opcode bytes at RIP 0xffffffd7.
[ 37.004091] Call Trace:
[ 37.004351] <TASK>
[ 37.004576] ? bpf_loop+0x4d/0x70
[ 37.004932] ? bpf_prog_3899083f75e4c5de_F+0xe3/0x13b
The jit blinding logic didn't recognize that ld_imm64 with an address
of bpf subprogram is a special instruction and proceeded to randomize it.
By itself it wouldn't have been an issue, but jit_subprogs() logic
relies on two step process to JIT all subprogs and then JIT them
again when addresses of all subprogs are known.
Blinding process in the first JIT phase caused second JIT to miss
adjustment of special ld_imm64.
Fix this issue by ignoring special ld_imm64 instructions that don't have
user controlled constants and shouldn't be blinded.
Fixes: 1ddf7323058b ("bpf: Add bpf_for_each_map_elem() helper") Reported-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Acked-by: Andrii Nakryiko <andrii@kernel.org> Acked-by: Martin KaFai Lau <kafai@fb.com> Link: https://lore.kernel.org/bpf/20220513011025.13344-1-alexei.starovoitov@gmail.com
Yuntao Wang [Sat, 30 Apr 2022 13:08:03 +0000 (21:08 +0800)]
bpf: Fix potential array overflow in bpf_trampoline_get_progs()
The cnt value in the 'cnt >= BPF_MAX_TRAMP_PROGS' check does not
include BPF_TRAMP_MODIFY_RETURN bpf programs, so the number of
the attached BPF_TRAMP_MODIFY_RETURN bpf programs in a trampoline
can exceed BPF_MAX_TRAMP_PROGS.
When this happens, the assignment '*progs++ = aux->prog' in
bpf_trampoline_get_progs() will cause progs array overflow as the
progs field in the bpf_tramp_progs struct can only hold at most
BPF_MAX_TRAMP_PROGS bpf programs.
Andrii Nakryiko [Wed, 11 May 2022 23:20:12 +0000 (16:20 -0700)]
selftests/bpf: make fexit_stress test run in serial mode
fexit_stress is attaching maximum allowed amount of fexit programs to
bpf_fentry_test1 kernel function, which is used by a bunch of other
parallel tests, thus pretty frequently interfering with their execution.
Given the test assumes nothing else is attaching to bpf_fentry_test1,
mark it serial.
Merge branch 'Introduce access remote cpu elem support in BPF percpu map'
Feng zhou says:
====================
From: Feng Zhou <zhoufeng.zf@bytedance.com>
Trace some functions, such as enqueue_task_fair, need to access the
corresponding cpu, not the current cpu, and bpf_map_lookup_elem percpu map
cannot do it. So add bpf_map_lookup_percpu_elem to accomplish it for
percpu_array_map, percpu_hash_map, lru_percpu_hash_map.
v1->v2: Addressed comments from Alexei Starovoitov.
- add a selftest for bpf_map_lookup_percpu_elem.
====================
Feng Zhou [Wed, 11 May 2022 09:38:53 +0000 (17:38 +0800)]
bpf: add bpf_map_lookup_percpu_elem for percpu map
Add new ebpf helpers bpf_map_lookup_percpu_elem.
The implementation method is relatively simple, refer to the implementation
method of map_lookup_elem of percpu map, increase the parameters of cpu, and
obtain it according to the specified cpu.
* Add Fixes tag to patch 1
* Fix test_progs-noalu32 failure in CI due to different alloc_insn (Alexei)
* Remove per-CPU struct, use global struct (Alexei)
====================
This uses the newly added SEC("?foo") naming to disable autoload of
programs, and then loads them one by one for the object and verifies
that loading fails and matches the returned error string from verifier.
This is similar to already existing verifier tests but provides coverage
for BPF C.
bpf: Prepare prog_test_struct kfuncs for runtime tests
In an effort to actually test the refcounting logic at runtime, add a
refcount_t member to prog_test_ref_kfunc and use it in selftests to
verify and test the whole logic more exhaustively.
The kfunc calls for prog_test_member do not require runtime refcounting,
as they are only used for verifier selftests, not during runtime
execution. Hence, their implementation now has a WARN_ON_ONCE as it is
not meant to be reachable code at runtime. It is strictly used in tests
triggering failure cases in the verifier. bpf_kfunc_call_memb_release is
called from map free path, since prog_test_member is embedded in map
value for some verifier tests, so we skip WARN_ON_ONCE for it.
Yonghong Song [Wed, 11 May 2022 18:47:35 +0000 (11:47 -0700)]
selftests/bpf: fix a few clang compilation errors
With latest clang, I got the following compilation errors:
.../prog_tests/test_tunnel.c:291:6: error: variable 'local_ip_map_fd' is used uninitialized
whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
if (attach_tc_prog(&tc_hook, -1, set_dst_prog_fd))
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.../bpf/prog_tests/test_tunnel.c:312:6: note: uninitialized use occurs here
if (local_ip_map_fd >= 0)
^~~~~~~~~~~~~~~
...
.../prog_tests/kprobe_multi_test.c:346:6: error: variable 'err' is used uninitialized
whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
if (IS_ERR(map))
^~~~~~~~~~~
.../prog_tests/kprobe_multi_test.c:388:6: note: uninitialized use occurs here
if (err) {
^~~
Daniel Müller [Wed, 11 May 2022 17:22:49 +0000 (17:22 +0000)]
selftests/bpf: Enable CONFIG_FPROBE for self tests
Some of the BPF selftests are failing when running with a rather bare
bones configuration based on tools/testing/selftests/bpf/config.
Specifically, we see a bunch of failures due to errno 95:
Merge branch 'selftests: xsk: add busy-poll testing plus various fixes'
Magnus Karlsson says:
====================
This patch set adds busy-poll testing to the xsk selftests. It runs
exactly the same tests as with regular softirq processing, but with
busy-poll enabled. I have also included a number of fixes to the
selftests that have been bugging me for a while or was discovered
while implementing the busy-poll support. In summary these are:
* Fix the error reporting of failed tests. Each failed test used to be
reported as both failed and passed, messing up things.
* Added a summary test printout at the end of the test suite so that
users do not have to scroll up and look at the result of both the
softirq run and the busy_poll run.
* Added a timeout to the tests, so that if a test locks up, we report
a fail and still get to run all the other tests.
* Made the stats test just look and feel like all the other
tests. Makes the code simpler and the test reporting more
consistent. These are the 3 last commits.
* Replaced zero length packets with packets of 64 byte length. This so
that some of the tests will pass after commit 0705d82fc4a7cb ("veth:
Ensure eth header is in skb's linear part").
* Added clean-up of the veth pair when terminating the test run.
* Some smaller clean-ups of unused stuff.
Note, to pass the busy-poll tests commit fcd419fbb6a1 ("xsk: Fix
l2fwd for copy mode + busy poll combo") need to be present. It is
present in bpf but not yet in bpf-next.
Thanks: Magnus
====================
Acked-by: Björn Töpel <bjorn@kernel.org> Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Magnus Karlsson [Tue, 10 May 2022 11:56:04 +0000 (13:56 +0200)]
selftests: xsk: make stat tests not spin on getsockopt
Convert the stats tests from spinning on the getsockopt to just check
getsockopt once when the Rx thread has received all the packets. The
actual completion of receiving the last packet forms a natural point
in time when the receiver is ready to call the getsockopt to check the
stats. In the previous version , we just span on the getsockopt until
we received the right answer. This could be forever or just getting
the "correct" answer by shear luck.
The pacing_on variable can now be dropped since all test can now
handle pacing properly.
Magnus Karlsson [Tue, 10 May 2022 11:56:03 +0000 (13:56 +0200)]
selftests: xsk: make the stats tests normal tests
Make the stats tests look and feel just like normal tests instead of
bunched under the umbrella of TEST_STATS. This means we will always
run each of them even if one fails. Also gets rid of some special case
code.
Magnus Karlsson [Tue, 10 May 2022 11:56:02 +0000 (13:56 +0200)]
selftests: xsk: introduce validation functions
Introduce validation functions that can be optionally called by the Rx
and Tx threads. These are then used to replace the Rx and Tx stats
dispatchers. This so that we in the next commit can make the stats
tests proper normal tests and not be some special case, as today.
Magnus Karlsson [Tue, 10 May 2022 11:56:01 +0000 (13:56 +0200)]
selftests: xsk: cleanup veth pair at ctrl-c
Remove the veth pair when the tests are aborted by pressing
ctrl-c. Currently in this situation, the veth pair is left on the
system polluting the netdev space.
Magnus Karlsson [Tue, 10 May 2022 11:56:00 +0000 (13:56 +0200)]
selftests: xsk: add timeout to tests
Add a timeout to the tests so that if all packets have not been
received within 3 seconds, fail the ongoing test. Hinders a test from
dead-locking if there is something wrong.
Magnus Karlsson [Tue, 10 May 2022 11:55:59 +0000 (13:55 +0200)]
selftests: xsk: fix reporting of failed tests
Fix the reporting of failed tests as it was broken in several
ways. First, a failed test was reported as both failed and passed
messing up the count. Second, tests were not aborted after a failure
and could generate more "failures" messing up the count even
more. Third, the failure reporting from the application to the shell
script was wrong. It always reported pass. And finally, the handling
of the failures in the launch script was not correct.
Correct all this by propagating the failure up through the function
calls to a calling function that can abort the test. A receiver or
sender thread will mark the new variable in the test spec called fail,
if a test has failed. This is then picked up by the main thread when
everyone else has exited and this is then marked and propagated up to
the calling script.
Also add a summary function in the calling script so that a user
does not have to go through the sub tests to see if something has
failed.
Magnus Karlsson [Tue, 10 May 2022 11:55:58 +0000 (13:55 +0200)]
selftests: xsk: run all tests for busy-poll
Execute all xsk selftests for busy-poll mode too. Currently they were
only run for the standard interrupt driven softirq mode. Replace the
unused option queue-id with the new option busy-poll.
Magnus Karlsson [Tue, 10 May 2022 11:55:57 +0000 (13:55 +0200)]
selftests: xsk: do not send zero-length packets
Do not try to send packets of zero length since they are dropped by
veth after commit 0705d82fc4a7cb ("veth: Ensure eth header is in skb's
linear part"). Replace these two packets with packets of length 60 so
that they are not dropped.
Also clean up the confusing naming. MIN_PKT_SIZE was really
MIN_ETH_PKT_SIZE and PKT_SIZE was both MIN_ETH_SIZE and the default
packet size called just PKT_SIZE. Make it consistent by using the
right define in the right place.
Kui-Feng Lee [Tue, 10 May 2022 20:59:21 +0000 (13:59 -0700)]
bpf, x86: Attach a cookie to fentry/fexit/fmod_ret/lsm.
Pass a cookie along with BPF_LINK_CREATE requests.
Add a bpf_cookie field to struct bpf_tracing_link to attach a cookie.
The cookie of a bpf_tracing_link is available by calling
bpf_get_attach_cookie when running the BPF program of the attached
link.
The value of a cookie will be set at bpf_tramp_run_ctx by the
trampoline of the link.
Kui-Feng Lee [Tue, 10 May 2022 20:59:19 +0000 (13:59 -0700)]
bpf, x86: Generate trampolines from bpf_tramp_links
Replace struct bpf_tramp_progs with struct bpf_tramp_links to collect
struct bpf_tramp_link(s) for a trampoline. struct bpf_tramp_link
extends bpf_link to act as a linked list node.
arch_prepare_bpf_trampoline() accepts a struct bpf_tramp_links to
collects all bpf_tramp_link(s) that a trampoline should call.
Change BPF trampoline and bpf_struct_ops to pass bpf_tramp_links
instead of bpf_tramp_progs.
v3 changes:
- renamed kallsyms_lookup_names to ftrace_lookup_symbols
and moved it to ftrace.c [Masami]
- added ack [Andrii]
- couple small test fixes [Andrii]
v2 changes (first version [2]):
- removed the 2 seconds check [Alexei]
- moving/forcing symbols sorting out of kallsyms_lookup_names function [Alexei]
- skipping one array allocation and copy_from_user [Andrii]
- several small fixes [Masami,Andrii]
- build fix [kernel test robot]
Jiri Olsa [Tue, 10 May 2022 12:26:15 +0000 (14:26 +0200)]
bpf: Resolve symbols with ftrace_lookup_symbols for kprobe multi link
Using kallsyms_lookup_names function to speed up symbols lookup in
kprobe multi link attachment and replacing with it the current
kprobe_multi_resolve_syms function.
Jiri Olsa [Tue, 10 May 2022 12:26:13 +0000 (14:26 +0200)]
ftrace: Add ftrace_lookup_symbols function
Adding ftrace_lookup_symbols function that resolves array of symbols
with single pass over kallsyms.
The user provides array of string pointers with count and pointer to
allocated array for resolved values.
int ftrace_lookup_symbols(const char **sorted_syms, size_t cnt,
unsigned long *addrs)
It iterates all kallsyms symbols and tries to loop up each in provided
symbols array with bsearch. The symbols array needs to be sorted by
name for this reason.
We also check each symbol to pass ftrace_location, because this API
will be used for fprobe symbols resolving.
Bpf links seem to be one of the important structures for which no
iterator is provided. Such iterator could be useful in those cases when
generic 'task/file' is not suitable or better performance is needed.
The implementation is mostly copied from prog iterator. This time tests were
executed, although I still had to exclude test_bpf_nf (failed to find BTF info
for global/extern symbol 'bpf_skb_ct_lookup') -- since it's unrelated, I hope
it's a minor issue.
Per suggestion from the previous discussion, there is a new patch for
converting CHECK to corresponding ASSERT_* macro. Such replacement is done only
if the final result would be the same, e.g. CHECK with important-looking custom
formatting strings are still in place -- from what I understand ASSERT_*
doesn't allow to specify such format.
The third small patch fixes what looks like a copy-paste error in the condition
checking.
====================
Dmitrii Dolgov [Tue, 10 May 2022 15:52:32 +0000 (17:52 +0200)]
selftests/bpf: Use ASSERT_* instead of CHECK
Replace usage of CHECK with a corresponding ASSERT_* macro for bpf_iter
tests. Only done if the final result is equivalent, no changes when
replacement means loosing some information, e.g. from formatting string.
Dmitrii Dolgov [Tue, 10 May 2022 15:52:30 +0000 (17:52 +0200)]
bpf: Add bpf_link iterator
Implement bpf_link iterator to traverse links via bpf_seq_file
operations. The changeset is mostly shamelessly copied from
commit 01807685454d ("bpf: Add bpf_prog iterator")
====================
From: Kaixi Fan <fankaixi.li@bytedance.com>
Now bpf code could not set tunnel source ip address of ip tunnel. So it
could not support flow based tunnel mode completely. Because flow based
tunnel mode could set tunnel source, destination ip address and tunnel
key simultaneously.
Flow based tunnel is useful for overlay networks. And by configuring tunnel
source ip address, user could make their networks more elastic.
For example, tunnel source ip could be used to select different egress
nic interface for different flows with same tunnel destination ip. Another
example, user could choose one of multiple ip address of the egress nic
interface as the packet's tunnel source ip.
Add tunnel and tunnel source testcases in test_progs. Other types of
tunnel testcases would be moved to test_progs step by step in the
future.
v6:
- use libbpf api to attach tc progs and remove some shell commands to reduce
test runtime based on Alexei Starovoitov's suggestion
v5:
- fix some code format errors
- use bpf kernel code at namespace at_ns0 to set tunnel metadata
v4:
- fix subject error of first patch
v3:
- move vxlan tunnel testcases to test_progs
- replace bpf_trace_printk with bpf_printk
- rename bpf kernel prog section name to tic
v2:
- merge vxlan tunnel and tunnel source ip testcases in test_tunnel.sh
====================
Kaixi Fan [Sat, 30 Apr 2022 07:48:43 +0000 (15:48 +0800)]
selftests/bpf: Move vxlan tunnel testcases to test_progs
Move vxlan tunnel testcases from test_tunnel.sh to test_progs.
And add vxlan tunnel source testcases also. Other tunnel testcases
will be moved to test_progs step by step in the future.
Rename bpf program section name as SEC("tc") because test_progs
bpf loader could not load sections with name SEC("gre_set_tunnel").
Because of this, add bpftool to load bpf programs in test_tunnel.sh.
KP Singh [Mon, 9 May 2022 21:49:05 +0000 (21:49 +0000)]
bpftool: bpf_link_get_from_fd support for LSM programs in lskel
bpf_link_get_from_fd currently returns a NULL fd for LSM programs.
LSM programs are similar to tracing programs and can also use
skel_raw_tracepoint_open.
Takshak Chahande [Tue, 10 May 2022 08:22:21 +0000 (01:22 -0700)]
selftests/bpf: Handle batch operations for map-in-map bpf-maps
This patch adds up test cases that handles 4 combinations:
a) outer map: BPF_MAP_TYPE_ARRAY_OF_MAPS
inner maps: BPF_MAP_TYPE_ARRAY and BPF_MAP_TYPE_HASH
b) outer map: BPF_MAP_TYPE_HASH_OF_MAPS
inner maps: BPF_MAP_TYPE_ARRAY and BPF_MAP_TYPE_HASH
Takshak Chahande [Tue, 10 May 2022 08:22:20 +0000 (01:22 -0700)]
bpf: Extend batch operations for map-in-map bpf-maps
This patch extends batch operations support for map-in-map map-types:
BPF_MAP_TYPE_HASH_OF_MAPS and BPF_MAP_TYPE_ARRAY_OF_MAPS
A usecase where outer HASH map holds hundred of VIP entries and its
associated reuse-ports per VIP stored in REUSEPORT_SOCKARRAY type
inner map, needs to do batch operation for performance gain.
This patch leverages the exiting generic functions for most of the batch
operations. As map-in-map's value contains the actual reference of the inner map,
for BPF_MAP_TYPE_HASH_OF_MAPS type, it needed an extra step to fetch the
map_id from the reference value.
Jerome Marchand [Sat, 7 May 2022 16:16:35 +0000 (18:16 +0200)]
samples: bpf: Don't fail for a missing VMLINUX_BTF when VMLINUX_H is provided
samples/bpf build currently always fails if it can't generate
vmlinux.h from vmlinux, even when vmlinux.h is directly provided by
VMLINUX_H variable, which makes VMLINUX_H pointless.
Only fails when neither method works.
Andrii Nakryiko [Tue, 10 May 2022 00:16:05 +0000 (17:16 -0700)]
Merge branch 'bpftool: fix feature output when helper probes fail'
Milan Landaverde says:
====================
Currently in bpftool's feature probe, we incorrectly tell the user that
all of the helper functions are supported for program types where helper
probing fails or is explicitly unsupported[1]:
$ bpftool feature probe
...
eBPF helpers supported for program type tracing:
- bpf_map_lookup_elem
- bpf_map_update_elem
- bpf_map_delete_elem
...
- bpf_redirect_neigh
- bpf_check_mtu
- bpf_sys_bpf
- bpf_sys_close
This patch adjusts bpftool to relay to the user when helper support
can't be determined:
$ bpftool feature probe
...
eBPF helpers supported for program type lirc_mode2:
Program type not supported
eBPF helpers supported for program type tracing:
Could not determine which helpers are available
eBPF helpers supported for program type struct_opts:
Could not determine which helpers are available
eBPF helpers supported for program type ext:
Could not determine which helpers are available
Rather than imply that no helpers are available for the program type, we
let the user know that helper function probing failed entirely.
bpftool: Output message if no helpers found in feature probing
Currently in libbpf, we have hardcoded program types that are not
supported for helper function probing (e.g. tracing, ext, lsm).
Due to this (and other legitimate failures), bpftool feature probe returns
empty for those program type helper functions.
Instead of implying to the user that there are no helper functions
available for a program type, we output a message to the user explaining
that helper function probing failed for that program type.
bpftool: Adjust for error codes from libbpf probes
Originally [1], libbpf's (now deprecated) probe functions returned a bool
to acknowledge support but the new APIs return an int with a possible
negative error code to reflect probe failure. This change decides for
bpftool to declare maps and helpers are not available on probe failures.
Andrii Nakryiko [Mon, 9 May 2022 00:41:47 +0000 (17:41 -0700)]
libbpf: Automatically fix up BPF_MAP_TYPE_RINGBUF size, if necessary
Kernel imposes a pretty particular restriction on ringbuf map size. It
has to be a power-of-2 multiple of page size. While generally this isn't
hard for user to satisfy, sometimes it's impossible to do this
declaratively in BPF source code or just plain inconvenient to do at
runtime.
One such example might be BPF libraries that are supposed to work on
different architectures, which might not agree on what the common page
size is.
Let libbpf find the right size for user instead, if it turns out to not
satisfy kernel requirements. If user didn't set size at all, that's most
probably a mistake so don't upsize such zero size to one full page,
though. Also we need to be careful about not overflowing __u32
max_entries.
Andrii Nakryiko [Mon, 9 May 2022 00:41:46 +0000 (17:41 -0700)]
libbpf: Provide barrier() and barrier_var() in bpf_helpers.h
Add barrier() and barrier_var() macros into bpf_helpers.h to be used by
end users. While a bit advanced and specialized instruments, they are
sometimes indispensable. Instead of requiring each user to figure out
exact asm volatile incantations for themselves, provide them from
bpf_helpers.h.
Also remove conflicting definitions from selftests. Some tests rely on
barrier_var() definition being nothing, those will still work as libbpf
does the #ifndef/#endif guarding for barrier() and barrier_var(),
allowing users to redefine them, if necessary.
Andrii Nakryiko [Mon, 9 May 2022 00:41:44 +0000 (17:41 -0700)]
libbpf: Complete field-based CO-RE helpers with field offset helper
Add bpf_core_field_offset() helper to complete field-based CO-RE
helpers. This helper can be useful for feature-detection and for some
more advanced cases of field reading (e.g., reading flexible array members).
Andrii Nakryiko [Mon, 9 May 2022 00:41:43 +0000 (17:41 -0700)]
selftests/bpf: Use both syntaxes for field-based CO-RE helpers
Excercise both supported forms of bpf_core_field_exists() and
bpf_core_field_size() helpers: variable-based field reference and
type/field name-based one.
Andrii Nakryiko [Mon, 9 May 2022 00:41:42 +0000 (17:41 -0700)]
libbpf: Improve usability of field-based CO-RE helpers
Allow to specify field reference in two ways:
- if user has variable of necessary type, they can use variable-based
reference (my_var.my_field or my_var_ptr->my_field). This was the
only supported syntax up till now.
- now, bpf_core_field_exists() and bpf_core_field_size() support also
specifying field in a fashion similar to offsetof() macro, by
specifying type of the containing struct/union separately and field
name separately: bpf_core_field_exists(struct my_type, my_field).
This forms is quite often more convenient in practice and it matches
type-based CO-RE helpers that support specifying type by its name
without requiring any variables.
Andrii Nakryiko [Mon, 9 May 2022 00:41:41 +0000 (17:41 -0700)]
libbpf: Make __kptr and __kptr_ref unconditionally use btf_type_tag() attr
It will be annoying and surprising for users of __kptr and __kptr_ref if
libbpf silently ignores them just because Clang used for compilation
didn't support btf_type_tag(). It's much better to get clear compiler
error than debug BPF verifier failures later on.
Andrii Nakryiko [Mon, 9 May 2022 00:41:40 +0000 (17:41 -0700)]
selftests/bpf: Prevent skeleton generation race
Prevent "classic" and light skeleton generation rules from stomping on
each other's toes due to the use of the same <obj>.linked{1,2,3}.o
naming pattern. There is no coordination and synchronizataion between
.skel.h and .lskel.h rules, so they can easily overwrite each other's
intermediate object files, leading to errors like:
/bin/sh: line 1: 170928 Bus error (core dumped)
/data/users/andriin/linux/tools/testing/selftests/bpf/tools/sbin/bpftool gen skeleton
/data/users/andriin/linux/tools/testing/selftests/bpf/test_ksyms_weak.linked3.o
name test_ksyms_weak
> /data/users/andriin/linux/tools/testing/selftests/bpf/test_ksyms_weak.skel.h
make: *** [Makefile:507: /data/users/andriin/linux/tools/testing/selftests/bpf/test_ksyms_weak.skel.h] Error 135
make: *** Deleting file '/data/users/andriin/linux/tools/testing/selftests/bpf/test_ksyms_weak.skel.h'
Fix by using different suffix for light skeleton rule.
projects / kernel.git / log