Updated elgohr/Publish-Docker-Github-Action to a supported version (v5) #1
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ng msg CONFIG_PREEMPTIRQ_TRACEPOINTS depends on TRACE_PREEMPT_TOGGLE or TRACE_IRQFLAGS, TRACE_PREEMPT_TOGGLE will also turn PREEMPT_TRACER on but NOT TRACE_IRQFLAGS. If you enable TRACE_IRQFLAGS for PREEMPTIRQ_TRACEPOINTS, you need to enable PREEMPT_TRACER as well. Signed-off-by: Edward Wu <edwardwu@realtek.com>
…andler _add_kprobe_fd() uses a <ev_name, fd> map to store fd of attached function, but the current implementation can only store the last fd if we attach multiple handler functions on the same kprobe event. This patch uses a <ev_name, <fn_name, fd>> map to build the corresponding relationship among the kprobe event, handler function names, and fds. Then we can detach any single handler function, which is pretty helpful if the developer wants to enable and disable kprobes/kretprobes dynamically. For example: We want to measure both the execution count, execution time, and some other metrics of a kernel function. For flexibility, we want to use separate handlers for each metric to disable them individually if any of them incur some performance penalties. Without this interface, we have to disable all handlers on the kernel function. The uprobe also has a similar problem. I will fix it in a subsequent patch. Signed-off-by: Hao Lee <haolee@didiglobal.com>
Add missing descriptions for detach_kprobe and detach_kretprobe. Signed-off-by: Hao Lee <haolee@didiglobal.com>
This commit adds the ability to print out tcp sequence numbers while running the tool in normal mode by reading the appropriate fields from skb. skb is not readily available for TLP, thus the output for that mode is set to 0. Signed-off-by: Michael Gugino <mgugino@redhat.com>
…ng BTF
BPF Type Format (BTF) provides a description of kernel data structures.
libbpf support was recently added - btf_dump__dump_type_data() -
that uses the BTF id of the associated type to create a string
representation of the data provided. For example, to create a string
representation of a "struct sk_buff", the pointer to the skb
data is provided along with the type id of "struct sk_buff".
Here that functionality is utilized to support tracing kernel
function entry and return using k[ret]probes. The "struct pt_regs"
context can be used to derive arguments and return values, and
when the user supplies a function name we
- look it up in /proc/kallsyms to find its address/module
- look it up in the BTF kernel/module data to get types of arguments
and return value
- store a map representation of the trace information, keyed by
function address
On function entry/return we look up info about the arguments (is
it a pointer? what size of data do we copy?) and call bpf_probe_read()
to copy the data into our trace buffers. These are then sent via
perf event to userspace, and since we know the associated BTF id,
we can dump the typed data using btf_dump__dump_type_data().
ksnoop can be used to show function signatures; for example:
$ ksnoop info ip_send_skb
int ip_send_skb(struct net * net, struct sk_buff * skb);
Then we can trace the function, for example:
$ ksnoop trace ip_send_skb
TIME CPU PID FUNCTION/ARGS
78101668506811 1 2813 ip_send_skb(
net = *(0xffffffffb5959840)
(struct net){
.passive = (refcount_t){
.refs = (atomic_t){
.counter = (int)0x2,
},
},
.dev_base_seq = (unsigned int)0x18,
.ifindex = (int)0xf,
.list = (struct list_head){
.next = (struct list_head *)0xffff9895
.prev = (struct list_head *)0xffffffff
},
[output truncated]
78178228354796 1 2813 ip_send_skb(
return =
(int)0x0
);
We see the raw value of pointers along with the typed representation
of the data they point to.
Up to five arguments are supported.
The arguments are referred to via name (e.g. skb, net), and
the return value is referred to as "return" (using the keyword
ensures we can never clash with an argument name).
ksnoop can select specific arguments/return value rather
than tracing everything; for example:
$ ksnoop "ip_send_skb(skb)"
...will only trace the skb argument. A single level of
reference is supported also, for example:
$ ksnoop "ip_send_skb(skb->sk)"
or
Simple predicates (==, !=, <, <=, >, >=) can also be specified;
for example, to show skbs where the length is > 255:
$ ksnoop "ip_rcv(skb->len > 0xff,skb)"
TIME CPU PID FUNCTION/ARGS
32461869484376 1 2955 ip_rcv(
skb->len =
(unsigned int)0x127,
skb = *(0xffff89c99623a000)
(struct sk_buff){
(union){
.sk = (struct sock *)0xffff89c880b37000,
.ip_defrag_offset = (int)0x80b37000,
},
We can also specify a combination of entry/return predicates;
when such a combination is specified, data on entry (assuming
it matches the predicate) is "stashed" for retrieval on return.
This allows us to ask questions like "show entry arguments for
function foo when it returned a non-zero value indicating error";
$ ksnoop "sock_sendmsg(skb, return != 0)"
Multiple functions can be specified also.
In addition, using "stack" (-s) mode, it is possible to specify that
a sequence of functions should be traced, but only if function
A calls function B (either directly or indirectly). For example,
in specifying
$ ksnoop -s tcp_sendmsg __tcp_transmit_skb ip_output
...we are saying we are only interested in tcp_sendmsg() function
calls that in turn issue calls to __tcp_transmit_skb(), and these
in turn eventually call ip_output(), and that we only want to
see their entry and return. This mode is useful for investigating
behaviour with a specific stack signature, allowing us to see
function/argument information for specific call chains only.
Finally, module support is included too, provided module BTF is
present in /sys/kernel/btf :
$ ksnoop iwl_trans_send_cmd
TIME CPU PID FUNCTION/ARGS
80046971419383 3 1038 iwl_trans_send_cmd(
trans = *(0xffff989564d20028)
(struct iwl_trans){
.ops = (struct iwl_trans_ops *)0xffffff
.op_mode = (struct iwl_op_mode *)0xffff
.trans_cfg = (struct iwl_cfg_trans_para
The goal pursued here is not to add another tracer to the world -
there are plenty of those - but rather to demonstrate feature usage
for deep data display in the hope that other tracing technologies
make use of this functionality. In the meantime, having a simple
tracer like this plugs the gap and can be quite helpful for kernel
debugging.
Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
Signed-off-by: Wenbo Zhang <ethercflow@gmail.com>
…space. Signed-off-by: Francis Laniel <laniel_francis@privacyrequired.com>
- fix commit link and version required for bpf_get_netns_cookie() - fix version required for bpf_get_ns_current_pid_tgid() Signed-off-by: Mauricio Vásquez <mauricio@kinvolk.io>
The snapcraft file is out of date, update to pick up latest commands Signed-off-by: Colin Ian King <colin.king@canonical.com>
Enable remove build without needing to specifiy architectures on the command line Signed-off-by: Colin Ian King <colin.king@canonical.com>
It's helpful to measure argdist in multi-thread case, so we can distinguish workload is balanced of not. Signed-off-by: zhenwei pi <pizhenwei@bytedance.com>
Signed-off-by: Wei Fu <fuweid89@gmail.com>
This is a better check than just checking the presence of the build dir.
In Gentoo Linux, when you remove the kernel source package, the leftover
build directory is intentionally left in place. Which means the
/lib/modules/$(uname -r)/build symlink still remains valid, but there's
no kconfig.h there anymore[1]. This prevents bcc from using the kheaders
(/sys/kernel/kheaders.tar.xz) fallback, instead making it fail later on:
<built-in>:1:10: fatal error: './include/linux/kconfig.h' file not found
#include "./include/linux/kconfig.h"
^~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
[1] https://bugs.gentoo.org/809347
Signed-off-by: WGH <wgh@torlan.ru>
The filtering by mount namespace logic has to access current_task->nsproxy->mnt_ns->ns.inum to get the mount namespace id. Before this commit, that line was written in C natural syntax and we're relying on the BCC rewriter to transform that to valid eBPF code by emitting some bpf_probe_read calls. This support was not working when using opensnoop in systems supporting kfuncs because in this case the BCC rewriter doesn't transform that line and the verifier claims about an invalid memory access: 7: (85) call bpf_get_current_task#35; return current_task->nsproxy->mnt_ns->ns.inum; 8: (79) r1 = *(u64 *)(r0 +2896) R0 invalid mem access 'inv' This commit fixes that by explicitly using bpf_probe_kernel_read() instead of the C natural syntax. Signed-off-by: Mauricio Vásquez <mauricio@kinvolk.io>
Sync with latest libbpf repo upto the following commit: 5579664205e4 libbpf: Fix build with latest gcc/binutils with LTO Signed-off-by: Yonghong Song <yhs@fb.com>
Kernel commit [1] used ->iter_type and ->data_source instead of ->type [1] 8cd54c1c8480 iov_iter: separate direction from flavour Signed-off-by: Sina Radmehr <sina_rad@hamravesh.com>
* Support for kernel up to 5.14 * add ipv4/ipv6 filter support for tcp trace tools * add python interface to attach raw perf events * fix tcpstates for incorrect display of dport * new options for bcc tools runqslower, argdist * new libbpf-tools: filetop, exitsnoop, tcprtt * doc update, bug fixes and other tools improvement Signed-off-by: Yonghong Song <yhs@fb.com>
Since glibc 2.34, pthread features are integrated in libc directly. Look for pthread_create there too when it is not found in libpthread. Fixes iovisor#3623
there's no `llvm-8.0`, but instead it's `llvm-8` in Ubuntu.
No more support timestamp and COMM, procedure as follows:
1. startup UNIX socket server:
```bash
[rongtao@bogon ~]$ nc --unixsock ./unix-sock -l
```
2. start UNIX socket client:
```bash
[rongtao@bogon ~]$ nc --unixsock ./unix-sock
```
3. startup undump.py script
```
[rongtao@bogon study]$ sudo ./undump2.py -p 41147
Tracing PID=41147 UNIX socket packets ... Hit Ctrl-C to end
```
4. send some packets
```
[rongtao@bogon ~]$ nc --unixsock ./unix-sock
abcdefg
1234567890
```
5. capture these packets
server recv:
```
[rongtao@bogon ~]$ nc --unixsock ./unix-sock -l
abcdefg
1234567890
```
undump.py capture
```
[rongtao@bogon study]$ sudo ./undump2.py -p 41147
Tracing PID=41147 UNIX socket packets ... Hit Ctrl-C to end
PID 41147 Recv 8 bytes
61 62 63 64 65 66 67 0a
PID 41147 Recv 11 bytes
31 32 33 34 35 36 37 38 39 30 0a
```
Signed-off-by: Hengqi Chen <chenhengqi@outlook.com>
`ksnoop` is the only libbpf tool which is including both `<linux/bpf.h>` and `<bpf/bpf.h>` - the rest of the tools just include the latter build fails for me because of redefinition errors as a result. Let's use `<bpf/bpf.h>` like the rest of the tools
…noop_include ksnoop: remove duplicate include
1. Most of the tools that use perf_buffer__poll() were not handling the case when it was interrupted by a signal, they were just ending. We noticed this issue by running the tools inside a container, after some seconds they will finish: ``` $ time /execsnoop ... runc 210198 939 0 /usr/sbin/runc --version docker-init 210205 939 0 /usr/bin/docker-init --version Error polling perf buffer: -4 real 0m48.913s user 0m0.020s sys 0m0.033s ``` This commit fixes that by checking if errno is EINTR after calling perf_buffer__poll(). 2. Many tools were returning non zero when ended by SIG_INT. ``` $ sudo ./execsnoop PCOMM PID PPID RET ARGS runc 203967 939 0 /usr/sbin/runc --version docker-init 203973 939 0 /usr/bin/docker-init --version calico 203974 724 0 /opt/cni/bin/calico portmap 203985 724 0 /opt/cni/bin/portmap bandwidth 203990 724 0 /opt/cni/bin/bandwidth ^C $ echo $? 130 ``` 3. Some tools were missing the SIG_INT handler Signed-off-by: Mauricio Vásquez <mauricio@kinvolk.io>
As per [github](actions/runner-images#3287), this is no longer supported. CI fails with errors like ``` This request was automatically failed because there were no enabled runners online to process the request for more than 1 days. ``` I will add 20.04 in a followup PR to match `publish.yml`. Want to keep it separate in case adding 20.04 causes issues, removing 16.04 should be much less likely to.
sync up to the following libbpf commit: eaea2bce024f sync: remove redundant test on $BPF_BRANCH Signed-off-by: Yonghong Song <yhs@fb.com>
Add a new function kernel_struct_has_field, which allows user to check that whether a kernel struct has a specific field. This enable us to deal with some kernel changes like in 2f064a59a1 ([0]) of the linux kernel. [0]: torvalds/linux@2f064a59a1 Signed-off-by: Hengqi Chen <chenhengqi@outlook.com>
Kernel commit 2f064a59a1 ("sched: Change task_struct::state") changes
the name of task_struct::state to task_struct::__state, which breaks
several bcc tools. Fix this issue by checking field existence in vmlinux
BTF. Since this change was intruduce in kernel v5.14, we should have
BTF support. Closes iovisor#3658 .
Signed-off-by: Hengqi Chen <chenhengqi@outlook.com>
libbpf-tools: Fix memory leaks in ksnoop/gethostlatency
Fix issue iovisor#3687. The tool cachestat.py doesn't work with 5.15 kernel due to kprobe function renaming. Adapt to the new function. Also added a comment that static functions might get inlined and the result may not be accurate if this happens. More work can be done in the future to make the tool more robust. Signed-off-by: Yonghong Song <yhs@fb.com>
The tool cachetop.py doesn't work with 5.15 kernel due to kprobe function renaming. Adapt to the new function. Commit 61087b9 ("tools: fix cachestat.py with 5.15 kernel") fixed a similar issue for cachestat.py. Signed-off-by: Yonghong Song <yhs@fb.com>
After this fix, the output may look like this: NAME/TYPE # ALLOCS # BYTES [B 1 1016 [D 1 8016
* Support for kernel up to 5.15 * bcc tools: update for kvmexit.py, tcpv4connect.py, cachetop.py, cachestat.py, etc. * libbpf tools: update for update for mountsnoop, ksnoop, gethostlatency, etc. * fix renaming of task_struct->state * get pid namespace properly for a number of tools * initial work for more libbpf utilization (less section names) * doc update, bug fixes and other tools improvement Signed-off-by: Yonghong Song <yhs@fb.com>
…-actions-1 gh actions: run test and publish actions on pull_request, not push
resending
it wasn't running on ubuntu-18.04 test runner b/c of the kernel version check and is failing now as I try to add ubuntu-20.04 test runner Will investigate separately from GH actions changes
Since commit d5299b67dd59 ("bpf: Memcg-based memory accounting for bpf
maps"), memory locked by bpf maps is no longer counted against rlimit.
Ubuntu 20.04's 5.11 kernel has this commit, so we should skip this test
there. When we add future distros to github actions it may be necessary
to modify the version check here.
add ubuntu-20.04 to bcc-test.yml
Sync with latest libbpf repo upto commit: 94a49850c5ee Makefile: enforce gnu89 standard Signed-off-by: Yonghong Song <yhs@fb.com>
The test send a udp packet to test tailcalls. The test may fail due to udp packet loss. Let us mark the test as mayFail. Signed-off-by: Yonghong Song <yhs@fb.com>
…sor#3713) set CMP0074 to allow the use of `LLVM_ROOT` env var
Create examples/tracing/undump.py examples text file and update permission (+x) for undump.py.
By just running `$ sudo amazon-linux-extras install BCC`, dependencies are install. ``` $ sudo amazon-linux-extras install BCC ... ================================================================================================================================================================== Package Arch Version Repository Size ================================================================================================================================================================== Installing: bcc x86_64 0.18.0-1.amzn2.0.3 amzn2-core 28 M Installing for dependencies: bcc-tools x86_64 0.18.0-1.amzn2.0.3 amzn2-core 557 k clang-libs x86_64 11.1.0-1.amzn2.0.2 amzn2-core 22 M clang-resource-filesystem x86_64 11.1.0-1.amzn2.0.2 amzn2-core 17 k cpp10 x86_64 10.3.1-1.amzn2.0.1 amzn2-core 9.5 M elfutils-libelf-devel x86_64 0.176-2.amzn2 amzn2-core 40 k gcc10 x86_64 10.3.1-1.amzn2.0.1 amzn2-core 38 M gcc10-binutils x86_64 2.35-21.amzn2.0.1 amzn2-core 2.9 M gcc10-binutils-gold x86_64 2.35-21.amzn2.0.1 amzn2-core 795 k glibc-devel x86_64 2.26-56.amzn2 amzn2-core 994 k glibc-headers x86_64 2.26-56.amzn2 amzn2-core 514 k isl x86_64 0.16.1-6.amzn2 amzn2-core 833 k kernel-devel x86_64 5.10.75-79.358.amzn2 amzn2extra-kernel-5.10 16 M kernel-headers x86_64 5.10.75-79.358.amzn2 amzn2extra-kernel-5.10 1.3 M libbpf x86_64 0.3.0-2.amzn2.0.3 amzn2-core 102 k libmpc x86_64 1.0.1-3.amzn2.0.2 amzn2-core 52 k libzstd x86_64 1.3.3-1.amzn2.0.1 amzn2-core 203 k llvm-libs x86_64 11.1.0-1.amzn2.0.2 amzn2-core 22 M mpfr x86_64 3.1.1-4.amzn2.0.2 amzn2-core 208 k python3-bcc noarch 0.18.0-1.amzn2.0.3 amzn2-core 86 k python3-netaddr noarch 0.7.18-3.amzn2.0.2 amzn2-core 1.3 M zlib-devel x86_64 1.2.7-18.amzn2 amzn2-core 50 k ... ```
The data_loc field (defined as __string in kernel source) should be treated as string NOT a fixed-size array, add a new macro TP_DATA_LOC_READ_STR which use bpf_probe_read_str to reflect this. Signed-off-by: Hengqi Chen <chenhengqi@outlook.com>
Fixes iovisor#3720. Signed-off-by: Hengqi Chen <chenhengqi@outlook.com>
This makes few improvements:
* This can send much larger data payload and also adds
--max-buffer-size CLI option which allow changing this param.
* Fixes dealing with non ASCII protocols, previously struct was
defined as array of chars which made python ctypes treat it as
NULL terminated string and it prevents from displaying any data
past the null byte (which is very common for http2).
* Adds more filtering and displaying options (--print-uid,
--print-tid, --uid <uid>)
This also deals correctly with rare cases when bpf_probe_read_user fails
(so buffer should be empty and should not be displayed).
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
elgohr/Publish-Docker-Github-Action@master is not supported anymore