Bpf/optimized usdt ci #9219
Closed
Bpf/optimized usdt ci #9219
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
kernel-patches-daemon-bpf
bot
force-pushed
the
bpf-next_base
branch
from
4b7c0fe to
29dabce
Compare
olsajiri
force-pushed
the
bpf/optimized_usdt_ci
branch
5 times, most recently
from
8745062 to
201471b
Compare
kernel-patches-daemon-bpf
bot
force-pushed
the
bpf-next_base
branch
7 times, most recently
from
d0d73c5 to
6fbb3d3
Compare
Fix misspelling reported by codespell. Signed-off-by: Sumanth Gavini <[email protected]> Link: https://patch.msgid.link/[email protected] [ rjw: Subject rewrite ] Signed-off-by: Rafael J. Wysocki <[email protected]>
kernel-patches-daemon-bpf
bot
force-pushed
the
bpf-next_base
branch
2 times, most recently
from
08dbf36 to
cc01c15
Compare
olsajiri
force-pushed
the
bpf/optimized_usdt_ci
branch
from
55364d7 to
388a43e
Compare
Certain classful qdiscs may invoke their classes' dequeue handler on an
enqueue operation. This may unexpectedly empty the child qdisc and thus
make an in-flight class passive via qlen_notify(). Most qdiscs do not
expect such behaviour at this point in time and may re-activate the
class eventually anyways which will lead to a use-after-free.
The referenced fix commit attempted to fix this behavior for the HFSC
case by moving the backlog accounting around, though this turned out to
be incomplete since the parent's parent may run into the issue too.
The following reproducer demonstrates this use-after-free:
tc qdisc add dev lo root handle 1: drr
tc filter add dev lo parent 1: basic classid 1:1
tc class add dev lo parent 1: classid 1:1 drr
tc qdisc add dev lo parent 1:1 handle 2: hfsc def 1
tc class add dev lo parent 2: classid 2:1 hfsc rt m1 8 d 1 m2 0
tc qdisc add dev lo parent 2:1 handle 3: netem
tc qdisc add dev lo parent 3:1 handle 4: blackhole
echo 1 | socat -u STDIN UDP4-DATAGRAM:127.0.0.1:8888
tc class delete dev lo classid 1:1
echo 1 | socat -u STDIN UDP4-DATAGRAM:127.0.0.1:8888
Since backlog accounting issues leading to a use-after-frees on stale
class pointers is a recurring pattern at this point, this patch takes
a different approach. Instead of trying to fix the accounting, the patch
ensures that qdisc_tree_reduce_backlog always calls qlen_notify when
the child qdisc is empty. This solves the problem because deletion of
qdiscs always involves a call to qdisc_reset() and / or
qdisc_purge_queue() which ultimately resets its qlen to 0 thus causing
the following qdisc_tree_reduce_backlog() to report to the parent. Note
that this may call qlen_notify on passive classes multiple times. This
is not a problem after the recent patch series that made all the
classful qdiscs qlen_notify() handlers idempotent.
Fixes: 3f98113 ("sch_hfsc: Fix qlen accounting bug when using peek in hfsc_enqueue()")
Signed-off-by: Lion Ackermann <[email protected]>
Reviewed-by: Jamal Hadi Salim <[email protected]>
Acked-by: Cong Wang <[email protected]>
Acked-by: Jamal Hadi Salim <[email protected]>
Link: https://patch.msgid.link/[email protected]
Signed-off-by: Jakub Kicinski <[email protected]>
The link status is latched low so that momentary link drops can be detected. Always double-reading the status defeats this design feature. Only double read if link was already down This prevents unnecessary duplicate readings of the link status. Fixes: 4f3b20b ("amd-xgbe: add support for rx-adaptation") Signed-off-by: Raju Rangoju <[email protected]> Reviewed-by: Simon Horman <[email protected]> Link: https://patch.msgid.link/[email protected] Signed-off-by: Jakub Kicinski <[email protected]>
When setting "ethtool -L eth0 combined 1", the number of RX/TX queue is changed to be 1. RSS is disabled at this moment, and the indices of FDIR have not be changed in wx_set_rss_queues(). So the combined count still shows the previous value. This issue was introduced when supporting FDIR. Fix it for those devices that support FDIR. Fixes: 34744a7 ("net: txgbe: add FDIR info to ethtool ops") Cc: [email protected] Signed-off-by: Jiawen Wu <[email protected]> Reviewed-by: Simon Horman <[email protected]> Link: https://patch.msgid.link/[email protected] Signed-off-by: Jakub Kicinski <[email protected]>
udp_v4_early_demux now returns drop reasons as it either returns 0 or ip_mc_validate_source, which returns itself a drop reason. However its use was not converted in ip_rcv_finish_core and the drop reason is ignored, leading to potentially skipping increasing LINUX_MIB_IPRPFILTER if the drop reason is SKB_DROP_REASON_IP_RPFILTER. This is a fix and we're not converting udp_v4_early_demux to explicitly return a drop reason to ease backports; this can be done as a follow-up. Fixes: d46f827 ("net: ip: make ip_mc_validate_source() return drop reason") Cc: Menglong Dong <[email protected]> Reported-by: Sabrina Dubroca <[email protected]> Signed-off-by: Antoine Tenart <[email protected]> Reviewed-by: Sabrina Dubroca <[email protected]> Link: https://patch.msgid.link/[email protected] Signed-off-by: Jakub Kicinski <[email protected]>
…t/tnguy/net-queue Tony Nguyen says: ==================== Intel Wired LAN Driver Updates 2025-07-01 (idpf, igc) For idpf: Michal returns 0 for key size when RSS is not supported. Ahmed changes control queue to a spinlock due to sleeping calls. For igc: Vitaly disables L1.2 PCI-E link substate on I226 devices to resolve performance issues. * '200GbE' of git://git.kernel.org/pub/scm/linux/kernel/git/tnguy/net-queue: igc: disable L1.2 PCI-E link substate to avoid performance issue idpf: convert control queue mutex to a spinlock idpf: return 0 size for RSS key if not supported ==================== Link: https://patch.msgid.link/[email protected] Signed-off-by: Jakub Kicinski <[email protected]>
Currently, an interrupt can be triggered during a GPU reset, which can lead to GPU hangs and NULL pointer dereference in an interrupt context as shown in the following trace: [ 314.035040] Unable to handle kernel NULL pointer dereference at virtual address 00000000000000c0 [ 314.043822] Mem abort info: [ 314.046606] ESR = 0x0000000096000005 [ 314.050347] EC = 0x25: DABT (current EL), IL = 32 bits [ 314.055651] SET = 0, FnV = 0 [ 314.058695] EA = 0, S1PTW = 0 [ 314.061826] FSC = 0x05: level 1 translation fault [ 314.066694] Data abort info: [ 314.069564] ISV = 0, ISS = 0x00000005, ISS2 = 0x00000000 [ 314.075039] CM = 0, WnR = 0, TnD = 0, TagAccess = 0 [ 314.080080] GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0 [ 314.085382] user pgtable: 4k pages, 39-bit VAs, pgdp=0000000102728000 [ 314.091814] [00000000000000c0] pgd=0000000000000000, p4d=0000000000000000, pud=0000000000000000 [ 314.100511] Internal error: Oops: 0000000096000005 [kernel-patches#1] PREEMPT SMP [ 314.106770] Modules linked in: v3d i2c_brcmstb vc4 snd_soc_hdmi_codec gpu_sched drm_shmem_helper drm_display_helper cec drm_dma_helper drm_kms_helper drm drm_panel_orientation_quirks snd_soc_core snd_compress snd_pcm_dmaengine snd_pcm snd_timer snd backlight [ 314.129654] CPU: 0 UID: 0 PID: 0 Comm: swapper/0 Not tainted 6.12.25+rpt-rpi-v8 kernel-patches#1 Debian 1:6.12.25-1+rpt1 [ 314.139388] Hardware name: Raspberry Pi 4 Model B Rev 1.4 (DT) [ 314.145211] pstate: 600000c5 (nZCv daIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--) [ 314.152165] pc : v3d_irq+0xec/0x2e0 [v3d] [ 314.156187] lr : v3d_irq+0xe0/0x2e0 [v3d] [ 314.160198] sp : ffffffc080003ea0 [ 314.163502] x29: ffffffc080003ea0 x28: ffffffec1f184980 x27: 021202b000000000 [ 314.170633] x26: ffffffec1f17f630 x25: ffffff8101372000 x24: ffffffec1f17d9f0 [ 314.177764] x23: 000000000000002a x22: 000000000000002a x21: ffffff8103252000 [ 314.184895] x20: 0000000000000001 x19: 00000000deadbeef x18: 0000000000000000 [ 314.192026] x17: ffffff94e51d2000 x16: ffffffec1dac3cb0 x15: c306000000000000 [ 314.199156] x14: 0000000000000000 x13: b2fc982e03cc5168 x12: 0000000000000001 [ 314.206286] x11: ffffff8103f8bcc0 x10: ffffffec1f196868 x9 : ffffffec1dac3874 [ 314.213416] x8 : 0000000000000000 x7 : 0000000000042a3a x6 : ffffff810017a180 [ 314.220547] x5 : ffffffec1ebad400 x4 : ffffffec1ebad320 x3 : 00000000000bebeb [ 314.227677] x2 : 0000000000000000 x1 : 0000000000000000 x0 : 0000000000000000 [ 314.234807] Call trace: [ 314.237243] v3d_irq+0xec/0x2e0 [v3d] [ 314.240906] __handle_irq_event_percpu+0x58/0x218 [ 314.245609] handle_irq_event+0x54/0xb8 [ 314.249439] handle_fasteoi_irq+0xac/0x240 [ 314.253527] handle_irq_desc+0x48/0x68 [ 314.257269] generic_handle_domain_irq+0x24/0x38 [ 314.261879] gic_handle_irq+0x48/0xd8 [ 314.265533] call_on_irq_stack+0x24/0x58 [ 314.269448] do_interrupt_handler+0x88/0x98 [ 314.273624] el1_interrupt+0x34/0x68 [ 314.277193] el1h_64_irq_handler+0x18/0x28 [ 314.281281] el1h_64_irq+0x64/0x68 [ 314.284673] default_idle_call+0x3c/0x168 [ 314.288675] do_idle+0x1fc/0x230 [ 314.291895] cpu_startup_entry+0x3c/0x50 [ 314.295810] rest_init+0xe4/0xf0 [ 314.299030] start_kernel+0x5e8/0x790 [ 314.302684] __primary_switched+0x80/0x90 [ 314.306691] Code: 940029eb 360ffc13 f9442ea0 52800001 (f9406017) [ 314.312775] ---[ end trace 0000000000000000 ]--- [ 314.317384] Kernel panic - not syncing: Oops: Fatal exception in interrupt [ 314.324249] SMP: stopping secondary CPUs [ 314.328167] Kernel Offset: 0x2b9da00000 from 0xffffffc080000000 [ 314.334076] PHYS_OFFSET: 0x0 [ 314.336946] CPU features: 0x08,00002013,c0200000,0200421b [ 314.342337] Memory Limit: none [ 314.345382] ---[ end Kernel panic - not syncing: Oops: Fatal exception in interrupt ]--- Before resetting the GPU, it's necessary to disable all interrupts and deal with any interrupt handler still in-flight. Otherwise, the GPU might reset with jobs still running, or yet, an interrupt could be handled during the reset. Cc: [email protected] Fixes: 57692c9 ("drm/v3d: Introduce a new DRM driver for Broadcom V3D V3.x+") Reviewed-by: Juan A. Suarez <[email protected]> Reviewed-by: Iago Toral Quiroga <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Maíra Canal <[email protected]>
olsajiri
force-pushed
the
bpf/optimized_usdt_ci
branch
from
388a43e to
c44bf1c
Compare
In appletb_kbd_probe an input handler is initialised and then registered with input core through input_register_handler(). When this happens input core will add the input handler (specifically its node) to the global input_handler_list. The input_handler_list is central to the functionality of input core and is traversed in various places in input core. An example of this is when a new input device is plugged in and gets registered with input core. The input_handler in probe is allocated as device managed memory. If a probe failure occurs after input_register_handler() the input_handler memory is freed, yet it will remain in the input_handler_list. This effectively means the input_handler_list contains a dangling pointer to data belonging to a freed input handler. This causes an issue when any other input device is plugged in - in my case I had an old PixArt HP USB optical mouse and I decided to plug it in after a failure occurred after input_register_handler(). This lead to the registration of this input device via input_register_device which involves traversing over every handler in the corrupted input_handler_list and calling input_attach_handler(), giving each handler a chance to bind to newly registered device. The core of this bug is a UAF which causes memory corruption of input_handler_list and to fix it we must ensure the input handler is unregistered from input core, this is done through input_unregister_handler(). [ 63.191597] ================================================================== [ 63.192094] BUG: KASAN: slab-use-after-free in input_attach_handler.isra.0+0x1a9/0x1e0 [ 63.192094] Read of size 8 at addr ffff888105ea7c80 by task kworker/0:2/54 [ 63.192094] [ 63.192094] CPU: 0 UID: 0 PID: 54 Comm: kworker/0:2 Not tainted 6.16.0-rc2-00321-g2aa6621d [ 63.192094] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.2-debian-1.164 [ 63.192094] Workqueue: usb_hub_wq hub_event [ 63.192094] Call Trace: [ 63.192094] <TASK> [ 63.192094] dump_stack_lvl+0x53/0x70 [ 63.192094] print_report+0xce/0x670 [ 63.192094] kasan_report+0xce/0x100 [ 63.192094] input_attach_handler.isra.0+0x1a9/0x1e0 [ 63.192094] input_register_device+0x76c/0xd00 [ 63.192094] hidinput_connect+0x686d/0xad60 [ 63.192094] hid_connect+0xf20/0x1b10 [ 63.192094] hid_hw_start+0x83/0x100 [ 63.192094] hid_device_probe+0x2d1/0x680 [ 63.192094] really_probe+0x1c3/0x690 [ 63.192094] __driver_probe_device+0x247/0x300 [ 63.192094] driver_probe_device+0x49/0x210 [ 63.192094] __device_attach_driver+0x160/0x320 [ 63.192094] bus_for_each_drv+0x10f/0x190 [ 63.192094] __device_attach+0x18e/0x370 [ 63.192094] bus_probe_device+0x123/0x170 [ 63.192094] device_add+0xd4d/0x1460 [ 63.192094] hid_add_device+0x30b/0x910 [ 63.192094] usbhid_probe+0x920/0xe00 [ 63.192094] usb_probe_interface+0x363/0x9a0 [ 63.192094] really_probe+0x1c3/0x690 [ 63.192094] __driver_probe_device+0x247/0x300 [ 63.192094] driver_probe_device+0x49/0x210 [ 63.192094] __device_attach_driver+0x160/0x320 [ 63.192094] bus_for_each_drv+0x10f/0x190 [ 63.192094] __device_attach+0x18e/0x370 [ 63.192094] bus_probe_device+0x123/0x170 [ 63.192094] device_add+0xd4d/0x1460 [ 63.192094] usb_set_configuration+0xd14/0x1880 [ 63.192094] usb_generic_driver_probe+0x78/0xb0 [ 63.192094] usb_probe_device+0xaa/0x2e0 [ 63.192094] really_probe+0x1c3/0x690 [ 63.192094] __driver_probe_device+0x247/0x300 [ 63.192094] driver_probe_device+0x49/0x210 [ 63.192094] __device_attach_driver+0x160/0x320 [ 63.192094] bus_for_each_drv+0x10f/0x190 [ 63.192094] __device_attach+0x18e/0x370 [ 63.192094] bus_probe_device+0x123/0x170 [ 63.192094] device_add+0xd4d/0x1460 [ 63.192094] usb_new_device+0x7b4/0x1000 [ 63.192094] hub_event+0x234d/0x3fa0 [ 63.192094] process_one_work+0x5bf/0xfe0 [ 63.192094] worker_thread+0x777/0x13a0 [ 63.192094] </TASK> [ 63.192094] [ 63.192094] Allocated by task 54: [ 63.192094] kasan_save_stack+0x33/0x60 [ 63.192094] kasan_save_track+0x14/0x30 [ 63.192094] __kasan_kmalloc+0x8f/0xa0 [ 63.192094] __kmalloc_node_track_caller_noprof+0x195/0x420 [ 63.192094] devm_kmalloc+0x74/0x1e0 [ 63.192094] appletb_kbd_probe+0x39/0x440 [ 63.192094] hid_device_probe+0x2d1/0x680 [ 63.192094] really_probe+0x1c3/0x690 [ 63.192094] __driver_probe_device+0x247/0x300 [ 63.192094] driver_probe_device+0x49/0x210 [ 63.192094] __device_attach_driver+0x160/0x320 [...] [ 63.192094] [ 63.192094] Freed by task 54: [ 63.192094] kasan_save_stack+0x33/0x60 [ 63.192094] kasan_save_track+0x14/0x30 [ 63.192094] kasan_save_free_info+0x3b/0x60 [ 63.192094] __kasan_slab_free+0x37/0x50 [ 63.192094] kfree+0xcf/0x360 [ 63.192094] devres_release_group+0x1f8/0x3c0 [ 63.192094] hid_device_probe+0x315/0x680 [ 63.192094] really_probe+0x1c3/0x690 [ 63.192094] __driver_probe_device+0x247/0x300 [ 63.192094] driver_probe_device+0x49/0x210 [ 63.192094] __device_attach_driver+0x160/0x320 [...] Fixes: 7d62ba8 ("HID: hid-appletb-kbd: add support for fn toggle between media and function mode") Cc: [email protected] Reviewed-by: Aditya Garg <[email protected]> Signed-off-by: Qasim Ijaz <[email protected]> Signed-off-by: Jiri Kosina <[email protected]>
The ELECOM M-HT1DRBK trackball has an additional device ID (056E:019B) not yet recognized by the driver, despite using the same report descriptor as earlier variants. This patch adds the new ID and applies the same fixups, enabling all 8 buttons to function properly. Signed-off-by: Leonard Dizon <[email protected]> Signed-off-by: Jiri Kosina <[email protected]>
The name of BTN_GEAR_DOWN was WheelBtn and BTN_WHEEL was missing. Further, BTN_GEAR_UP had a space in its name and no Btn, which is against convention. This makes the names BtnGearDown, BtnGearUp, and BtnWheel, fixing the errors and matching convention. Signed-off-by: Vicki Pfau <[email protected]> Signed-off-by: Jiri Kosina <[email protected]>
…ux/kernel/git/boqun/linux into sched/core Rust task & schedule changes for v6.17: - Make Task, CondVar and PollCondVar methods inline to avoid unnecessary function calls - Add might_sleep() support for Rust code: Rust's "#[track_caller]" mechanism is used so that Rust's might_sleep() doesn't need to be defined as a macro Signed-off-by: Peter Zijlstra <[email protected]> # -----BEGIN PGP SIGNATURE----- # # iQEzBAABCAAdFiEEj5IosQTPz8XU1wRHSXnow7UH+rgFAmhbLOgACgkQSXnow7UH # +riscwf/e/+KmJTox5/JOqs6yxxQdCHMaGnMK62E5AII7NsiUI8+XB9z6efzCMmy # kS2W7aCmBZX67Y1B/xRL/ArHMBAJBi/CrCedZJcmzfB9aMa4Lj4mgiPkbUXkxE6Q # F5CDQK21ftu+0Q7Hhlq92ec17ZWodOvNxCFBBmjtQqUvBzj0dY45jcG7brs+N+1Z # t9UO3YokzukNqpIXTpG0HFP+XNafWWCgn9iIQ44lRxIaAoPI44uJjh1OXLTrZ1M9 # EMWYIrsY3b71Im78l6pzr+UOzJdJLI+QCBiz7ySLYz3kZ5dEfFdJOsumbc0G8A69 # VSGDFPEbJxZYuMxrH0E44XmxH4rJdA== # =gD/Y # -----END PGP SIGNATURE----- # gpg: Signature made Wed 25 Jun 2025 12:55:36 AM CEST # gpg: using RSA key 8F9228B104CFCFC5D4D704474979E8C3B507FAB8 # gpg: Can't check signature: No public key
Jann reports that uprobes can be used destructively when used in the middle of an instruction. The kernel only verifies there is a valid instruction at the requested offset, but due to variable instruction length cannot determine if this is an instruction as seen by the intended execution stream. Additionally, Mark Rutland notes that on architectures that mix data in the text segment (like arm64), a similar things can be done if the data word is 'mistaken' for an instruction. As such, require CAP_SYS_ADMIN for uprobes. Fixes: c9e0924 ("perf/core: open access to probes for CAP_PERFMON privileged process") Reported-by: Jann Horn <[email protected]> Signed-off-by: Peter Zijlstra (Intel) <[email protected]> Link: https://lkml.kernel.org/r/CAG48ez1n4520sq0XrWYDHKiKxE_+WCfAK+qt9qkY4ZiBGmL-5g@mail.gmail.com
In xdp_linearize_page, when reading the following buffers from the ring, we forget to check the received length with the true allocate size. This can lead to an out-of-bound read. This commit adds that missing check. Cc: <[email protected]> Fixes: 4941d47 ("virtio-net: do not reset during XDP set") Signed-off-by: Bui Quang Minh <[email protected]> Acked-by: Jason Wang <[email protected]> Link: https://patch.msgid.link/[email protected] Signed-off-by: Paolo Abeni <[email protected]>
* tip/x86/platform: x86/itmt: Add debugfs file to show core priorities platform/x86/amd: hfi: Add debugfs support platform/x86/amd: hfi: Set ITMT priority from ranking data cpufreq/amd-pstate: Disable preferred cores on designs with workload classification x86/process: Clear hardware feedback history for AMD processors platform/x86: hfi: Add power management callback platform/x86: hfi: Add online and offline callback support platform/x86: hfi: Init per-cpu scores for each class platform/x86: hfi: Parse CPU core ranking data from shared memory platform/x86: hfi: Introduce AMD Hardware Feedback Interface Driver x86/msr-index: Add AMD workload classification MSRs MAINTAINERS: Add maintainer entry for AMD Hardware Feedback Driver Documentation/x86: Add AMD Hardware Feedback Interface documentation Signed-off-by: Borislav Petkov (AMD) <[email protected]>
Currently unapply_uprobe takes mmap_read_lock, but it might call remove_breakpoint which eventually changes user pages. Current code writes either breakpoint or original instruction, so it can go away with read lock as explained in here [1]. But with the upcoming change that writes multiple instructions on the probed address we need to ensure that any update to mm's pages is exclusive. [1] https://lore.kernel.org/all/[email protected]/ Acked-by: Oleg Nesterov <[email protected]> Signed-off-by: Jiri Olsa <[email protected]>
We are about to add uprobe trampoline, so cleaning up the namespace. Acked-by: Andrii Nakryiko <[email protected]> Acked-by: Oleg Nesterov <[email protected]> Signed-off-by: Jiri Olsa <[email protected]>
Making copy_from_page global and adding uprobe prefix. Adding the uprobe prefix to copy_to_page as well for symmetry. Acked-by: Andrii Nakryiko <[email protected]> Acked-by: Oleg Nesterov <[email protected]> Signed-off-by: Jiri Olsa <[email protected]>
Adding uprobe_write function that does what uprobe_write_opcode did so far, but allows to pass verify callback function that checks the memory location before writing the opcode. It will be used in following changes to implement specific checking logic for instruction update. The uprobe_write_opcode now calls uprobe_write with verify_opcode as the verify callback. Acked-by: Oleg Nesterov <[email protected]> Signed-off-by: Jiri Olsa <[email protected]>
Adding nbytes argument to uprobe_write and related functions as preparation for writing whole instructions in following changes. Also renaming opcode arguments to insn, which seems to fit better. Acked-by: Andrii Nakryiko <[email protected]> Acked-by: Oleg Nesterov <[email protected]> Signed-off-by: Jiri Olsa <[email protected]>
…code The uprobe_write has special path to restore the original page when we write original instruction back. This happens when uprobe_write detects that we want to write anything else but breakpoint instruction. Moving the detection away and passing it to uprobe_write as argument, so it's possible to write different instructions (other than just breakpoint and rest). Acked-by: Andrii Nakryiko <[email protected]> Acked-by: Oleg Nesterov <[email protected]> Signed-off-by: Jiri Olsa <[email protected]>
Making update_ref_ctr call in uprobe_write conditional based on do_ref_ctr argument. This way we can use uprobe_write for instruction update without doing ref_ctr_offset update. Acked-by: Oleg Nesterov <[email protected]> Signed-off-by: Jiri Olsa <[email protected]>
Adding support to add special mapping for user space trampoline with following functions: uprobe_trampoline_get - find or add uprobe_trampoline uprobe_trampoline_put - remove or destroy uprobe_trampoline The user space trampoline is exported as arch specific user space special mapping through tramp_mapping, which is initialized in following changes with new uprobe syscall. The uprobe trampoline needs to be callable/reachable from the probed address, so while searching for available address we use is_reachable_by_call function to decide if the uprobe trampoline is callable from the probe address. All uprobe_trampoline objects are stored in uprobes_state object and are cleaned up when the process mm_struct goes down. Adding new arch hooks for that, because this change is x86_64 specific. Locking is provided by callers in following changes. Acked-by: Oleg Nesterov <[email protected]> Signed-off-by: Jiri Olsa <[email protected]>
Adding new uprobe syscall that calls uprobe handlers for given 'breakpoint' address. The idea is that the 'breakpoint' address calls the user space trampoline which executes the uprobe syscall. The syscall handler reads the return address of the initial call to retrieve the original 'breakpoint' address. With this address we find the related uprobe object and call its consumers. Adding the arch_uprobe_trampoline_mapping function that provides uprobe trampoline mapping. This mapping is backed with one global page initialized at __init time and shared by the all the mapping instances. We do not allow to execute uprobe syscall if the caller is not from uprobe trampoline mapping. The uprobe syscall ensures the consumer (bpf program) sees registers values in the state before the trampoline was called. Acked-by: Andrii Nakryiko <[email protected]> Acked-by: Oleg Nesterov <[email protected]> Signed-off-by: Jiri Olsa <[email protected]>
Putting together all the previously added pieces to support optimized
uprobes on top of 5-byte nop instruction.
The current uprobe execution goes through following:
- installs breakpoint instruction over original instruction
- exception handler hit and calls related uprobe consumers
- and either simulates original instruction or does out of line single step
execution of it
- returns to user space
The optimized uprobe path does following:
- checks the original instruction is 5-byte nop (plus other checks)
- adds (or uses existing) user space trampoline with uprobe syscall
- overwrites original instruction (5-byte nop) with call to user space
trampoline
- the user space trampoline executes uprobe syscall that calls related uprobe
consumers
- trampoline returns back to next instruction
This approach won't speed up all uprobes as it's limited to using nop5 as
original instruction, but we plan to use nop5 as USDT probe instruction
(which currently uses single byte nop) and speed up the USDT probes.
The arch_uprobe_optimize triggers the uprobe optimization and is called after
first uprobe hit. I originally had it called on uprobe installation but then
it clashed with elf loader, because the user space trampoline was added in a
place where loader might need to put elf segments, so I decided to do it after
first uprobe hit when loading is done.
The uprobe is un-optimized in arch specific set_orig_insn call.
The instruction overwrite is x86 arch specific and needs to go through 3 updates:
(on top of nop5 instruction)
- write int3 into 1st byte
- write last 4 bytes of the call instruction
- update the call instruction opcode
And cleanup goes though similar reverse stages:
- overwrite call opcode with breakpoint (int3)
- write last 4 bytes of the nop5 instruction
- write the nop5 first instruction byte
We do not unmap and release uprobe trampoline when it's no longer needed,
because there's no easy way to make sure none of the threads is still
inside the trampoline. But we do not waste memory, because there's just
single page for all the uprobe trampoline mappings.
We do waste frame on page mapping for every 4GB by keeping the uprobe
trampoline page mapped, but that seems ok.
We take the benefit from the fact that set_swbp and set_orig_insn are
called under mmap_write_lock(mm), so we can use the current instruction
as the state the uprobe is in - nop5/breakpoint/call trampoline -
and decide the needed action (optimize/un-optimize) based on that.
Attaching the speed up from benchs/run_bench_uprobes.sh script:
current:
usermode-count : 152.604 ± 0.044M/s
syscall-count : 13.359 ± 0.042M/s
--> uprobe-nop : 3.229 ± 0.002M/s
uprobe-push : 3.086 ± 0.004M/s
uprobe-ret : 1.114 ± 0.004M/s
uprobe-nop5 : 1.121 ± 0.005M/s
uretprobe-nop : 2.145 ± 0.002M/s
uretprobe-push : 2.070 ± 0.001M/s
uretprobe-ret : 0.931 ± 0.001M/s
uretprobe-nop5 : 0.957 ± 0.001M/s
after the change:
usermode-count : 152.448 ± 0.244M/s
syscall-count : 14.321 ± 0.059M/s
uprobe-nop : 3.148 ± 0.007M/s
uprobe-push : 2.976 ± 0.004M/s
uprobe-ret : 1.068 ± 0.003M/s
--> uprobe-nop5 : 7.038 ± 0.007M/s
uretprobe-nop : 2.109 ± 0.004M/s
uretprobe-push : 2.035 ± 0.001M/s
uretprobe-ret : 0.908 ± 0.001M/s
uretprobe-nop5 : 3.377 ± 0.009M/s
I see bit more speed up on Intel (above) compared to AMD. The big nop5
speed up is partly due to emulating nop5 and partly due to optimization.
The key speed up we do this for is the USDT switch from nop to nop5:
uprobe-nop : 3.148 ± 0.007M/s
uprobe-nop5 : 7.038 ± 0.007M/s
Acked-by: Andrii Nakryiko <[email protected]>
Acked-by: Oleg Nesterov <[email protected]>
Signed-off-by: Jiri Olsa <[email protected]>
Importing usdt.h from libbpf/usdt project. Suggested-by: Andrii Nakryiko <[email protected]> Acked-by: Andrii Nakryiko <[email protected]> Signed-off-by: Jiri Olsa <[email protected]>
Adding __test_uprobe_syscall with non x86_64 stub to execute all the tests, so we don't need to keep adding non x86_64 stub functions for new tests. Acked-by: Andrii Nakryiko <[email protected]> Signed-off-by: Jiri Olsa <[email protected]>
…multi Renaming uprobe_syscall_executed prog to test_uretprobe_multi to fit properly in the following changes that add more programs. Plus adding pid filter and increasing executed variable. Acked-by: Andrii Nakryiko <[email protected]> Signed-off-by: Jiri Olsa <[email protected]>
Adding tests for optimized uprobe/usdt probes. Checking that we get expected trampoline and attached bpf programs get executed properly. Signed-off-by: Jiri Olsa <[email protected]>
Adding test that makes sure parallel execution of the uprobe and attach/detach of optimized uprobe on it works properly. By default the test runs for 500ms, which is adjustable by using BPF_SELFTESTS_UPROBE_SYSCALL_RACE_MSEC env variable. Signed-off-by: Jiri Olsa <[email protected]>
Make sure that calling uprobe syscall from outside uprobe trampoline results in sigill signal. Signed-off-by: Jiri Olsa <[email protected]>
Adding optimized usdt variant for basic usdt test to check that usdt arguments are properly passed in optimized code path. Acked-by: Andrii Nakryiko <[email protected]> Signed-off-by: Jiri Olsa <[email protected]>
Changing uretprobe_regs_trigger to allow the test for both uprobe and uretprobe and renaming it to uprobe_regs_equal. We check that both uprobe and uretprobe probes (bpf programs) see expected registers with few exceptions. Signed-off-by: Jiri Olsa <[email protected]>
…robe Changing the test_uretprobe_regs_change test to test both uprobe and uretprobe by adding entry consumer handler to the testmod and making it to change one of the registers. Making sure that changed values both uprobe and uretprobe handlers propagate to the user space. Signed-off-by: Jiri Olsa <[email protected]>
Adding uprobe as another exception to the seccomp filter alongside with the uretprobe syscall. Same as the uretprobe the uprobe syscall is installed by kernel as replacement for the breakpoint exception and is limited to x86_64 arch and isn't expected to ever be supported in i386. Cc: Kees Cook <[email protected]> Cc: Eyal Birger <[email protected]> Reviewed-by: Kees Cook <[email protected]> Signed-off-by: Jiri Olsa <[email protected]>
Adding uprobe checks into the current uretprobe tests. All the related tests are now executed with attached uprobe or uretprobe or without any probe. Renaming the test fixture to uprobe, because it seems better. Cc: Kees Cook <[email protected]> Cc: Eyal Birger <[email protected]> Reviewed-by: Kees Cook <[email protected]> Signed-off-by: Jiri Olsa <[email protected]>
…to bpf/optimized_usdt_ci
olsajiri
force-pushed
the
bpf/optimized_usdt_ci
branch
from
919c917 to
7d46c95
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
test