linux.git/arch, branch contended-release-qspinlock

x86/paravirt: Trace contended_release on unlock

2026-07-29T13:07:29Z

On PARAVIRT_SPINLOCKS=y kernels queued_spin_unlock() is dispatched through a static_call(), and those kernels are quite common. Gating contended_release behind a static branch there would leave a NOP on the unlock path even while the tracepoint is disabled. Since the static_call() is already present, swap its target to a traced unlock while the tracepoint is enabled instead. When it is disabled the target is the plain unlock (an inline store on native), so the unlock path is unchanged and the tracepoint is truly zero-cost. Implement the arch_contended_release_trace_reg() and arch_contended_release_trace_unreg() hooks to update the queued_spin_unlock static_call() on the tracepoint's enable and disable transitions. Provide two traced variants, native_queued_spin_unlock_traced() and pv_queued_spin_unlock_traced(), so each tail-calls its own base unlock directly rather than recursing through the now-traced static_call(). Teach pv_is_native_spin_unlock() that the traced native variant still counts as native. Only PARAVIRT_SPINLOCKS=y is affected. PARAVIRT_SPINLOCKS=n keeps the generic static-branch path. Signed-off-by: Dmitry Ilvokhin

locking: Factor out queued_spin_release()

2026-07-29T13:07:28Z

The contended_release tracepoint needs to hook queued_spin_unlock(), but architectures with a custom unlock define queued_spin_unlock() directly, leaving no single generic place to add the tracing. Introduce queued_spin_release() as the arch-overridable release primitive and make queued_spin_unlock() a generic wrapper around it. An architecture that only customizes the release can then override queued_spin_release() and inherit the generic wrapper. Rename the MIPS override to queued_spin_release() accordingly. x86 paravirt overrides queued_spin_unlock() directly and is left unchanged. No functional change intended. Signed-off-by: Dmitry Ilvokhin

x86/paravirt: Use static_call() for the paravirt spinlock ops

2026-07-29T13:07:28Z

From: Peter Zijlstra queued_spin_lock_slowpath() and queued_spin_unlock() are dispatched through pv_ops_lock via the paravirt-ops ALTERNATIVE machinery, which picks the target (native inline store / hypervisor call) once at boot and cannot change at runtime. Convert both to static_call(). The site becomes a direct call patched in place (one byte smaller), and on native the unlock still collapses to the inline "movb $0, (%rdi)" store, so the fast path is unchanged. Unlike the ALTERNATIVE mechanism, a static_call() target can also be updated at runtime via static_call_update(). This is a prerequisite for the contended_release tracepoint, which has to swap in a traced unlock while the system is running. Drop the now-unused pv_ops_lock.{queued_spin_lock_slowpath,queued_spin_unlock} members and the X86_FEATURE_PVUNLOCK feature bit. KVM, Xen and Hyper-V install their backends with static_call_update(), and pv_is_native_spin_unlock() reads the current target with static_call_query(). The native unlock is emitted inline at the call site from __static_call_transform(), mirroring the existing __static_call_return0 and __WARN_trap special cases. [ ilvokhin: fix PARAVIRT_SPINLOCKS=n build; teach __static_call_validate() about the inline unlock insn; make the slowpath site module-safe: static_call_mod() + EXPORT_STATIC_CALL_TRAMP(); pass @lock to the callee-save unlock, fixing a boot hang under CALL_DEPTH_TRACKING. Boot tested native + KVM PV guest. ] Link: https://lore.kernel.org/all/20260603120811.GW3493090@noisy.programming.kicks-ass.net/ Co-developed-by: Dmitry Ilvokhin Signed-off-by: Dmitry Ilvokhin

s390/runtime-const: Introduce runtime_const_mask_32()

2026-07-29T11:17:52Z

riscv/runtime-const: Introduce runtime_const_mask_32()

2026-07-29T11:17:45Z

Futex hash computation requires a mask operation with read-only after init data that will be converted to a runtime constant in the subsequent commit. Introduce runtime_const_mask_32 to further optimize the mask operation in the futex hash computation hot path. Since all the current use-cases are of the form GENMASK(n, 0), with n > 0, following sequence: srli a0, a1, imm slli a0, a0, imm is used for RISC-V where imm = (31 - width) to improve instruction density and performance. "The RISC-V Instruction Set Manual, Volume I - Unprivileged Architecture" [1] Sec. 2.4.1 "Integer Register-Immediate Instructions" notes the immediate shift for SRLI and SLLI are 5 bits wide starting at bit #10. __runtime_fixup_shift() is reused to patch the immediate shifts for the two instructions. If a future use case arises that needs to tackle arbitrary mask, consider using: lui a0, 0x12346 # upper; +0x800 then >>12 for correct rounding addi a0, a0, 0x678 # lower 12 bits to patch the 32-bit mask in the asm block and return "__ret & (val)" from runtime_const_mask_32() which allows compiler to further optimize the logical and operation. __runtime_fixup_ptr() already patches a lui + addi sequence which can be reused when the need arises. A possible implementation for this alternate scheme can be found at [2]. Suggested-by: Samuel Holland Suggested-by: Charlie Jenkins Assisted-by: Claude:claude-sonnet-4-5 Signed-off-by: K Prateek Nayak Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: Charlie Jenkins Tested-by: Charlie Jenkins Link: https://docs.riscv.org/reference/isa/_attachments/riscv-unprivileged.pdf [1] Link: https://lore.kernel.org/lkml/20260430094730.31624-6-kprateek.nayak@amd.com/ [2] Link: https://patch.msgid.link/20260728052540.4728-6-kprateek.nayak@amd.com

riscv/runtime-const: Replace open-coded placeholder with RUNTIME_MAGIC

2026-07-28T11:07:15Z

Define the placeholder used for lui + addi[w] patching sequence as RUNTIME_MAGIC and use that instead of open coding the constants in the inline assembly. No functional changes intended. Suggested-by: Guo Ren Signed-off-by: K Prateek Nayak Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: Charlie Jenkins Reviewed-by: Guo Ren Tested-by: Charlie Jenkins Link: https://patch.msgid.link/20260728052540.4728-5-kprateek.nayak@amd.com

arm64/runtime-const: Introduce runtime_const_mask_32()

2026-07-28T11:07:14Z

Futex hash computation requires a mask operation with read-only after init data that will be converted to a runtime constant in the subsequent commit. Introduce runtime_const_mask_32 to further optimize the mask operation in the futex hash computation hot path. Since all the current use-cases are of the form GENMASK(n, 0), with n > 0, a single: ubfx w0, w0, #0, #widthm1 // w0 = w0 [widthm1:0] instruction is used for amd64 to improve instruction dinsity and performance. "Arm A-profile A64 Instruction Set Architecture" manual, Sec. "A64 -- Base Instructions" [1] for UBFX instruction highlights the immediate "width" is encoded as width minus 1 in imms (Bits [15:10]) which is patched by __runtime_fixup_mask() once the mask is known. If a future use case arises that needs to tackle arbitrary mask, consider using: movz w1, #lo16, lsl #0 movk w1, #hi16, lsl #16 to patch the 32-bit mask in the asm block and return "__ret & (val)" from runtime_const_mask_32() which allows compiler to further optimize the logical and operation. __runtime_fixup_ptr() already patches a "movz, + movk lsl #16" sequence which can be reused when the need arises. A possible implementation for this alternate scheme can be found at [2]. Suggested-by: Samuel Holland Suggested-by: Charlie Jenkins Assisted-by: Claude:claude-sonnet-4-6 Signed-off-by: K Prateek Nayak Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: Charlie Jenkins Tested-by: Charlie Jenkins Link: https://developer.arm.com/documentation/ddi0602/2026-03/Base-Instructions/ [1] Link: https://lore.kernel.org/lkml/20260430094730.31624-4-kprateek.nayak@amd.com/ [2] Link: https://patch.msgid.link/20260728052540.4728-4-kprateek.nayak@amd.com

arm64/runtime-const: Use aarch64_insn_patch_text_nosync() for patching

2026-07-28T11:07:14Z

The current scheme to directly patch the kernel text for runtime constants runs into the following issue with futex adapted to using runtime constants on arm64: Unable to handle kernel write to read-only memory at virtual address ... The pc points to the *p assignment in the following call chain: futex_init() runtime_const_init(shift, __futex_shift) __runtime_fixup_shift() *p = cpu_to_le32(insn); which suggests that core_initcall() is too late to patch the kernel text directly unlike the "d_hash_shift" which is initialized during vfs_caches_init_early() before the protections are in place. Use aarch64_insn_patch_text_nosync() to patch the runtime constants instead of doing it directly to allow runtime_const_init() slightly later into the boot. Since aarch64_insn_patch_text_nosync() calls caches_clean_inval_pou() internally, __runtime_fixup_caches() ends up being redundant. runtime_const_init() are rare and the overheads of multiple calls to caches_clean_inval_pou() instead of batching them together should be negligible in practice. The cpu_to_le32() conversion of instruction isn't necessary since it is handled later in the aarch64_insn_patch_text_nosync() call-chain: aarch64_insn_patch_text_nosync(addr, insn) aarch64_insn_write(addr, insn) __aarch64_insn_write(addr, cpu_to_le32(insn)) Sashiko noted that aarch64_insn_patch_text_nosync() does not expect a lm_alias() address and Catalin suggested it is safe to drop the lm_alias() for runtime patching since the kernel text is readable. The address passed to fixup function is interpreted as a __le32 and dereferenced as is to read the opcode at the patch site. No functional changes are intended. Signed-off-by: K Prateek Nayak Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: Catalin Marinas Reviewed-by: Charlie Jenkins Tested-by: Charlie Jenkins Link: https://patch.msgid.link/20260728052540.4728-3-kprateek.nayak@amd.com

x86/runtime-const: Introduce runtime_const_mask_32()

2026-07-28T11:07:14Z

Merge tag 'x86-urgent-2026-07-05' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

2026-07-05T15:37:46Z

Pull x86 fix from Ingo Molnar: - Prevent OOB access in the resctrl code while offlining CPUs when Intel SNC (Sub-NUMA Clustering) is enabled (Reinette Chatre) * tag 'x86-urgent-2026-07-05' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: x86,fs/resctrl: Prevent out-of-bounds access while offlining CPU when SNC enabled