x86/kprobes: Fix crash when probing CS CALL instructionsgrafted

When using eBPF to probe CS CALL instructions within a function,
a crash can be triggered.

The eBPF tool probes offset 257 of the __hrtimer_run_queues()
function:

<__hrtimer_run_queues+249>:  nopl   0x0(%rax,%rax,1)
<__hrtimer_run_queues+254>:  mov    %r14,%rdi
<__hrtimer_run_queues+257>:  cs call <__x86_indirect_thunk_r12>
<__hrtimer_run_queues+263>:  mov    %eax,%r12d
<__hrtimer_run_queues+266>:  xchg   %ax,%ax
<__hrtimer_run_queues+268>:  mov    %r13,%rdi

Which triggers this crash:

  BUG: unable to handle page fault for address: 00000000000f41c9
  #PF: supervisor write access in kernel mode
  #PF: error_code(0x0002) - not-present page
  PGD 0 P4D 0
  Oops: 0002 [#1] SMP NOPTI
  CPU: 1 PID: 0 Comm: swapper/1 Kdump: loaded Tainted: P
  RIP: 0010:__hrtimer_run_queues+0x106/0x230

Note that __hrtimer_run_queues+0x106 is __hrtimer_run_queues+262, which is
at the 6th byte of the above CS CALL instruction. Since the CS CALL
instruction occupies 6 bytes, the exception occurred in the middle of that
call instruction.

The root cause is that when using eBPF tools to probe in the middle of a
function, a kprobe with INT3 is used as the underlying implementation.

During single-step emulation of the original CALL instruction,
int3_emulate_call() assumes that the probed CALL instruction is 5 bytes
long. However, the actual CS-prefixed CALL instruction occupies 6 bytes,
so it constructs an incorrect exception return address. When the CPU
returns from the kprobe handler, the next instruction to be executed is at
the address of the last byte of that CS CALL instruction. Coincidentally,
starting from that address, the CPU fetches and decodes a completely
different instruction, which ultimately triggers a kernel crash.

Fix the issue by using the actual instruction length obtained from
the instruction decoder when constructing the exception return
address, rather than relying on the hardcoded CALL_INSN_SIZE macro.

[ mingo: Refined the changelog ]

Fixes: 6256e668b7af ("x86/kprobes: Use int3 instead of debug trap for single-step")
Suggested-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Signed-off-by: Jinke Han <jinkehan@didiglobal.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Acked-by: Yafang Shao <laoar.shao@gmail.com>
Acked-by: Borislav Petkov <bp@alien8.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: https://patch.msgid.link/20260908073742.GA10517@didi-ThinkCentre-M920t-N000

1 files changed, 60 insertions, 0 deletions

diff --git a/Documentation/gpu/amdgpu/process-isolation.rst b/Documentation/gpu/amdgpu/process-isolation.rst
new file mode 100644
index 000000000..03c4288aa
--- /dev/null
+++ b/Documentation/gpu/amdgpu/process-isolation.rst
@@ -0,0 +1,60 @@
+.. _amdgpu-process-isolation:
+.. SPDX-License-Identifier: GPL-2.0
+
+=========================
+ AMDGPU Process Isolation
+=========================
+
+The AMDGPU driver includes a feature that enables automatic process isolation on the graphics engine. This feature serializes access to the graphics engine and adds a cleaner shader which clears the Local Data Store (LDS) and General Purpose Registers (GPRs) between jobs. All processes using the GPU, including both graphics and compute workloads, are serialized when this feature is enabled. On GPUs that support partitionable graphics engines, this feature can be enabled on a per-partition basis.
+
+In addition, there is an interface to manually run the cleaner shader when the use of the GPU is complete. This may be preferable in some use cases, such as a single-user system where the login manager triggers the cleaner shader when the user logs out.
+
+Process Isolation
+=================
+
+The `run_cleaner_shader` and `enforce_isolation` sysfs interfaces allow users to manually execute the cleaner shader and control the process isolation feature, respectively.
+
+Partition Handling
+------------------
+
+The `enforce_isolation` file in sysfs can be used to enable process isolation and automatic shader cleanup between processes. On GPUs that support graphics engine partitioning, this can be enabled per partition. The partition and its current setting (0 disabled, 1 enabled) can be read from sysfs. On GPUs that do not support graphics engine partitioning, only a single partition will be present. Writing 1 to the partition position enables enforce isolation, writing 0 disables it.
+
+Example of enabling enforce isolation on a GPU with multiple partitions:
+
+.. code-block:: console
+
+    $ echo 1 0 1 0 > /sys/class/drm/card0/device/enforce_isolation
+    $ cat /sys/class/drm/card0/device/enforce_isolation
+    1 0 1 0
+
+The output indicates that enforce isolation is enabled on zeroth and second partition and disabled on first and third partition.
+
+For devices with a single partition or those that do not support partitions, there will be only one element:
+
+.. code-block:: console
+
+    $ echo 1 > /sys/class/drm/card0/device/enforce_isolation
+    $ cat /sys/class/drm/card0/device/enforce_isolation
+    1
+
+Cleaner Shader Execution
+========================
+
+The driver can trigger a cleaner shader to clean up the LDS and GPR state on the graphics engine. When process isolation is enabled, this happens automatically between processes. In addition, there is a sysfs file to manually trigger cleaner shader execution.
+
+To manually trigger the execution of the cleaner shader, write `0` to the `run_cleaner_shader` sysfs file:
+
+.. code-block:: console
+
+    $ echo 0 > /sys/class/drm/card0/device/run_cleaner_shader
+
+For multi-partition devices, you can specify the partition index when triggering the cleaner shader:
+
+.. code-block:: console
+
+    $ echo 0 > /sys/class/drm/card0/device/run_cleaner_shader # For partition 0
+    $ echo 1 > /sys/class/drm/card0/device/run_cleaner_shader # For partition 1
+    $ echo 2 > /sys/class/drm/card0/device/run_cleaner_shader # For partition 2
+    # ... and so on for each partition
+
+This command initiates the cleaner shader, which will run and complete before any new tasks are scheduled on the GPU.