From 9cc63f8bcd560c760d0b12e15bba8f81c86237cf Mon Sep 17 00:00:00 2001 From: "Paul E. McKenney" Date: Fri, 14 Aug 2026 15:43:51 -0700 Subject: Merge branches 'expcb.2026.07.24a', 'misc.2026.07.30a', 'rcu-tasks.2026.07.30a', 'srcu.2026.08.11a' and 'torture.2026.08.14a' into HEAD Changes: Make expedited grace periods expedite normal RCU callbacks Miscellaneous fixes: * Improve diagnostic output with character task states. * Mark accesses to inform KCSAN of concurrency design. * Move from kmalloc() to kmalloc_obj(). * Documentation updates. * Improve handling of RCU deferred quiescent states. * Clean up unused function arguments and structure fields. * Reduce show_rcu_gp_kthreads() stack space. Tasks RCU updates: * Clean up after SRCU re-implementation of Tasks Trace RCU. * Mark accesses to inform KCSAN of concurrency design. * Add ->lazy_timer status to diagnostic output. * Remove an unnecessary memory barrier. * Fix a data race, courtesy of KCSAN. * Documentation updates. * Convert cond_resched_tasks_rcu_qs() from macro to static inline function. SRCU updates: * Add Rust helpers for SRCU. * Avoid losing queued work at cleanup_srcu_struct() time. Torture-test updates: * Preparation work for immediate RCU priority deboosting. * Test RCU readers from real interrupt handlers (as opposed to softirq). * Simplify code through use of cpumask_next_wrap(). * Improve diagnostic output with character task states. * Add rcutorture.nwriters parameter to allow lightweight stall testing, and rcutorture.stall_only to make doing so easier. * Test an RCU Tasks Trace grace period implying an RCU grace period. * Make RCU Tasks Trace torturing track reader batches. * Fix a data race, courtesy of KCSAN. * Plug a shuffle_tmp_mask memory leak on kthread spawn failure. --- Documentation/gpu/amdgpu/ring-buffer.rst | 95 ++++++++++++++++++++++++++++++++ 1 file changed, 95 insertions(+) create mode 100644 Documentation/gpu/amdgpu/ring-buffer.rst (limited to 'Documentation/gpu/amdgpu/ring-buffer.rst') diff --git a/Documentation/gpu/amdgpu/ring-buffer.rst b/Documentation/gpu/amdgpu/ring-buffer.rst new file mode 100644 index 000000000..cc642c213 --- /dev/null +++ b/Documentation/gpu/amdgpu/ring-buffer.rst @@ -0,0 +1,95 @@ +============= + Ring Buffer +============= + +To handle communication between user space and kernel space, AMD GPUs use a +ring buffer design to feed the engines (GFX, Compute, SDMA, UVD, VCE, VCN, VPE, +etc.). See the figure below that illustrates how this communication works: + +.. kernel-figure:: ring_buffers.svg + +Ring buffers in the amdgpu work as a producer-consumer model, where userspace +acts as the producer, constantly filling the ring buffer with GPU commands to +be executed. Meanwhile, the GPU retrieves the information from the ring, parses +it, and distributes the specific set of instructions between the different +amdgpu blocks. + +Notice from the diagram that the ring has a Read Pointer (rptr), which +indicates where the engine is currently reading packets from the ring, and a +Write Pointer (wptr), which indicates how many packets software has added to +the ring. When the rptr and wptr are equal, the ring is idle. When software +adds packets to the ring, it updates the wptr, this causes the engine to start +fetching and processing packets. As the engine processes packets, the rptr gets +updates until the rptr catches up to the wptr and they are equal again. + +Usually, ring buffers in the driver have a limited size (search for occurrences +of `amdgpu_ring_init()`). One of the reasons for the small ring buffer size is +that CP (Command Processor) is capable of following addresses inserted into the +ring; this is illustrated in the image by the reference to the IB (Indirect +Buffer). The IB gives userspace the possibility to have an area in memory that +CP can read and feed the hardware with extra instructions. + +All ASICs pre-GFX11 use what is called a kernel queue, which means +the ring is allocated in kernel space and has some restrictions, such as not +being able to be :ref:`preempted directly by the scheduler`. GFX11 +and newer support kernel queues, but also provide a new mechanism named +:ref:`user queues`, where the queue is moved to the user space +and can be mapped and unmapped via the scheduler. In practice, both queues +insert user-space-generated GPU commands from different jobs into the requested +component ring. + +Enforce Isolation +================= + +.. note:: After reading this section, you might want to check the + :ref:`Process Isolation` page for more details. + +Before examining the Enforce Isolation mechanism in the ring buffer context, it +is helpful to briefly discuss how instructions from the ring buffer are +processed in the graphics pipeline. Let’s expand on this topic by checking the +diagram below that illustrates the graphics pipeline: + +.. kernel-figure:: gfx_pipeline_seq.svg + +In terms of executing instructions, the GFX pipeline follows the sequence: +Shader Export (SX), Geometry Engine (GE), Shader Process or Input (SPI), Scan +Converter (SC), Primitive Assembler (PA), and cache manipulation (which may +vary across ASICs). Another common way to describe the pipeline is to use Pixel +Shader (PS), raster, and Vertex Shader (VS) to symbolize the two shader stages. +Now, with this pipeline in mind, let's assume that Job B causes a hang issue, +but Job C's instruction might already be executing, leading developers to +incorrectly identify Job C as the problematic one. This problem can be +mitigated on multiple levels; the diagram below illustrates how to minimize +part of this problem: + +.. kernel-figure:: no_enforce_isolation.svg + +Note from the diagram that there is no guarantee of order or a clear separation +between instructions, which is not a problem most of the time, and is also good +for performance. Furthermore, notice some circles between jobs in the diagram +that represent a **fence wait** used to avoid overlapping work in the ring. At +the end of the fence, a cache flush occurs, ensuring that when the next job +starts, it begins in a clean state and, if issues arise, the developer can +pinpoint the problematic process more precisely. + +To increase the level of isolation between jobs, there is the "Enforce +Isolation" method described in the picture below: + +.. kernel-figure:: enforce_isolation.svg + +As shown in the diagram, enforcing isolation introduces ordering between +submissions, since the access to GFX/Compute is serialized, think about it as +single process at a time mode for gfx/compute. Notice that this approach has a +significant performance impact, as it allows only one job to submit commands at +a time. However, this option can help pinpoint the job that caused the problem. +Although enforcing isolation improves the situation, it does not fully resolve +the issue of precisely pinpointing bad jobs, since isolation might mask the +problem. In summary, identifying which job caused the issue may not be precise, +but enforcing isolation might help with the debugging. + +Ring Operations +=============== + +.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c + :internal: + -- cgit v1.3.1