From a9e3760b0838299649c0d57cca44daaf40ba3c33 Mon Sep 17 00:00:00 2001 From: Tejun Heo Date: Mon, 14 Sep 2026 22:12:34 -1000 Subject: sched_ext: Maintain an online cid mask in the scheduler arena Schedulers on the default cid mapping treat [0, nr_online_cids) as the online set and restart on hotplug. Schedulers that install their own mapping with scx_bpf_cid_override() have no way to learn which cids are online: the count no longer identifies members and the CPU-form cpumask is unusable from cid programs. This is an obvious hole in the cid API. Add scx_bpf_online_cmask(), a kernel-maintained cmask in the scheduler's arena, allocated alongside the per-CPU scratch masks and populated after the cid mapping is finalized and before ops.init(), for child schedulers too. The pointer stays valid through ops.exit() with no reference to take. It is the arena offset as a void pointer, the same form struct_ops arena arguments arrive in. The verifier types the void return as a scalar for the program's arena cast. The mask follows the SCX hotplug notifications: seeded from cpu_active_mask and updated before ops.cid_online/offline() runs, so it lags cpu_online_mask only inside a hotplug transition. Updates walk the scheduler list under the lock that also serializes unlinking. Reads are live, not atomic snapshots. Root initialization excludes hotplug. v2: Reworded the getter kerneldoc (Andrea Righi). Signed-off-by: Tejun Heo Reviewed-by: Andrea Righi --- Documentation/gpu/amdgpu/gc/index.rst | 54 +++++++++++++++++++++++++++++++++++ Documentation/gpu/amdgpu/gc/mes.rst | 38 ++++++++++++++++++++++++ 2 files changed, 92 insertions(+) create mode 100644 Documentation/gpu/amdgpu/gc/index.rst create mode 100644 Documentation/gpu/amdgpu/gc/mes.rst (limited to 'Documentation/gpu/amdgpu/gc') diff --git a/Documentation/gpu/amdgpu/gc/index.rst b/Documentation/gpu/amdgpu/gc/index.rst new file mode 100644 index 000000000..b6b583c9d --- /dev/null +++ b/Documentation/gpu/amdgpu/gc/index.rst @@ -0,0 +1,54 @@ +.. _amdgpu-gc: + +======================================== + drm/amdgpu - Graphics and Compute (GC) +======================================== + +The relationship between the CPU and GPU can be described as the +producer-consumer problem, where the CPU fills out a buffer with operations +(producer) to be executed by the GPU (consumer). The requested operations in +the buffer are called **Command Packets**, which can be summarized as a +compressed way of transmitting command information to the graphics controller. + +The component that acts as the front end between the CPU and the GPU is called +**Command Processor (CP)**. This component is responsible for providing greater +flexibility to the **Graphics and Compute (GC)** since CP makes it possible to +program various aspects of the GPU pipeline. CP also coordinates the +communication between the CPU and GPU via a mechanism named **Ring Buffers**, +where the CPU appends information to the buffer while the GPU removes +operations. CP is also responsible for handling **Indirect Buffers (IB)**. + +For reference, internally the CP consists of several sub-blocks (CPC - CP +compute, CPG - CP graphics, and CPF - CP fetcher). Some of these acronyms +appear in register names, but this is more of an implementation detail and not +something that directly impacts driver programming or debugging. + +Graphics (GFX) and Compute Microcontrollers +------------------------------------------- + +GC is a large block, and as a result, it has multiple firmware associated with +it. Some of them are: + +CP (Command Processor) + The name for the hardware block that encompasses the front end of the + GFX/Compute pipeline. Consists mainly of a bunch of microcontrollers + (PFP, ME, CE, MEC). The firmware that runs on these microcontrollers + provides the driver interface to interact with the GFX/Compute engine. + + MEC (MicroEngine Compute) + This is the microcontroller that controls the compute queues on the + GFX/compute engine. + + MES (MicroEngine Scheduler) + This is the engine for managing queues. For more details check + :ref:`MicroEngine Scheduler (MES) `. + +RLC (RunList Controller) + This is another microcontroller in the GFX/Compute engine. It handles + power management related functionality within the GFX/Compute engine. + The name is a vestige of old hardware where it was originally added + and doesn't really have much relation to what the engine does now. + +.. toctree:: + + mes.rst diff --git a/Documentation/gpu/amdgpu/gc/mes.rst b/Documentation/gpu/amdgpu/gc/mes.rst new file mode 100644 index 000000000..b99eb211b --- /dev/null +++ b/Documentation/gpu/amdgpu/gc/mes.rst @@ -0,0 +1,38 @@ +.. _amdgpu-mes: + +============================= + MicroEngine Scheduler (MES) +============================= + +.. note:: + Queue and ring buffer are used as a synonymous. + +.. note:: + This section assumes that you are familiar with the concept of Pipes, Queues, and GC. + If not, check :ref:`GFX, Compute, and SDMA Overall Behavior` + and :ref:`drm/amdgpu - Graphics and Compute (GC) `. + +Every GFX has a pipe component with one or more hardware queues. Pipes can +switch between queues depending on certain conditions, and one of the +components that can request a queue switch to a pipe is the MicroEngine +Scheduler (MES). Whenever the driver is initialized, it creates one MQD per +hardware queue, and then the MQDs are handed to the MES firmware for mapping +to: + +1. Kernel Queues (legacy): This queue is statically mapped to HQDs and never + preempted. Even though this is a legacy feature, it is the current default, and + most existing hardware supports it. When an application submits work to the + kernel driver, it submits all of the application command buffers to the kernel + queues. The CS IOCTL takes the command buffer from the applications and + schedules them on the kernel queue. + +2. User Queues: These queues are dynamically mapped to the HQDs. Regarding the + utilization of User Queues, the userspace application will create its user + queues and submit work directly to its user queues with no need to IOCTL for + each submission and no need to share a single kernel queue. + +In terms of User Queues, MES can dynamically map them to the HQD. If there are +more MQDs than HQDs, the MES firmware will preempt other user queues to make +sure each queues get a time slice; in other words, MES is a microcontroller +that handles the mapping and unmapping of MQDs into HQDs, as well as the +priorities and oversubscription of MQDs. -- cgit v1.3.1