sched/cache: Keep nr_pref_llc_running in the runnable domain, to fix LLC mis-scheduling buggrafted

alb_break_llc() decides whether to break LLC preference during active
load balance. It does so by testing that every runnable fair task on the
source rq prefers its LLC:

	env->src_rq->nr_pref_llc_running == env->src_rq->cfs.h_nr_runnable

But the two counters cover different sets. nr_pref_llc_running is updated
in account_llc_enqueue()/account_llc_dequeue(), next to cfs_rq->nr_queued,
so it follows queued tasks. h_nr_runnable is updated in set_delayed()/
clear_delayed() and drops delay-dequeued tasks.

So under DELAY_DEQUEUE, a preferring task that goes to sleep stays counted
in nr_pref_llc_running while h_nr_runnable falls. The equality then breaks,
alb_break_llc() returns false, and active balance is free to pull a task
off its preferred LLC. Active balance only moves runnable tasks, and this
is the only LLC check it consults: once the stopper runs, LBF_ACTIVE_LB
skips the per-task test in can_migrate_task(). The runnable set is the one
we want.

Fix it on the counter side. A task should be counted in
nr_pref_llc_running exactly while it is both queued on its preferred LLC
(pref_llc_queued) and runnable (!sched_delayed). Define that membership
once in task_pref_llc_runnable(), and adjust the counter only through
pref_llc_running_inc()/pref_llc_running_dec() from the four sites that
change either input: account_llc_enqueue(), account_llc_dequeue(),
set_delayed() and clear_delayed(). Gating every update on the same
predicate keeps the delay, wake and dequeue paths from double-counting
or underflowing; see the comments at those sites for the ordering.

nr_llc_running and sd->llc_counts are not touched and stay on queued
semantics.

Fixes: 714059f79ff0 ("sched/cache: Handle moving single tasks to/from their preferred LLC")
Closes: https://lore.kernel.org/lkml/20260827135000.735138-1-zhanxusheng@xiaomi.com/
Reported-by: Zhan Xusheng <zhanxusheng@xiaomi.com>
Suggested-by: Chen Yu <yu.c.chen@intel.com>
Signed-off-by: Tim Chen <tim.c.chen@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Kayra Cizmeci <kayracizmeci@gmail.com>
Cc: <stable@kernel.org> # v7.2.x
Link: https://patch.msgid.link/06af61afedac32e6477f57feb4d658f6c411c3af.1790035273.git.tim.c.chen@linux.intel.com

1 files changed, 242 insertions, 0 deletions

diff --git a/Documentation/ABI/testing/sysfs-class-mtd b/Documentation/ABI/testing/sysfs-class-mtd
new file mode 100644
index 000000000..f77fa4f6d
--- /dev/null
+++ b/Documentation/ABI/testing/sysfs-class-mtd
@@ -0,0 +1,242 @@
+What:		/sys/class/mtd/
+Date:		April 2009
+KernelVersion:	2.6.29
+Contact:	linux-mtd@lists.infradead.org
+Description:
+		The mtd/ class subdirectory belongs to the MTD subsystem
+		(MTD core).
+
+What:		/sys/class/mtd/mtdX/
+Date:		April 2009
+KernelVersion:	2.6.29
+Contact:	linux-mtd@lists.infradead.org
+Description:
+		The /sys/class/mtd/mtd{0,1,2,3,...} directories correspond
+		to each /dev/mtdX character device.  These may represent
+		physical/simulated flash devices, partitions on a flash
+		device, or concatenated flash devices.
+
+What:		/sys/class/mtd/mtdXro/
+Date:		April 2009
+KernelVersion:	2.6.29
+Contact:	linux-mtd@lists.infradead.org
+Description:
+		These directories provide the corresponding read-only device
+		nodes for /sys/class/mtd/mtdX/ .
+
+What:		/sys/class/mtd/mtdX/dev
+Date:		April 2009
+KernelVersion:	2.6.29
+Contact:	linux-mtd@lists.infradead.org
+Description:
+		Major and minor numbers of the character device corresponding
+		to this MTD device (in <major>:<minor> format).  This is the
+		read-write device so <minor> will be even.
+
+What:		/sys/class/mtd/mtdXro/dev
+Date:		April 2009
+KernelVersion:	2.6.29
+Contact:	linux-mtd@lists.infradead.org
+Description:
+		Major and minor numbers of the character device corresponding
+		to the read-only variant of the MTD device (in
+		<major>:<minor> format).  In this case <minor> will be odd.
+
+What:		/sys/class/mtd/mtdX/erasesize
+Date:		April 2009
+KernelVersion:	2.6.29
+Contact:	linux-mtd@lists.infradead.org
+Description:
+		"Major" erase size for the device.  If numeraseregions is
+		zero, this is the eraseblock size for the entire device.
+		Otherwise, the MEMGETREGIONCOUNT/MEMGETREGIONINFO ioctls
+		can be used to determine the actual eraseblock layout.
+
+What:		/sys/class/mtd/mtdX/flags
+Date:		April 2009
+KernelVersion:	2.6.29
+Contact:	linux-mtd@lists.infradead.org
+Description:
+		A hexadecimal value representing the device flags, ORed
+		together:
+
+		0x0400: MTD_WRITEABLE - device is writable
+		0x0800: MTD_BIT_WRITEABLE - single bits can be flipped
+		0x1000: MTD_NO_ERASE - no erase necessary
+		0x2000: MTD_POWERUP_LOCK - always locked after reset
+
+What:		/sys/class/mtd/mtdX/name
+Date:		April 2009
+KernelVersion:	2.6.29
+Contact:	linux-mtd@lists.infradead.org
+Description:
+		A human-readable ASCII name for the device or partition.
+		This will match the name in /proc/mtd .
+
+What:		/sys/class/mtd/mtdX/numeraseregions
+Date:		April 2009
+KernelVersion:	2.6.29
+Contact:	linux-mtd@lists.infradead.org
+Description:
+		For devices that have variable eraseblock sizes, this
+		provides the total number of erase regions.  Otherwise,
+		it will read back as zero.
+
+What:		/sys/class/mtd/mtdX/oobsize
+Date:		April 2009
+KernelVersion:	2.6.29
+Contact:	linux-mtd@lists.infradead.org
+Description:
+		Number of OOB bytes per page.
+
+What:		/sys/class/mtd/mtdX/size
+Date:		April 2009
+KernelVersion:	2.6.29
+Contact:	linux-mtd@lists.infradead.org
+Description:
+		Total size of the device/partition, in bytes.
+
+What:		/sys/class/mtd/mtdX/type
+Date:		April 2009
+KernelVersion:	2.6.29
+Contact:	linux-mtd@lists.infradead.org
+Description:
+		One of the following ASCII strings, representing the device
+		type:
+
+		absent, ram, rom, nor, nand, mlc-nand, dataflash, ubi, unknown
+
+What:		/sys/class/mtd/mtdX/writesize
+Date:		April 2009
+KernelVersion:	2.6.29
+Contact:	linux-mtd@lists.infradead.org
+Description:
+		Minimal writable flash unit size.  This will always be
+		a positive integer.
+
+		In the case of NOR flash it is 1 (even though individual
+		bits can be cleared).
+
+		In the case of NAND flash it is one NAND page (or a
+		half page, or a quarter page).
+
+		In the case of ECC NOR, it is the ECC block size.
+
+What:		/sys/class/mtd/mtdX/ecc_strength
+Date:		April 2012
+KernelVersion:	3.4
+Contact:	linux-mtd@lists.infradead.org
+Description:
+		Maximum number of bit errors that the device is capable of
+		correcting within each region covering an ECC step (see
+		ecc_step_size).  This will always be a non-negative integer.
+
+		In the case of devices lacking any ECC capability, it is 0.
+
+What:		/sys/class/mtd/mtdX/bitflip_threshold
+Date:		April 2012
+KernelVersion:	3.4
+Contact:	linux-mtd@lists.infradead.org
+Description:
+		This allows the user to examine and adjust the criteria by which
+		mtd returns -EUCLEAN from mtd_read() and mtd_read_oob().  If the
+		maximum number of bit errors that were corrected on any single
+		region comprising an ecc step (as reported by the driver) equals
+		or exceeds this value, -EUCLEAN is returned.  Otherwise, absent
+		an error, 0 is returned.  Higher layers (e.g., UBI) use this
+		return code as an indication that an erase block may be
+		degrading and should be scrutinized as a candidate for being
+		marked as bad.
+
+		The initial value may be specified by the flash device driver.
+		If not, then the default value is ecc_strength.
+
+		The introduction of this feature brings a subtle change to the
+		meaning of the -EUCLEAN return code.  Previously, it was
+		interpreted to mean simply "one or more bit errors were
+		corrected".  Its new interpretation can be phrased as "a
+		dangerously high number of bit errors were corrected on one or
+		more regions comprising an ecc step".  The precise definition of
+		"dangerously high" can be adjusted by the user with
+		bitflip_threshold.  Users are discouraged from doing this,
+		however, unless they know what they are doing and have intimate
+		knowledge of the properties of their device.  Broadly speaking,
+		bitflip_threshold should be low enough to detect genuine erase
+		block degradation, but high enough to avoid the consequences of
+		a persistent return value of -EUCLEAN on devices where sticky
+		bitflips occur.  Note that if bitflip_threshold exceeds
+		ecc_strength, -EUCLEAN is never returned by the read operations.
+		Conversely, if bitflip_threshold is zero, -EUCLEAN is always
+		returned, absent a hard error.
+
+		This is generally applicable only to NAND flash devices with ECC
+		capability.  It is ignored on devices lacking ECC capability;
+		i.e., devices for which ecc_strength is zero.
+
+What:		/sys/class/mtd/mtdX/ecc_step_size
+Date:		May 2013
+KernelVersion:	3.10
+Contact:	linux-mtd@lists.infradead.org
+Description:
+		The size of a single region covered by ECC, known as the ECC
+		step.  Devices may have several equally sized ECC steps within
+		each writesize region.
+
+		It will always be a non-negative integer.  In the case of
+		devices lacking any ECC capability, it is 0.
+
+What:		/sys/class/mtd/mtdX/ecc_failures
+Date:		June 2014
+KernelVersion:	3.17
+Contact:	linux-mtd@lists.infradead.org
+Description:
+		The number of failures reported by this device's ECC. Typically,
+		these failures are associated with failed read operations.
+
+		It will always be a non-negative integer.  In the case of
+		devices lacking any ECC capability, it is 0.
+
+What:		/sys/class/mtd/mtdX/corrected_bits
+Date:		June 2014
+KernelVersion:	3.17
+Contact:	linux-mtd@lists.infradead.org
+Description:
+		The number of bits that have been corrected by means of the
+		device's ECC.
+
+		It will always be a non-negative integer.  In the case of
+		devices lacking any ECC capability, it is 0.
+
+What:		/sys/class/mtd/mtdX/bad_blocks
+Date:		June 2014
+KernelVersion:	3.17
+Contact:	linux-mtd@lists.infradead.org
+Description:
+		The number of blocks marked as bad, if any, in this partition.
+
+What:		/sys/class/mtd/mtdX/bbt_blocks
+Date:		June 2014
+KernelVersion:	3.17
+Contact:	linux-mtd@lists.infradead.org
+Description:
+		The number of blocks that are marked as reserved, if any, in
+		this partition. These are typically used to store the in-flash
+		bad block table (BBT).
+
+What:		/sys/class/mtd/mtdX/offset
+Date:		March 2015
+KernelVersion:	4.1
+Contact:	linux-mtd@lists.infradead.org
+Description:
+		For a partition, the offset of that partition from the start
+		of the parent (another partition or a flash device) in bytes.
+		This attribute is absent on flash devices, so it can be used
+		to distinguish them from partitions.
+
+What:		/sys/class/mtd/mtdX/oobavail
+Date:		April 2018
+KernelVersion:	4.16
+Contact:	linux-mtd@lists.infradead.org
+Description:
+		Number of bytes available for a client to place data into
+		the out of band area.