sched/cache: Keep nr_pref_llc_running in the runnable domain, to fix LLC mis-scheduling buggrafted

alb_break_llc() decides whether to break LLC preference during active
load balance. It does so by testing that every runnable fair task on the
source rq prefers its LLC:

	env->src_rq->nr_pref_llc_running == env->src_rq->cfs.h_nr_runnable

But the two counters cover different sets. nr_pref_llc_running is updated
in account_llc_enqueue()/account_llc_dequeue(), next to cfs_rq->nr_queued,
so it follows queued tasks. h_nr_runnable is updated in set_delayed()/
clear_delayed() and drops delay-dequeued tasks.

So under DELAY_DEQUEUE, a preferring task that goes to sleep stays counted
in nr_pref_llc_running while h_nr_runnable falls. The equality then breaks,
alb_break_llc() returns false, and active balance is free to pull a task
off its preferred LLC. Active balance only moves runnable tasks, and this
is the only LLC check it consults: once the stopper runs, LBF_ACTIVE_LB
skips the per-task test in can_migrate_task(). The runnable set is the one
we want.

Fix it on the counter side. A task should be counted in
nr_pref_llc_running exactly while it is both queued on its preferred LLC
(pref_llc_queued) and runnable (!sched_delayed). Define that membership
once in task_pref_llc_runnable(), and adjust the counter only through
pref_llc_running_inc()/pref_llc_running_dec() from the four sites that
change either input: account_llc_enqueue(), account_llc_dequeue(),
set_delayed() and clear_delayed(). Gating every update on the same
predicate keeps the delay, wake and dequeue paths from double-counting
or underflowing; see the comments at those sites for the ordering.

nr_llc_running and sd->llc_counts are not touched and stay on queued
semantics.

Fixes: 714059f79ff0 ("sched/cache: Handle moving single tasks to/from their preferred LLC")
Closes: https://lore.kernel.org/lkml/20260827135000.735138-1-zhanxusheng@xiaomi.com/
Reported-by: Zhan Xusheng <zhanxusheng@xiaomi.com>
Suggested-by: Chen Yu <yu.c.chen@intel.com>
Signed-off-by: Tim Chen <tim.c.chen@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Kayra Cizmeci <kayracizmeci@gmail.com>
Cc: <stable@kernel.org> # v7.2.x
Link: https://patch.msgid.link/06af61afedac32e6477f57feb4d658f6c411c3af.1790035273.git.tim.c.chen@linux.intel.com

1 files changed, 134 insertions, 0 deletions

diff --git a/arch/arm64/tools/cpucaps b/arch/arm64/tools/cpucaps
new file mode 100644
index 000000000..2775ba335
--- /dev/null
+++ b/arch/arm64/tools/cpucaps
@@ -0,0 +1,134 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Internal CPU capabilities constants
+
+ALWAYS_BOOT
+ALWAYS_SYSTEM
+BTI
+# Unreliable: use system_supports_32bit_el0() instead.
+HAS_32BIT_EL0_DO_NOT_USE
+HAS_32BIT_EL1
+HAS_ADDRESS_AUTH
+HAS_ADDRESS_AUTH_ARCH_QARMA3
+HAS_ADDRESS_AUTH_ARCH_QARMA5
+HAS_ADDRESS_AUTH_IMP_DEF
+HAS_AMU_EXTN
+HAS_ARMv8_4_TTL
+HAS_BBML3
+HAS_CACHE_DIC
+HAS_CACHE_IDC
+HAS_CNP
+HAS_CRC32
+HAS_DCPODP
+HAS_DCPOP
+HAS_DIT
+HAS_E0PD
+HAS_ECV
+HAS_ECV_CNTPOFF
+HAS_EPAN
+HAS_EVT
+HAS_FPMR
+HAS_FGT
+HAS_FGT2
+HAS_FPSIMD
+HAS_GCS
+HAS_GENERIC_AUTH
+HAS_GENERIC_AUTH_ARCH_QARMA3
+HAS_GENERIC_AUTH_ARCH_QARMA5
+HAS_GENERIC_AUTH_IMP_DEF
+HAS_GICV3_CPUIF
+HAS_GICV5_CPUIF
+HAS_GICV5_LEGACY
+HAS_GIC_PRIO_MASKING
+HAS_GIC_PRIO_RELAXED_SYNC
+HAS_ICH_HCR_EL2_TDIR
+HAS_HCR_NV1
+HAS_HCX
+HAS_LDAPR
+HAS_LPA2
+HAS_LSE_ATOMICS
+HAS_LS64
+HAS_LS64_V
+HAS_LSUI
+HAS_MOPS
+HAS_NESTED_VIRT
+HAS_NV2P1
+HAS_NV3
+HAS_PAN
+HAS_PMUV3
+HAS_S1PIE
+HAS_S1POE
+HAS_SCTLR2
+HAS_RAS_EXTN
+HAS_RASV1P1_EXTN
+HAS_RNG
+HAS_SB
+HAS_STAGE2_FWB
+HAS_TCR2
+HAS_TIDCP1
+HAS_TLB_RANGE
+HAS_VA52
+HAS_VIRT_HOST_EXTN
+HAS_WFXT
+HAS_XNX
+HAFT
+HW_DBM
+KVM_HVHE
+KVM_PROTECTED_MODE
+MISMATCHED_CACHE_TYPE
+MPAM
+MPAM_HCR
+MTE
+MTE_ASYMM
+MTE_FAR
+MTE_STORE_ONLY
+SME
+SME_FA64
+SME2
+SPECTRE_V2
+SPECTRE_V3A
+SPECTRE_V4
+SPECTRE_BHB
+SSBS
+SVE
+UNMAP_KERNEL_AT_EL0
+WORKAROUND_834220
+WORKAROUND_843419
+WORKAROUND_845719
+WORKAROUND_858921
+WORKAROUND_1418040
+WORKAROUND_1463225
+WORKAROUND_1508412
+WORKAROUND_1542419
+WORKAROUND_1742098
+WORKAROUND_1902691
+WORKAROUND_2038923
+WORKAROUND_2064142
+WORKAROUND_2077057
+WORKAROUND_2457168
+WORKAROUND_2645198
+WORKAROUND_2658417
+WORKAROUND_4193714
+WORKAROUND_4311569
+WORKAROUND_AMPERE_AC03_CPU_38
+WORKAROUND_AMPERE_AC04_CPU_23
+WORKAROUND_TRBE_OVERWRITE_FILL_MODE
+WORKAROUND_TSB_FLUSH_FAILURE
+WORKAROUND_TRBE_WRITE_OUT_OF_RANGE
+WORKAROUND_CAVIUM_23154
+WORKAROUND_CAVIUM_27456
+WORKAROUND_CAVIUM_30115
+WORKAROUND_CAVIUM_TX2_219_PRFM
+WORKAROUND_CAVIUM_TX2_219_TVM
+WORKAROUND_CLEAN_CACHE
+WORKAROUND_DEVICE_LOAD_ACQUIRE
+WORKAROUND_DISABLE_CNP
+WORKAROUND_GICv3_BROKEN_SEIS
+WORKAROUND_NVIDIA_OLYMPUS_1027
+WORKAROUND_PMUV3_IMPDEF_TRAPS
+WORKAROUND_QCOM_FALKOR_E1003
+WORKAROUND_QCOM_ORYON_CNTVOFF
+WORKAROUND_REPEAT_TLBI_SYNC
+WORKAROUND_SPECULATIVE_AT
+WORKAROUND_SPECULATIVE_SSBS
+WORKAROUND_SPECULATIVE_UNPRIV_LOAD