sched/cache: Keep nr_pref_llc_running in the runnable domain, to fix LLC mis-scheduling buggrafted

alb_break_llc() decides whether to break LLC preference during active
load balance. It does so by testing that every runnable fair task on the
source rq prefers its LLC:

	env->src_rq->nr_pref_llc_running == env->src_rq->cfs.h_nr_runnable

But the two counters cover different sets. nr_pref_llc_running is updated
in account_llc_enqueue()/account_llc_dequeue(), next to cfs_rq->nr_queued,
so it follows queued tasks. h_nr_runnable is updated in set_delayed()/
clear_delayed() and drops delay-dequeued tasks.

So under DELAY_DEQUEUE, a preferring task that goes to sleep stays counted
in nr_pref_llc_running while h_nr_runnable falls. The equality then breaks,
alb_break_llc() returns false, and active balance is free to pull a task
off its preferred LLC. Active balance only moves runnable tasks, and this
is the only LLC check it consults: once the stopper runs, LBF_ACTIVE_LB
skips the per-task test in can_migrate_task(). The runnable set is the one
we want.

Fix it on the counter side. A task should be counted in
nr_pref_llc_running exactly while it is both queued on its preferred LLC
(pref_llc_queued) and runnable (!sched_delayed). Define that membership
once in task_pref_llc_runnable(), and adjust the counter only through
pref_llc_running_inc()/pref_llc_running_dec() from the four sites that
change either input: account_llc_enqueue(), account_llc_dequeue(),
set_delayed() and clear_delayed(). Gating every update on the same
predicate keeps the delay, wake and dequeue paths from double-counting
or underflowing; see the comments at those sites for the ordering.

nr_llc_running and sd->llc_counts are not touched and stay on queued
semantics.

Fixes: 714059f79ff0 ("sched/cache: Handle moving single tasks to/from their preferred LLC")
Closes: https://lore.kernel.org/lkml/20260827135000.735138-1-zhanxusheng@xiaomi.com/
Reported-by: Zhan Xusheng <zhanxusheng@xiaomi.com>
Suggested-by: Chen Yu <yu.c.chen@intel.com>
Signed-off-by: Tim Chen <tim.c.chen@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Kayra Cizmeci <kayracizmeci@gmail.com>
Cc: <stable@kernel.org> # v7.2.x
Link: https://patch.msgid.link/06af61afedac32e6477f57feb4d658f6c411c3af.1790035273.git.tim.c.chen@linux.intel.com

1 files changed, 212 insertions, 0 deletions

diff --git a/scripts/setlocalversion b/scripts/setlocalversion
new file mode 100755
index 000000000..28169d7e1
--- /dev/null
+++ b/scripts/setlocalversion
@@ -0,0 +1,212 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+#
+# This scripts adds local version information from the version
+# control system git.
+#
+# If something goes wrong, send a mail the kernel build mailinglist
+# (see MAINTAINERS) and CC Nico Schottelius
+# <nico-linuxsetlocalversion -at- schottelius.org>.
+#
+#
+
+set -e
+
+usage() {
+	echo "Usage: $0 [--no-local] [srctree]" >&2
+	exit 1
+}
+
+no_local=false
+if test "$1" = "--no-local"; then
+	no_local=true
+	shift
+fi
+
+srctree=.
+if test $# -gt 0; then
+	srctree=$1
+	shift
+fi
+if test $# -gt 0 -o ! -d "$srctree"; then
+	usage
+fi
+
+try_tag() {
+	tag="$1"
+
+	# Is $tag an annotated tag?
+	if [ "$(git cat-file -t "$tag" 2> /dev/null)" != tag ]; then
+		 return
+	fi
+
+	# Is it an ancestor of HEAD, and if so, how many commits are in $tag..HEAD?
+	# shellcheck disable=SC2046 # word splitting is the point here
+	set -- $(git rev-list --count --left-right "$tag"...HEAD 2> /dev/null)
+
+	# $1 is 0 if and only if $tag is an ancestor of HEAD. Use
+	# string comparison, because $1 is empty if the 'git rev-list'
+	# command somehow failed.
+	if [ "$1" != 0 ]; then
+		return
+	fi
+
+	# $2 is the number of commits in the range $tag..HEAD, possibly 0.
+	count="$2"
+}
+
+scm_version()
+{
+	local short=false
+	local no_dirty=false
+	local tag
+
+	while [ $# -gt 0 ];
+	do
+		case "$1" in
+		--short)
+			short=true;;
+		--no-dirty)
+			no_dirty=true;;
+		esac
+		shift
+	done
+
+	cd "$srctree"
+
+	if test -n "$(git rev-parse --show-cdup 2>/dev/null)"; then
+		return
+	fi
+
+	if ! head=$(git rev-parse --verify HEAD 2>/dev/null); then
+		return
+	fi
+
+	# mainline kernel:  6.2.0-rc5  ->  v6.2-rc5
+	# stable kernel:    6.1.7      ->  v6.1.7
+	version_tag=v$(echo "${KERNELVERSION}" | sed -E 's/^([0-9]+\.[0-9]+)\.0(.*)$/\1\2/')
+
+	# try_tag initializes count if the tag is usable.
+	count=
+
+	# If a localversion* file exists, and the corresponding
+	# annotated tag exists and is an ancestor of HEAD, use
+	# it. This is the case in linux-next.
+	if [ -n "${file_localversion#-}" ] ; then
+		try_tag "${file_localversion#-}"
+	fi
+
+	# Otherwise, if a localversion* file exists, and the tag
+	# obtained by appending it to the tag derived from
+	# KERNELVERSION exists and is an ancestor of HEAD, use
+	# it. This is e.g. the case in linux-rt.
+	if [ -z "${count}" ] && [ -n "${file_localversion}" ]; then
+		try_tag "${version_tag}${file_localversion}"
+	fi
+
+	# Otherwise, default to the annotated tag derived from KERNELVERSION.
+	if [ -z "${count}" ]; then
+		try_tag "${version_tag}"
+	fi
+
+	# If we are at the tagged commit, we ignore it because the
+	# version is well-defined. If none of the attempted tags exist
+	# or were usable, $count is still empty.
+	if [ -z "${count}" ] || [ "${count}" -gt 0 ]; then
+
+		# If only the short version is requested, don't bother
+		# running further git commands
+		if $short; then
+			echo "+"
+			return
+		fi
+
+		# If we are past the tagged commit, we pretty print it.
+		# (like 6.1.0-14595-g292a089d78d3)
+		if [ -n "${count}" ]; then
+			printf "%s%05d" "-" "${count}"
+		fi
+
+		# Add -g and exactly 12 hex chars.
+		printf '%s%.12s' -g "$head"
+	fi
+
+	if ${no_dirty}; then
+		return
+	fi
+
+	# Check for uncommitted changes.
+	# This script must avoid any write attempt to the source tree, which
+	# might be read-only.
+	# You cannot use 'git describe --dirty' because it tries to create
+	# .git/index.lock .
+	# First, with git-status, but --no-optional-locks is only supported in
+	# git >= 2.14, so fall back to git-diff-index if it fails. Note that
+	# git-diff-index does not refresh the index, so it may give misleading
+	# results.
+	# See git-update-index(1), git-diff-index(1), and git-status(1).
+	if {
+		git --no-optional-locks status -uno --porcelain 2>/dev/null ||
+		git diff-index --name-only HEAD
+	} | read dummy; then
+		printf '%s' -dirty
+	fi
+}
+
+collect_files()
+{
+	local file res=
+
+	for file; do
+		case "$file" in
+		*\~*)
+			continue
+			;;
+		esac
+		if test -e "$file"; then
+			res="$res$(cat "$file")"
+		fi
+	done
+	echo "$res"
+}
+
+if [ -z "${KERNELVERSION}" ]; then
+	echo "KERNELVERSION is not set" >&2
+	exit 1
+fi
+
+# localversion* files in the build and source directory
+file_localversion="$(collect_files localversion*)"
+if test ! "$srctree" -ef .; then
+	file_localversion="${file_localversion}$(collect_files "$srctree"/localversion*)"
+fi
+
+if ${no_local}; then
+	echo "${KERNELVERSION}$(scm_version --no-dirty)"
+	exit 0
+fi
+
+if ! test -e include/config/auto.conf; then
+	echo "Error: kernelrelease not valid - run 'make prepare' to update it" >&2
+	exit 1
+fi
+
+# version string from CONFIG_LOCALVERSION
+config_localversion=$(sed -n 's/^CONFIG_LOCALVERSION=\(.*\)$/\1/p' include/config/auto.conf)
+
+# scm version string if not at the kernel version tag or at the file_localversion
+if grep -q "^CONFIG_LOCALVERSION_AUTO=y$" include/config/auto.conf; then
+	# full scm version string
+	scm_version="$(scm_version)"
+elif [ "${LOCALVERSION+set}" != "set" ]; then
+	# If the variable LOCALVERSION is not set, append a plus
+	# sign if the repository is not in a clean annotated or
+	# signed tagged state (as git describe only looks at signed
+	# or annotated tags - git tag -a/-s).
+	#
+	# If the variable LOCALVERSION is set (including being set
+	# to an empty string), we don't want to append a plus sign.
+	scm_version="$(scm_version --short)"
+fi
+
+echo "${KERNELVERSION}${file_localversion}${config_localversion}${LOCALVERSION}${scm_version}"