sched/cache: Keep nr_pref_llc_running in the runnable domain, to fix LLC mis-scheduling buggrafted

alb_break_llc() decides whether to break LLC preference during active
load balance. It does so by testing that every runnable fair task on the
source rq prefers its LLC:

	env->src_rq->nr_pref_llc_running == env->src_rq->cfs.h_nr_runnable

But the two counters cover different sets. nr_pref_llc_running is updated
in account_llc_enqueue()/account_llc_dequeue(), next to cfs_rq->nr_queued,
so it follows queued tasks. h_nr_runnable is updated in set_delayed()/
clear_delayed() and drops delay-dequeued tasks.

So under DELAY_DEQUEUE, a preferring task that goes to sleep stays counted
in nr_pref_llc_running while h_nr_runnable falls. The equality then breaks,
alb_break_llc() returns false, and active balance is free to pull a task
off its preferred LLC. Active balance only moves runnable tasks, and this
is the only LLC check it consults: once the stopper runs, LBF_ACTIVE_LB
skips the per-task test in can_migrate_task(). The runnable set is the one
we want.

Fix it on the counter side. A task should be counted in
nr_pref_llc_running exactly while it is both queued on its preferred LLC
(pref_llc_queued) and runnable (!sched_delayed). Define that membership
once in task_pref_llc_runnable(), and adjust the counter only through
pref_llc_running_inc()/pref_llc_running_dec() from the four sites that
change either input: account_llc_enqueue(), account_llc_dequeue(),
set_delayed() and clear_delayed(). Gating every update on the same
predicate keeps the delay, wake and dequeue paths from double-counting
or underflowing; see the comments at those sites for the ordering.

nr_llc_running and sd->llc_counts are not touched and stay on queued
semantics.

Fixes: 714059f79ff0 ("sched/cache: Handle moving single tasks to/from their preferred LLC")
Closes: https://lore.kernel.org/lkml/20260827135000.735138-1-zhanxusheng@xiaomi.com/
Reported-by: Zhan Xusheng <zhanxusheng@xiaomi.com>
Suggested-by: Chen Yu <yu.c.chen@intel.com>
Signed-off-by: Tim Chen <tim.c.chen@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Kayra Cizmeci <kayracizmeci@gmail.com>
Cc: <stable@kernel.org> # v7.2.x
Link: https://patch.msgid.link/06af61afedac32e6477f57feb4d658f6c411c3af.1790035273.git.tim.c.chen@linux.intel.com

1 files changed, 138 insertions, 0 deletions

diff --git a/rust/syn/ext.rs b/rust/syn/ext.rs
new file mode 100644
index 000000000..a9be9af42
--- /dev/null
+++ b/rust/syn/ext.rs
@@ -0,0 +1,138 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
+//! Extension traits to provide parsing methods on foreign types.
+
+use crate::buffer::Cursor;
+use crate::error::Result;
+use crate::parse::ParseStream;
+use crate::parse::Peek;
+use crate::sealed::lookahead;
+use crate::token::CustomToken;
+use proc_macro2::Ident;
+
+/// Additional methods for `Ident` not provided by proc-macro2 or libproc_macro.
+///
+/// This trait is sealed and cannot be implemented for types outside of Syn. It
+/// is implemented only for `proc_macro2::Ident`.
+pub trait IdentExt: Sized + private::Sealed {
+    /// Parses any identifier including keywords.
+    ///
+    /// This is useful when parsing macro input which allows Rust keywords as
+    /// identifiers.
+    ///
+    /// # Example
+    ///
+    /// ```
+    /// use syn::{Error, Ident, Result, Token};
+    /// use syn::ext::IdentExt;
+    /// use syn::parse::ParseStream;
+    ///
+    /// mod kw {
+    ///     syn::custom_keyword!(name);
+    /// }
+    ///
+    /// // Parses input that looks like `name = NAME` where `NAME` can be
+    /// // any identifier.
+    /// //
+    /// // Examples:
+    /// //
+    /// //     name = anything
+    /// //     name = impl
+    /// fn parse_dsl(input: ParseStream) -> Result<Ident> {
+    ///     input.parse::<kw::name>()?;
+    ///     input.parse::<Token![=]>()?;
+    ///     let name = input.call(Ident::parse_any)?;
+    ///     Ok(name)
+    /// }
+    /// ```
+    fn parse_any(input: ParseStream) -> Result<Self>;
+
+    /// Peeks any identifier including keywords. Usage:
+    /// `input.peek(Ident::peek_any)`
+    ///
+    /// This is different from `input.peek(Ident)` which only returns true in
+    /// the case of an ident which is not a Rust keyword.
+    #[allow(non_upper_case_globals)]
+    const peek_any: private::PeekFn = private::PeekFn;
+
+    /// Strips the raw marker `r#`, if any, from the beginning of an ident.
+    ///
+    ///   - unraw(`x`) = `x`
+    ///   - unraw(`move`) = `move`
+    ///   - unraw(`r#move`) = `move`
+    ///
+    /// # Example
+    ///
+    /// In the case of interop with other languages like Python that have a
+    /// different set of keywords than Rust, we might come across macro input
+    /// that involves raw identifiers to refer to ordinary variables in the
+    /// other language with a name that happens to be a Rust keyword.
+    ///
+    /// The function below appends an identifier from the caller's input onto a
+    /// fixed prefix. Without using `unraw()`, this would tend to produce
+    /// invalid identifiers like `__pyo3_get_r#move`.
+    ///
+    /// ```
+    /// use proc_macro2::Span;
+    /// use syn::Ident;
+    /// use syn::ext::IdentExt;
+    ///
+    /// fn ident_for_getter(variable: &Ident) -> Ident {
+    ///     let getter = format!("__pyo3_get_{}", variable.unraw());
+    ///     Ident::new(&getter, Span::call_site())
+    /// }
+    /// ```
+    fn unraw(&self) -> Ident;
+}
+
+impl IdentExt for Ident {
+    fn parse_any(input: ParseStream) -> Result<Self> {
+        input.step(|cursor| match cursor.ident() {
+            Some((ident, rest)) => Ok((ident, rest)),
+            None => Err(cursor.error("expected ident")),
+        })
+    }
+
+    fn unraw(&self) -> Ident {
+        let string = self.to_string();
+        if let Some(string) = string.strip_prefix("r#") {
+            Ident::new(string, self.span())
+        } else {
+            self.clone()
+        }
+    }
+}
+
+impl Peek for private::PeekFn {
+    type Token = private::IdentAny;
+}
+
+impl CustomToken for private::IdentAny {
+    fn peek(cursor: Cursor) -> bool {
+        cursor.ident().is_some()
+    }
+
+    fn display() -> &'static str {
+        "identifier"
+    }
+}
+
+impl lookahead::Sealed for private::PeekFn {}
+
+mod private {
+    use proc_macro2::Ident;
+
+    pub trait Sealed {}
+
+    impl Sealed for Ident {}
+
+    pub struct PeekFn;
+    pub struct IdentAny;
+
+    impl Copy for PeekFn {}
+    impl Clone for PeekFn {
+        fn clone(&self) -> Self {
+            *self
+        }
+    }
+}