diff options
| author | Jinke Han <jinkehan@didiglobal.com> | 2026-09-08 15:37:42 +0800 |
|---|---|---|
| committer | Ingo Molnar <mingo@kernel.org> | 2026-09-17 09:48:42 +0200 |
| commit | a5f7a5bb3b7f28ba7e4fa246775b29a0e5537255 (patch) | |
| tree | 73fef824f9a846cf935869d0dbc5070ef6e2f741 /rust/macros/lib.rs | |
| download | linux-stable-a5f7a5bb3b7f28ba7e4fa246775b29a0e5537255.tar.gz linux-stable-a5f7a5bb3b7f28ba7e4fa246775b29a0e5537255.zip | |
x86/kprobes: Fix crash when probing CS CALL instructionsgrafted
When using eBPF to probe CS CALL instructions within a function,
a crash can be triggered.
The eBPF tool probes offset 257 of the __hrtimer_run_queues()
function:
<__hrtimer_run_queues+249>: nopl 0x0(%rax,%rax,1)
<__hrtimer_run_queues+254>: mov %r14,%rdi
<__hrtimer_run_queues+257>: cs call <__x86_indirect_thunk_r12>
<__hrtimer_run_queues+263>: mov %eax,%r12d
<__hrtimer_run_queues+266>: xchg %ax,%ax
<__hrtimer_run_queues+268>: mov %r13,%rdi
Which triggers this crash:
BUG: unable to handle page fault for address: 00000000000f41c9
#PF: supervisor write access in kernel mode
#PF: error_code(0x0002) - not-present page
PGD 0 P4D 0
Oops: 0002 [#1] SMP NOPTI
CPU: 1 PID: 0 Comm: swapper/1 Kdump: loaded Tainted: P
RIP: 0010:__hrtimer_run_queues+0x106/0x230
Note that __hrtimer_run_queues+0x106 is __hrtimer_run_queues+262, which is
at the 6th byte of the above CS CALL instruction. Since the CS CALL
instruction occupies 6 bytes, the exception occurred in the middle of that
call instruction.
The root cause is that when using eBPF tools to probe in the middle of a
function, a kprobe with INT3 is used as the underlying implementation.
During single-step emulation of the original CALL instruction,
int3_emulate_call() assumes that the probed CALL instruction is 5 bytes
long. However, the actual CS-prefixed CALL instruction occupies 6 bytes,
so it constructs an incorrect exception return address. When the CPU
returns from the kprobe handler, the next instruction to be executed is at
the address of the last byte of that CS CALL instruction. Coincidentally,
starting from that address, the CPU fetches and decodes a completely
different instruction, which ultimately triggers a kernel crash.
Fix the issue by using the actual instruction length obtained from
the instruction decoder when constructing the exception return
address, rather than relying on the hardcoded CALL_INSN_SIZE macro.
[ mingo: Refined the changelog ]
Fixes: 6256e668b7af ("x86/kprobes: Use int3 instead of debug trap for single-step")
Suggested-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Signed-off-by: Jinke Han <jinkehan@didiglobal.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Acked-by: Yafang Shao <laoar.shao@gmail.com>
Acked-by: Borislav Petkov <bp@alien8.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: https://patch.msgid.link/20260908073742.GA10517@didi-ThinkCentre-M920t-N000
Diffstat (limited to 'rust/macros/lib.rs')
| -rw-r--r-- | rust/macros/lib.rs | 539 |
1 files changed, 539 insertions, 0 deletions
diff --git a/rust/macros/lib.rs b/rust/macros/lib.rs new file mode 100644 index 000000000..24f96feae --- /dev/null +++ b/rust/macros/lib.rs @@ -0,0 +1,539 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! Crate for all kernel procedural macros. + +// When fixdep scans this, it will find this string `CONFIG_RUSTC_VERSION_TEXT` +// and thus add a dependency on `include/config/RUSTC_VERSION_TEXT`, which is +// touched by Kconfig when the version string from the compiler changes. + +// Stable since Rust 1.87.0. +#![feature(extract_if)] +// +// Stable since Rust 1.88.0 under a different name, `proc_macro_span_file`, +// which was added in Rust 1.88.0. This is why `cfg_attr` is used here, i.e. +// to avoid depending on the full `proc_macro_span` on Rust >= 1.88.0. +#![cfg_attr(not(CONFIG_RUSTC_HAS_SPAN_FILE), feature(proc_macro_span))] + +mod concat_idents; +mod export; +mod fmt; +mod for_lt; +mod helpers; +mod kunit; +mod module; +mod paste; +mod vtable; + +use proc_macro::TokenStream; + +use syn::parse_macro_input; + +/// Declares a kernel module. +/// +/// The `type` argument should be a type which implements the [`Module`] +/// trait. Also accepts various forms of kernel metadata. +/// +/// The `params` field describe module parameters. Each entry has the form +/// +/// ```ignore +/// parameter_name: type { +/// default: default_value, +/// description: "Description", +/// } +/// ``` +/// +/// `type` may be one of +/// +/// - [`i8`] +/// - [`u8`] +/// - [`i8`] +/// - [`u8`] +/// - [`i16`] +/// - [`u16`] +/// - [`i32`] +/// - [`u32`] +/// - [`i64`] +/// - [`u64`] +/// - [`isize`] +/// - [`usize`] +/// - [`bool`] +/// +/// C header: [`include/linux/moduleparam.h`](srctree/include/linux/moduleparam.h) +/// +/// [`Module`]: ../kernel/trait.Module.html +/// +/// # Examples +/// +/// ```ignore +/// use kernel::prelude::*; +/// +/// module!{ +/// type: MyModule, +/// name: "my_kernel_module", +/// authors: ["Rust for Linux Contributors"], +/// description: "My very own kernel module!", +/// license: "GPL", +/// alias: ["alternate_module_name"], +/// params: { +/// my_parameter: i64 { +/// default: 1, +/// description: "This parameter has a default of 1", +/// }, +/// }, +/// } +/// +/// struct MyModule(i32); +/// +/// impl kernel::Module for MyModule { +/// fn init(_module: &'static ThisModule) -> Result<Self> { +/// let foo: i32 = 42; +/// pr_info!("I contain: {}\n", foo); +/// pr_info!("i32 param is: {}\n", module_parameters::my_parameter.read()); +/// Ok(Self(foo)) +/// } +/// } +/// # fn main() {} +/// ``` +/// +/// ## Firmware +/// +/// The following example shows how to declare a kernel module that needs +/// to load binary firmware files. You need to specify the file names of +/// the firmware in the `firmware` field. The information is embedded +/// in the `modinfo` section of the kernel module. For example, a tool to +/// build an initramfs uses this information to put the firmware files into +/// the initramfs image. +/// +/// ``` +/// use kernel::prelude::*; +/// +/// module!{ +/// type: MyDeviceDriverModule, +/// name: "my_device_driver_module", +/// authors: ["Rust for Linux Contributors"], +/// description: "My device driver requires firmware", +/// license: "GPL", +/// firmware: ["my_device_firmware1.bin", "my_device_firmware2.bin"], +/// } +/// +/// struct MyDeviceDriverModule; +/// +/// impl kernel::Module for MyDeviceDriverModule { +/// fn init(_module: &'static ThisModule) -> Result<Self> { +/// Ok(Self) +/// } +/// } +/// # fn main() {} +/// ``` +/// +/// # Supported argument types +/// - `type`: type which implements the [`Module`] trait (required). +/// - `name`: ASCII string literal of the name of the kernel module (required). +/// - `authors`: array of ASCII string literals of the authors of the kernel module. +/// - `description`: string literal of the description of the kernel module. +/// - `license`: ASCII string literal of the license of the kernel module (required). +/// - `alias`: array of ASCII string literals of the alias names of the kernel module. +/// - `firmware`: array of ASCII string literals of the firmware files of +/// the kernel module. +#[proc_macro] +pub fn module(input: TokenStream) -> TokenStream { + module::module(parse_macro_input!(input)) + .unwrap_or_else(|e| e.into_compile_error()) + .into() +} + +/// Declares or implements a vtable trait. +/// +/// Linux's use of pure vtables is very close to Rust traits, but they differ +/// in how unimplemented functions are represented. In Rust, traits can provide +/// default implementation for all non-required methods (and the default +/// implementation could just return `Error::EINVAL`); Linux typically use C +/// `NULL` pointers to represent these functions. +/// +/// This attribute closes that gap. A trait can be annotated with the +/// `#[vtable]` attribute. Implementers of the trait will then also have to +/// annotate the trait with `#[vtable]`. This attribute generates a `HAS_*` +/// associated constant bool for each method in the trait that is set to true if +/// the implementer has overridden the associated method. +/// +/// For a trait method to be optional, it must have a default implementation. +/// This is also the case for traits annotated with `#[vtable]`, but in this +/// case the default implementation will never be executed. The reason for this +/// is that the functions will be called through function pointers installed in +/// C side vtables. When an optional method is not implemented on a `#[vtable]` +/// trait, a `NULL` entry is installed in the vtable. Thus the default +/// implementation is never called. Since these traits are not designed to be +/// used on the Rust side, it should not be possible to call the default +/// implementation. This is done to ensure that we call the vtable methods +/// through the C vtable, and not through the Rust vtable. Therefore, the +/// default implementation should call `build_error!`, which prevents +/// calls to this function at compile time: +/// +/// ```compile_fail +/// # // Intentionally missing `use`s to simplify `rusttest`. +/// build_error!(VTABLE_DEFAULT_ERROR) +/// ``` +/// +/// Note that you might need to import [`kernel::error::VTABLE_DEFAULT_ERROR`]. +/// +/// This macro should not be used when all functions are required. +/// +/// Additionally, this macro automatically handles the `OwnerModule` +/// associated type: on the trait side, `type OwnerModule: ModuleMetadata;` +/// is added as a required associated type if not already defined; on the +/// impl side, `type OwnerModule = LocalModule;` is automatically inserted +/// if not explicitly defined. +/// +/// # Examples +/// +/// ``` +/// use kernel::error::VTABLE_DEFAULT_ERROR; +/// use kernel::prelude::*; +/// +/// # struct LocalModule; +/// # impl kernel::ModuleMetadata for LocalModule { +/// # const NAME: &'static kernel::str::CStr = c"vtable_doctest"; +/// # +/// # // SAFETY: This doctest runs on the host: there is no `THIS_MODULE`. +/// # const THIS_MODULE: kernel::ThisModule = unsafe { +/// # kernel::ThisModule::from_ptr(core::ptr::null_mut()) +/// # }; +/// # } +/// # +/// # fn main() { +/// // Declares a `#[vtable]` trait +/// #[vtable] +/// pub trait Operations: Send + Sync + Sized { +/// fn foo(&self) -> Result<()> { +/// build_error!(VTABLE_DEFAULT_ERROR) +/// } +/// +/// fn bar(&self) -> Result<()> { +/// build_error!(VTABLE_DEFAULT_ERROR) +/// } +/// } +/// +/// struct Foo; +/// +/// // Implements the `#[vtable]` trait +/// #[vtable] +/// impl Operations for Foo { +/// fn foo(&self) -> Result<()> { +/// # Err(EINVAL) +/// // ... +/// } +/// } +/// +/// assert_eq!(<Foo as Operations>::HAS_FOO, true); +/// assert_eq!(<Foo as Operations>::HAS_BAR, false); +/// # } +/// ``` +/// +/// [`kernel::error::VTABLE_DEFAULT_ERROR`]: ../kernel/error/constant.VTABLE_DEFAULT_ERROR.html +#[proc_macro_attribute] +pub fn vtable(attr: TokenStream, input: TokenStream) -> TokenStream { + parse_macro_input!(attr as syn::parse::Nothing); + vtable::vtable(parse_macro_input!(input)) + .unwrap_or_else(|e| e.into_compile_error()) + .into() +} + +/// Export a function so that C code can call it via a header file. +/// +/// Functions exported using this macro can be called from C code using the declaration in the +/// appropriate header file. It should only be used in cases where C calls the function through a +/// header file; cases where C calls into Rust via a function pointer in a vtable (such as +/// `file_operations`) should not use this macro. +/// +/// This macro has the following effect: +/// +/// * Disables name mangling for this function. +/// * Verifies at compile-time that the function signature matches the declaration in the header +/// file. +/// +/// You must declare the signature of the Rust function in a header file that is included by +/// `rust/bindings/bindings_helper.h`. +/// +/// This macro is *not* the same as the C macros `EXPORT_SYMBOL_*`. All Rust symbols are currently +/// automatically exported with `EXPORT_SYMBOL_GPL`. +#[proc_macro_attribute] +pub fn export(attr: TokenStream, input: TokenStream) -> TokenStream { + parse_macro_input!(attr as syn::parse::Nothing); + export::export(parse_macro_input!(input)).into() +} + +/// Like [`core::format_args!`], but automatically wraps arguments in [`kernel::fmt::Adapter`]. +/// +/// This macro allows generating `fmt::Arguments` while ensuring that each argument is wrapped with +/// `::kernel::fmt::Adapter`, which customizes formatting behavior for kernel logging. +/// +/// Named arguments used in the format string (e.g. `{foo}`) are detected and resolved from local +/// bindings. All positional and named arguments are automatically wrapped. +/// +/// This macro is an implementation detail of other kernel logging macros like [`pr_info!`] and +/// should not typically be used directly. +/// +/// [`kernel::fmt::Adapter`]: ../kernel/fmt/struct.Adapter.html +/// [`pr_info!`]: ../kernel/macro.pr_info.html +#[proc_macro] +pub fn fmt(input: TokenStream) -> TokenStream { + fmt::fmt(input.into()).into() +} + +/// Concatenate two identifiers. +/// +/// This is useful in macros that need to declare or reference items with names +/// starting with a fixed prefix and ending in a user specified name. The resulting +/// identifier has the span of the second argument. +/// +/// # Examples +/// +/// ``` +/// # const binder_driver_return_protocol_BR_OK: u32 = 0; +/// # const binder_driver_return_protocol_BR_ERROR: u32 = 1; +/// # const binder_driver_return_protocol_BR_TRANSACTION: u32 = 2; +/// # const binder_driver_return_protocol_BR_REPLY: u32 = 3; +/// # const binder_driver_return_protocol_BR_DEAD_REPLY: u32 = 4; +/// # const binder_driver_return_protocol_BR_TRANSACTION_COMPLETE: u32 = 5; +/// # const binder_driver_return_protocol_BR_INCREFS: u32 = 6; +/// # const binder_driver_return_protocol_BR_ACQUIRE: u32 = 7; +/// # const binder_driver_return_protocol_BR_RELEASE: u32 = 8; +/// # const binder_driver_return_protocol_BR_DECREFS: u32 = 9; +/// # const binder_driver_return_protocol_BR_NOOP: u32 = 10; +/// # const binder_driver_return_protocol_BR_SPAWN_LOOPER: u32 = 11; +/// # const binder_driver_return_protocol_BR_DEAD_BINDER: u32 = 12; +/// # const binder_driver_return_protocol_BR_CLEAR_DEATH_NOTIFICATION_DONE: u32 = 13; +/// # const binder_driver_return_protocol_BR_FAILED_REPLY: u32 = 14; +/// use kernel::macros::concat_idents; +/// +/// macro_rules! pub_no_prefix { +/// ($prefix:ident, $($newname:ident),+) => { +/// $(pub(crate) const $newname: u32 = concat_idents!($prefix, $newname);)+ +/// }; +/// } +/// +/// pub_no_prefix!( +/// binder_driver_return_protocol_, +/// BR_OK, +/// BR_ERROR, +/// BR_TRANSACTION, +/// BR_REPLY, +/// BR_DEAD_REPLY, +/// BR_TRANSACTION_COMPLETE, +/// BR_INCREFS, +/// BR_ACQUIRE, +/// BR_RELEASE, +/// BR_DECREFS, +/// BR_NOOP, +/// BR_SPAWN_LOOPER, +/// BR_DEAD_BINDER, +/// BR_CLEAR_DEATH_NOTIFICATION_DONE, +/// BR_FAILED_REPLY +/// ); +/// +/// assert_eq!(BR_OK, binder_driver_return_protocol_BR_OK); +/// ``` +#[proc_macro] +pub fn concat_idents(input: TokenStream) -> TokenStream { + concat_idents::concat_idents(parse_macro_input!(input)).into() +} + +/// Paste identifiers together. +/// +/// Within the `paste!` macro, identifiers inside `[<` and `>]` are concatenated together to form a +/// single identifier. +/// +/// This is similar to the [`paste`] crate, but with pasting feature limited to identifiers and +/// literals (lifetimes and documentation strings are not supported). There is a difference in +/// supported modifiers as well. +/// +/// # Examples +/// +/// ``` +/// # const binder_driver_return_protocol_BR_OK: u32 = 0; +/// # const binder_driver_return_protocol_BR_ERROR: u32 = 1; +/// # const binder_driver_return_protocol_BR_TRANSACTION: u32 = 2; +/// # const binder_driver_return_protocol_BR_REPLY: u32 = 3; +/// # const binder_driver_return_protocol_BR_DEAD_REPLY: u32 = 4; +/// # const binder_driver_return_protocol_BR_TRANSACTION_COMPLETE: u32 = 5; +/// # const binder_driver_return_protocol_BR_INCREFS: u32 = 6; +/// # const binder_driver_return_protocol_BR_ACQUIRE: u32 = 7; +/// # const binder_driver_return_protocol_BR_RELEASE: u32 = 8; +/// # const binder_driver_return_protocol_BR_DECREFS: u32 = 9; +/// # const binder_driver_return_protocol_BR_NOOP: u32 = 10; +/// # const binder_driver_return_protocol_BR_SPAWN_LOOPER: u32 = 11; +/// # const binder_driver_return_protocol_BR_DEAD_BINDER: u32 = 12; +/// # const binder_driver_return_protocol_BR_CLEAR_DEATH_NOTIFICATION_DONE: u32 = 13; +/// # const binder_driver_return_protocol_BR_FAILED_REPLY: u32 = 14; +/// macro_rules! pub_no_prefix { +/// ($prefix:ident, $($newname:ident),+) => { +/// ::kernel::macros::paste! { +/// $(pub(crate) const $newname: u32 = [<$prefix $newname>];)+ +/// } +/// }; +/// } +/// +/// pub_no_prefix!( +/// binder_driver_return_protocol_, +/// BR_OK, +/// BR_ERROR, +/// BR_TRANSACTION, +/// BR_REPLY, +/// BR_DEAD_REPLY, +/// BR_TRANSACTION_COMPLETE, +/// BR_INCREFS, +/// BR_ACQUIRE, +/// BR_RELEASE, +/// BR_DECREFS, +/// BR_NOOP, +/// BR_SPAWN_LOOPER, +/// BR_DEAD_BINDER, +/// BR_CLEAR_DEATH_NOTIFICATION_DONE, +/// BR_FAILED_REPLY +/// ); +/// +/// assert_eq!(BR_OK, binder_driver_return_protocol_BR_OK); +/// ``` +/// +/// # Modifiers +/// +/// For each identifier, it is possible to attach one or multiple modifiers to +/// it. +/// +/// Currently supported modifiers are: +/// * `span`: change the span of concatenated identifier to the span of the specified token. By +/// default the span of the `[< >]` group is used. +/// * `lower`: change the identifier to lower case. +/// * `upper`: change the identifier to upper case. +/// +/// ``` +/// # const binder_driver_return_protocol_BR_OK: u32 = 0; +/// # const binder_driver_return_protocol_BR_ERROR: u32 = 1; +/// # const binder_driver_return_protocol_BR_TRANSACTION: u32 = 2; +/// # const binder_driver_return_protocol_BR_REPLY: u32 = 3; +/// # const binder_driver_return_protocol_BR_DEAD_REPLY: u32 = 4; +/// # const binder_driver_return_protocol_BR_TRANSACTION_COMPLETE: u32 = 5; +/// # const binder_driver_return_protocol_BR_INCREFS: u32 = 6; +/// # const binder_driver_return_protocol_BR_ACQUIRE: u32 = 7; +/// # const binder_driver_return_protocol_BR_RELEASE: u32 = 8; +/// # const binder_driver_return_protocol_BR_DECREFS: u32 = 9; +/// # const binder_driver_return_protocol_BR_NOOP: u32 = 10; +/// # const binder_driver_return_protocol_BR_SPAWN_LOOPER: u32 = 11; +/// # const binder_driver_return_protocol_BR_DEAD_BINDER: u32 = 12; +/// # const binder_driver_return_protocol_BR_CLEAR_DEATH_NOTIFICATION_DONE: u32 = 13; +/// # const binder_driver_return_protocol_BR_FAILED_REPLY: u32 = 14; +/// macro_rules! pub_no_prefix { +/// ($prefix:ident, $($newname:ident),+) => { +/// ::kernel::macros::paste! { +/// $(pub(crate) const fn [<$newname:lower:span>]() -> u32 { [<$prefix $newname:span>] })+ +/// } +/// }; +/// } +/// +/// pub_no_prefix!( +/// binder_driver_return_protocol_, +/// BR_OK, +/// BR_ERROR, +/// BR_TRANSACTION, +/// BR_REPLY, +/// BR_DEAD_REPLY, +/// BR_TRANSACTION_COMPLETE, +/// BR_INCREFS, +/// BR_ACQUIRE, +/// BR_RELEASE, +/// BR_DECREFS, +/// BR_NOOP, +/// BR_SPAWN_LOOPER, +/// BR_DEAD_BINDER, +/// BR_CLEAR_DEATH_NOTIFICATION_DONE, +/// BR_FAILED_REPLY +/// ); +/// +/// assert_eq!(br_ok(), binder_driver_return_protocol_BR_OK); +/// ``` +/// +/// # Literals +/// +/// Literals can also be concatenated with other identifiers: +/// +/// ``` +/// macro_rules! create_numbered_fn { +/// ($name:literal, $val:literal) => { +/// ::kernel::macros::paste! { +/// fn [<some_ $name _fn $val>]() -> u32 { $val } +/// } +/// }; +/// } +/// +/// create_numbered_fn!("foo", 100); +/// +/// assert_eq!(some_foo_fn100(), 100) +/// ``` +/// +/// [`paste`]: https://docs.rs/paste/ +#[proc_macro] +pub fn paste(input: TokenStream) -> TokenStream { + let mut tokens = proc_macro2::TokenStream::from(input).into_iter().collect(); + paste::expand(&mut tokens); + tokens + .into_iter() + .collect::<proc_macro2::TokenStream>() + .into() +} + +/// Registers a KUnit test suite and its test cases using a user-space like syntax. +/// +/// This macro should be used on modules. If `CONFIG_KUNIT` (in `.config`) is `n`, the target module +/// is ignored. +/// +/// # Examples +/// +/// ```ignore +/// # use kernel::prelude::*; +/// #[kunit_tests(kunit_test_suit_name)] +/// mod tests { +/// #[test] +/// fn foo() { +/// assert_eq!(1, 1); +/// } +/// +/// #[test] +/// fn bar() { +/// assert_eq!(2, 2); +/// } +/// } +/// ``` +#[proc_macro_attribute] +pub fn kunit_tests(attr: TokenStream, input: TokenStream) -> TokenStream { + kunit::kunit_tests(parse_macro_input!(attr), parse_macro_input!(input)) + .unwrap_or_else(|e| e.into_compile_error()) + .into() +} + +/// Obtain a type that implements [`ForLt`] for the given higher-ranked type. +/// +/// Please refer to the documentation of the [`ForLt`] trait. +/// +/// [`ForLt`]: trait.ForLt.html +#[proc_macro] +#[allow(non_snake_case)] +pub fn ForLt(input: TokenStream) -> TokenStream { + for_lt::for_lt(parse_macro_input!(input)).into() +} + +/// Obtain a type that implements [`CovariantForLt`] (and [`ForLt`]) for the given higher-ranked +/// type. +/// +/// Unlike [`ForLt!`], this macro additionally proves that the type is covariant over the lifetime, +/// providing a safe [`CovariantForLt::cast_ref`] method. +/// +/// Please refer to the documentation of the [`CovariantForLt`] trait. +/// +/// [`CovariantForLt`]: trait.CovariantForLt.html +/// [`CovariantForLt::cast_ref`]: trait.CovariantForLt.html#method.cast_ref +/// [`ForLt`]: trait.ForLt.html +#[proc_macro] +#[allow(non_snake_case)] +pub fn CovariantForLt(input: TokenStream) -> TokenStream { + for_lt::covariant_for_lt(parse_macro_input!(input)).into() +} |
