Merge tag 'driver-core-7.3-rc2' of git://git.kernel.org/pub/scm/linux/kernel/git/driver-core/driver-coregrafted

Pull driver core fixes from Danilo Krummrich:

 - Fix kernfs listxattr() not returning security xattr names (e.g.
   SELinux labels) when the kernfs node has no allocated kernfs_iattrs

 - Fix silent truncation of IRQ vector indices in the Rust PCI
   abstractions

 - Don't select OF from DRIVER_PE_KUNIT_TEST; skip the test when OF is
   disabled instead of silently enabling extra kernel functionality

 - Russ Weight is retiring from kernel development; update the Firmware
   Loader sysfs contact to the driver-core mailing list, add a CREDITS
   entry for Firmware Upload, and update MAINTAINERS accordingly

* tag 'driver-core-7.3-rc2' of git://git.kernel.org/pub/scm/linux/kernel/git/driver-core/driver-core:
  MAINTAINERS: Remove Russ Weight from Firmware Loader
  CREDITS: Add CREDITS entry for Firmware Upload
  firmware_loader: Change contact for sysfs nodes
  rust: pci: reject IRQ vector indices that do not fit in u32
  kernfs: preserve security xattrs without allocating iattrs
  drivers: base: test: DRIVER_PE_KUNIT_TEST should not select OF

17 files changed, 4715 insertions, 0 deletions

diff --git a/rust/pin-init/CONTRIBUTING.md b/rust/pin-init/CONTRIBUTING.md
new file mode 100644
index 000000000..16c899a7a
--- /dev/null
+++ b/rust/pin-init/CONTRIBUTING.md
@@ -0,0 +1,72 @@
+# Contributing to `pin-init`
+
+Thanks for showing interest in contributing to `pin-init`! This document outlines the guidelines for
+contributing to `pin-init`.
+
+All contributions are double-licensed under Apache 2.0 and MIT. You can find the respective licenses
+in the `LICENSE-APACHE` and `LICENSE-MIT` files.
+
+## Non-Code Contributions
+
+### Bug Reports
+
+For any type of bug report, please submit an issue using the bug report issue template.
+
+If the issue is a soundness issue, please privately report it as a security vulnerability via the
+GitHub web interface.
+
+### Feature Requests
+
+If you have any feature requests, please submit an issue using the feature request issue template.
+
+### Questions and Getting Help
+
+You can ask questions in the Discussions page of the GitHub repository. If you're encountering
+problems or just have questions related to `pin-init` in the Linux kernel, you can also ask your
+questions in the [Rust-for-Linux Zulip](https://rust-for-linux.zulipchat.com/) or see
+<https://rust-for-linux.com/contact>.
+
+## Contributing Code
+
+### Linux Kernel
+
+`pin-init` is used by the Linux kernel and all commits are synchronized to it. For this reason, the
+same requirements for commits apply to `pin-init`. See [the kernel's documentation] for details. The
+rest of this document will also cover some of the rules listed there and additional ones.
+
+[the kernel's documentation]: https://docs.kernel.org/process/submitting-patches.html
+
+Contributions to `pin-init` ideally go through the [GitHub repository], because that repository runs
+a CI with lots of tests not present in the kernel. However, patches are also accepted (though not
+preferred). Do note that there are some files that are only present in the GitHub repository such as
+tests, licenses and cargo related files. Making changes to them can only happen via GitHub.
+
+[GitHub repository]: https://github.com/Rust-for-Linux/pin-init
+
+### Commit Style
+
+Everything must compile without errors or warnings and all tests must pass after **every commit**.
+This is important for bisection and also required by the kernel.
+
+Each commit should be a single, logically cohesive change. Of course it's best to keep the changes
+small and digestible, but logically linked changes should be made in the same commit. For example,
+when fixing typos, create a single commit that fixes all of them instead of one commit per typo.
+
+Commits must have a meaningful commit title. Commits with changes to files in the `internal`
+directory should have a title prefixed with `internal:`. The commit message should explain the
+change and its rationale. You also have to add your `Signed-off-by` tag, see [Developer's
+Certificate of Origin]. This has to be done for both mailing list submissions as well as GitHub
+submissions.
+
+[Developer's Certificate of Origin]: https://docs.kernel.org/process/submitting-patches.html#sign-your-work-the-developer-s-certificate-of-origin
+
+Any changes made to public APIs must be documented not only in the commit message, but also in the
+`CHANGELOG.md` file. This is especially important for breaking changes, as those warrant a major
+version bump.
+
+If you make changes to the top-level crate documentation, you also need to update the `README.md`
+via `cargo rdme`.
+
+Some of these rules can be ignored if the change is done solely to files that are not present in the
+kernel version of this library. Those files are documented in the `sync-kernel.sh` script at the
+very bottom in the `--exclude` flag given to the `git am` command.
diff --git a/rust/pin-init/README.md b/rust/pin-init/README.md
new file mode 100644
index 000000000..2312c9e75
--- /dev/null
+++ b/rust/pin-init/README.md
@@ -0,0 +1,247 @@
+[![Crates.io](https://img.shields.io/crates/v/pin-init.svg)](https://crates.io/crates/pin-init)
+[![Documentation](https://docs.rs/pin-init/badge.svg)](https://docs.rs/pin-init/)
+[![Dependency status](https://deps.rs/repo/github/Rust-for-Linux/pin-init/status.svg)](https://deps.rs/repo/github/Rust-for-Linux/pin-init)
+![License](https://img.shields.io/crates/l/pin-init)
+[![Toolchain](https://img.shields.io/badge/toolchain-nightly-red)](#nightly-only)
+![GitHub Workflow Status](https://img.shields.io/github/actions/workflow/status/Rust-for-Linux/pin-init/ci.yml)
+# `pin-init`
+
+> [!NOTE]
+> 
+> This crate was originally named [`pinned-init`], but the migration to
+> `pin-init` is not yet complete. The `legacy` branch contains the current
+> version of the `pinned-init` crate & the `main` branch already incorporates
+> the rename to `pin-init`.
+>
+> There are still some changes needed on the kernel side before the migration
+> can be completed.
+
+[`pinned-init`]: https://crates.io/crates/pinned-init
+
+<!-- cargo-rdme start -->
+
+Library to safely and fallibly initialize pinned `struct`s using in-place constructors.
+
+[Pinning][pinning] is Rust's way of ensuring data does not move.
+
+It also allows in-place initialization of big `struct`s that would otherwise produce a stack
+overflow.
+
+This library's main use-case is in [Rust-for-Linux]. Although this version can be used
+standalone.
+
+There are cases when you want to in-place initialize a struct. For example when it is very big
+and moving it from the stack is not an option, because it is bigger than the stack itself.
+Another reason would be that you need the address of the object to initialize it. This stands
+in direct conflict with Rust's normal process of first initializing an object and then moving
+it into it's final memory location. For more information, see
+<https://rust-for-linux.com/the-safe-pinned-initialization-problem>.
+
+This library allows you to do in-place initialization safely.
+
+### Nightly Needed for `alloc` feature
+
+This library requires the [`allocator_api` unstable feature] when the `alloc` feature is
+enabled and thus this feature can only be used with a nightly compiler. When enabling the
+`alloc` feature, the user will be required to activate `allocator_api` as well.
+
+[`allocator_api` unstable feature]: https://doc.rust-lang.org/nightly/unstable-book/library-features/allocator-api.html
+
+The feature is enabled by default, thus by default `pin-init` will require a nightly compiler.
+However, using the crate on stable compilers is possible by disabling `alloc`. In practice this
+will require the `std` feature, because stable compilers have neither `Box` nor `Arc` in no-std
+mode.
+
+### Nightly needed for `unsafe-pinned` feature
+
+This feature enables the `Wrapper` implementation on the unstable `core::pin::UnsafePinned` type.
+This requires the [`unsafe_pinned` unstable feature](https://github.com/rust-lang/rust/issues/125735)
+and therefore a nightly compiler. Note that this feature is not enabled by default.
+
+## Overview
+
+To initialize a `struct` with an in-place constructor you will need two things:
+- an in-place constructor,
+- a memory location that can hold your `struct` (this can be the [stack], an [`Arc<T>`],
+  [`Box<T>`] or any other smart pointer that supports this library).
+
+To get an in-place constructor there are generally three options:
+- directly creating an in-place constructor using the [`pin_init!`] macro,
+- a custom function/macro returning an in-place constructor provided by someone else,
+- using the unsafe function [`pin_init_from_closure()`] to manually create an initializer.
+
+Aside from pinned initialization, this library also supports in-place construction without
+pinning, the macros/types/functions are generally named like the pinned variants without the
+`pin_` prefix.
+
+## Examples
+
+Throughout the examples we will often make use of the `CMutex` type which can be found in
+`../examples/mutex.rs`. It is essentially a userland rebuild of the `struct mutex` type from
+the Linux kernel. It also uses a wait list and a basic spinlock. Importantly the wait list
+requires it to be pinned to be locked and thus is a prime candidate for using this library.
+
+### Using the [`pin_init!`] macro
+
+If you want to use [`PinInit`], then you will have to annotate your `struct` with
+`#[`[`pin_data`]`]`. It is a macro that uses `#[pin]` as a marker for
+[structurally pinned fields]. After doing this, you can then create an in-place constructor via
+[`pin_init!`]. The syntax is almost the same as normal `struct` initializers. The difference is
+that you need to write `<-` instead of `:` for fields that you want to initialize in-place.
+
+```rust
+use pin_init::{pin_data, pin_init, InPlaceInit};
+
+#[pin_data]
+struct Foo {
+    #[pin]
+    a: CMutex<usize>,
+    b: u32,
+}
+
+let foo = pin_init!(Foo {
+    a <- CMutex::new(42),
+    b: 24,
+});
+```
+
+`foo` now is of the type [`impl PinInit<Foo>`]. We can now use any smart pointer that we like
+(or just the stack) to actually initialize a `Foo`:
+
+```rust
+let foo: Result<Pin<Box<Foo>>, AllocError> = Box::pin_init(foo);
+```
+
+For more information see the [`pin_init!`] macro.
+
+### Using a custom function/macro that returns an initializer
+
+Many types that use this library supply a function/macro that returns an initializer, because
+the above method only works for types where you can access the fields.
+
+```rust
+let mtx: Result<Pin<Arc<CMutex<usize>>>, _> = Arc::pin_init(CMutex::new(42));
+```
+
+To declare an init macro/function you just return an [`impl PinInit<T, E>`]:
+
+```rust
+#[pin_data]
+struct DriverData {
+    #[pin]
+    status: CMutex<i32>,
+    buffer: Box<[u8; 1_000_000]>,
+}
+
+impl DriverData {
+    fn new() -> impl PinInit<Self, Error> {
+        pin_init!(Self {
+            status <- CMutex::new(0),
+            buffer: Box::init(pin_init::init_zeroed())?,
+        }? Error)
+    }
+}
+```
+
+### Manual creation of an initializer
+
+Often when working with primitives the previous approaches are not sufficient. That is where
+[`pin_init_from_closure()`] comes in. This `unsafe` function allows you to create a
+[`impl PinInit<T, E>`] directly from a closure. Of course you have to ensure that the closure
+actually does the initialization in the correct way. Here are the things to look out for
+(we are calling the parameter to the closure `slot`):
+- when the closure returns `Ok(())`, then it has completed the initialization successfully, so
+  `slot` now contains a valid bit pattern for the type `T`,
+- when the closure returns `Err(e)`, then the caller may deallocate the memory at `slot`, so
+  you need to take care to clean up anything if your initialization fails mid-way,
+- you may assume that `slot` will stay pinned even after the closure returns until `drop` of
+  `slot` gets called.
+
+```rust
+use pin_init::{pin_data, pinned_drop, PinInit, PinnedDrop, pin_init_from_closure};
+use core::{
+    marker::PhantomPinned,
+    cell::UnsafeCell,
+    pin::Pin,
+    mem::MaybeUninit,
+};
+mod bindings {
+    #[repr(C)]
+    pub struct foo {
+        /* fields from C ... */
+    }
+    extern "C" {
+        pub fn init_foo(ptr: *mut foo);
+        pub fn destroy_foo(ptr: *mut foo);
+        #[must_use = "you must check the error return code"]
+        pub fn enable_foo(ptr: *mut foo, flags: u32) -> i32;
+    }
+}
+
+/// # Invariants
+///
+/// `foo` is always initialized
+#[pin_data(PinnedDrop)]
+pub struct RawFoo {
+    #[pin]
+    _p: PhantomPinned,
+    #[pin]
+    foo: UnsafeCell<MaybeUninit<bindings::foo>>,
+}
+
+impl RawFoo {
+    pub fn new(flags: u32) -> impl PinInit<Self, i32> {
+        // SAFETY:
+        // - when the closure returns `Ok(())`, then it has successfully initialized and
+        //   enabled `foo`,
+        // - when it returns `Err(e)`, then it has cleaned up before
+        unsafe {
+            pin_init_from_closure(move |slot: *mut Self| {
+                // `slot` contains uninit memory, avoid creating a reference.
+                let foo = &raw mut (*slot).foo;
+                let foo = UnsafeCell::raw_get(foo).cast::<bindings::foo>();
+
+                // Initialize the `foo`
+                bindings::init_foo(foo);
+
+                // Try to enable it.
+                let err = bindings::enable_foo(foo, flags);
+                if err != 0 {
+                    // Enabling has failed, first clean up the foo and then return the error.
+                    bindings::destroy_foo(foo);
+                    Err(err)
+                } else {
+                    // All fields of `RawFoo` have been initialized, since `_p` is a ZST.
+                    Ok(())
+                }
+            })
+        }
+    }
+}
+
+#[pinned_drop]
+impl PinnedDrop for RawFoo {
+    fn drop(self: Pin<&mut Self>) {
+        // SAFETY: Since `foo` is initialized, destroying is safe.
+        unsafe { bindings::destroy_foo(self.foo.get().cast::<bindings::foo>()) };
+    }
+}
+```
+
+For more information on how to use [`pin_init_from_closure()`], take a look at the uses inside
+the `kernel` crate. The [`sync`] module is a good starting point.
+
+[`sync`]: https://rust.docs.kernel.org/kernel/sync/index.html
+[pinning]: https://doc.rust-lang.org/std/pin/index.html
+[structurally pinned fields]: https://doc.rust-lang.org/std/pin/index.html#projections-and-structural-pinning
+[stack]: https://docs.rs/pin-init/latest/pin_init/macro.stack_pin_init.html
+[`impl PinInit<Foo>`]: https://docs.rs/pin-init/latest/pin_init/trait.PinInit.html
+[`impl PinInit<T, E>`]: https://docs.rs/pin-init/latest/pin_init/trait.PinInit.html
+[`impl Init<T, E>`]: https://docs.rs/pin-init/latest/pin_init/trait.Init.html
+[Rust-for-Linux]: https://rust-for-linux.com/
+
+<!-- cargo-rdme end -->
+
+<!-- These links are not picked up by cargo-rdme, since they are behind cfgs... -->
+[`Arc<T>`]: https://doc.rust-lang.org/stable/alloc/sync/struct.Arc.html
+[`Box<T>`]: https://doc.rust-lang.org/stable/alloc/boxed/struct.Box.html
diff --git a/rust/pin-init/examples/big_struct_in_place.rs b/rust/pin-init/examples/big_struct_in_place.rs
new file mode 100644
index 000000000..c05139927
--- /dev/null
+++ b/rust/pin-init/examples/big_struct_in_place.rs
@@ -0,0 +1,43 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
+use pin_init::*;
+
+// Struct with size over 1GiB
+#[derive(Debug)]
+#[allow(dead_code)]
+pub struct BigStruct {
+    buf: [u8; 1024 * 1024 * 1024],
+    a: u64,
+    b: u64,
+    c: u64,
+    d: u64,
+    managed_buf: ManagedBuf,
+}
+
+#[derive(Debug)]
+pub struct ManagedBuf {
+    buf: [u8; 1024 * 1024],
+}
+
+impl ManagedBuf {
+    pub fn new() -> impl Init<Self> {
+        init!(ManagedBuf { buf <- init_zeroed() })
+    }
+}
+
+fn main() {
+    #[cfg(any(feature = "std", feature = "alloc"))]
+    {
+        // we want to initialize the struct in-place, otherwise we would get a stackoverflow
+        let buf: Box<BigStruct> = Box::init(init!(BigStruct {
+            buf <- init_zeroed(),
+            a: 7,
+            b: 186,
+            c: 7789,
+            d: 34,
+            managed_buf <- ManagedBuf::new(),
+        }))
+        .unwrap();
+        println!("{}", core::mem::size_of_val(&*buf));
+    }
+}
diff --git a/rust/pin-init/examples/error.rs b/rust/pin-init/examples/error.rs
new file mode 100644
index 000000000..96f095398
--- /dev/null
+++ b/rust/pin-init/examples/error.rs
@@ -0,0 +1,31 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
+#![cfg_attr(feature = "alloc", feature(allocator_api))]
+
+use core::convert::Infallible;
+
+#[cfg(feature = "alloc")]
+use std::alloc::AllocError;
+
+#[derive(Debug)]
+pub struct Error;
+
+impl From<Infallible> for Error {
+    #[inline]
+    fn from(e: Infallible) -> Self {
+        match e {}
+    }
+}
+
+#[cfg(feature = "alloc")]
+impl From<AllocError> for Error {
+    #[inline]
+    fn from(_: AllocError) -> Self {
+        Self
+    }
+}
+
+#[allow(dead_code)]
+fn main() {
+    let _ = Error;
+}
diff --git a/rust/pin-init/examples/linked_list.rs b/rust/pin-init/examples/linked_list.rs
new file mode 100644
index 000000000..424585fe2
--- /dev/null
+++ b/rust/pin-init/examples/linked_list.rs
@@ -0,0 +1,168 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
+#![allow(clippy::undocumented_unsafe_blocks)]
+#![cfg_attr(feature = "alloc", feature(allocator_api))]
+
+use core::{
+    cell::Cell,
+    marker::PhantomPinned,
+    pin::Pin,
+    ptr::{self, NonNull},
+};
+
+use pin_init::*;
+
+#[allow(unused_attributes)]
+mod error;
+#[allow(unused_imports)]
+use error::Error;
+
+#[pin_data(PinnedDrop)]
+#[repr(C)]
+#[derive(Debug)]
+pub struct ListHead {
+    next: Link,
+    prev: Link,
+    #[pin]
+    pin: PhantomPinned,
+}
+
+impl ListHead {
+    #[inline]
+    pub fn new() -> impl PinInit<Self> {
+        pin_init!(&this in Self {
+            next: unsafe { Link::new_unchecked(this) },
+            prev: unsafe { Link::new_unchecked(this) },
+            pin: PhantomPinned,
+        })
+    }
+
+    #[inline]
+    #[allow(dead_code)]
+    pub fn insert_next(list: &ListHead) -> impl PinInit<Self> + '_ {
+        pin_init!(&this in Self {
+            prev: list.next.prev().replace(unsafe { Link::new_unchecked(this)}),
+            next: list.next.replace(unsafe { Link::new_unchecked(this)}),
+            pin: PhantomPinned,
+        })
+    }
+
+    #[inline]
+    pub fn insert_prev(list: &ListHead) -> impl PinInit<Self> + '_ {
+        pin_init!(&this in Self {
+            next: list.prev.next().replace(unsafe { Link::new_unchecked(this)}),
+            prev: list.prev.replace(unsafe { Link::new_unchecked(this)}),
+            pin: PhantomPinned,
+        })
+    }
+
+    #[inline]
+    pub fn next(&self) -> Option<NonNull<Self>> {
+        if ptr::eq(self.next.as_ptr(), self) {
+            None
+        } else {
+            Some(unsafe { NonNull::new_unchecked(self.next.as_ptr() as *mut Self) })
+        }
+    }
+
+    #[allow(dead_code)]
+    pub fn size(&self) -> usize {
+        let mut size = 1;
+        let mut cur = self.next.clone();
+        while !ptr::eq(self, cur.cur()) {
+            cur = cur.next().clone();
+            size += 1;
+        }
+        size
+    }
+}
+
+#[pinned_drop]
+impl PinnedDrop for ListHead {
+    //#[inline]
+    fn drop(self: Pin<&mut Self>) {
+        if !ptr::eq(self.next.as_ptr(), &*self) {
+            let next = unsafe { &*self.next.as_ptr() };
+            let prev = unsafe { &*self.prev.as_ptr() };
+            next.prev.set(&self.prev);
+            prev.next.set(&self.next);
+        }
+    }
+}
+
+#[repr(transparent)]
+#[derive(Clone, Debug)]
+struct Link(Cell<NonNull<ListHead>>);
+
+impl Link {
+    /// # Safety
+    ///
+    /// The contents of the pointer should form a consistent circular
+    /// linked list; for example, a "next" link should be pointed back
+    /// by the target `ListHead`'s "prev" link and a "prev" link should be
+    /// pointed back by the target `ListHead`'s "next" link.
+    #[inline]
+    unsafe fn new_unchecked(ptr: NonNull<ListHead>) -> Self {
+        Self(Cell::new(ptr))
+    }
+
+    #[inline]
+    fn next(&self) -> &Link {
+        unsafe { &(*self.0.get().as_ptr()).next }
+    }
+
+    #[inline]
+    #[allow(dead_code)]
+    fn prev(&self) -> &Link {
+        unsafe { &(*self.0.get().as_ptr()).prev }
+    }
+
+    #[allow(dead_code)]
+    fn cur(&self) -> &ListHead {
+        unsafe { &*self.0.get().as_ptr() }
+    }
+
+    #[inline]
+    fn replace(&self, other: Link) -> Link {
+        unsafe { Link::new_unchecked(self.0.replace(other.0.get())) }
+    }
+
+    #[inline]
+    fn as_ptr(&self) -> *const ListHead {
+        self.0.get().as_ptr()
+    }
+
+    #[inline]
+    fn set(&self, val: &Link) {
+        self.0.set(val.0.get());
+    }
+}
+
+#[allow(dead_code)]
+#[cfg(not(any(feature = "std", feature = "alloc")))]
+fn main() {}
+
+#[allow(dead_code)]
+#[cfg_attr(test, test)]
+#[cfg(any(feature = "std", feature = "alloc"))]
+fn main() -> Result<(), Error> {
+    let a = Box::pin_init(ListHead::new())?;
+    stack_pin_init!(let b = ListHead::insert_next(&a));
+    stack_pin_init!(let c = ListHead::insert_next(&a));
+    stack_pin_init!(let d = ListHead::insert_next(&b));
+    let e = Box::pin_init(ListHead::insert_next(&b))?;
+    println!("a ({a:p}): {a:?}");
+    println!("b ({b:p}): {b:?}");
+    println!("c ({c:p}): {c:?}");
+    println!("d ({d:p}): {d:?}");
+    println!("e ({e:p}): {e:?}");
+    let mut inspect = &*a;
+    while let Some(next) = inspect.next() {
+        println!("({inspect:p}): {inspect:?}");
+        inspect = unsafe { &*next.as_ptr() };
+        if core::ptr::eq(inspect, &*a) {
+            break;
+        }
+    }
+    Ok(())
+}
diff --git a/rust/pin-init/examples/mutex.rs b/rust/pin-init/examples/mutex.rs
new file mode 100644
index 000000000..e8d4dbb66
--- /dev/null
+++ b/rust/pin-init/examples/mutex.rs
@@ -0,0 +1,217 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
+#![allow(clippy::undocumented_unsafe_blocks)]
+#![cfg_attr(feature = "alloc", feature(allocator_api))]
+#![allow(clippy::missing_safety_doc)]
+
+use core::{
+    cell::{Cell, UnsafeCell},
+    marker::PhantomPinned,
+    ops::{Deref, DerefMut},
+    pin::Pin,
+    sync::atomic::{AtomicBool, Ordering},
+};
+#[cfg(feature = "std")]
+use std::{
+    sync::Arc,
+    thread::{self, sleep, Builder, Thread},
+    time::Duration,
+};
+
+use pin_init::*;
+#[allow(unused_attributes)]
+#[path = "./linked_list.rs"]
+pub mod linked_list;
+use linked_list::*;
+
+pub struct SpinLock {
+    inner: AtomicBool,
+}
+
+impl SpinLock {
+    #[inline]
+    pub fn acquire(&self) -> SpinLockGuard<'_> {
+        while self
+            .inner
+            .compare_exchange(false, true, Ordering::Acquire, Ordering::Relaxed)
+            .is_err()
+        {
+            #[cfg(feature = "std")]
+            while self.inner.load(Ordering::Relaxed) {
+                thread::yield_now();
+            }
+        }
+        SpinLockGuard(self)
+    }
+
+    #[inline]
+    #[allow(clippy::new_without_default)]
+    pub const fn new() -> Self {
+        Self {
+            inner: AtomicBool::new(false),
+        }
+    }
+}
+
+pub struct SpinLockGuard<'a>(&'a SpinLock);
+
+impl Drop for SpinLockGuard<'_> {
+    #[inline]
+    fn drop(&mut self) {
+        self.0.inner.store(false, Ordering::Release);
+    }
+}
+
+#[pin_data]
+pub struct CMutex<T> {
+    #[pin]
+    wait_list: ListHead,
+    spin_lock: SpinLock,
+    locked: Cell<bool>,
+    #[pin]
+    data: UnsafeCell<T>,
+}
+
+impl<T> CMutex<T> {
+    #[inline]
+    pub fn new(val: impl PinInit<T>) -> impl PinInit<Self> {
+        pin_init!(CMutex {
+            wait_list <- ListHead::new(),
+            spin_lock: SpinLock::new(),
+            locked: Cell::new(false),
+            data <- UnsafeCell::pin_init(val),
+        })
+    }
+
+    #[inline]
+    pub fn lock(&self) -> Pin<CMutexGuard<'_, T>> {
+        let mut sguard = self.spin_lock.acquire();
+        if self.locked.get() {
+            stack_pin_init!(let _wait_entry = WaitEntry::insert_new(&self.wait_list));
+            // println!("wait list length: {}", self.wait_list.size());
+            while self.locked.get() {
+                drop(sguard);
+                #[cfg(feature = "std")]
+                thread::park();
+                sguard = self.spin_lock.acquire();
+            }
+        }
+        self.locked.set(true);
+        unsafe {
+            Pin::new_unchecked(CMutexGuard {
+                mtx: self,
+                _pin: PhantomPinned,
+            })
+        }
+    }
+
+    #[allow(dead_code)]
+    pub fn get_data_mut(self: Pin<&mut Self>) -> &mut T {
+        // SAFETY: we have an exclusive reference and thus nobody has access to data.
+        unsafe { &mut *self.data.get() }
+    }
+}
+
+unsafe impl<T: Send> Send for CMutex<T> {}
+unsafe impl<T: Send> Sync for CMutex<T> {}
+
+pub struct CMutexGuard<'a, T> {
+    mtx: &'a CMutex<T>,
+    _pin: PhantomPinned,
+}
+
+impl<T> Drop for CMutexGuard<'_, T> {
+    #[inline]
+    fn drop(&mut self) {
+        let sguard = self.mtx.spin_lock.acquire();
+        self.mtx.locked.set(false);
+        if let Some(list_field) = self.mtx.wait_list.next() {
+            let _wait_entry = list_field.as_ptr().cast::<WaitEntry>();
+            #[cfg(feature = "std")]
+            unsafe {
+                (*_wait_entry).thread.unpark()
+            };
+        }
+        drop(sguard);
+    }
+}
+
+impl<T> Deref for CMutexGuard<'_, T> {
+    type Target = T;
+
+    #[inline]
+    fn deref(&self) -> &Self::Target {
+        unsafe { &*self.mtx.data.get() }
+    }
+}
+
+impl<T> DerefMut for CMutexGuard<'_, T> {
+    #[inline]
+    fn deref_mut(&mut self) -> &mut Self::Target {
+        unsafe { &mut *self.mtx.data.get() }
+    }
+}
+
+#[pin_data]
+#[repr(C)]
+struct WaitEntry {
+    #[pin]
+    wait_list: ListHead,
+    #[cfg(feature = "std")]
+    thread: Thread,
+}
+
+impl WaitEntry {
+    #[inline]
+    fn insert_new(list: &ListHead) -> impl PinInit<Self> + '_ {
+        #[cfg(feature = "std")]
+        {
+            pin_init!(Self {
+                thread: thread::current(),
+                wait_list <- ListHead::insert_prev(list),
+            })
+        }
+        #[cfg(not(feature = "std"))]
+        {
+            pin_init!(Self {
+                wait_list <- ListHead::insert_prev(list),
+            })
+        }
+    }
+}
+
+#[cfg_attr(test, test)]
+#[allow(dead_code)]
+fn main() {
+    #[cfg(feature = "std")]
+    {
+        let mtx: Pin<Arc<CMutex<usize>>> = Arc::pin_init(CMutex::new(0)).unwrap();
+        let mut handles = vec![];
+        let thread_count = 20;
+        let workload = if cfg!(miri) { 100 } else { 1_000 };
+        for i in 0..thread_count {
+            let mtx = mtx.clone();
+            handles.push(
+                Builder::new()
+                    .name(format!("worker #{i}"))
+                    .spawn(move || {
+                        for _ in 0..workload {
+                            *mtx.lock() += 1;
+                        }
+                        println!("{i} halfway");
+                        sleep(Duration::from_millis((i as u64) * 10));
+                        for _ in 0..workload {
+                            *mtx.lock() += 1;
+                        }
+                        println!("{i} finished");
+                    })
+                    .expect("should not fail"),
+            );
+        }
+        for h in handles {
+            h.join().expect("thread panicked");
+        }
+        println!("{:?}", *mtx.lock());
+        assert_eq!(*mtx.lock(), workload * thread_count * 2);
+    }
+}
diff --git a/rust/pin-init/examples/pthread_mutex.rs b/rust/pin-init/examples/pthread_mutex.rs
new file mode 100644
index 000000000..00f457e68
--- /dev/null
+++ b/rust/pin-init/examples/pthread_mutex.rs
@@ -0,0 +1,183 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
+// inspired by <https://github.com/nbdd0121/pin-init/blob/trunk/examples/pthread_mutex.rs>
+#![allow(clippy::undocumented_unsafe_blocks)]
+#![cfg_attr(feature = "alloc", feature(allocator_api))]
+
+#[cfg(not(windows))]
+mod pthread_mtx {
+    #[cfg(feature = "alloc")]
+    use core::alloc::AllocError;
+    use core::{
+        cell::UnsafeCell,
+        marker::PhantomPinned,
+        mem::MaybeUninit,
+        ops::{Deref, DerefMut},
+        pin::Pin,
+    };
+    use pin_init::*;
+    use std::convert::Infallible;
+
+    #[pin_data(PinnedDrop)]
+    pub struct PThreadMutex<T> {
+        #[pin]
+        raw: UnsafeCell<libc::pthread_mutex_t>,
+        data: UnsafeCell<T>,
+        #[pin]
+        pin: PhantomPinned,
+    }
+
+    unsafe impl<T: Send> Send for PThreadMutex<T> {}
+    unsafe impl<T: Send> Sync for PThreadMutex<T> {}
+
+    #[pinned_drop]
+    impl<T> PinnedDrop for PThreadMutex<T> {
+        fn drop(self: Pin<&mut Self>) {
+            unsafe {
+                libc::pthread_mutex_destroy(self.raw.get());
+            }
+        }
+    }
+
+    #[derive(Debug)]
+    pub enum Error {
+        #[allow(dead_code)]
+        IO(std::io::Error),
+        #[allow(dead_code)]
+        Alloc,
+    }
+
+    impl From<Infallible> for Error {
+        fn from(e: Infallible) -> Self {
+            match e {}
+        }
+    }
+
+    #[cfg(feature = "alloc")]
+    impl From<AllocError> for Error {
+        fn from(_: AllocError) -> Self {
+            Self::Alloc
+        }
+    }
+
+    impl<T> PThreadMutex<T> {
+        #[allow(dead_code)]
+        pub fn new(data: T) -> impl PinInit<Self, Error> {
+            fn init_raw() -> impl PinInit<UnsafeCell<libc::pthread_mutex_t>, Error> {
+                let init = |slot: *mut UnsafeCell<libc::pthread_mutex_t>| {
+                    // we can cast, because `UnsafeCell` has the same layout as T.
+                    let slot: *mut libc::pthread_mutex_t = slot.cast();
+                    let mut attr = MaybeUninit::uninit();
+                    let attr = attr.as_mut_ptr();
+                    // SAFETY: ptr is valid
+                    let ret = unsafe { libc::pthread_mutexattr_init(attr) };
+                    if ret != 0 {
+                        return Err(Error::IO(std::io::Error::from_raw_os_error(ret)));
+                    }
+                    // SAFETY: attr is initialized
+                    let ret = unsafe {
+                        libc::pthread_mutexattr_settype(attr, libc::PTHREAD_MUTEX_NORMAL)
+                    };
+                    if ret != 0 {
+                        // SAFETY: attr is initialized
+                        unsafe { libc::pthread_mutexattr_destroy(attr) };
+                        return Err(Error::IO(std::io::Error::from_raw_os_error(ret)));
+                    }
+                    // SAFETY: slot is valid
+                    unsafe { slot.write(libc::PTHREAD_MUTEX_INITIALIZER) };
+                    // SAFETY: attr and slot are valid ptrs and attr is initialized
+                    let ret = unsafe { libc::pthread_mutex_init(slot, attr) };
+                    // SAFETY: attr was initialized
+                    unsafe { libc::pthread_mutexattr_destroy(attr) };
+                    if ret != 0 {
+                        return Err(Error::IO(std::io::Error::from_raw_os_error(ret)));
+                    }
+                    Ok(())
+                };
+                // SAFETY: mutex has been initialized
+                unsafe { pin_init_from_closure(init) }
+            }
+            pin_init!(Self {
+                data: UnsafeCell::new(data),
+                raw <- init_raw(),
+                pin: PhantomPinned,
+            }? Error)
+        }
+
+        #[allow(dead_code)]
+        pub fn lock(&self) -> PThreadMutexGuard<'_, T> {
+            // SAFETY: raw is always initialized
+            unsafe { libc::pthread_mutex_lock(self.raw.get()) };
+            PThreadMutexGuard { mtx: self }
+        }
+    }
+
+    pub struct PThreadMutexGuard<'a, T> {
+        mtx: &'a PThreadMutex<T>,
+    }
+
+    impl<T> Drop for PThreadMutexGuard<'_, T> {
+        fn drop(&mut self) {
+            // SAFETY: raw is always initialized
+            unsafe { libc::pthread_mutex_unlock(self.mtx.raw.get()) };
+        }
+    }
+
+    impl<T> Deref for PThreadMutexGuard<'_, T> {
+        type Target = T;
+
+        fn deref(&self) -> &Self::Target {
+            unsafe { &*self.mtx.data.get() }
+        }
+    }
+
+    impl<T> DerefMut for PThreadMutexGuard<'_, T> {
+        fn deref_mut(&mut self) -> &mut Self::Target {
+            unsafe { &mut *self.mtx.data.get() }
+        }
+    }
+}
+
+#[cfg_attr(test, test)]
+#[cfg_attr(all(test, miri), ignore)]
+fn main() {
+    #[cfg(all(any(feature = "std", feature = "alloc"), not(windows)))]
+    {
+        use core::pin::Pin;
+        use pin_init::*;
+        use pthread_mtx::*;
+        use std::{
+            sync::Arc,
+            thread::{sleep, Builder},
+            time::Duration,
+        };
+        let mtx: Pin<Arc<PThreadMutex<usize>>> = Arc::try_pin_init(PThreadMutex::new(0)).unwrap();
+        let mut handles = vec![];
+        let thread_count = 20;
+        let workload = 1_000_000;
+        for i in 0..thread_count {
+            let mtx = mtx.clone();
+            handles.push(
+                Builder::new()
+                    .name(format!("worker #{i}"))
+                    .spawn(move || {
+                        for _ in 0..workload {
+                            *mtx.lock() += 1;
+                        }
+                        println!("{i} halfway");
+                        sleep(Duration::from_millis((i as u64) * 10));
+                        for _ in 0..workload {
+                            *mtx.lock() += 1;
+                        }
+                        println!("{i} finished");
+                    })
+                    .expect("should not fail"),
+            );
+        }
+        for h in handles {
+            h.join().expect("thread panicked");
+        }
+        println!("{:?}", *mtx.lock());
+        assert_eq!(*mtx.lock(), workload * thread_count * 2);
+    }
+}
diff --git a/rust/pin-init/examples/static_init.rs b/rust/pin-init/examples/static_init.rs
new file mode 100644
index 000000000..8dd52313c
--- /dev/null
+++ b/rust/pin-init/examples/static_init.rs
@@ -0,0 +1,121 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
+#![allow(clippy::undocumented_unsafe_blocks)]
+#![cfg_attr(feature = "alloc", feature(allocator_api))]
+#![allow(unused_imports)]
+
+use core::{
+    cell::{Cell, UnsafeCell},
+    mem::MaybeUninit,
+    ops,
+    pin::Pin,
+    time::Duration,
+};
+use pin_init::*;
+#[cfg(feature = "std")]
+use std::{
+    sync::Arc,
+    thread::{sleep, Builder},
+};
+
+#[allow(unused_attributes)]
+mod mutex;
+use mutex::*;
+
+pub struct StaticInit<T, I> {
+    cell: UnsafeCell<MaybeUninit<T>>,
+    init: Cell<Option<I>>,
+    lock: SpinLock,
+    present: Cell<bool>,
+}
+
+unsafe impl<T: Sync, I> Sync for StaticInit<T, I> {}
+unsafe impl<T: Send, I> Send for StaticInit<T, I> {}
+
+impl<T, I: PinInit<T>> StaticInit<T, I> {
+    pub const fn new(init: I) -> Self {
+        Self {
+            cell: UnsafeCell::new(MaybeUninit::uninit()),
+            init: Cell::new(Some(init)),
+            lock: SpinLock::new(),
+            present: Cell::new(false),
+        }
+    }
+}
+
+impl<T, I: PinInit<T>> ops::Deref for StaticInit<T, I> {
+    type Target = T;
+    fn deref(&self) -> &Self::Target {
+        if self.present.get() {
+            unsafe { (*self.cell.get()).assume_init_ref() }
+        } else {
+            println!("acquire spinlock on static init");
+            let _guard = self.lock.acquire();
+            println!("rechecking present...");
+            std::thread::sleep(std::time::Duration::from_millis(200));
+            if self.present.get() {
+                return unsafe { (*self.cell.get()).assume_init_ref() };
+            }
+            println!("doing init");
+            let ptr = self.cell.get().cast::<T>();
+            match self.init.take() {
+                Some(f) => unsafe { pin_init::raw_init(ptr, f) },
+                None => unsafe { core::hint::unreachable_unchecked() },
+            }
+            self.present.set(true);
+            unsafe { (*self.cell.get()).assume_init_ref() }
+        }
+    }
+}
+
+pub struct CountInit;
+
+unsafe impl PinInit<CMutex<usize>> for CountInit {
+    unsafe fn __init(self, slot: *mut CMutex<usize>) -> Result<(), core::convert::Infallible> {
+        let init = CMutex::new(0);
+        std::thread::sleep(std::time::Duration::from_millis(1000));
+        unsafe { pin_init::raw_init(slot, init) };
+        Ok(())
+    }
+}
+
+pub static COUNT: StaticInit<CMutex<usize>, CountInit> = StaticInit::new(CountInit);
+
+fn main() {
+    #[cfg(feature = "std")]
+    {
+        let mtx: Pin<Arc<CMutex<usize>>> = Arc::pin_init(CMutex::new(0)).unwrap();
+        let mut handles = vec![];
+        let thread_count = 20;
+        let workload = 1_000;
+        for i in 0..thread_count {
+            let mtx = mtx.clone();
+            handles.push(
+                Builder::new()
+                    .name(format!("worker #{i}"))
+                    .spawn(move || {
+                        for _ in 0..workload {
+                            *COUNT.lock() += 1;
+                            std::thread::sleep(std::time::Duration::from_millis(10));
+                            *mtx.lock() += 1;
+                            std::thread::sleep(std::time::Duration::from_millis(10));
+                            *COUNT.lock() += 1;
+                        }
+                        println!("{i} halfway");
+                        sleep(Duration::from_millis((i as u64) * 10));
+                        for _ in 0..workload {
+                            std::thread::sleep(std::time::Duration::from_millis(10));
+                            *mtx.lock() += 1;
+                        }
+                        println!("{i} finished");
+                    })
+                    .expect("should not fail"),
+            );
+        }
+        for h in handles {
+            h.join().expect("thread panicked");
+        }
+        println!("{:?}, {:?}", *mtx.lock(), *COUNT.lock());
+        assert_eq!(*mtx.lock(), workload * thread_count * 2);
+    }
+}
diff --git a/rust/pin-init/internal/src/diagnostics.rs b/rust/pin-init/internal/src/diagnostics.rs
new file mode 100644
index 000000000..c7d9b3e62
--- /dev/null
+++ b/rust/pin-init/internal/src/diagnostics.rs
@@ -0,0 +1,44 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
+use std::fmt::Display;
+
+use proc_macro2::TokenStream;
+use quote::quote_spanned;
+use syn::{spanned::Spanned, Error};
+
+pub(crate) struct DiagCtxt(TokenStream);
+pub(crate) struct ErrorGuaranteed(());
+
+impl DiagCtxt {
+    pub(crate) fn error(&mut self, span: impl Spanned, msg: impl Display) -> ErrorGuaranteed {
+        let error = Error::new(span.span(), msg);
+        self.0.extend(error.into_compile_error());
+        ErrorGuaranteed(())
+    }
+
+    pub(crate) fn warn(&mut self, span: impl Spanned, msg: impl Display) {
+        // Have the message start on a new line for visual clarity.
+        let msg = format!("\n{}", msg);
+        self.0.extend(quote_spanned!(span.span() =>
+            // Approximate using deprecated warning while `proc_macro_diagnostic` is unstable.
+            const _: () = {
+                #[deprecated = #msg]
+                const fn warn() {}
+                warn();
+            };
+        ));
+    }
+
+    pub(crate) fn with(
+        fun: impl FnOnce(&mut DiagCtxt) -> Result<TokenStream, ErrorGuaranteed>,
+    ) -> TokenStream {
+        let mut dcx = Self(TokenStream::new());
+        match fun(&mut dcx) {
+            Ok(mut stream) => {
+                stream.extend(dcx.0);
+                stream
+            }
+            Err(ErrorGuaranteed(())) => dcx.0,
+        }
+    }
+}
diff --git a/rust/pin-init/internal/src/init.rs b/rust/pin-init/internal/src/init.rs
new file mode 100644
index 000000000..fd0b5ea4a
--- /dev/null
+++ b/rust/pin-init/internal/src/init.rs
@@ -0,0 +1,482 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
+use proc_macro2::{Span, TokenStream};
+use quote::{format_ident, quote};
+use syn::{
+    braced,
+    parse::{End, Parse},
+    parse_quote,
+    punctuated::Punctuated,
+    spanned::Spanned,
+    token, Attribute, Block, Expr, ExprCall, ExprPath, Ident, Path, Token, Type,
+};
+
+use crate::diagnostics::{DiagCtxt, ErrorGuaranteed};
+
+pub(crate) struct Initializer {
+    attrs: Vec<InitializerAttribute>,
+    this: Option<This>,
+    path: Path,
+    brace_token: token::Brace,
+    fields: Punctuated<InitializerField, Token![,]>,
+    rest: Option<(Token![..], Expr)>,
+    error: Option<(Token![?], Type)>,
+}
+
+struct This {
+    _and_token: Token![&],
+    ident: Ident,
+    _in_token: Token![in],
+}
+
+struct InitializerField {
+    attrs: Vec<Attribute>,
+    kind: InitializerKind,
+}
+
+enum InitializerKind {
+    Value {
+        ident: Ident,
+        value: Option<(Token![:], Expr)>,
+    },
+    Init {
+        ident: Ident,
+        _left_arrow_token: Token![<-],
+        value: Expr,
+    },
+    Code {
+        _underscore_token: Token![_],
+        _colon_token: Token![:],
+        block: Block,
+    },
+}
+
+impl InitializerKind {
+    fn ident(&self) -> Option<&Ident> {
+        match self {
+            Self::Value { ident, .. } | Self::Init { ident, .. } => Some(ident),
+            Self::Code { .. } => None,
+        }
+    }
+}
+
+enum InitializerAttribute {
+    DefaultError(DefaultErrorAttribute),
+}
+
+struct DefaultErrorAttribute {
+    ty: Box<Type>,
+}
+
+pub(crate) fn expand(
+    Initializer {
+        attrs,
+        this,
+        path,
+        brace_token,
+        fields,
+        rest,
+        error,
+    }: Initializer,
+    default_error: Option<&'static str>,
+    pinned: bool,
+    dcx: &mut DiagCtxt,
+) -> Result<TokenStream, ErrorGuaranteed> {
+    let error = error.map_or_else(
+        || {
+            if let Some(default_error) = attrs.iter().fold(None, |acc, attr| {
+                #[expect(irrefutable_let_patterns)]
+                if let InitializerAttribute::DefaultError(DefaultErrorAttribute { ty }) = attr {
+                    Some(ty.clone())
+                } else {
+                    acc
+                }
+            }) {
+                default_error
+            } else if let Some(default_error) = default_error {
+                syn::parse_str(default_error).unwrap()
+            } else {
+                dcx.error(brace_token.span.close(), "expected `? <type>` after `}`");
+                parse_quote!(::core::convert::Infallible)
+            }
+        },
+        |(_, err)| Box::new(err),
+    );
+    let slot = format_ident!("slot");
+    let (has_data_trait, get_data, init_from_closure) = if pinned {
+        (
+            format_ident!("HasPinData"),
+            format_ident!("__pin_data"),
+            format_ident!("pin_init_from_closure"),
+        )
+    } else {
+        (
+            format_ident!("HasInitData"),
+            format_ident!("__init_data"),
+            format_ident!("init_from_closure"),
+        )
+    };
+    let init_kind = get_init_kind(rest, dcx);
+    let zeroable_check = match init_kind {
+        InitKind::Normal => quote!(),
+        InitKind::Zeroing => quote! {
+            // The user specified `..Zeroable::zeroed()` at the end of the list of fields.
+            // Therefore we check if the struct implements `Zeroable` and then zero the memory.
+            // This allows us to also remove the check that all fields are present (since we
+            // already set the memory to zero and that is a valid bit pattern).
+            fn assert_zeroable<T: ?::core::marker::Sized>(_: *mut T)
+            where T: ::pin_init::Zeroable
+            {}
+            // Ensure that the struct is indeed `Zeroable`.
+            assert_zeroable(#slot);
+            // SAFETY: The type implements `Zeroable` by the check above.
+            unsafe { ::core::ptr::write_bytes(#slot, 0, 1) };
+        },
+    };
+    let this = match this {
+        None => quote!(),
+        Some(This { ident, .. }) => quote! {
+            // Create the `this` so it can be referenced by the user inside of the
+            // expressions creating the individual fields.
+            let #ident = unsafe { ::core::ptr::NonNull::new_unchecked(slot) };
+        },
+    };
+    // `mixed_site` ensures that the data is not accessible to the user-controlled code.
+    let data = Ident::new("__data", Span::mixed_site());
+    let init_fields = init_fields(&fields, pinned, &data, &slot);
+    let field_check = make_field_check(&fields, init_kind, &path);
+    Ok(quote! {{
+        // Get the data about fields from the supplied type.
+        // SAFETY: TODO
+        let #data = unsafe {
+            use ::pin_init::__internal::#has_data_trait;
+            // Can't use `<#path as #has_data_trait>::#get_data`, since the user is able to omit
+            // generics (which need to be present with that syntax).
+            #path::#get_data()
+        };
+        // Ensure that `#data` really is of type `#data` and help with type inference:
+        let init = #data.__make_closure::<_, #error>(
+            move |slot| {
+                #zeroable_check
+                #this
+                #init_fields
+                #field_check
+                // SAFETY: we are the `init!` macro that is allowed to call this.
+                Ok(unsafe { ::pin_init::__internal::InitOk::new() })
+            }
+        );
+        let init = move |slot| -> ::core::result::Result<(), #error> {
+            init(slot).map(|__InitOk| ())
+        };
+        // SAFETY: TODO
+        unsafe { ::pin_init::#init_from_closure::<_, #error>(init) }
+    }})
+}
+
+enum InitKind {
+    Normal,
+    Zeroing,
+}
+
+fn get_init_kind(rest: Option<(Token![..], Expr)>, dcx: &mut DiagCtxt) -> InitKind {
+    let Some((dotdot, expr)) = rest else {
+        return InitKind::Normal;
+    };
+    match &expr {
+        Expr::Call(ExprCall { func, args, .. }) if args.is_empty() => match &**func {
+            Expr::Path(ExprPath {
+                attrs,
+                qself: None,
+                path:
+                    Path {
+                        leading_colon: None,
+                        segments,
+                    },
+            }) if attrs.is_empty()
+                && segments.len() == 2
+                && segments[0].ident == "Zeroable"
+                && segments[0].arguments.is_none()
+                && segments[1].ident == "init_zeroed"
+                && segments[1].arguments.is_none() =>
+            {
+                return InitKind::Zeroing;
+            }
+            _ => {}
+        },
+        _ => {}
+    }
+    dcx.error(
+        dotdot.span().join(expr.span()).unwrap_or(expr.span()),
+        "expected nothing or `..Zeroable::init_zeroed()`.",
+    );
+    InitKind::Normal
+}
+
+/// Generate the code that initializes the fields of the struct using the initializers in `field`.
+fn init_fields(
+    fields: &Punctuated<InitializerField, Token![,]>,
+    pinned: bool,
+    data: &Ident,
+    slot: &Ident,
+) -> TokenStream {
+    let mut guards = vec![];
+    let mut guard_attrs = vec![];
+    let mut res = TokenStream::new();
+    for InitializerField { attrs, kind } in fields {
+        let cfgs = {
+            let mut cfgs = attrs.clone();
+            cfgs.retain(|attr| attr.path().is_ident("cfg"));
+            cfgs
+        };
+
+        let ident = match kind {
+            InitializerKind::Value { ident, .. } => ident,
+            InitializerKind::Init { ident, .. } => ident,
+            InitializerKind::Code { block, .. } => {
+                let stmt = &block.stmts;
+                res.extend(quote! {
+                    #(#attrs)*
+                    {
+                        #(#stmt)*
+                    }
+                });
+                continue;
+            }
+        };
+
+        let slot = if pinned {
+            quote! {
+                // SAFETY:
+                // - `slot` is valid and properly aligned.
+                // - `make_field_check` checks that `&raw mut (*slot).#ident` is properly aligned.
+                // - `make_field_check` prevents `#ident` from being used twice, therefore
+                //   `(*slot).#ident` is exclusively accessed and has not been initialized.
+                (unsafe { #data.#ident(#slot) })
+            }
+        } else {
+            quote! {
+                // For `init!()` macro, everything is unpinned.
+                // SAFETY:
+                // - `&raw mut (*slot).#ident` is valid.
+                // - `make_field_check` checks that `&raw mut (*slot).#ident` is properly aligned.
+                // - `make_field_check` prevents `#ident` from being used twice, therefore
+                //   `(*slot).#ident` is exclusively accessed and has not been initialized.
+                (unsafe {
+                    ::pin_init::__internal::Slot::<::pin_init::__internal::Unpinned, _>::new(
+                        &raw mut (*#slot).#ident
+                    )
+                })
+            }
+        };
+
+        // `mixed_site` ensures that the guard is not accessible to the user-controlled code.
+        let guard = format_ident!("__{ident}_guard", span = Span::mixed_site());
+
+        let init = match kind {
+            InitializerKind::Value { ident, value } => {
+                let value = value
+                    .as_ref()
+                    .map(|(_, value)| quote!(#value))
+                    .unwrap_or_else(|| quote!(#ident));
+
+                quote! {
+                    #(#attrs)*
+                    let mut #guard = #slot.write(#value);
+
+                }
+            }
+            InitializerKind::Init { value, .. } => {
+                quote! {
+                    #(#attrs)*
+                    let mut #guard = #slot.init(#value)?;
+                }
+            }
+            InitializerKind::Code { .. } => unreachable!(),
+        };
+
+        res.extend(quote! {
+            #init
+
+            #(#cfgs)*
+            // Allow `non_snake_case` since the same warning is going to be reported for the struct
+            // field.
+            #[allow(unused_variables, non_snake_case)]
+            let #ident = #guard.let_binding();
+        });
+
+        guards.push(guard);
+        guard_attrs.push(cfgs);
+    }
+    quote! {
+        #res
+        // If execution reaches this point, all fields have been initialized. Therefore we can now
+        // dismiss the guards by forgetting them.
+        #(
+            #(#guard_attrs)*
+            ::core::mem::forget(#guards);
+        )*
+    }
+}
+
+/// Generate the check for ensuring that every field has been initialized and aligned.
+fn make_field_check(
+    fields: &Punctuated<InitializerField, Token![,]>,
+    init_kind: InitKind,
+    path: &Path,
+) -> TokenStream {
+    let field_attrs: Vec<_> = fields
+        .iter()
+        .filter_map(|f| f.kind.ident().map(|_| &f.attrs))
+        .collect();
+    let field_name: Vec<_> = fields.iter().filter_map(|f| f.kind.ident()).collect();
+    let zeroing_trailer = match init_kind {
+        InitKind::Normal => None,
+        InitKind::Zeroing => Some(quote! {
+            ..::core::mem::zeroed()
+        }),
+    };
+    quote! {
+        #[allow(unreachable_code)]
+        // We use unreachable code to perform field checks. They're still checked by the compiler.
+        // SAFETY: this code is never executed.
+        let _ = || unsafe {
+            // Create references to ensure that the initialized field is properly aligned.
+            // Unaligned fields will cause the compiler to emit E0793. We do not support
+            // unaligned fields since `Init::__init` requires an aligned pointer; the call to
+            // `ptr::write` for value-initialization case has the same requirement.
+            #(
+                #(#field_attrs)*
+                let _ = &(*slot).#field_name;
+            )*
+
+            // If the zeroing trailer is not present, this checks that all fields have been
+            // mentioned exactly once. If the zeroing trailer is present, all missing fields will be
+            // zeroed, so this checks that all fields have been mentioned at most once. The use of
+            // struct initializer will still generate very natural error messages for any misuse.
+            ::core::ptr::write(slot, #path {
+                #(
+                    #(#field_attrs)*
+                    #field_name: loop {},
+                )*
+                #zeroing_trailer
+            })
+        };
+    }
+}
+
+impl Parse for Initializer {
+    fn parse(input: syn::parse::ParseStream<'_>) -> syn::Result<Self> {
+        let attrs = input.call(Attribute::parse_outer)?;
+        let this = input.peek(Token![&]).then(|| input.parse()).transpose()?;
+        let path = input.parse()?;
+        let content;
+        let brace_token = braced!(content in input);
+        let mut fields = Punctuated::new();
+        loop {
+            let lh = content.lookahead1();
+            if lh.peek(End) || lh.peek(Token![..]) {
+                break;
+            } else if lh.peek(Ident) || lh.peek(Token![_]) || lh.peek(Token![#]) {
+                fields.push_value(content.parse()?);
+                let lh = content.lookahead1();
+                if lh.peek(End) {
+                    break;
+                } else if lh.peek(Token![,]) {
+                    fields.push_punct(content.parse()?);
+                } else {
+                    return Err(lh.error());
+                }
+            } else {
+                return Err(lh.error());
+            }
+        }
+        let rest = content
+            .peek(Token![..])
+            .then(|| Ok::<_, syn::Error>((content.parse()?, content.parse()?)))
+            .transpose()?;
+        let error = input
+            .peek(Token![?])
+            .then(|| Ok::<_, syn::Error>((input.parse()?, input.parse()?)))
+            .transpose()?;
+        let attrs = attrs
+            .into_iter()
+            .map(|a| {
+                if a.path().is_ident("default_error") {
+                    a.parse_args::<DefaultErrorAttribute>()
+                        .map(InitializerAttribute::DefaultError)
+                } else {
+                    Err(syn::Error::new_spanned(a, "unknown initializer attribute"))
+                }
+            })
+            .collect::<Result<Vec<_>, _>>()?;
+        Ok(Self {
+            attrs,
+            this,
+            path,
+            brace_token,
+            fields,
+            rest,
+            error,
+        })
+    }
+}
+
+impl Parse for DefaultErrorAttribute {
+    fn parse(input: syn::parse::ParseStream<'_>) -> syn::Result<Self> {
+        Ok(Self { ty: input.parse()? })
+    }
+}
+
+impl Parse for This {
+    fn parse(input: syn::parse::ParseStream<'_>) -> syn::Result<Self> {
+        Ok(Self {
+            _and_token: input.parse()?,
+            ident: input.parse()?,
+            _in_token: input.parse()?,
+        })
+    }
+}
+
+impl Parse for InitializerField {
+    fn parse(input: syn::parse::ParseStream<'_>) -> syn::Result<Self> {
+        let attrs = input.call(Attribute::parse_outer)?;
+        Ok(Self {
+            attrs,
+            kind: input.parse()?,
+        })
+    }
+}
+
+impl Parse for InitializerKind {
+    fn parse(input: syn::parse::ParseStream<'_>) -> syn::Result<Self> {
+        let lh = input.lookahead1();
+        if lh.peek(Token![_]) {
+            Ok(Self::Code {
+                _underscore_token: input.parse()?,
+                _colon_token: input.parse()?,
+                block: input.parse()?,
+            })
+        } else if lh.peek(Ident) {
+            let ident = input.parse()?;
+            let lh = input.lookahead1();
+            if lh.peek(Token![<-]) {
+                Ok(Self::Init {
+                    ident,
+                    _left_arrow_token: input.parse()?,
+                    value: input.parse()?,
+                })
+            } else if lh.peek(Token![:]) {
+                Ok(Self::Value {
+                    ident,
+                    value: Some((input.parse()?, input.parse()?)),
+                })
+            } else if lh.peek(Token![,]) || lh.peek(End) {
+                Ok(Self::Value { ident, value: None })
+            } else {
+                Err(lh.error())
+            }
+        } else {
+            Err(lh.error())
+        }
+    }
+}
diff --git a/rust/pin-init/internal/src/lib.rs b/rust/pin-init/internal/src/lib.rs
new file mode 100644
index 000000000..60d5093f3
--- /dev/null
+++ b/rust/pin-init/internal/src/lib.rs
@@ -0,0 +1,59 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
+// When fixdep scans this, it will find this string `CONFIG_RUSTC_VERSION_TEXT`
+// and thus add a dependency on `include/config/RUSTC_VERSION_TEXT`, which is
+// touched by Kconfig when the version string from the compiler changes.
+
+//! `pin-init` proc macros.
+
+// Documentation is done in the pin-init crate instead.
+#![allow(missing_docs)]
+
+use proc_macro::TokenStream;
+use syn::parse_macro_input;
+
+use crate::diagnostics::DiagCtxt;
+
+mod diagnostics;
+mod init;
+mod pin_data;
+mod pinned_drop;
+mod zeroable;
+
+#[proc_macro_attribute]
+pub fn pin_data(args: TokenStream, input: TokenStream) -> TokenStream {
+    let args = parse_macro_input!(args);
+    let input = parse_macro_input!(input);
+    DiagCtxt::with(|dcx| pin_data::pin_data(args, input, dcx)).into()
+}
+
+#[proc_macro_attribute]
+pub fn pinned_drop(args: TokenStream, input: TokenStream) -> TokenStream {
+    let args = parse_macro_input!(args);
+    let input = parse_macro_input!(input);
+    DiagCtxt::with(|dcx| pinned_drop::pinned_drop(args, input, dcx)).into()
+}
+
+#[proc_macro_derive(Zeroable)]
+pub fn derive_zeroable(input: TokenStream) -> TokenStream {
+    let input = parse_macro_input!(input);
+    DiagCtxt::with(|dcx| zeroable::derive(input, dcx)).into()
+}
+
+#[proc_macro_derive(MaybeZeroable)]
+pub fn maybe_derive_zeroable(input: TokenStream) -> TokenStream {
+    let input = parse_macro_input!(input);
+    DiagCtxt::with(|dcx| zeroable::maybe_derive(input, dcx)).into()
+}
+#[proc_macro]
+pub fn init(input: TokenStream) -> TokenStream {
+    let input = parse_macro_input!(input);
+    DiagCtxt::with(|dcx| init::expand(input, Some("::core::convert::Infallible"), false, dcx))
+        .into()
+}
+
+#[proc_macro]
+pub fn pin_init(input: TokenStream) -> TokenStream {
+    let input = parse_macro_input!(input);
+    DiagCtxt::with(|dcx| init::expand(input, Some("::core::convert::Infallible"), true, dcx)).into()
+}
diff --git a/rust/pin-init/internal/src/pin_data.rs b/rust/pin-init/internal/src/pin_data.rs
new file mode 100644
index 000000000..ff194d275
--- /dev/null
+++ b/rust/pin-init/internal/src/pin_data.rs
@@ -0,0 +1,537 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
+use proc_macro2::TokenStream;
+use quote::{format_ident, quote, ToTokens};
+use syn::{
+    parse::{End, Nothing, Parse},
+    parse_quote, parse_quote_spanned,
+    spanned::Spanned,
+    visit_mut::VisitMut,
+    Field, Fields, Generics, Ident, Item, PathSegment, Type, TypePath, Visibility, WhereClause,
+};
+
+use crate::diagnostics::{DiagCtxt, ErrorGuaranteed};
+
+pub(crate) mod kw {
+    syn::custom_keyword!(PinnedDrop);
+}
+
+pub(crate) enum Args {
+    Nothing(Nothing),
+    #[allow(dead_code)]
+    PinnedDrop(kw::PinnedDrop),
+}
+
+impl Parse for Args {
+    fn parse(input: syn::parse::ParseStream<'_>) -> syn::Result<Self> {
+        let lh = input.lookahead1();
+        if lh.peek(End) {
+            input.parse().map(Self::Nothing)
+        } else if lh.peek(kw::PinnedDrop) {
+            input.parse().map(Self::PinnedDrop)
+        } else {
+            Err(lh.error())
+        }
+    }
+}
+
+impl ToTokens for Args {
+    fn to_tokens(&self, tokens: &mut TokenStream) {
+        match self {
+            Self::Nothing(_) => (),
+            Self::PinnedDrop(kw) => kw.to_tokens(tokens),
+        }
+    }
+}
+
+struct FieldInfo<'a> {
+    field: &'a Field,
+    pinned: bool,
+}
+
+pub(crate) fn pin_data(
+    args: Args,
+    input: Item,
+    dcx: &mut DiagCtxt,
+) -> Result<TokenStream, ErrorGuaranteed> {
+    let mut struct_ = match input {
+        Item::Struct(struct_) => struct_,
+        Item::Enum(enum_) => {
+            return Err(dcx.error(
+                enum_.enum_token,
+                "`#[pin_data]` only supports structs for now",
+            ));
+        }
+        Item::Union(union) => {
+            return Err(dcx.error(
+                union.union_token,
+                "`#[pin_data]` only supports structs for now",
+            ));
+        }
+        rest => {
+            return Err(dcx.error(
+                rest,
+                "`#[pin_data]` can only be applied to struct, enum and union definitions",
+            ));
+        }
+    };
+
+    // Handling cfg can gets very complicated, especially for tuple structs. Therefore, resolve all
+    // field cfgs first before continuing.
+    //
+    // We need to perform this after parsing so we can reliably detect field cfgs.
+    for (field_idx, field) in struct_.fields.iter_mut().enumerate() {
+        let cfg: Vec<_> = field
+            .attrs
+            .iter()
+            .filter(|a| a.path().is_ident("cfg"))
+            .map(|a| {
+                a.parse_args::<TokenStream>()
+                    .expect("parse as token stream cannot fail")
+            })
+            .collect();
+
+        if cfg.is_empty() {
+            continue;
+        }
+
+        field.attrs.retain(|a| !a.path().is_ident("cfg"));
+        let cfg_true_struct = quote!(#struct_);
+
+        let punctuated = match &mut struct_.fields {
+            Fields::Named(fields) => &mut fields.named,
+            Fields::Unnamed(fields) => &mut fields.unnamed,
+            Fields::Unit => unreachable!(),
+        };
+        *punctuated = std::mem::take(punctuated)
+            .into_pairs()
+            .enumerate()
+            .filter(|&(i, _)| i != field_idx)
+            .map(|(_, p)| p)
+            .collect();
+        let cfg_false_struct = quote!(#struct_);
+
+        // Resolve one field at a time until we've got no more field cfgs.
+        //
+        // This is linear time because macro invocations with false cfg will not be expanded.
+        return Ok(quote!(
+            #[cfg(all(#(#cfg,)*))]
+            #[::pin_init::pin_data(#args)]
+            #cfg_true_struct
+
+            #[cfg(not(all(#(#cfg,)*)))]
+            #[::pin_init::pin_data(#args)]
+            #cfg_false_struct
+        ));
+    }
+
+    // The generics might contain the `Self` type. Since this macro will define a new type with the
+    // same generics and bounds, this poses a problem: `Self` will refer to the new type as opposed
+    // to this struct definition. Therefore we have to replace `Self` with the concrete name.
+    let mut replacer = {
+        let name = &struct_.ident;
+        let (_, ty_generics, _) = struct_.generics.split_for_impl();
+        SelfReplacer(parse_quote!(#name #ty_generics))
+    };
+    replacer.visit_generics_mut(&mut struct_.generics);
+    replacer.visit_fields_mut(&mut struct_.fields);
+
+    let fields: Vec<FieldInfo<'_>> = struct_
+        .fields
+        .iter_mut()
+        .map(|field| {
+            let len = field.attrs.len();
+            field.attrs.retain(|a| !a.path().is_ident("pin"));
+            let pinned_count = len - field.attrs.len();
+            if pinned_count > 1 {
+                dcx.error(&field, "#[pin] attribute specified more than once");
+            }
+
+            assert!(
+                !field.attrs.iter().any(|a| a.path().is_ident("cfg")),
+                "cfgs should be all resolved at this point"
+            );
+
+            FieldInfo {
+                field: &*field,
+                pinned: pinned_count != 0,
+            }
+        })
+        .collect();
+
+    for field in &fields {
+        let ident = field.field.ident.as_ref().unwrap();
+
+        if !field.pinned && is_phantom_pinned(&field.field.ty) {
+            dcx.warn(
+                field.field,
+                format!(
+                    "The field `{ident}` of type `PhantomPinned` only has an effect \
+                    if it has the `#[pin]` attribute",
+                ),
+            );
+        }
+    }
+
+    let unpin_impl = generate_unpin_impl(&struct_.ident, &struct_.generics, &fields);
+    let drop_impl = generate_drop_impl(&struct_.ident, &struct_.generics, args);
+    let projections =
+        generate_projections(&struct_.vis, &struct_.ident, &struct_.generics, &fields);
+    let the_pin_data =
+        generate_the_pin_data(&struct_.vis, &struct_.ident, &struct_.generics, &fields);
+
+    Ok(quote! {
+        #struct_
+        #projections
+        // We put the rest into this const item, because it then will not be accessible to anything
+        // outside.
+        const _: () = {
+            #the_pin_data
+            #unpin_impl
+            #drop_impl
+        };
+    })
+}
+
+fn is_phantom_pinned(ty: &Type) -> bool {
+    match ty {
+        Type::Path(TypePath { qself: None, path }) => {
+            // Cannot possibly refer to `PhantomPinned` (except alias, but that's on the user).
+            if path.segments.len() > 3 {
+                return false;
+            }
+            // If there is a `::`, then the path needs to be `::core::marker::PhantomPinned` or
+            // `::std::marker::PhantomPinned`.
+            if path.leading_colon.is_some() && path.segments.len() != 3 {
+                return false;
+            }
+            let expected: Vec<&[&str]> = vec![&["PhantomPinned"], &["marker"], &["core", "std"]];
+            for (actual, expected) in path.segments.iter().rev().zip(expected) {
+                if !actual.arguments.is_empty() || expected.iter().all(|e| actual.ident != e) {
+                    return false;
+                }
+            }
+            true
+        }
+        _ => false,
+    }
+}
+
+fn generate_unpin_impl(
+    ident: &Ident,
+    generics: &Generics,
+    fields: &[FieldInfo<'_>],
+) -> TokenStream {
+    let (_, ty_generics, _) = generics.split_for_impl();
+    let mut generics_with_pin_lt = generics.clone();
+    generics_with_pin_lt.params.insert(0, parse_quote!('__pin));
+    generics_with_pin_lt.make_where_clause();
+    let (
+        impl_generics_with_pin_lt,
+        ty_generics_with_pin_lt,
+        Some(WhereClause {
+            where_token,
+            predicates,
+        }),
+    ) = generics_with_pin_lt.split_for_impl()
+    else {
+        unreachable!()
+    };
+    let pinned_fields = fields.iter().filter(|f| f.pinned).map(|f| {
+        let ident = f.field.ident.as_ref().unwrap();
+        let ty = &f.field.ty;
+        quote!(
+            #ident: #ty
+        )
+    });
+    quote! {
+        // This struct will be used for the unpin analysis. It is needed, because only structurally
+        // pinned fields are relevant whether the struct should implement `Unpin`.
+        #[allow(
+            dead_code, // The fields below are never used.
+            non_snake_case // The warning will be emitted on the struct definition.
+        )]
+        struct __Unpin #generics_with_pin_lt
+        #where_token
+            #predicates
+        {
+            __phantom_pin: ::pin_init::__internal::PhantomInvariantLifetime<'__pin>,
+            __phantom: ::pin_init::__internal::PhantomInvariant<#ident #ty_generics>,
+            #(#pinned_fields),*
+        }
+
+        #[doc(hidden)]
+        impl #impl_generics_with_pin_lt ::core::marker::Unpin for #ident #ty_generics
+        #where_token
+            __Unpin #ty_generics_with_pin_lt: ::core::marker::Unpin,
+            #predicates
+        {}
+    }
+}
+
+fn generate_drop_impl(ident: &Ident, generics: &Generics, args: Args) -> TokenStream {
+    let (impl_generics, ty_generics, whr) = generics.split_for_impl();
+    let has_pinned_drop = matches!(args, Args::PinnedDrop(_));
+    // We need to disallow normal `Drop` implementation, the exact behavior depends on whether
+    // `PinnedDrop` was specified in `args`.
+    if has_pinned_drop {
+        // When `PinnedDrop` was specified we just implement `Drop` and delegate.
+        quote! {
+            impl #impl_generics ::core::ops::Drop for #ident #ty_generics
+                #whr
+            {
+                fn drop(&mut self) {
+                    // SAFETY: Since this is a destructor, `self` will not move after this function
+                    // terminates, since it is inaccessible.
+                    let pinned = unsafe { ::core::pin::Pin::new_unchecked(self) };
+                    // SAFETY: Since this is a drop function, we can create this token to call the
+                    // pinned destructor of this type.
+                    let token = unsafe { ::pin_init::__internal::OnlyCallFromDrop::new() };
+                    ::pin_init::PinnedDrop::drop(pinned, token);
+                }
+            }
+        }
+    } else {
+        // When no `PinnedDrop` was specified, then we have to prevent implementing drop.
+        quote! {
+            // We prevent this by creating a trait that will be implemented for all types implementing
+            // `Drop`. Additionally we will implement this trait for the struct leading to a conflict,
+            // if it also implements `Drop`
+            trait MustNotImplDrop {}
+            impl<T: ::core::ops::Drop + ?::core::marker::Sized> MustNotImplDrop for T {}
+            impl #impl_generics MustNotImplDrop for #ident #ty_generics
+                #whr
+            {}
+            // We also take care to prevent users from writing a useless `PinnedDrop` implementation.
+            // They might implement `PinnedDrop` correctly for the struct, but forget to give
+            // `PinnedDrop` as the parameter to `#[pin_data]`.
+            trait UselessPinnedDropImpl_you_need_to_specify_PinnedDrop {}
+            impl<T: ::pin_init::PinnedDrop + ?::core::marker::Sized>
+                UselessPinnedDropImpl_you_need_to_specify_PinnedDrop for T {}
+            impl #impl_generics
+                UselessPinnedDropImpl_you_need_to_specify_PinnedDrop for #ident #ty_generics
+                #whr
+            {}
+        }
+    }
+}
+
+fn generate_projections(
+    vis: &Visibility,
+    ident: &Ident,
+    generics: &Generics,
+    fields: &[FieldInfo<'_>],
+) -> TokenStream {
+    let (impl_generics, ty_generics, _) = generics.split_for_impl();
+    let mut generics_with_pin_lt = generics.clone();
+    generics_with_pin_lt.params.insert(0, parse_quote!('__pin));
+    let (_, ty_generics_with_pin_lt, whr) = generics_with_pin_lt.split_for_impl();
+    let projection = format_ident!("{ident}Projection");
+    let this = format_ident!("this");
+
+    let (fields_decl, fields_proj): (Vec<_>, Vec<_>) = fields
+        .iter()
+        .map(|field| {
+            let Field { vis, ident, ty, .. } = &field.field;
+
+            let ident = ident
+                .as_ref()
+                .expect("only structs with named fields are supported");
+            if field.pinned {
+                (
+                    quote!(
+                        #vis #ident: ::core::pin::Pin<&'__pin mut #ty>,
+                    ),
+                    quote!(
+                        // SAFETY: this field is structurally pinned.
+                        #ident: unsafe { ::core::pin::Pin::new_unchecked(&mut #this.#ident) },
+                    ),
+                )
+            } else {
+                (
+                    quote!(
+                        #vis #ident: &'__pin mut #ty,
+                    ),
+                    quote!(
+                        #ident: &mut #this.#ident,
+                    ),
+                )
+            }
+        })
+        .collect();
+    let structurally_pinned_fields_docs = fields
+        .iter()
+        .filter(|f| f.pinned)
+        .map(|f| format!(" - `{}`", f.field.ident.as_ref().unwrap()));
+    let not_structurally_pinned_fields_docs = fields
+        .iter()
+        .filter(|f| !f.pinned)
+        .map(|f| format!(" - `{}`", f.field.ident.as_ref().unwrap()));
+    let docs = format!(" Pin-projections of [`{ident}`]");
+    quote! {
+        #[doc = #docs]
+        // Allow `non_snake_case` since the same warning will be emitted on
+        // the struct definition.
+        #[allow(dead_code, non_snake_case)]
+        #[doc(hidden)]
+        #vis struct #projection #generics_with_pin_lt
+            #whr
+        {
+            #(#fields_decl)*
+            ___pin_phantom_data: ::core::marker::PhantomData<&'__pin mut ()>,
+        }
+
+        impl #impl_generics #ident #ty_generics
+            #whr
+        {
+            /// Pin-projects all fields of `Self`.
+            ///
+            /// These fields are structurally pinned:
+            #(#[doc = #structurally_pinned_fields_docs])*
+            ///
+            /// These fields are **not** structurally pinned:
+            #(#[doc = #not_structurally_pinned_fields_docs])*
+            #[inline]
+            #vis fn project<'__pin>(
+                self: ::core::pin::Pin<&'__pin mut Self>,
+            ) -> #projection #ty_generics_with_pin_lt {
+                // SAFETY: we only give access to `&mut` for fields not structurally pinned.
+                let #this = unsafe { ::core::pin::Pin::get_unchecked_mut(self) };
+                #projection {
+                    #(#fields_proj)*
+                    ___pin_phantom_data: ::core::marker::PhantomData,
+                }
+            }
+        }
+    }
+}
+
+fn generate_the_pin_data(
+    vis: &Visibility,
+    struct_name: &Ident,
+    generics: &Generics,
+    fields: &[FieldInfo<'_>],
+) -> TokenStream {
+    let (impl_generics, ty_generics, whr) = generics.split_for_impl();
+
+    // For every field, we create an initializing projection function according to its projection
+    // type. If a field is structurally pinned, we create a `Slot` with `Pinned` which must be
+    // initialized via `PinInit`; if it is not structurally pinned, then we create a `Slot` with
+    // `Unpinned` which allows initialization via `Init`.
+    let field_accessors = fields
+        .iter()
+        .map(|f| {
+            let Field { vis, ident, ty, .. } = f.field;
+
+            let field_name = ident
+                .as_ref()
+                .expect("only structs with named fields are supported");
+            let pin_marker = if f.pinned {
+                quote!(Pinned)
+            } else {
+                quote!(Unpinned)
+            };
+            quote! {
+                /// # Safety
+                ///
+                /// - `slot` is valid and properly aligned.
+                /// - `(*slot).#field_name` is properly aligned.
+                /// - `(*slot).#field_name` points to uninitialized and exclusively accessed
+                ///   memory.
+                // Allow `non_snake_case` since the same warning will be emitted on
+                // the struct definition.
+                #[allow(non_snake_case)]
+                #[inline(always)]
+                #vis unsafe fn #field_name(
+                    self,
+                    slot: *mut #struct_name #ty_generics,
+                ) -> ::pin_init::__internal::Slot<::pin_init::__internal::#pin_marker, #ty> {
+                    // SAFETY:
+                    // - If `#pin_marker` is `Pinned`, the corresponding field is structurally
+                    //   pinned.
+                    // - Other safety requirements follows the safety requirement.
+                    unsafe { ::pin_init::__internal::Slot::new(&raw mut (*slot).#field_name) }
+                }
+            }
+        })
+        .collect::<TokenStream>();
+    quote! {
+        // We declare this struct which will host all of the projection function for our type. It
+        // will be invariant over all generic parameters which are inherited from the struct.
+        #[doc(hidden)]
+        #vis struct __ThePinData #generics
+            #whr
+        {
+            __phantom: ::pin_init::__internal::PhantomInvariant<#struct_name #ty_generics>,
+        }
+
+        impl #impl_generics ::core::clone::Clone for __ThePinData #ty_generics
+            #whr
+        {
+            #[inline]
+            fn clone(&self) -> Self { *self }
+        }
+
+        impl #impl_generics ::core::marker::Copy for __ThePinData #ty_generics
+            #whr
+        {}
+
+        #[allow(dead_code)] // Some functions might never be used and private.
+        impl #impl_generics __ThePinData #ty_generics
+            #whr
+        {
+            /// Type inference helper function.
+            #[inline(always)]
+            #vis fn __make_closure<__F, __E>(self, f: __F) -> __F
+            where
+                __F: FnOnce(*mut #struct_name #ty_generics) ->
+                    ::core::result::Result<::pin_init::__internal::InitOk, __E>,
+            {
+                f
+            }
+
+            #field_accessors
+        }
+
+        // SAFETY: We have added the correct projection functions above to `__ThePinData` and
+        // we also use the least restrictive generics possible.
+        unsafe impl #impl_generics ::pin_init::__internal::HasPinData for #struct_name #ty_generics
+            #whr
+        {
+            type PinData = __ThePinData #ty_generics;
+
+            #[inline]
+            unsafe fn __pin_data() -> Self::PinData {
+                __ThePinData { __phantom: ::pin_init::__internal::PhantomInvariant::new() }
+            }
+        }
+    }
+}
+
+struct SelfReplacer(PathSegment);
+
+impl VisitMut for SelfReplacer {
+    fn visit_path_mut(&mut self, i: &mut syn::Path) {
+        if i.is_ident("Self") {
+            let span = i.span();
+            let seg = &self.0;
+            *i = parse_quote_spanned!(span=> #seg);
+        } else {
+            syn::visit_mut::visit_path_mut(self, i);
+        }
+    }
+
+    fn visit_path_segment_mut(&mut self, seg: &mut PathSegment) {
+        if seg.ident == "Self" {
+            let span = seg.span();
+            let this = &self.0;
+            *seg = parse_quote_spanned!(span=> #this);
+        } else {
+            syn::visit_mut::visit_path_segment_mut(self, seg);
+        }
+    }
+
+    fn visit_item_mut(&mut self, _: &mut Item) {
+        // Do not descend into items, since items reset/change what `Self` refers to.
+    }
+}
diff --git a/rust/pin-init/internal/src/pinned_drop.rs b/rust/pin-init/internal/src/pinned_drop.rs
new file mode 100644
index 000000000..a20ac314c
--- /dev/null
+++ b/rust/pin-init/internal/src/pinned_drop.rs
@@ -0,0 +1,61 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
+use proc_macro2::TokenStream;
+use quote::quote;
+use syn::{parse::Nothing, parse_quote, spanned::Spanned, ImplItem, ItemImpl, Token};
+
+use crate::diagnostics::{DiagCtxt, ErrorGuaranteed};
+
+pub(crate) fn pinned_drop(
+    _args: Nothing,
+    mut input: ItemImpl,
+    dcx: &mut DiagCtxt,
+) -> Result<TokenStream, ErrorGuaranteed> {
+    if let Some(unsafety) = input.unsafety {
+        dcx.error(unsafety, "implementing `PinnedDrop` is safe");
+    }
+    input.unsafety = Some(Token![unsafe](input.impl_token.span));
+    match &mut input.trait_ {
+        Some((not, path, _for)) => {
+            if let Some(not) = not {
+                dcx.error(not, "cannot implement `!PinnedDrop`");
+            }
+            for (seg, expected) in path
+                .segments
+                .iter()
+                .rev()
+                .zip(["PinnedDrop", "pin_init", ""])
+            {
+                if expected.is_empty() || seg.ident != expected {
+                    dcx.error(seg, "bad import path for `PinnedDrop`");
+                }
+                if !seg.arguments.is_none() {
+                    dcx.error(&seg.arguments, "unexpected arguments for `PinnedDrop` path");
+                }
+            }
+            *path = parse_quote!(::pin_init::PinnedDrop);
+        }
+        None => {
+            let span = input
+                .impl_token
+                .span
+                .join(input.self_ty.span())
+                .unwrap_or(input.impl_token.span);
+            dcx.error(
+                span,
+                "expected `impl ... PinnedDrop for ...`, got inherent impl",
+            );
+        }
+    }
+    for item in &mut input.items {
+        if let ImplItem::Fn(fn_item) = item {
+            if fn_item.sig.ident == "drop" {
+                fn_item
+                    .sig
+                    .inputs
+                    .push(parse_quote!(_: ::pin_init::__internal::OnlyCallFromDrop));
+            }
+        }
+    }
+    Ok(quote!(#input))
+}
diff --git a/rust/pin-init/internal/src/zeroable.rs b/rust/pin-init/internal/src/zeroable.rs
new file mode 100644
index 000000000..b11feaeb1
--- /dev/null
+++ b/rust/pin-init/internal/src/zeroable.rs
@@ -0,0 +1,78 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
+use proc_macro2::TokenStream;
+use quote::quote;
+use syn::{parse_quote, Data, DeriveInput, Field, Fields};
+
+use crate::{diagnostics::ErrorGuaranteed, DiagCtxt};
+
+pub(crate) fn derive(
+    input: DeriveInput,
+    dcx: &mut DiagCtxt,
+) -> Result<TokenStream, ErrorGuaranteed> {
+    let fields = match input.data {
+        Data::Struct(data_struct) => data_struct.fields,
+        Data::Union(data_union) => Fields::Named(data_union.fields),
+        Data::Enum(data_enum) => {
+            return Err(dcx.error(data_enum.enum_token, "cannot derive `Zeroable` for an enum"));
+        }
+    };
+    let name = input.ident;
+    let mut generics = input.generics;
+    for param in generics.type_params_mut() {
+        param.bounds.insert(0, parse_quote!(::pin_init::Zeroable));
+    }
+    let (impl_gen, ty_gen, whr) = generics.split_for_impl();
+    let field_type = fields.iter().map(|field| &field.ty);
+    Ok(quote! {
+        // SAFETY: Every field type implements `Zeroable` and padding bytes may be zero.
+        #[automatically_derived]
+        unsafe impl #impl_gen ::pin_init::Zeroable for #name #ty_gen
+            #whr
+        {}
+        const _: () = {
+            fn assert_zeroable<T: ?::core::marker::Sized + ::pin_init::Zeroable>() {}
+            fn ensure_zeroable #impl_gen ()
+                #whr
+            {
+                #(
+                    assert_zeroable::<#field_type>();
+                )*
+            }
+        };
+    })
+}
+
+pub(crate) fn maybe_derive(
+    input: DeriveInput,
+    dcx: &mut DiagCtxt,
+) -> Result<TokenStream, ErrorGuaranteed> {
+    let fields = match input.data {
+        Data::Struct(data_struct) => data_struct.fields,
+        Data::Union(data_union) => Fields::Named(data_union.fields),
+        Data::Enum(data_enum) => {
+            return Err(dcx.error(data_enum.enum_token, "cannot derive `Zeroable` for an enum"));
+        }
+    };
+    let name = input.ident;
+    let mut generics = input.generics;
+    for param in generics.type_params_mut() {
+        param.bounds.insert(0, parse_quote!(::pin_init::Zeroable));
+    }
+    for Field { ty, .. } in fields {
+        generics
+            .make_where_clause()
+            .predicates
+            // the `for<'__dummy>` HRTB makes this not error without the `trivial_bounds`
+            // feature <https://github.com/rust-lang/rust/issues/48214#issuecomment-2557829956>.
+            .push(parse_quote!(#ty: for<'__dummy> ::pin_init::Zeroable));
+    }
+    let (impl_gen, ty_gen, whr) = generics.split_for_impl();
+    Ok(quote! {
+        // SAFETY: Every field type implements `Zeroable` and padding bytes may be zero.
+        #[automatically_derived]
+        unsafe impl #impl_gen ::pin_init::Zeroable for #name #ty_gen
+            #whr
+        {}
+    })
+}
diff --git a/rust/pin-init/src/__internal.rs b/rust/pin-init/src/__internal.rs
new file mode 100644
index 000000000..8e9fd18b9
--- /dev/null
+++ b/rust/pin-init/src/__internal.rs
@@ -0,0 +1,409 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
+//! This module contains library internal items.
+//!
+//! These items must not be used outside of this crate and the pin-init-internal crate located at
+//! `../internal`.
+
+use super::*;
+
+/// Zero-sized type used to mark a type as invariant.
+///
+/// This is a polyfill for the [unstable type] in the standard library of the same name.
+///
+/// See the [nomicon] for what subtyping is. See also [this table].
+///
+/// [unstable type]: https://doc.rust-lang.org/nightly/std/marker/struct.PhantomInvariant.html
+/// [nomicon]: https://doc.rust-lang.org/nomicon/subtyping.html
+/// [this table]: https://doc.rust-lang.org/nomicon/phantom-data.html#table-of-phantomdata-patterns
+#[repr(transparent)]
+pub struct PhantomInvariant<T: ?Sized>(PhantomData<fn(T) -> T>);
+
+impl<T: ?Sized> Clone for PhantomInvariant<T> {
+    #[inline(always)]
+    fn clone(&self) -> Self {
+        *self
+    }
+}
+
+impl<T: ?Sized> Copy for PhantomInvariant<T> {}
+
+impl<T: ?Sized> Default for PhantomInvariant<T> {
+    #[inline(always)]
+    fn default() -> Self {
+        Self::new()
+    }
+}
+
+impl<T: ?Sized> PhantomInvariant<T> {
+    #[inline(always)]
+    pub const fn new() -> Self {
+        Self(PhantomData)
+    }
+}
+
+/// Zero-sized type used to mark a lifetime as invariant.
+///
+/// This is a polyfill for the [unstable type] in the standard library of the same name.
+///
+/// [unstable type]: https://doc.rust-lang.org/nightly/std/marker/struct.PhantomInvariantLifetime.html
+#[repr(transparent)]
+#[derive(Clone, Copy, Default)]
+pub struct PhantomInvariantLifetime<'a>(PhantomInvariant<&'a ()>);
+
+impl PhantomInvariantLifetime<'_> {
+    #[inline(always)]
+    pub const fn new() -> Self {
+        Self(PhantomInvariant::new())
+    }
+}
+
+/// Token type to signify successful initialization.
+///
+/// Can only be constructed via the unsafe [`Self::new`] function. The initializer macros use this
+/// token type to prevent returning `Ok` from an initializer without initializing all fields.
+pub struct InitOk(());
+
+impl InitOk {
+    /// Creates a new token.
+    ///
+    /// # Safety
+    ///
+    /// This function may only be called from the `init!` macro in `../internal/src/init.rs`.
+    #[inline(always)]
+    pub unsafe fn new() -> Self {
+        Self(())
+    }
+}
+
+/// This trait is only implemented via the `#[pin_data]` proc-macro. It is used to facilitate
+/// the pin projections within the initializers.
+///
+/// # Safety
+///
+/// Only the `init` module is allowed to use this trait.
+pub unsafe trait HasPinData {
+    type PinData;
+
+    #[expect(clippy::missing_safety_doc)]
+    unsafe fn __pin_data() -> Self::PinData;
+}
+
+/// This trait is automatically implemented for every type. It aims to provide the same type
+/// inference help as `HasPinData`.
+///
+/// # Safety
+///
+/// Only the `init` module is allowed to use this trait.
+pub unsafe trait HasInitData {
+    type InitData;
+
+    #[expect(clippy::missing_safety_doc)]
+    unsafe fn __init_data() -> Self::InitData;
+}
+
+pub struct AllData<T: ?Sized>(PhantomInvariant<T>);
+
+impl<T: ?Sized> Clone for AllData<T> {
+    #[inline]
+    fn clone(&self) -> Self {
+        *self
+    }
+}
+
+impl<T: ?Sized> Copy for AllData<T> {}
+
+impl<T: ?Sized> AllData<T> {
+    /// Type inference helper function.
+    #[inline(always)]
+    pub fn __make_closure<F, E>(self, f: F) -> F
+    where
+        F: FnOnce(*mut T) -> Result<InitOk, E>,
+    {
+        f
+    }
+}
+
+// SAFETY: TODO.
+unsafe impl<T: ?Sized> HasInitData for T {
+    type InitData = AllData<T>;
+
+    #[inline]
+    unsafe fn __init_data() -> Self::InitData {
+        AllData(PhantomInvariant::new())
+    }
+}
+
+/// Stack initializer helper type. Use [`stack_pin_init`] instead of this primitive.
+///
+/// # Invariants
+///
+/// If `self.is_init` is true, then `self.value` is initialized.
+///
+/// [`stack_pin_init`]: crate::stack_pin_init
+pub struct StackInit<T> {
+    value: MaybeUninit<T>,
+    is_init: bool,
+}
+
+impl<T> Drop for StackInit<T> {
+    #[inline]
+    fn drop(&mut self) {
+        if self.is_init {
+            // SAFETY: As we are being dropped, we only call this once. And since `self.is_init` is
+            // true, `self.value` is initialized.
+            unsafe { self.value.assume_init_drop() };
+        }
+    }
+}
+
+impl<T> StackInit<T> {
+    /// Creates a new [`StackInit<T>`] that is uninitialized. Use [`stack_pin_init`] instead of this
+    /// primitive.
+    ///
+    /// [`stack_pin_init`]: crate::stack_pin_init
+    #[inline]
+    pub fn uninit() -> Self {
+        Self {
+            value: MaybeUninit::uninit(),
+            is_init: false,
+        }
+    }
+
+    /// Initializes the contents and returns the result.
+    #[inline]
+    pub fn init<E>(self: Pin<&mut Self>, init: impl PinInit<T, E>) -> Result<Pin<&mut T>, E> {
+        // SAFETY: We never move out of `this`.
+        let this = unsafe { Pin::into_inner_unchecked(self) };
+        // The value is currently initialized, so it needs to be dropped before we can reuse
+        // the memory (this is a safety guarantee of `Pin`).
+        if this.is_init {
+            this.is_init = false;
+            // SAFETY: `this.is_init` was true and therefore `this.value` is initialized.
+            unsafe { this.value.assume_init_drop() };
+        }
+        // SAFETY: The memory slot is valid and this type ensures that it will stay pinned.
+        unsafe { init.__init(this.value.as_mut_ptr())? };
+        // INVARIANT: `this.value` is initialized above.
+        this.is_init = true;
+        // SAFETY: The slot is now pinned, since we will never give access to `&mut T`.
+        Ok(unsafe { Pin::new_unchecked(this.value.assume_init_mut()) })
+    }
+}
+
+#[test]
+#[cfg(feature = "std")]
+fn stack_init_reuse() {
+    use ::std::{borrow::ToOwned, println, string::String};
+    use core::pin::pin;
+
+    #[derive(Debug)]
+    struct Foo {
+        a: usize,
+        b: String,
+    }
+    let mut slot: Pin<&mut StackInit<Foo>> = pin!(StackInit::uninit());
+    let value: Result<Pin<&mut Foo>, core::convert::Infallible> =
+        slot.as_mut().init(crate::init!(Foo {
+            a: 42,
+            b: "Hello".to_owned(),
+        }));
+    let value = value.unwrap();
+    println!("{value:?}");
+    let value: Result<Pin<&mut Foo>, core::convert::Infallible> =
+        slot.as_mut().init(crate::init!(Foo {
+            a: 24,
+            b: "world!".to_owned(),
+        }));
+    let value = value.unwrap();
+    println!("{value:?}");
+}
+
+// Marker types that determines type of `DropGuard`'s let bindings.
+pub struct Pinned;
+pub struct Unpinned;
+
+/// Represent an uninitialized field.
+///
+/// # Invariants
+///
+/// - `ptr` is valid, properly aligned and points to uninitialized and exclusively accessed memory.
+/// - If `P` is `Pinned`, then `ptr` is structurally pinned.
+pub struct Slot<P, T: ?Sized> {
+    ptr: *mut T,
+    _phantom: PhantomData<P>,
+}
+
+impl<P, T: ?Sized> Slot<P, T> {
+    /// # Safety
+    ///
+    /// - `ptr` is valid, properly aligned and points to uninitialized and exclusively accessed
+    ///   memory.
+    /// - If `P` is `Pinned`, then `ptr` is structurally pinned.
+    #[inline(always)]
+    pub unsafe fn new(ptr: *mut T) -> Self {
+        // INVARIANT: Per safety requirement.
+        Self {
+            ptr,
+            _phantom: PhantomData,
+        }
+    }
+
+    /// Initialize the field by value.
+    #[inline(always)]
+    pub fn write(self, value: T) -> DropGuard<P, T>
+    where
+        T: Sized,
+    {
+        // SAFETY: `self.ptr` is a valid and aligned pointer for write.
+        unsafe { self.ptr.write(value) }
+        // SAFETY:
+        // - `self.ptr` is valid and properly aligned per type invariant.
+        // - `*self.ptr` is initialized above and the ownership is transferred to the guard.
+        // - If `P` is `Pinned`, `self.ptr` is pinned.
+        unsafe { DropGuard::new(self.ptr) }
+    }
+}
+
+impl<T: ?Sized> Slot<Unpinned, T> {
+    /// Initialize the field.
+    #[inline(always)]
+    pub fn init<E>(self, init: impl Init<T, E>) -> Result<DropGuard<Unpinned, T>, E> {
+        // SAFETY:
+        // - `self.ptr` is valid and properly aligned.
+        // - when `Err` is returned, we also propagate the error without touching `slot`;
+        //   also `self` is consumed so it cannot be touched further.
+        unsafe { init.__init(self.ptr)? };
+
+        // SAFETY:
+        // - `self.ptr` is valid and properly aligned per type invariant.
+        // - `*self.ptr` is initialized above and the ownership is transferred to the guard.
+        Ok(unsafe { DropGuard::new(self.ptr) })
+    }
+}
+
+impl<T: ?Sized> Slot<Pinned, T> {
+    /// Initialize the field.
+    #[inline(always)]
+    pub fn init<E>(self, init: impl PinInit<T, E>) -> Result<DropGuard<Pinned, T>, E> {
+        // SAFETY:
+        // - `self.ptr` is valid and properly aligned.
+        // - when `Err` is returned, we also propagate the error without touching `ptr`;
+        //   also `self` is consumed so it cannot be touched further.
+        // - the drop guard will not hand out `&mut` (only `Pin<&mut T>`).
+        unsafe { init.__init(self.ptr)? };
+
+        // SAFETY:
+        // - `self.ptr` is valid, properly aligned and pinned per type invariant.
+        // - `*self.ptr` is initialized above and the ownership is transferred to the guard.
+        Ok(unsafe { DropGuard::new(self.ptr) })
+    }
+}
+
+/// When a value of this type is dropped, it drops a `T`.
+///
+/// Can be forgotten to prevent the drop.
+///
+/// # Invariants
+///
+/// - `ptr` is valid and properly aligned.
+/// - `*ptr` is initialized and owned by this guard.
+/// - if `P` is `Pinned`, `ptr` is pinned.
+pub struct DropGuard<P, T: ?Sized> {
+    ptr: *mut T,
+    phantom: PhantomData<P>,
+}
+
+impl<P, T: ?Sized> DropGuard<P, T> {
+    /// Creates a drop guard and transfer the ownership of the pointer content.
+    ///
+    /// The ownership is only relinguished if the guard is forgotten via [`core::mem::forget`].
+    ///
+    /// # Safety
+    ///
+    /// - `ptr` is valid and properly aligned.
+    /// - `*ptr` is initialized, and the ownership is transferred to this guard.
+    /// - if `P` is `Pinned`, `ptr` is pinned.
+    #[inline]
+    pub unsafe fn new(ptr: *mut T) -> Self {
+        // INVARIANT: By safety requirement.
+        Self {
+            ptr,
+            phantom: PhantomData,
+        }
+    }
+}
+
+impl<T: ?Sized> DropGuard<Unpinned, T> {
+    /// Create a let binding for accessor use.
+    #[inline]
+    pub fn let_binding(&mut self) -> &mut T {
+        // SAFETY: Per type invariant.
+        unsafe { &mut *self.ptr }
+    }
+}
+
+impl<T: ?Sized> DropGuard<Pinned, T> {
+    /// Create a let binding for accessor use.
+    #[inline]
+    pub fn let_binding(&mut self) -> Pin<&mut T> {
+        // SAFETY: `self.ptr` is valid, properly aligned, initialized, exclusively accessible and
+        // pinned per type invariant.
+        unsafe { Pin::new_unchecked(&mut *self.ptr) }
+    }
+}
+
+impl<P, T: ?Sized> Drop for DropGuard<P, T> {
+    #[inline]
+    fn drop(&mut self) {
+        // SAFETY: `self.ptr` is valid, properly aligned and `*self.ptr` is owned by this guard.
+        unsafe { ptr::drop_in_place(self.ptr) }
+    }
+}
+
+/// Token used by `PinnedDrop` to prevent calling the function without creating this unsafely
+/// created struct. This is needed, because the `drop` function is safe, but should not be called
+/// manually.
+pub struct OnlyCallFromDrop(());
+
+impl OnlyCallFromDrop {
+    /// # Safety
+    ///
+    /// This function should only be called from the [`Drop::drop`] function and only be used to
+    /// delegate the destruction to the pinned destructor [`PinnedDrop::drop`] of the same type.
+    pub unsafe fn new() -> Self {
+        Self(())
+    }
+}
+
+/// Initializer that always fails.
+///
+/// Used by [`assert_pinned!`].
+///
+/// [`assert_pinned!`]: crate::assert_pinned
+pub struct AlwaysFail<T: ?Sized> {
+    _t: PhantomData<T>,
+}
+
+impl<T: ?Sized> AlwaysFail<T> {
+    /// Creates a new initializer that always fails.
+    #[inline]
+    pub fn new() -> Self {
+        Self { _t: PhantomData }
+    }
+}
+
+impl<T: ?Sized> Default for AlwaysFail<T> {
+    #[inline]
+    fn default() -> Self {
+        Self::new()
+    }
+}
+
+// SAFETY: `__init` always fails, which is always okay.
+unsafe impl<T: ?Sized> PinInit<T, ()> for AlwaysFail<T> {
+    #[inline]
+    unsafe fn __init(self, _slot: *mut T) -> Result<(), ()> {
+        Err(())
+    }
+}
diff --git a/rust/pin-init/src/alloc.rs b/rust/pin-init/src/alloc.rs
new file mode 100644
index 000000000..471652e86
--- /dev/null
+++ b/rust/pin-init/src/alloc.rs
@@ -0,0 +1,160 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
+#[cfg(all(feature = "alloc", not(feature = "std")))]
+use alloc::{boxed::Box, sync::Arc};
+#[cfg(feature = "alloc")]
+use core::alloc::AllocError;
+use core::{mem::MaybeUninit, pin::Pin};
+#[cfg(feature = "std")]
+use std::sync::Arc;
+
+#[cfg(not(feature = "alloc"))]
+type AllocError = core::convert::Infallible;
+
+use crate::{
+    init_from_closure, pin_init_from_closure, InPlaceWrite, Init, PinInit, ZeroableOption,
+};
+
+pub extern crate alloc;
+
+// SAFETY: All zeros is equivalent to `None` (option layout optimization guarantee:
+// <https://doc.rust-lang.org/stable/std/option/index.html#representation>).
+unsafe impl<T> ZeroableOption for Box<T> {}
+
+/// Smart pointer that can initialize memory in-place.
+pub trait InPlaceInit<T>: Sized {
+    /// Use the given pin-initializer to pin-initialize a `T` inside of a new smart pointer of this
+    /// type.
+    ///
+    /// If `T: !Unpin` it will not be able to move afterwards.
+    fn try_pin_init<E>(init: impl PinInit<T, E>) -> Result<Pin<Self>, E>
+    where
+        E: From<AllocError>;
+
+    /// Use the given pin-initializer to pin-initialize a `T` inside of a new smart pointer of this
+    /// type.
+    ///
+    /// If `T: !Unpin` it will not be able to move afterwards.
+    #[inline]
+    fn pin_init(init: impl PinInit<T>) -> Result<Pin<Self>, AllocError> {
+        // SAFETY: We delegate to `init` and only change the error type.
+        let init = unsafe {
+            pin_init_from_closure(|slot| match init.__init(slot) {
+                Ok(()) => Ok(()),
+                Err(i) => match i {},
+            })
+        };
+        Self::try_pin_init(init)
+    }
+
+    /// Use the given initializer to in-place initialize a `T`.
+    fn try_init<E>(init: impl Init<T, E>) -> Result<Self, E>
+    where
+        E: From<AllocError>;
+
+    /// Use the given initializer to in-place initialize a `T`.
+    #[inline]
+    fn init(init: impl Init<T>) -> Result<Self, AllocError> {
+        // SAFETY: We delegate to `init` and only change the error type.
+        let init = unsafe {
+            init_from_closure(|slot| match init.__init(slot) {
+                Ok(()) => Ok(()),
+                Err(i) => match i {},
+            })
+        };
+        Self::try_init(init)
+    }
+}
+
+#[cfg(feature = "alloc")]
+macro_rules! try_new_uninit {
+    ($type:ident) => {
+        $type::try_new_uninit()?
+    };
+}
+#[cfg(all(feature = "std", not(feature = "alloc")))]
+macro_rules! try_new_uninit {
+    ($type:ident) => {
+        $type::new_uninit()
+    };
+}
+
+impl<T> InPlaceInit<T> for Box<T> {
+    #[inline]
+    fn try_pin_init<E>(init: impl PinInit<T, E>) -> Result<Pin<Self>, E>
+    where
+        E: From<AllocError>,
+    {
+        try_new_uninit!(Box).write_pin_init(init)
+    }
+
+    #[inline]
+    fn try_init<E>(init: impl Init<T, E>) -> Result<Self, E>
+    where
+        E: From<AllocError>,
+    {
+        try_new_uninit!(Box).write_init(init)
+    }
+}
+
+impl<T> InPlaceInit<T> for Arc<T> {
+    #[inline]
+    fn try_pin_init<E>(init: impl PinInit<T, E>) -> Result<Pin<Self>, E>
+    where
+        E: From<AllocError>,
+    {
+        let mut this = try_new_uninit!(Arc);
+        let Some(slot) = Arc::get_mut(&mut this) else {
+            // SAFETY: the Arc has just been created and has no external references
+            unsafe { core::hint::unreachable_unchecked() }
+        };
+        let slot = slot.as_mut_ptr();
+        // SAFETY: When init errors/panics, slot will get deallocated but not dropped,
+        // slot is valid and will not be moved, because we pin it later.
+        unsafe { init.__init(slot)? };
+        // SAFETY: All fields have been initialized and this is the only `Arc` to that data.
+        Ok(unsafe { Pin::new_unchecked(this.assume_init()) })
+    }
+
+    #[inline]
+    fn try_init<E>(init: impl Init<T, E>) -> Result<Self, E>
+    where
+        E: From<AllocError>,
+    {
+        let mut this = try_new_uninit!(Arc);
+        let Some(slot) = Arc::get_mut(&mut this) else {
+            // SAFETY: the Arc has just been created and has no external references
+            unsafe { core::hint::unreachable_unchecked() }
+        };
+        let slot = slot.as_mut_ptr();
+        // SAFETY: When init errors/panics, slot will get deallocated but not dropped,
+        // slot is valid.
+        unsafe { init.__init(slot)? };
+        // SAFETY: All fields have been initialized.
+        Ok(unsafe { this.assume_init() })
+    }
+}
+
+impl<T> InPlaceWrite<T> for Box<MaybeUninit<T>> {
+    type Initialized = Box<T>;
+
+    #[inline]
+    fn write_init<E>(mut self, init: impl Init<T, E>) -> Result<Self::Initialized, E> {
+        let slot = self.as_mut_ptr();
+        // SAFETY: When init errors/panics, slot will get deallocated but not dropped,
+        // slot is valid.
+        unsafe { init.__init(slot)? };
+        // SAFETY: All fields have been initialized.
+        Ok(unsafe { self.assume_init() })
+    }
+
+    #[inline]
+    fn write_pin_init<E>(mut self, init: impl PinInit<T, E>) -> Result<Pin<Self::Initialized>, E> {
+        let slot = self.as_mut_ptr();
+        // SAFETY: When init errors/panics, slot will get deallocated but not dropped,
+        // slot is valid and will not be moved, because we pin it later.
+        unsafe { init.__init(slot)? };
+        // SAFETY: All fields have been initialized.
+        Ok(unsafe { self.assume_init() }.into())
+    }
+}
diff --git a/rust/pin-init/src/lib.rs b/rust/pin-init/src/lib.rs
new file mode 100644
index 000000000..7600cdbbb
--- /dev/null
+++ b/rust/pin-init/src/lib.rs
@@ -0,0 +1,1803 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
+//! Library to safely and fallibly initialize pinned `struct`s using in-place constructors.
+//!
+//! [Pinning][pinning] is Rust's way of ensuring data does not move.
+//!
+//! It also allows in-place initialization of big `struct`s that would otherwise produce a stack
+//! overflow.
+//!
+//! This library's main use-case is in [Rust-for-Linux]. Although this version can be used
+//! standalone.
+//!
+//! There are cases when you want to in-place initialize a struct. For example when it is very big
+//! and moving it from the stack is not an option, because it is bigger than the stack itself.
+//! Another reason would be that you need the address of the object to initialize it. This stands
+//! in direct conflict with Rust's normal process of first initializing an object and then moving
+//! it into it's final memory location. For more information, see
+//! <https://rust-for-linux.com/the-safe-pinned-initialization-problem>.
+//!
+//! This library allows you to do in-place initialization safely.
+//!
+//! ## Nightly Needed for `alloc` feature
+//!
+//! This library requires the [`allocator_api` unstable feature] when the `alloc` feature is
+//! enabled and thus this feature can only be used with a nightly compiler. When enabling the
+//! `alloc` feature, the user will be required to activate `allocator_api` as well.
+//!
+//! [`allocator_api` unstable feature]: https://doc.rust-lang.org/nightly/unstable-book/library-features/allocator-api.html
+//!
+//! The feature is enabled by default, thus by default `pin-init` will require a nightly compiler.
+//! However, using the crate on stable compilers is possible by disabling `alloc`. In practice this
+//! will require the `std` feature, because stable compilers have neither `Box` nor `Arc` in no-std
+//! mode.
+//!
+//! ## Nightly needed for `unsafe-pinned` feature
+//!
+//! This feature enables the `Wrapper` implementation on the unstable `core::pin::UnsafePinned` type.
+//! This requires the [`unsafe_pinned` unstable feature](https://github.com/rust-lang/rust/issues/125735)
+//! and therefore a nightly compiler. Note that this feature is not enabled by default.
+//!
+//! # Overview
+//!
+//! To initialize a `struct` with an in-place constructor you will need two things:
+//! - an in-place constructor,
+//! - a memory location that can hold your `struct` (this can be the [stack], an [`Arc<T>`],
+//!   [`Box<T>`] or any other smart pointer that supports this library).
+//!
+//! To get an in-place constructor there are generally three options:
+//! - directly creating an in-place constructor using the [`pin_init!`] macro,
+//! - a custom function/macro returning an in-place constructor provided by someone else,
+//! - using the unsafe function [`pin_init_from_closure()`] to manually create an initializer.
+//!
+//! Aside from pinned initialization, this library also supports in-place construction without
+//! pinning, the macros/types/functions are generally named like the pinned variants without the
+//! `pin_` prefix.
+//!
+//! # Examples
+//!
+//! Throughout the examples we will often make use of the `CMutex` type which can be found in
+//! `../examples/mutex.rs`. It is essentially a userland rebuild of the `struct mutex` type from
+//! the Linux kernel. It also uses a wait list and a basic spinlock. Importantly the wait list
+//! requires it to be pinned to be locked and thus is a prime candidate for using this library.
+//!
+//! ## Using the [`pin_init!`] macro
+//!
+//! If you want to use [`PinInit`], then you will have to annotate your `struct` with
+//! `#[`[`pin_data`]`]`. It is a macro that uses `#[pin]` as a marker for
+//! [structurally pinned fields]. After doing this, you can then create an in-place constructor via
+//! [`pin_init!`]. The syntax is almost the same as normal `struct` initializers. The difference is
+//! that you need to write `<-` instead of `:` for fields that you want to initialize in-place.
+//!
+//! ```rust
+//! # #![feature(allocator_api)]
+//! # #[path = "../examples/mutex.rs"] mod mutex; use mutex::*;
+//! # use core::pin::Pin;
+//! use pin_init::{pin_data, pin_init, InPlaceInit};
+//!
+//! #[pin_data]
+//! struct Foo {
+//!     #[pin]
+//!     a: CMutex<usize>,
+//!     b: u32,
+//! }
+//!
+//! let foo = pin_init!(Foo {
+//!     a <- CMutex::new(42),
+//!     b: 24,
+//! });
+//! # let _ = Box::pin_init(foo);
+//! ```
+//!
+//! `foo` now is of the type [`impl PinInit<Foo>`]. We can now use any smart pointer that we like
+//! (or just the stack) to actually initialize a `Foo`:
+//!
+//! ```rust
+//! # #![feature(allocator_api)]
+//! # #[path = "../examples/mutex.rs"] mod mutex; use mutex::*;
+//! # use core::{alloc::AllocError, pin::Pin};
+//! # use pin_init::*;
+//! #
+//! # #[pin_data]
+//! # struct Foo {
+//! #     #[pin]
+//! #     a: CMutex<usize>,
+//! #     b: u32,
+//! # }
+//! #
+//! # let foo = pin_init!(Foo {
+//! #     a <- CMutex::new(42),
+//! #     b: 24,
+//! # });
+//! let foo: Result<Pin<Box<Foo>>, AllocError> = Box::pin_init(foo);
+//! ```
+//!
+//! For more information see the [`pin_init!`] macro.
+//!
+//! ## Using a custom function/macro that returns an initializer
+//!
+//! Many types that use this library supply a function/macro that returns an initializer, because
+//! the above method only works for types where you can access the fields.
+//!
+//! ```rust
+//! # #![feature(allocator_api)]
+//! # #[path = "../examples/mutex.rs"] mod mutex; use mutex::*;
+//! # use pin_init::*;
+//! # use std::sync::Arc;
+//! # use core::pin::Pin;
+//! let mtx: Result<Pin<Arc<CMutex<usize>>>, _> = Arc::pin_init(CMutex::new(42));
+//! ```
+//!
+//! To declare an init macro/function you just return an [`impl PinInit<T, E>`]:
+//!
+//! ```rust
+//! # #![feature(allocator_api)]
+//! # use pin_init::*;
+//! # #[path = "../examples/error.rs"] mod error; use error::Error;
+//! # #[path = "../examples/mutex.rs"] mod mutex; use mutex::*;
+//! #[pin_data]
+//! struct DriverData {
+//!     #[pin]
+//!     status: CMutex<i32>,
+//!     buffer: Box<[u8; 1_000_000]>,
+//! }
+//!
+//! impl DriverData {
+//!     fn new() -> impl PinInit<Self, Error> {
+//!         pin_init!(Self {
+//!             status <- CMutex::new(0),
+//!             buffer: Box::init(pin_init::init_zeroed())?,
+//!         }? Error)
+//!     }
+//! }
+//! ```
+//!
+//! ## Manual creation of an initializer
+//!
+//! Often when working with primitives the previous approaches are not sufficient. That is where
+//! [`pin_init_from_closure()`] comes in. This `unsafe` function allows you to create a
+//! [`impl PinInit<T, E>`] directly from a closure. Of course you have to ensure that the closure
+//! actually does the initialization in the correct way. Here are the things to look out for
+//! (we are calling the parameter to the closure `slot`):
+//! - when the closure returns `Ok(())`, then it has completed the initialization successfully, so
+//!   `slot` now contains a valid bit pattern for the type `T`,
+//! - when the closure returns `Err(e)`, then the caller may deallocate the memory at `slot`, so
+//!   you need to take care to clean up anything if your initialization fails mid-way,
+//! - you may assume that `slot` will stay pinned even after the closure returns until `drop` of
+//!   `slot` gets called.
+//!
+//! ```rust
+//! # #![feature(extern_types)]
+//! use pin_init::{pin_data, pinned_drop, PinInit, PinnedDrop, pin_init_from_closure};
+//! use core::{
+//!     marker::PhantomPinned,
+//!     cell::UnsafeCell,
+//!     pin::Pin,
+//!     mem::MaybeUninit,
+//! };
+//! mod bindings {
+//!     #[repr(C)]
+//!     pub struct foo {
+//!         /* fields from C ... */
+//!     }
+//!     extern "C" {
+//!         pub fn init_foo(ptr: *mut foo);
+//!         pub fn destroy_foo(ptr: *mut foo);
+//!         #[must_use = "you must check the error return code"]
+//!         pub fn enable_foo(ptr: *mut foo, flags: u32) -> i32;
+//!     }
+//! }
+//!
+//! /// # Invariants
+//! ///
+//! /// `foo` is always initialized
+//! #[pin_data(PinnedDrop)]
+//! pub struct RawFoo {
+//!     #[pin]
+//!     _p: PhantomPinned,
+//!     #[pin]
+//!     foo: UnsafeCell<MaybeUninit<bindings::foo>>,
+//! }
+//!
+//! impl RawFoo {
+//!     pub fn new(flags: u32) -> impl PinInit<Self, i32> {
+//!         // SAFETY:
+//!         // - when the closure returns `Ok(())`, then it has successfully initialized and
+//!         //   enabled `foo`,
+//!         // - when it returns `Err(e)`, then it has cleaned up before
+//!         unsafe {
+//!             pin_init_from_closure(move |slot: *mut Self| {
+//!                 // `slot` contains uninit memory, avoid creating a reference.
+//!                 let foo = &raw mut (*slot).foo;
+//!                 let foo = UnsafeCell::raw_get(foo).cast::<bindings::foo>();
+//!
+//!                 // Initialize the `foo`
+//!                 bindings::init_foo(foo);
+//!
+//!                 // Try to enable it.
+//!                 let err = bindings::enable_foo(foo, flags);
+//!                 if err != 0 {
+//!                     // Enabling has failed, first clean up the foo and then return the error.
+//!                     bindings::destroy_foo(foo);
+//!                     Err(err)
+//!                 } else {
+//!                     // All fields of `RawFoo` have been initialized, since `_p` is a ZST.
+//!                     Ok(())
+//!                 }
+//!             })
+//!         }
+//!     }
+//! }
+//!
+//! #[pinned_drop]
+//! impl PinnedDrop for RawFoo {
+//!     fn drop(self: Pin<&mut Self>) {
+//!         // SAFETY: Since `foo` is initialized, destroying is safe.
+//!         unsafe { bindings::destroy_foo(self.foo.get().cast::<bindings::foo>()) };
+//!     }
+//! }
+//! ```
+//!
+//! For more information on how to use [`pin_init_from_closure()`], take a look at the uses inside
+//! the `kernel` crate. The [`sync`] module is a good starting point.
+//!
+//! [`sync`]: https://rust.docs.kernel.org/kernel/sync/index.html
+//! [pinning]: https://doc.rust-lang.org/std/pin/index.html
+//! [structurally pinned fields]:
+//!     https://doc.rust-lang.org/std/pin/index.html#projections-and-structural-pinning
+//! [stack]: crate::stack_pin_init
+#![cfg_attr(
+    kernel,
+    doc = "[`Arc<T>`]: https://rust.docs.kernel.org/kernel/sync/struct.Arc.html"
+)]
+#![cfg_attr(
+    kernel,
+    doc = "[`Box<T>`]: https://rust.docs.kernel.org/kernel/alloc/kbox/struct.Box.html"
+)]
+#![cfg_attr(not(kernel), doc = "[`Arc<T>`]: alloc::alloc::sync::Arc")]
+#![cfg_attr(not(kernel), doc = "[`Box<T>`]: alloc::alloc::boxed::Box")]
+//! [`impl PinInit<Foo>`]: crate::PinInit
+//! [`impl PinInit<T, E>`]: crate::PinInit
+//! [`impl Init<T, E>`]: crate::Init
+//! [Rust-for-Linux]: https://rust-for-linux.com/
+
+#![forbid(missing_docs, unsafe_op_in_unsafe_fn)]
+#![cfg_attr(not(feature = "std"), no_std)]
+#![cfg_attr(feature = "alloc", feature(allocator_api))]
+#![cfg_attr(
+    all(feature = "unsafe-pinned", CONFIG_RUSTC_HAS_UNSAFE_PINNED),
+    feature(unsafe_pinned)
+)]
+#![cfg_attr(all(USE_RUSTC_FEATURES, doc), allow(internal_features))]
+#![cfg_attr(all(USE_RUSTC_FEATURES, doc), feature(rustdoc_internals))]
+
+use core::{
+    cell::UnsafeCell,
+    convert::Infallible,
+    marker::PhantomData,
+    mem::MaybeUninit,
+    num::*,
+    pin::Pin,
+    ptr::{self, NonNull},
+};
+
+// This is used by doc-tests -- the proc-macros expand to `::pin_init::...` and without this the
+// doc-tests wouldn't have an extern crate named `pin_init`.
+#[allow(unused_extern_crates)]
+extern crate self as pin_init;
+
+#[doc(hidden)]
+pub mod __internal;
+
+#[cfg(any(feature = "std", feature = "alloc"))]
+mod alloc;
+#[cfg(any(feature = "std", feature = "alloc"))]
+pub use alloc::InPlaceInit;
+
+/// Used to specify the pinning information of the fields of a struct.
+///
+/// This is somewhat similar in purpose as
+/// [pin-project-lite](https://crates.io/crates/pin-project-lite).
+/// Place this macro on a struct definition and then `#[pin]` in front of the attributes of each
+/// field you want to structurally pin.
+///
+/// This macro enables the use of the [`pin_init!`] macro. When pin-initializing a `struct`,
+/// then `#[pin]` directs the type of initializer that is required.
+///
+/// If your `struct` implements `Drop`, then you need to add `PinnedDrop` as arguments to this
+/// macro, and change your `Drop` implementation to `PinnedDrop` annotated with
+/// `#[`[`macro@pinned_drop`]`]`, since dropping pinned values requires extra care.
+///
+/// # Examples
+///
+/// ```
+/// # #![feature(allocator_api)]
+/// # #[path = "../examples/mutex.rs"] mod mutex; use mutex::*;
+/// use pin_init::pin_data;
+///
+/// enum Command {
+///     /* ... */
+/// }
+///
+/// #[pin_data]
+/// struct DriverData {
+///     #[pin]
+///     queue: CMutex<Vec<Command>>,
+///     buf: Box<[u8; 1024 * 1024]>,
+/// }
+/// ```
+///
+/// ```
+/// # #![feature(allocator_api)]
+/// # #[path = "../examples/mutex.rs"] mod mutex; use mutex::*;
+/// # mod bindings { pub struct info; pub unsafe fn destroy_info(_: *mut info) {} }
+/// use core::pin::Pin;
+/// use pin_init::{pin_data, pinned_drop, PinnedDrop};
+///
+/// enum Command {
+///     /* ... */
+/// }
+///
+/// #[pin_data(PinnedDrop)]
+/// struct DriverData {
+///     #[pin]
+///     queue: CMutex<Vec<Command>>,
+///     buf: Box<[u8; 1024 * 1024]>,
+///     raw_info: *mut bindings::info,
+/// }
+///
+/// #[pinned_drop]
+/// impl PinnedDrop for DriverData {
+///     fn drop(self: Pin<&mut Self>) {
+///         unsafe { bindings::destroy_info(self.raw_info) };
+///     }
+/// }
+/// ```
+pub use ::pin_init_internal::pin_data;
+
+/// Used to implement `PinnedDrop` safely.
+///
+/// Only works on structs that are annotated via `#[`[`macro@pin_data`]`]`.
+///
+/// # Examples
+///
+/// ```
+/// # #![feature(allocator_api)]
+/// # #[path = "../examples/mutex.rs"] mod mutex; use mutex::*;
+/// # mod bindings { pub struct info; pub unsafe fn destroy_info(_: *mut info) {} }
+/// use core::pin::Pin;
+/// use pin_init::{pin_data, pinned_drop, PinnedDrop};
+///
+/// enum Command {
+///     /* ... */
+/// }
+///
+/// #[pin_data(PinnedDrop)]
+/// struct DriverData {
+///     #[pin]
+///     queue: CMutex<Vec<Command>>,
+///     buf: Box<[u8; 1024 * 1024]>,
+///     raw_info: *mut bindings::info,
+/// }
+///
+/// #[pinned_drop]
+/// impl PinnedDrop for DriverData {
+///     fn drop(self: Pin<&mut Self>) {
+///         unsafe { bindings::destroy_info(self.raw_info) };
+///     }
+/// }
+/// ```
+pub use ::pin_init_internal::pinned_drop;
+
+/// Derives the [`Zeroable`] trait for the given `struct` or `union`.
+///
+/// This can only be used for `struct`s/`union`s where every field implements the [`Zeroable`]
+/// trait.
+///
+/// # Examples
+///
+/// ```
+/// use pin_init::Zeroable;
+///
+/// #[derive(Zeroable)]
+/// pub struct DriverData {
+///     pub(crate) id: i64,
+///     buf_ptr: *mut u8,
+///     len: usize,
+/// }
+/// ```
+///
+/// ```
+/// use pin_init::Zeroable;
+///
+/// #[derive(Zeroable)]
+/// pub union SignCast {
+///     signed: i64,
+///     unsigned: u64,
+/// }
+/// ```
+pub use ::pin_init_internal::Zeroable;
+
+/// Derives the [`Zeroable`] trait for the given `struct` or `union` if all fields implement
+/// [`Zeroable`].
+///
+/// Contrary to the derive macro named [`macro@Zeroable`], this one silently fails when a field
+/// doesn't implement [`Zeroable`].
+///
+/// # Examples
+///
+/// ```
+/// use pin_init::MaybeZeroable;
+///
+/// // implements `Zeroable`
+/// #[derive(MaybeZeroable)]
+/// pub struct DriverData {
+///     pub(crate) id: i64,
+///     buf_ptr: *mut u8,
+///     len: usize,
+/// }
+///
+/// // does not implement `Zeroable`
+/// #[derive(MaybeZeroable)]
+/// pub struct DriverData2 {
+///     pub(crate) id: i64,
+///     buf_ptr: *mut u8,
+///     len: usize,
+///     // this field doesn't implement `Zeroable`
+///     other_data: &'static i32,
+/// }
+/// ```
+pub use ::pin_init_internal::MaybeZeroable;
+
+/// Initialize and pin a type directly on the stack.
+///
+/// # Examples
+///
+/// ```rust
+/// # #![feature(allocator_api)]
+/// # #[path = "../examples/mutex.rs"] mod mutex; use mutex::*;
+/// # use pin_init::*;
+/// # use core::pin::Pin;
+/// #[pin_data]
+/// struct Foo {
+///     #[pin]
+///     a: CMutex<usize>,
+///     b: Bar,
+/// }
+///
+/// #[pin_data]
+/// struct Bar {
+///     x: u32,
+/// }
+///
+/// stack_pin_init!(let foo = pin_init!(Foo {
+///     a <- CMutex::new(42),
+///     b: Bar {
+///         x: 64,
+///     },
+/// }));
+/// let foo: Pin<&mut Foo> = foo;
+/// println!("a: {}", &*foo.a.lock());
+/// ```
+///
+/// # Syntax
+///
+/// A normal `let` binding with optional type annotation. The expression is expected to implement
+/// [`PinInit`]/[`Init`] with the error type [`Infallible`]. If you want to use a different error
+/// type, then use [`stack_try_pin_init!`].
+#[macro_export]
+macro_rules! stack_pin_init {
+    (let $var:ident $(: $t:ty)? = $val:expr) => {
+        let val = $val;
+        let mut $var = ::core::pin::pin!($crate::__internal::StackInit$(::<$t>)?::uninit());
+        let mut $var = match $crate::__internal::StackInit::init($var, val) {
+            Ok(res) => res,
+            Err(x) => {
+                let x: ::core::convert::Infallible = x;
+                match x {}
+            }
+        };
+    };
+}
+
+/// Initialize and pin a type directly on the stack.
+///
+/// # Examples
+///
+/// ```rust
+/// # #![feature(allocator_api)]
+/// # #[path = "../examples/error.rs"] mod error; use error::Error;
+/// # #[path = "../examples/mutex.rs"] mod mutex; use mutex::*;
+/// # use pin_init::*;
+/// #[pin_data]
+/// struct Foo {
+///     #[pin]
+///     a: CMutex<usize>,
+///     b: Box<Bar>,
+/// }
+///
+/// struct Bar {
+///     x: u32,
+/// }
+///
+/// stack_try_pin_init!(let foo: Foo = pin_init!(Foo {
+///     a <- CMutex::new(42),
+///     b: Box::try_new(Bar {
+///         x: 64,
+///     })?,
+/// }? Error));
+/// let foo = foo.unwrap();
+/// println!("a: {}", &*foo.a.lock());
+/// ```
+///
+/// ```rust
+/// # #![feature(allocator_api)]
+/// # #[path = "../examples/error.rs"] mod error; use error::Error;
+/// # #[path = "../examples/mutex.rs"] mod mutex; use mutex::*;
+/// # use pin_init::*;
+/// #[pin_data]
+/// struct Foo {
+///     #[pin]
+///     a: CMutex<usize>,
+///     b: Box<Bar>,
+/// }
+///
+/// struct Bar {
+///     x: u32,
+/// }
+///
+/// stack_try_pin_init!(let foo: Foo =? pin_init!(Foo {
+///     a <- CMutex::new(42),
+///     b: Box::try_new(Bar {
+///         x: 64,
+///     })?,
+/// }? Error));
+/// println!("a: {}", &*foo.a.lock());
+/// # Ok::<_, Error>(())
+/// ```
+///
+/// # Syntax
+///
+/// A normal `let` binding with optional type annotation. The expression is expected to implement
+/// [`PinInit`]/[`Init`]. This macro assigns a result to the given variable, adding a `?` after the
+/// `=` will propagate this error.
+#[macro_export]
+macro_rules! stack_try_pin_init {
+    (let $var:ident $(: $t:ty)? = $val:expr) => {
+        let val = $val;
+        let mut $var = ::core::pin::pin!($crate::__internal::StackInit$(::<$t>)?::uninit());
+        let mut $var = $crate::__internal::StackInit::init($var, val);
+    };
+    (let $var:ident $(: $t:ty)? =? $val:expr) => {
+        let val = $val;
+        let mut $var = ::core::pin::pin!($crate::__internal::StackInit$(::<$t>)?::uninit());
+        let mut $var = $crate::__internal::StackInit::init($var, val)?;
+    };
+}
+
+/// Construct an in-place, fallible pinned initializer for `struct`s.
+///
+/// The error type defaults to [`Infallible`]; if you need a different one, write `? Error` at the
+/// end, after the struct initializer.
+///
+/// The syntax is almost identical to that of a normal `struct` initializer:
+///
+/// ```rust
+/// # use pin_init::*;
+/// # use core::pin::Pin;
+/// #[pin_data]
+/// struct Foo {
+///     a: usize,
+///     b: Bar,
+/// }
+///
+/// #[pin_data]
+/// struct Bar {
+///     x: u32,
+/// }
+///
+/// # fn demo() -> impl PinInit<Foo> {
+/// let a = 42;
+///
+/// let initializer = pin_init!(Foo {
+///     a,
+///     b: Bar {
+///         x: 64,
+///     },
+/// });
+/// # initializer }
+/// # Box::pin_init(demo()).unwrap();
+/// ```
+///
+/// Arbitrary Rust expressions can be used to set the value of a variable.
+///
+/// The fields are initialized in the order that they appear in the initializer. So it is possible
+/// to read already initialized fields using raw pointers.
+///
+/// IMPORTANT: You are not allowed to create references to fields of the struct inside of the
+/// initializer.
+///
+/// # Init-functions
+///
+/// When working with this library it is often desired to let others construct your types without
+/// giving access to all fields. This is where you would normally write a plain function `new` that
+/// would return a new instance of your type. With this library that is also possible. However,
+/// there are a few extra things to keep in mind.
+///
+/// To create an initializer function, simply declare it like this:
+///
+/// ```rust
+/// # use pin_init::*;
+/// # use core::pin::Pin;
+/// # #[pin_data]
+/// # struct Foo {
+/// #     a: usize,
+/// #     b: Bar,
+/// # }
+/// # #[pin_data]
+/// # struct Bar {
+/// #     x: u32,
+/// # }
+/// impl Foo {
+///     fn new() -> impl PinInit<Self> {
+///         pin_init!(Self {
+///             a: 42,
+///             b: Bar {
+///                 x: 64,
+///             },
+///         })
+///     }
+/// }
+/// ```
+///
+/// Users of `Foo` can now create it like this:
+///
+/// ```rust
+/// # use pin_init::*;
+/// # use core::pin::Pin;
+/// # #[pin_data]
+/// # struct Foo {
+/// #     a: usize,
+/// #     b: Bar,
+/// # }
+/// # #[pin_data]
+/// # struct Bar {
+/// #     x: u32,
+/// # }
+/// # impl Foo {
+/// #     fn new() -> impl PinInit<Self> {
+/// #         pin_init!(Self {
+/// #             a: 42,
+/// #             b: Bar {
+/// #                 x: 64,
+/// #             },
+/// #         })
+/// #     }
+/// # }
+/// let foo = Box::pin_init(Foo::new());
+/// ```
+///
+/// They can also easily embed it into their own `struct`s:
+///
+/// ```rust
+/// # use pin_init::*;
+/// # use core::pin::Pin;
+/// # #[pin_data]
+/// # struct Foo {
+/// #     a: usize,
+/// #     b: Bar,
+/// # }
+/// # #[pin_data]
+/// # struct Bar {
+/// #     x: u32,
+/// # }
+/// # impl Foo {
+/// #     fn new() -> impl PinInit<Self> {
+/// #         pin_init!(Self {
+/// #             a: 42,
+/// #             b: Bar {
+/// #                 x: 64,
+/// #             },
+/// #         })
+/// #     }
+/// # }
+/// #[pin_data]
+/// struct FooContainer {
+///     #[pin]
+///     foo1: Foo,
+///     #[pin]
+///     foo2: Foo,
+///     other: u32,
+/// }
+///
+/// impl FooContainer {
+///     fn new(other: u32) -> impl PinInit<Self> {
+///         pin_init!(Self {
+///             foo1 <- Foo::new(),
+///             foo2 <- Foo::new(),
+///             other,
+///         })
+///     }
+/// }
+/// ```
+///
+/// Here we see that when using `pin_init!` with `PinInit`, one needs to write `<-` instead of `:`.
+/// This signifies that the given field is initialized in-place. As with `struct` initializers, just
+/// writing the field (in this case `other`) without `:` or `<-` means `other: other,`.
+///
+/// # Syntax
+///
+/// As already mentioned in the examples above, inside of `pin_init!` a `struct` initializer with
+/// the following modifications is expected:
+/// - Fields that you want to initialize in-place have to use `<-` instead of `:`.
+/// - You can use `_: { /* run any user-code here */ },` anywhere where you can place fields in
+///   order to run arbitrary code.
+/// - In front of the initializer you can write `&this in` to have access to a [`NonNull<Self>`]
+///   pointer named `this` inside of the initializer.
+/// - Using struct update syntax one can place `..Zeroable::init_zeroed()` at the very end of the
+///   struct, this initializes every field with 0 and then runs all initializers specified in the
+///   body. This can only be done if [`Zeroable`] is implemented for the struct.
+///
+/// For instance:
+///
+/// ```rust
+/// # use pin_init::*;
+/// # use core::marker::PhantomPinned;
+/// #[pin_data]
+/// #[derive(Zeroable)]
+/// struct Buf {
+///     // `ptr` points into `buf`.
+///     ptr: *mut u8,
+///     buf: [u8; 64],
+///     #[pin]
+///     pin: PhantomPinned,
+/// }
+///
+/// let init = pin_init!(&this in Buf {
+///     buf: [0; 64],
+///     // SAFETY: TODO.
+///     ptr: unsafe { (&raw mut (*this.as_ptr()).buf).cast() },
+///     pin: PhantomPinned,
+/// });
+/// let init = pin_init!(Buf {
+///     buf: [1; 64],
+///     ..Zeroable::init_zeroed()
+/// });
+/// ```
+///
+/// [`NonNull<Self>`]: core::ptr::NonNull
+pub use pin_init_internal::pin_init;
+
+/// Construct an in-place, fallible initializer for `struct`s.
+///
+/// This macro defaults the error to [`Infallible`]; if you need a different one, write `? Error`
+/// at the end, after the struct initializer.
+///
+/// The syntax is identical to [`pin_init!`] and its safety caveats also apply:
+/// - `unsafe` code must guarantee either full initialization or return an error and allow
+///   deallocation of the memory.
+/// - the fields are initialized in the order given in the initializer.
+/// - no references to fields are allowed to be created inside of the initializer.
+///
+/// This initializer is for initializing data in-place that might later be moved. If you want to
+/// pin-initialize, use [`pin_init!`].
+///
+/// # Examples
+///
+/// ```rust
+/// # #![feature(allocator_api)]
+/// # #[path = "../examples/error.rs"] mod error; use error::Error;
+/// # #[path = "../examples/mutex.rs"] mod mutex; use mutex::*;
+/// # use pin_init::InPlaceInit;
+/// use pin_init::{init, Init, init_zeroed};
+///
+/// struct BigBuf {
+///     small: [u8; 1024 * 1024],
+/// }
+///
+/// impl BigBuf {
+///     fn new() -> impl Init<Self> {
+///         init!(Self {
+///             small <- init_zeroed(),
+///         })
+///     }
+/// }
+/// # let _ = Box::init(BigBuf::new());
+/// ```
+pub use pin_init_internal::init;
+
+/// Asserts that a field on a struct using `#[pin_data]` is marked with `#[pin]` ie. that it is
+/// structurally pinned.
+///
+/// # Examples
+///
+/// This will succeed:
+/// ```
+/// use pin_init::{pin_data, assert_pinned};
+///
+/// #[pin_data]
+/// struct MyStruct {
+///     #[pin]
+///     some_field: u64,
+/// }
+///
+/// assert_pinned!(MyStruct, some_field, u64);
+/// ```
+///
+/// This will fail:
+/// ```compile_fail
+/// use pin_init::{pin_data, assert_pinned};
+///
+/// #[pin_data]
+/// struct MyStruct {
+///     some_field: u64,
+/// }
+///
+/// assert_pinned!(MyStruct, some_field, u64);
+/// ```
+///
+/// Some uses of the macro may trigger the `can't use generic parameters from outer item` error. To
+/// work around this, you may pass the `inline` parameter to the macro. The `inline` parameter can
+/// only be used when the macro is invoked from a function body.
+/// ```
+/// # use core::pin::Pin;
+/// use pin_init::{pin_data, assert_pinned};
+///
+/// #[pin_data]
+/// struct Foo<T> {
+///     #[pin]
+///     elem: T,
+/// }
+///
+/// impl<T> Foo<T> {
+///     fn project_this(self: Pin<&mut Self>) -> Pin<&mut T> {
+///         assert_pinned!(Foo<T>, elem, T, inline);
+///
+///         // SAFETY: The field is structurally pinned.
+///         unsafe { self.map_unchecked_mut(|me| &mut me.elem) }
+///     }
+/// }
+/// ```
+#[macro_export]
+macro_rules! assert_pinned {
+    ($ty:ty, $field:ident, $field_ty:ty, inline) => {
+        // SAFETY: This code is unreachable.
+        let _ = move |ptr: *mut $ty| unsafe {
+            let data = <$ty as $crate::__internal::HasPinData>::__pin_data();
+            _ = data
+                .$field(ptr)
+                .init($crate::__internal::AlwaysFail::<$field_ty>::new());
+        };
+    };
+
+    ($ty:ty, $field:ident, $field_ty:ty) => {
+        const _: () = {
+            $crate::assert_pinned!($ty, $field, $field_ty, inline);
+        };
+    };
+}
+
+/// A pin-initializer for the type `T`.
+///
+/// To use this initializer, you will need a suitable memory location that can hold a `T`. This can
+/// be [`Box<T>`], [`Arc<T>`] or even the stack (see [`stack_pin_init!`]).
+///
+/// Also see the [module description](self).
+///
+/// # Safety
+///
+/// When implementing this trait you will need to take great care. Also there are probably very few
+/// cases where a manual implementation is necessary. Use [`pin_init_from_closure`] where possible.
+///
+/// The [`PinInit::__init`] function:
+/// - returns `Ok(())` if it initialized every field of `slot`,
+/// - returns `Err(err)` if it encountered an error and then cleaned `slot`, this means:
+///     - `slot` can be deallocated without UB occurring,
+///     - `slot` does not need to be dropped,
+///     - `slot` is not partially initialized.
+/// - while constructing the `T` at `slot` it upholds the pinning invariants of `T`.
+///
+#[cfg_attr(
+    kernel,
+    doc = "[`Arc<T>`]: https://rust.docs.kernel.org/kernel/sync/struct.Arc.html"
+)]
+#[cfg_attr(
+    kernel,
+    doc = "[`Box<T>`]: https://rust.docs.kernel.org/kernel/alloc/kbox/struct.Box.html"
+)]
+#[cfg_attr(not(kernel), doc = "[`Arc<T>`]: alloc::alloc::sync::Arc")]
+#[cfg_attr(not(kernel), doc = "[`Box<T>`]: alloc::alloc::boxed::Box")]
+#[must_use = "An initializer must be used in order to create its value."]
+pub unsafe trait PinInit<T: ?Sized, E = Infallible>: Sized {
+    /// Alias of [`PinInit::__init`].
+    ///
+    /// New code should use `__init` instead.
+    ///
+    /// # Safety
+    ///
+    /// Same as `__init`.
+    #[inline(always)]
+    #[cfg(not(kernel))]
+    #[deprecated = "use `raw_try_init` instead"]
+    unsafe fn __pinned_init(self, slot: *mut T) -> Result<(), E> {
+        // SAFETY: Per safety requirement.
+        unsafe { self.__init(slot) }
+    }
+
+    /// Initializes `slot`.
+    ///
+    /// It is not recommended to call this directly. Use [`raw_init`] or [`raw_try_init`].
+    ///
+    /// # Safety
+    ///
+    /// - `slot` is a valid pointer to uninitialized memory.
+    /// - the caller does not touch `slot` when `Err` is returned, they are only permitted to
+    ///   deallocate.
+    /// - `slot` will not move until it is dropped, i.e. it will be pinned.
+    ///   If `Self: Init<T, E>`, this requirement is cancelled and it may be moved.
+    unsafe fn __init(self, slot: *mut T) -> Result<(), E>;
+
+    /// First initializes the value using `self` then calls the function `f` with the initialized
+    /// value.
+    ///
+    /// If `f` returns an error the value is dropped and the initializer will forward the error.
+    ///
+    /// # Examples
+    ///
+    /// ```rust
+    /// # #![feature(allocator_api)]
+    /// # #[path = "../examples/mutex.rs"] mod mutex; use mutex::*;
+    /// # use pin_init::*;
+    /// let mtx_init = CMutex::new(42);
+    /// // Make the initializer print the value.
+    /// let mtx_init = mtx_init.pin_chain(|mtx| {
+    ///     println!("{:?}", mtx.get_data_mut());
+    ///     Ok(())
+    /// });
+    /// ```
+    #[inline]
+    fn pin_chain<F>(self, f: F) -> ChainPinInit<Self, F, T, E>
+    where
+        F: FnOnce(Pin<&mut T>) -> Result<(), E>,
+    {
+        ChainPinInit(self, f, __internal::PhantomInvariant::new())
+    }
+}
+
+/// Initializes `slot` with an initializer.
+///
+/// # Safety
+///
+/// - `slot` is a valid pointer to uninitialized memory.
+/// - `slot` will not move until it is dropped, i.e. it will be pinned.
+///   If `init` implements `Init<T, E>`, this requirement is cancelled and it may be moved.
+#[inline(always)]
+pub unsafe fn raw_init<T>(slot: *mut T, init: impl PinInit<T>) {
+    // SAFETY: Per safety requirement.
+    unsafe { init.__init(slot).unwrap_or_else(|e| match e {}) }
+}
+
+/// Fallibly initializes `slot` with an initializer.
+///
+/// # Safety
+///
+/// - `slot` is a valid pointer to uninitialized memory.
+/// - the caller does not touch `slot` when `Err` is returned, they are only permitted to
+///   deallocate.
+/// - `slot` will not move until it is dropped, i.e. it will be pinned.
+///   If `init` implements `Init<T, E>`, this requirement is cancelled and it may be moved.
+#[inline(always)]
+pub unsafe fn raw_try_init<T, E>(slot: *mut T, init: impl PinInit<T, E>) -> Result<(), E> {
+    // SAFETY: Per safety requirement.
+    unsafe { init.__init(slot) }
+}
+
+/// An initializer returned by [`PinInit::pin_chain`].
+pub struct ChainPinInit<I, F, T: ?Sized, E>(I, F, __internal::PhantomInvariant<(E, T)>);
+
+// SAFETY: The `__init` function is implemented such that it
+// - returns `Ok(())` on successful initialization,
+// - returns `Err(err)` on error and in this case `slot` will be dropped.
+// - considers `slot` pinned.
+unsafe impl<T: ?Sized, E, I, F> PinInit<T, E> for ChainPinInit<I, F, T, E>
+where
+    I: PinInit<T, E>,
+    F: FnOnce(Pin<&mut T>) -> Result<(), E>,
+{
+    #[inline]
+    unsafe fn __init(self, slot: *mut T) -> Result<(), E> {
+        // SAFETY: All requirements fulfilled since this function is `__init`.
+        let slot = unsafe { __internal::Slot::<__internal::Pinned, _>::new(slot) };
+        let mut guard = slot.init(self.0)?;
+        (self.1)(guard.let_binding())?;
+        core::mem::forget(guard);
+        Ok(())
+    }
+}
+
+/// An initializer for `T`.
+///
+/// To use this initializer, you will need a suitable memory location that can hold a `T`. This can
+/// be [`Box<T>`], [`Arc<T>`] or even the stack (see [`stack_pin_init!`]). Because
+/// [`PinInit<T, E>`] is a super trait, you can use every function that takes it as well.
+///
+/// Also see the [module description](self).
+///
+/// # Safety
+///
+/// When implementing this trait you will need to take great care. Also there are probably very few
+/// cases where a manual implementation is necessary. Use [`init_from_closure`] where possible.
+///
+/// The [`PinInit::__init`] function must work without the pinning requirement; the caller is
+/// allowed to move the pointee after initialization.
+///
+#[cfg_attr(
+    kernel,
+    doc = "[`Arc<T>`]: https://rust.docs.kernel.org/kernel/sync/struct.Arc.html"
+)]
+#[cfg_attr(
+    kernel,
+    doc = "[`Box<T>`]: https://rust.docs.kernel.org/kernel/alloc/kbox/struct.Box.html"
+)]
+#[cfg_attr(not(kernel), doc = "[`Arc<T>`]: alloc::alloc::sync::Arc")]
+#[cfg_attr(not(kernel), doc = "[`Box<T>`]: alloc::alloc::boxed::Box")]
+#[must_use = "An initializer must be used in order to create its value."]
+pub unsafe trait Init<T: ?Sized, E = Infallible>: PinInit<T, E> {
+    /// First initializes the value using `self` then calls the function `f` with the initialized
+    /// value.
+    ///
+    /// If `f` returns an error the value is dropped and the initializer will forward the error.
+    ///
+    /// # Examples
+    ///
+    /// ```rust
+    /// use pin_init::{init, init_zeroed, Init};
+    ///
+    /// struct Foo {
+    ///     buf: [u8; 1_000_000],
+    /// }
+    ///
+    /// impl Foo {
+    ///     fn setup(&mut self) {
+    ///         println!("Setting up foo");
+    ///     }
+    /// }
+    ///
+    /// let foo = init!(Foo {
+    ///     buf <- init_zeroed()
+    /// }).chain(|foo| {
+    ///     foo.setup();
+    ///     Ok(())
+    /// });
+    /// ```
+    #[inline]
+    fn chain<F>(self, f: F) -> ChainInit<Self, F, T, E>
+    where
+        F: FnOnce(&mut T) -> Result<(), E>,
+    {
+        ChainInit(self, f, __internal::PhantomInvariant::new())
+    }
+}
+
+/// An initializer returned by [`Init::chain`].
+pub struct ChainInit<I, F, T: ?Sized, E>(I, F, __internal::PhantomInvariant<(E, T)>);
+
+// SAFETY: The `__init` function does not rely on the pinning requirement.
+unsafe impl<T: ?Sized, E, I, F> Init<T, E> for ChainInit<I, F, T, E>
+where
+    I: Init<T, E>,
+    F: FnOnce(&mut T) -> Result<(), E>,
+{
+}
+
+// SAFETY: The `__init` function is implemented such that it
+// - returns `Ok(())` on successful initialization,
+// - returns `Err(err)` on error and in this case `slot` will be dropped.
+unsafe impl<T: ?Sized, E, I, F> PinInit<T, E> for ChainInit<I, F, T, E>
+where
+    I: Init<T, E>,
+    F: FnOnce(&mut T) -> Result<(), E>,
+{
+    #[inline]
+    unsafe fn __init(self, slot: *mut T) -> Result<(), E> {
+        // SAFETY: All requirements fulfilled since this function is `__init`.
+        let slot = unsafe { __internal::Slot::<__internal::Unpinned, _>::new(slot) };
+        let mut guard = slot.init(self.0)?;
+        (self.1)(guard.let_binding())?;
+        core::mem::forget(guard);
+        Ok(())
+    }
+}
+
+/// Implement `PinInit` and `Init` for closures.
+///
+/// It is unsafe to create this type, since the closure needs to fulfill the same safety
+/// requirement as the `__init` functions.
+struct InitClosure<F, T: ?Sized>(F, __internal::PhantomInvariant<T>);
+
+// SAFETY: When constructing via `init_from_closure`, the `__init` function does not rely on the
+// pinning requirement. When constructing via `pin_init_from_closure`, the opaque type prevents this
+// implementation from being visible.
+unsafe impl<T: ?Sized, F, E> Init<T, E> for InitClosure<F, T> where
+    F: FnOnce(*mut T) -> Result<(), E>
+{
+}
+
+// SAFETY: While constructing the `InitClosure`, the user promised that it upholds the
+// `__init` invariants.
+unsafe impl<T: ?Sized, F, E> PinInit<T, E> for InitClosure<F, T>
+where
+    F: FnOnce(*mut T) -> Result<(), E>,
+{
+    #[inline]
+    unsafe fn __init(self, slot: *mut T) -> Result<(), E> {
+        (self.0)(slot)
+    }
+}
+
+/// Creates a new [`PinInit<T, E>`] from the given closure.
+///
+/// # Safety
+///
+/// The closure:
+/// - returns `Ok(())` if it initialized every field of `slot`,
+/// - returns `Err(err)` if it encountered an error and then cleaned `slot`, this means:
+///     - `slot` can be deallocated without UB occurring,
+///     - `slot` does not need to be dropped,
+///     - `slot` is not partially initialized.
+/// - may assume that the `slot` does not move if `T: !Unpin`,
+/// - while constructing the `T` at `slot` it upholds the pinning invariants of `T`.
+#[inline]
+pub const unsafe fn pin_init_from_closure<T: ?Sized, E>(
+    f: impl FnOnce(*mut T) -> Result<(), E>,
+) -> impl PinInit<T, E> {
+    InitClosure(f, __internal::PhantomInvariant::new())
+}
+
+/// Creates a new [`Init<T, E>`] from the given closure.
+///
+/// # Safety
+///
+/// The closure:
+/// - returns `Ok(())` if it initialized every field of `slot`,
+/// - returns `Err(err)` if it encountered an error and then cleaned `slot`, this means:
+///     - `slot` can be deallocated without UB occurring,
+///     - `slot` does not need to be dropped,
+///     - `slot` is not partially initialized.
+/// - the `slot` may move after initialization.
+/// - while constructing the `T` at `slot` it upholds the pinning invariants of `T`.
+#[inline]
+pub const unsafe fn init_from_closure<T: ?Sized, E>(
+    f: impl FnOnce(*mut T) -> Result<(), E>,
+) -> impl Init<T, E> {
+    InitClosure(f, __internal::PhantomInvariant::new())
+}
+
+/// Changes the to be initialized type.
+///
+/// # Safety
+///
+/// - `*mut U` must be castable to `*mut T` and any value of type `T` written through such a
+///   pointer must result in a valid `U`.
+#[inline]
+pub const unsafe fn cast_pin_init<T, U, E>(init: impl PinInit<T, E>) -> impl PinInit<U, E> {
+    // SAFETY: initialization delegated to a valid initializer. Cast is valid by function safety
+    // requirements.
+    unsafe { pin_init_from_closure(|ptr: *mut U| init.__init(ptr.cast::<T>())) }
+}
+
+/// Changes the to be initialized type.
+///
+/// # Safety
+///
+/// - `*mut U` must be castable to `*mut T` and any value of type `T` written through such a
+///   pointer must result in a valid `U`.
+#[inline]
+pub const unsafe fn cast_init<T, U, E>(init: impl Init<T, E>) -> impl Init<U, E> {
+    // SAFETY: initialization delegated to a valid initializer. Cast is valid by function safety
+    // requirements.
+    unsafe { init_from_closure(|ptr: *mut U| init.__init(ptr.cast::<T>())) }
+}
+
+/// An initializer that leaves the memory uninitialized.
+///
+/// The initializer is a no-op. The `slot` memory is not changed.
+#[inline]
+pub fn uninit<T, E>() -> impl Init<MaybeUninit<T>, E> {
+    // SAFETY: The memory is allowed to be uninitialized.
+    unsafe { init_from_closure(|_| Ok(())) }
+}
+
+/// Array initializer from element initializer.
+struct ArrayInit<T: ?Sized, F>(F, __internal::PhantomInvariant<T>);
+
+// SAFETY: On success, all `N` elements of the array have been initialized. On error or panic, the
+// elements that have been initialized so far are dropped, thus leaving the array uninitialized and
+// ready to deallocate.
+unsafe impl<T, F, I, E, const N: usize> PinInit<[T; N], E> for ArrayInit<T, F>
+where
+    F: FnMut(usize) -> I,
+    I: PinInit<T, E>,
+{
+    unsafe fn __init(mut self, slot: *mut [T; N]) -> Result<(), E> {
+        /// # Invariants
+        ///
+        /// - `ptr[..num_init]` contains initialized elements of type `T`
+        /// - `ptr[num_init..N]` (where N is the size of the array) contains uninitialized memory
+        struct ArrayInitGuard<T> {
+            /// A pointer to the first element of the array.
+            ptr: *mut T,
+            /// The number of initialized elements in the array.
+            num_init: usize,
+        }
+
+        impl<T> Drop for ArrayInitGuard<T> {
+            #[inline]
+            fn drop(&mut self) {
+                // SAFETY: Per type invariant, `self.ptr[..self.num_init]` are initialized.
+                unsafe {
+                    core::ptr::drop_in_place(core::ptr::slice_from_raw_parts_mut(
+                        self.ptr,
+                        self.num_init,
+                    ))
+                };
+            }
+        }
+
+        // INVARIANT: nothing is initialized yet.
+        let mut guard = ArrayInitGuard {
+            ptr: slot.cast::<T>(),
+            num_init: 0,
+        };
+
+        for i in 0..N {
+            // INVARIANT: Elements `self.ptr[..self.num_init]` have been initialized
+            // thus far. This holds true for every `self.num_init = i`.
+            guard.num_init = i;
+
+            let init = (self.0)(i);
+            // SAFETY:
+            // - The subslot is derived from `slot` with a valid offset.
+            // - If `Err` is touched, the subslot is not touched further, the guard will drop
+            //   previously initialized elements only.
+            // - `slot` is pinned so is the subslot.
+            unsafe { init.__init(&raw mut (*slot)[i]) }?;
+        }
+
+        // Dismiss the drop guard now that all elements are initialized.
+        core::mem::forget(guard);
+        Ok(())
+    }
+}
+
+// SAFETY: `I: Init` cancels out the pinning requirement on subslots, which is the only place in the
+// `__init` function that relies on `slot` being pinned.
+unsafe impl<T, F, I, E, const N: usize> Init<[T; N], E> for ArrayInit<T, F>
+where
+    F: FnMut(usize) -> I,
+    I: Init<T, E>,
+{
+}
+
+/// Initializes an array by initializing each element via the provided initializer.
+///
+/// # Examples
+///
+/// ```rust
+/// # use pin_init::*;
+/// use pin_init::init_array_from_fn;
+/// let array: Box<[usize; 1_000]> = Box::init(init_array_from_fn(|i| i)).unwrap();
+/// assert_eq!(array.len(), 1_000);
+/// ```
+#[inline]
+pub fn init_array_from_fn<I, const N: usize, T, E>(
+    make_init: impl FnMut(usize) -> I,
+) -> impl Init<[T; N], E>
+where
+    I: Init<T, E>,
+{
+    ArrayInit(make_init, __internal::PhantomInvariant::new())
+}
+
+/// Initializes an array by initializing each element via the provided initializer.
+///
+/// # Examples
+///
+/// ```rust
+/// # #![feature(allocator_api)]
+/// # #[path = "../examples/mutex.rs"] mod mutex; use mutex::*;
+/// # use pin_init::*;
+/// # use core::pin::Pin;
+/// use pin_init::pin_init_array_from_fn;
+/// use std::sync::Arc;
+/// let array: Pin<Arc<[CMutex<usize>; 1_000]>> =
+///     Arc::pin_init(pin_init_array_from_fn(|i| CMutex::new(i))).unwrap();
+/// assert_eq!(array.len(), 1_000);
+/// ```
+#[inline]
+pub fn pin_init_array_from_fn<I, const N: usize, T, E>(
+    make_init: impl FnMut(usize) -> I,
+) -> impl PinInit<[T; N], E>
+where
+    I: PinInit<T, E>,
+{
+    ArrayInit(make_init, __internal::PhantomInvariant::new())
+}
+
+/// Construct an initializer in a closure and run it.
+///
+/// Returns an initializer that first runs the closure and then the initializer returned by it.
+///
+/// See also [`init_scope`].
+///
+/// # Examples
+///
+/// ```
+/// # use pin_init::*;
+/// # #[pin_data]
+/// # struct Foo { a: u64, b: isize }
+/// # struct Bar { a: u32, b: isize }
+/// # fn lookup_bar() -> Result<Bar, Error> { todo!() }
+/// # struct Error;
+/// fn init_foo() -> impl PinInit<Foo, Error> {
+///     pin_init_scope(|| {
+///         let bar = lookup_bar()?;
+///         Ok(pin_init!(Foo { a: bar.a.into(), b: bar.b }? Error))
+///     })
+/// }
+/// ```
+///
+/// This initializer will first execute `lookup_bar()`, match on it, if it returned an error, the
+/// initializer itself will fail with that error. If it returned `Ok`, then it will run the
+/// initializer returned by the [`pin_init!`] invocation.
+#[inline]
+pub fn pin_init_scope<T, E, F, I>(make_init: F) -> impl PinInit<T, E>
+where
+    F: FnOnce() -> Result<I, E>,
+    I: PinInit<T, E>,
+{
+    // SAFETY:
+    // - If `make_init` returns `Err`, `Err` is returned and `slot` is completely uninitialized,
+    // - If `make_init` returns `Ok`, safety requirement are fulfilled by `init.__init`.
+    // - The safety requirements of `init.__init` are fulfilled, since it's being called from an
+    //   initializer.
+    unsafe {
+        pin_init_from_closure(move |slot: *mut T| -> Result<(), E> {
+            let init = make_init()?;
+            init.__init(slot)
+        })
+    }
+}
+
+/// Construct an initializer in a closure and run it.
+///
+/// Returns an initializer that first runs the closure and then the initializer returned by it.
+///
+/// See also [`pin_init_scope`].
+///
+/// # Examples
+///
+/// ```
+/// # use pin_init::*;
+/// # struct Foo { a: u64, b: isize }
+/// # struct Bar { a: u32, b: isize }
+/// # fn lookup_bar() -> Result<Bar, Error> { todo!() }
+/// # struct Error;
+/// fn init_foo() -> impl Init<Foo, Error> {
+///     init_scope(|| {
+///         let bar = lookup_bar()?;
+///         Ok(init!(Foo { a: bar.a.into(), b: bar.b }? Error))
+///     })
+/// }
+/// ```
+///
+/// This initializer will first execute `lookup_bar()`, match on it, if it returned an error, the
+/// initializer itself will fail with that error. If it returned `Ok`, then it will run the
+/// initializer returned by the [`init!`] invocation.
+#[inline]
+pub fn init_scope<T, E, F, I>(make_init: F) -> impl Init<T, E>
+where
+    F: FnOnce() -> Result<I, E>,
+    I: Init<T, E>,
+{
+    // SAFETY:
+    // - If `make_init` returns `Err`, `Err` is returned and `slot` is completely uninitialized,
+    // - If `make_init` returns `Ok`, safety requirement are fulfilled by `init.__init`.
+    // - The safety requirements of `init.__init` are fulfilled, since it's being called from an
+    //   initializer.
+    unsafe {
+        init_from_closure(move |slot: *mut T| -> Result<(), E> {
+            let init = make_init()?;
+            init.__init(slot)
+        })
+    }
+}
+
+// SAFETY: The `__init` function does not rely on slot being pinned after it returns.
+unsafe impl<T> Init<T> for T {}
+
+// SAFETY: the `__init` function always returns `Ok(())` and initializes every field of
+// `slot`. Additionally, all pinning invariants of `T` are upheld.
+unsafe impl<T> PinInit<T> for T {
+    #[inline]
+    unsafe fn __init(self, slot: *mut T) -> Result<(), Infallible> {
+        // SAFETY: `slot` is valid for writes by the safety requirements of this function.
+        unsafe { slot.write(self) };
+        Ok(())
+    }
+}
+
+// SAFETY: The `__init` function does not rely on slot being pinned after it returns.
+unsafe impl<T, E> Init<T, E> for Result<T, E> {}
+
+// SAFETY: when the `__init` function returns with
+// - `Ok(())`, `slot` was initialized and all pinned invariants of `T` are upheld.
+// - `Err(err)`, slot was not written to.
+unsafe impl<T, E> PinInit<T, E> for Result<T, E> {
+    #[inline]
+    unsafe fn __init(self, slot: *mut T) -> Result<(), E> {
+        // SAFETY: `slot` is valid for writes by the safety requirements of this function.
+        unsafe { slot.write(self?) };
+        Ok(())
+    }
+}
+
+/// Smart pointer containing uninitialized memory and that can write a value.
+pub trait InPlaceWrite<T> {
+    /// The type `Self` turns into when the contents are initialized.
+    type Initialized;
+
+    /// Use the given initializer to write a value into `self`.
+    ///
+    /// Does not drop the current value and considers it as uninitialized memory.
+    fn write_init<E>(self, init: impl Init<T, E>) -> Result<Self::Initialized, E>;
+
+    /// Use the given pin-initializer to write a value into `self`.
+    ///
+    /// Does not drop the current value and considers it as uninitialized memory.
+    fn write_pin_init<E>(self, init: impl PinInit<T, E>) -> Result<Pin<Self::Initialized>, E>;
+}
+
+impl<T> InPlaceWrite<T> for &'static mut MaybeUninit<T> {
+    type Initialized = &'static mut T;
+
+    #[inline]
+    fn write_init<E>(self, init: impl Init<T, E>) -> Result<Self::Initialized, E> {
+        let slot = self.as_mut_ptr();
+
+        // SAFETY: `slot` is a valid pointer to uninitialized memory.
+        unsafe { init.__init(slot)? };
+
+        // SAFETY: The above call initialized the memory.
+        unsafe { Ok(self.assume_init_mut()) }
+    }
+
+    #[inline]
+    fn write_pin_init<E>(self, init: impl PinInit<T, E>) -> Result<Pin<Self::Initialized>, E> {
+        let slot = self.as_mut_ptr();
+
+        // SAFETY: `slot` is a valid pointer to uninitialized memory.
+        //
+        // The `'static` borrow guarantees the data will not be
+        // moved/invalidated until it gets dropped (which is never).
+        unsafe { init.__init(slot)? };
+
+        // SAFETY: The above call initialized the memory.
+        Ok(Pin::static_mut(unsafe { self.assume_init_mut() }))
+    }
+}
+
+/// Trait facilitating pinned destruction.
+///
+/// Use [`pinned_drop`] to implement this trait safely:
+///
+/// ```rust
+/// # #![feature(allocator_api)]
+/// # #[path = "../examples/mutex.rs"] mod mutex; use mutex::*;
+/// # use pin_init::*;
+/// use core::pin::Pin;
+/// #[pin_data(PinnedDrop)]
+/// struct Foo {
+///     #[pin]
+///     mtx: CMutex<usize>,
+/// }
+///
+/// #[pinned_drop]
+/// impl PinnedDrop for Foo {
+///     fn drop(self: Pin<&mut Self>) {
+///         println!("Foo is being dropped!");
+///     }
+/// }
+/// ```
+///
+/// # Safety
+///
+/// This trait must be implemented via the [`pinned_drop`] proc-macro attribute on the impl.
+pub unsafe trait PinnedDrop: __internal::HasPinData {
+    /// Executes the pinned destructor of this type.
+    ///
+    /// While this function is marked safe, it is actually unsafe to call it manually. For this
+    /// reason it takes an additional parameter. This type can only be constructed by `unsafe` code
+    /// and thus prevents this function from being called where it should not.
+    ///
+    /// This extra parameter will be generated by the `#[pinned_drop]` proc-macro attribute
+    /// automatically.
+    fn drop(self: Pin<&mut Self>, only_call_from_drop: __internal::OnlyCallFromDrop);
+}
+
+/// Marker trait for types that can be initialized by writing just zeroes.
+///
+/// # Safety
+///
+/// The bit pattern consisting of only zeroes is a valid bit pattern for this type. In other words,
+/// this is not UB:
+///
+/// ```rust,ignore
+/// let val: Self = unsafe { core::mem::zeroed() };
+/// ```
+pub unsafe trait Zeroable {
+    /// Create a new zeroed `Self`.
+    ///
+    /// The returned initializer will write `0x00` to every byte of the given `slot`.
+    #[inline]
+    fn init_zeroed() -> impl Init<Self>
+    where
+        Self: Sized,
+    {
+        init_zeroed()
+    }
+
+    /// Create a `Self` consisting of all zeroes.
+    ///
+    /// Whenever a type implements [`Zeroable`], this function should be preferred over
+    /// [`core::mem::zeroed()`] or using `MaybeUninit<T>::zeroed().assume_init()`.
+    ///
+    /// As const traits are not yet stable, [`pin_init::zeroed()`] can be used instead
+    /// when initialization is required in a `const` context.
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// use pin_init::Zeroable;
+    ///
+    /// #[derive(Zeroable)]
+    /// struct Point {
+    ///     x: u32,
+    ///     y: u32,
+    /// }
+    ///
+    /// let point: Point = Zeroable::zeroed();
+    /// assert_eq!(point.x, 0);
+    /// assert_eq!(point.y, 0);
+    /// ```
+    #[inline]
+    fn zeroed() -> Self
+    where
+        Self: Sized,
+    {
+        zeroed()
+    }
+}
+
+/// Create an initializer for a zeroed `T`.
+///
+/// The returned initializer will write `0x00` to every byte of the given `slot`.
+#[inline]
+pub fn init_zeroed<T: Zeroable>() -> impl Init<T> {
+    // SAFETY: Because `T: Zeroable`, all bytes zero is a valid bit pattern for `T`
+    // and because we write all zeroes, the memory is initialized.
+    unsafe {
+        init_from_closure(|slot: *mut T| {
+            slot.write_bytes(0, 1);
+            Ok(())
+        })
+    }
+}
+
+/// Create a `T` consisting of all zeroes.
+///
+/// Whenever a type implements [`Zeroable`], this function should be preferred over
+/// [`core::mem::zeroed()`] or using `MaybeUninit<T>::zeroed().assume_init()`.
+///
+/// While const traits remain unstable, this function serves as the `const` version of
+/// [`Zeroable::zeroed()`].
+///
+/// # Examples
+///
+/// ```
+/// use pin_init::{Zeroable, zeroed};
+///
+/// #[derive(Zeroable)]
+/// struct Point {
+///     x: u32,
+///     y: u32,
+/// }
+///
+/// let point: Point = zeroed();
+/// assert_eq!(point.x, 0);
+/// assert_eq!(point.y, 0);
+/// ```
+#[inline]
+pub const fn zeroed<T: Zeroable>() -> T {
+    // SAFETY:By the type invariants of `Zeroable`, all zeroes is a valid bit pattern for `T`.
+    unsafe { core::mem::zeroed() }
+}
+
+macro_rules! impl_zeroable {
+    ($($({$($generics:tt)*})? $t:ty, )*) => {
+        // SAFETY: Safety comments written in the macro invocation.
+        $(unsafe impl$($($generics)*)? Zeroable for $t {})*
+    };
+}
+
+impl_zeroable! {
+    // SAFETY: All primitives that are allowed to be zero.
+    bool,
+    char,
+    u8, u16, u32, u64, u128, usize,
+    i8, i16, i32, i64, i128, isize,
+    f32, f64,
+
+    // Note: do not add uninhabited types (such as `!` or `core::convert::Infallible`) to this list;
+    // creating an instance of an uninhabited type is immediate undefined behavior. For more on
+    // uninhabited/empty types, consult The Rustonomicon:
+    // <https://doc.rust-lang.org/stable/nomicon/exotic-sizes.html#empty-types>. The Rust Reference
+    // also has information on undefined behavior:
+    // <https://doc.rust-lang.org/stable/reference/behavior-considered-undefined.html>.
+    //
+    // SAFETY: These are inhabited ZSTs; there is nothing to zero and a valid value exists.
+    {<T: ?Sized>} PhantomData<T>, core::marker::PhantomPinned, (),
+
+    // SAFETY: Type is allowed to take any value, including all zeros.
+    {<T>} MaybeUninit<T>,
+
+    // SAFETY: `T: Zeroable` and `UnsafeCell` is `repr(transparent)`.
+    {<T: ?Sized + Zeroable>} UnsafeCell<T>,
+
+    // SAFETY: `null` pointer is valid.
+    //
+    // We cannot use `T: ?Sized`, since the VTABLE pointer part of fat pointers is not allowed to be
+    // null.
+    //
+    // When `Pointee` gets stabilized, we could use
+    // `T: ?Sized where <T as Pointee>::Metadata: Zeroable`
+    {<T>} *mut T, {<T>} *const T,
+
+    // SAFETY: `null` pointer is valid and the metadata part of these fat pointers is allowed to be
+    // zero.
+    {<T>} *mut [T], {<T>} *const [T], *mut str, *const str,
+
+    // SAFETY: `T` is `Zeroable`.
+    {<const N: usize, T: Zeroable>} [T; N], {<T: Zeroable>} Wrapping<T>,
+}
+
+macro_rules! impl_tuple_zeroable {
+    ($first:ident, $(,)?) => {
+        #[cfg_attr(all(USE_RUSTC_FEATURES, doc), doc(fake_variadic))]
+        /// Implemented for tuples up to 10 items long.
+        // SAFETY: All elements are zeroable and padding can be zero.
+        unsafe impl<$first: Zeroable> Zeroable for ($first,) {}
+    };
+    ($first:ident, $($t:ident),* $(,)?) => {
+        #[cfg_attr(doc, doc(hidden))]
+        // SAFETY: All elements are zeroable and padding can be zero.
+        unsafe impl<$first: Zeroable, $($t: Zeroable),*> Zeroable for ($first, $($t),*) {}
+        impl_tuple_zeroable!($($t),* ,);
+    }
+}
+
+impl_tuple_zeroable!(A, B, C, D, E, F, G, H, I, J);
+
+/// Marker trait for types that allow `Option<Self>` to be set to all zeroes in order to write
+/// `None` to that location.
+///
+/// # Safety
+///
+/// The implementer needs to ensure that `unsafe impl Zeroable for Option<Self> {}` is sound.
+pub unsafe trait ZeroableOption {}
+
+// SAFETY: by the safety requirement of `ZeroableOption`, this is valid.
+unsafe impl<T: ZeroableOption> Zeroable for Option<T> {}
+
+macro_rules! impl_fn_zeroable_option {
+    ([$($abi:literal),* $(,)?] $args:tt) => {
+        $(impl_fn_zeroable_option!({extern $abi} $args);)*
+        $(impl_fn_zeroable_option!({unsafe extern $abi} $args);)*
+    };
+    ({$($prefix:tt)*} {$(,)?}) => {};
+    ({$($prefix:tt)*} {$ret:ident, $arg:ident $(,)?}) => {
+        #[cfg_attr(all(USE_RUSTC_FEATURES, doc), doc(fake_variadic))]
+        /// Implemented for function pointers with up to 20 arity.
+        // SAFETY: function pointers are part of the option layout optimization:
+        // <https://doc.rust-lang.org/stable/std/option/index.html#representation>.
+        unsafe impl<$ret, $arg> ZeroableOption for $($prefix)* fn($arg) -> $ret {}
+        impl_fn_zeroable_option!({$($prefix)*} {$arg,});
+    };
+    ({$($prefix:tt)*} {$ret:ident, $($rest:ident),* $(,)?}) => {
+        #[cfg_attr(doc, doc(hidden))]
+        // SAFETY: function pointers are part of the option layout optimization:
+        // <https://doc.rust-lang.org/stable/std/option/index.html#representation>.
+        unsafe impl<$ret, $($rest),*> ZeroableOption for $($prefix)* fn($($rest),*) -> $ret {}
+        impl_fn_zeroable_option!({$($prefix)*} {$($rest),*,});
+    };
+}
+
+impl_fn_zeroable_option!(["Rust", "C"] { A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U });
+
+macro_rules! impl_zeroable_option {
+    ($($({$($generics:tt)*})? $t:ty, )*) => {
+        // SAFETY: Safety comments written in the macro invocation.
+        $(unsafe impl$($($generics)*)? ZeroableOption for $t {})*
+    };
+}
+
+impl_zeroable_option! {
+    // SAFETY: `Option<&T>` is part of the option layout optimization guarantee:
+    // <https://doc.rust-lang.org/stable/std/option/index.html#representation>.
+    {<T: ?Sized>} &T,
+    // SAFETY: `Option<&mut T>` is part of the option layout optimization guarantee:
+    // <https://doc.rust-lang.org/stable/std/option/index.html#representation>.
+    {<T: ?Sized>} &mut T,
+    // SAFETY: `Option<NonNull<T>>` is part of the option layout optimization guarantee:
+    // <https://doc.rust-lang.org/stable/std/option/index.html#representation>.
+    {<T: ?Sized>} NonNull<T>,
+    // SAFETY: All zeros is equivalent to `None` (option layout optimization guarantee:
+    // <https://doc.rust-lang.org/stable/std/option/index.html#representation>).
+    NonZero<u8>, NonZero<u16>, NonZero<u32>, NonZero<u64>, NonZero<u128>, NonZero<usize>,
+    NonZero<i8>, NonZero<i16>, NonZero<i32>, NonZero<i64>, NonZero<i128>, NonZero<isize>,
+}
+
+/// This trait allows creating an instance of `Self` which contains exactly one
+/// [structurally pinned value](https://doc.rust-lang.org/std/pin/index.html#projections-and-structural-pinning).
+///
+/// This is useful when using wrapper `struct`s like [`UnsafeCell`] or with new-type `struct`s.
+///
+/// # Examples
+///
+/// ```
+/// # use core::cell::UnsafeCell;
+/// # use pin_init::{pin_data, pin_init, Wrapper};
+///
+/// #[pin_data]
+/// struct Foo {}
+///
+/// #[pin_data]
+/// struct Bar {
+///     #[pin]
+///     content: UnsafeCell<Foo>
+/// };
+///
+/// let foo_initializer = pin_init!(Foo{});
+/// let initializer = pin_init!(Bar {
+///     content <- UnsafeCell::pin_init(foo_initializer)
+/// });
+/// ```
+pub trait Wrapper<T> {
+    /// Creates an pin-initializer for a [`Self`] containing `T` from the `value_init` initializer.
+    fn pin_init<E>(value_init: impl PinInit<T, E>) -> impl PinInit<Self, E>;
+}
+
+impl<T> Wrapper<T> for UnsafeCell<T> {
+    #[inline]
+    fn pin_init<E>(value_init: impl PinInit<T, E>) -> impl PinInit<Self, E> {
+        // SAFETY: `UnsafeCell<T>` has a compatible layout to `T`.
+        unsafe { cast_pin_init(value_init) }
+    }
+}
+
+impl<T> Wrapper<T> for MaybeUninit<T> {
+    #[inline]
+    fn pin_init<E>(value_init: impl PinInit<T, E>) -> impl PinInit<Self, E> {
+        // SAFETY: `MaybeUninit<T>` has a compatible layout to `T`.
+        unsafe { cast_pin_init(value_init) }
+    }
+}
+
+#[cfg(all(feature = "unsafe-pinned", CONFIG_RUSTC_HAS_UNSAFE_PINNED))]
+impl<T> Wrapper<T> for core::pin::UnsafePinned<T> {
+    #[inline]
+    fn pin_init<E>(init: impl PinInit<T, E>) -> impl PinInit<Self, E> {
+        // SAFETY: `UnsafePinned<T>` has a compatible layout to `T`.
+        unsafe { cast_pin_init(init) }
+    }
+}