Merge tag 'trace-v7.3-2' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-tracegrafted

Pull tracing fixes from Steven Rostedt:

 - Fix error output of boot instance creation failure

   Currently if a boot instance creation fails, instead of printing out
   the name of the instance that failed, it prints "(null)". That is
   because it prints "cur_str" that had already been processed by
   strsep(). Print the saved name instead.

   While at it, print the error code of the failure.

 - Fix use-after-free for same named historgrams

   Histograms can be named so that they can be used in multiple events.
   But if the named histogram has a variable attached, the second event
   that uses the named histogram which duplicates it and needs to free
   the original after duplication leaves the old variable in place and
   still visible. If another histogram uses than variable, it will use
   the stale one which will try to reference the freed duplicate
   histogram and crash the kernel.

   Free the duplicate variables along with the duplicated histogram
   data.

 - Check return value of kthread_run() in event self test

   The events self tests uses a kthread for testing but does not check
   if it succeeded in creating a kthread. If the kthread creation were
   to fail, the code will still try to call kthread_stop() on the error
   returned.

 - Fix race between reading trace_pipe and updating subbuffer size

   If a user is reading the trace_pipe file at the same time they update
   the ring buffer sub-buffer size, can cause the trace_pipe read to
   read stale data. Add trace_access_lock() around updating the ring
   buffer sub-buffer size.

 - Fix eventfs_inode on failure path in creation of the events directory

   In the creation of the "events" directory, if after allocating the
   eventfs_inode a failure is detected, it calls cleanup_ei() which
   calls free_ei(). The free_ei() will test if eventfs_inode being freed
   has no children. It is a bug if it does. But on the failure case of
   the creation of the "events" directory, the children lists have not
   yet been initialized and the free will trigger a warning because
   list_empty() on an uninitialized list returns false.

   Move the initialization into init_ei() where it makes more sense and
   makes sure that a created eventfs_inode has its lists initialized
   upon creation.

 - Check return value of kthread_run() in ftrace direct sample code

   The sample code that shows how to use the ftrace direct calls does
   not test the return of kthread_run() to see if it succeeds. Return a
   failure if the kthread_run() doesn't succeed.

 - Clear user events state on fork in case of alloc failure

   On fork, the child gets a pointer to the parent's user events state.
   It makes a copy of it then updates the child's pointer to it. But if
   the allocation fails, the duplication function leaves the child with
   a pointer to its parent's descriptor. When the child cleans up its
   data, it will free the parent's descriptor while the parent is still
   using it.

   In the duplication function, set the child's user_event_mm to NULL
   before testing if the allocation succeeded, and when it exits it will
   not free the parent's descriptor.

 - Fix retry exhaustion in simple ring buffer reader swap

   simple_ring_buffer_swap_reader_page() starts with retry set to 8 and
   post-decrements it only after a failed link replacement. On the final
   attempt, a successful replacement leaves retry at zero, while a
   failed replacement leaves it at -1.

   But the check for success expects the retry value to be non-zero and
   exits with an error on zero. This is the opposite result. Fix it.

 - Fail nicely when the remote swap_reader_page() returns an error

   Currently, if the swap_reader_page() of a remote buffer fails, it
   triggers a WARN_ON_ONCE() and continues normally. Instead, have it
   exit with an error and a pr_warn() print instead of a full WARNING.

* tag 'trace-v7.3-2' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace:
  ring-buffer: Stop remote reader update when page swap fails
  tracing: Fix retry exhaustion in simple ring buffer reader swap
  tracing/user_events: Clear copied tracing state before fork duplication
  samples/ftrace: Fix kthread_stop() on ERR_PTR in ftrace-direct-multi-modify
  samples/ftrace: Fix kthread_stop() on ERR_PTR in ftrace-direct-modify
  eventfs: Initialize ei->children and ei->list in init_ei()
  tracing: Fix use-after-free in trace_pipe read on sub-buffer order change
  tracing: Fix crash passing ERR_PTR to kthread_stop()
  tracing: Fix use-after-free with same-name named triggers
  tracing: Fix logged instance name on creation failure

17 files changed, 4715 insertions, 0 deletions

diff --git a/rust/pin-init/CONTRIBUTING.md b/rust/pin-init/CONTRIBUTING.md
new file mode 100644
index 000000000..16c899a7a
--- /dev/null
+++ b/rust/pin-init/CONTRIBUTING.md
@@ -0,0 +1,72 @@
+# Contributing to `pin-init`
+
+Thanks for showing interest in contributing to `pin-init`! This document outlines the guidelines for
+contributing to `pin-init`.
+
+All contributions are double-licensed under Apache 2.0 and MIT. You can find the respective licenses
+in the `LICENSE-APACHE` and `LICENSE-MIT` files.
+
+## Non-Code Contributions
+
+### Bug Reports
+
+For any type of bug report, please submit an issue using the bug report issue template.
+
+If the issue is a soundness issue, please privately report it as a security vulnerability via the
+GitHub web interface.
+
+### Feature Requests
+
+If you have any feature requests, please submit an issue using the feature request issue template.
+
+### Questions and Getting Help
+
+You can ask questions in the Discussions page of the GitHub repository. If you're encountering
+problems or just have questions related to `pin-init` in the Linux kernel, you can also ask your
+questions in the [Rust-for-Linux Zulip](https://rust-for-linux.zulipchat.com/) or see
+<https://rust-for-linux.com/contact>.
+
+## Contributing Code
+
+### Linux Kernel
+
+`pin-init` is used by the Linux kernel and all commits are synchronized to it. For this reason, the
+same requirements for commits apply to `pin-init`. See [the kernel's documentation] for details. The
+rest of this document will also cover some of the rules listed there and additional ones.
+
+[the kernel's documentation]: https://docs.kernel.org/process/submitting-patches.html
+
+Contributions to `pin-init` ideally go through the [GitHub repository], because that repository runs
+a CI with lots of tests not present in the kernel. However, patches are also accepted (though not
+preferred). Do note that there are some files that are only present in the GitHub repository such as
+tests, licenses and cargo related files. Making changes to them can only happen via GitHub.
+
+[GitHub repository]: https://github.com/Rust-for-Linux/pin-init
+
+### Commit Style
+
+Everything must compile without errors or warnings and all tests must pass after **every commit**.
+This is important for bisection and also required by the kernel.
+
+Each commit should be a single, logically cohesive change. Of course it's best to keep the changes
+small and digestible, but logically linked changes should be made in the same commit. For example,
+when fixing typos, create a single commit that fixes all of them instead of one commit per typo.
+
+Commits must have a meaningful commit title. Commits with changes to files in the `internal`
+directory should have a title prefixed with `internal:`. The commit message should explain the
+change and its rationale. You also have to add your `Signed-off-by` tag, see [Developer's
+Certificate of Origin]. This has to be done for both mailing list submissions as well as GitHub
+submissions.
+
+[Developer's Certificate of Origin]: https://docs.kernel.org/process/submitting-patches.html#sign-your-work-the-developer-s-certificate-of-origin
+
+Any changes made to public APIs must be documented not only in the commit message, but also in the
+`CHANGELOG.md` file. This is especially important for breaking changes, as those warrant a major
+version bump.
+
+If you make changes to the top-level crate documentation, you also need to update the `README.md`
+via `cargo rdme`.
+
+Some of these rules can be ignored if the change is done solely to files that are not present in the
+kernel version of this library. Those files are documented in the `sync-kernel.sh` script at the
+very bottom in the `--exclude` flag given to the `git am` command.
diff --git a/rust/pin-init/README.md b/rust/pin-init/README.md
new file mode 100644
index 000000000..2312c9e75
--- /dev/null
+++ b/rust/pin-init/README.md
@@ -0,0 +1,247 @@
+[![Crates.io](https://img.shields.io/crates/v/pin-init.svg)](https://crates.io/crates/pin-init)
+[![Documentation](https://docs.rs/pin-init/badge.svg)](https://docs.rs/pin-init/)
+[![Dependency status](https://deps.rs/repo/github/Rust-for-Linux/pin-init/status.svg)](https://deps.rs/repo/github/Rust-for-Linux/pin-init)
+![License](https://img.shields.io/crates/l/pin-init)
+[![Toolchain](https://img.shields.io/badge/toolchain-nightly-red)](#nightly-only)
+![GitHub Workflow Status](https://img.shields.io/github/actions/workflow/status/Rust-for-Linux/pin-init/ci.yml)
+# `pin-init`
+
+> [!NOTE]
+> 
+> This crate was originally named [`pinned-init`], but the migration to
+> `pin-init` is not yet complete. The `legacy` branch contains the current
+> version of the `pinned-init` crate & the `main` branch already incorporates
+> the rename to `pin-init`.
+>
+> There are still some changes needed on the kernel side before the migration
+> can be completed.
+
+[`pinned-init`]: https://crates.io/crates/pinned-init
+
+<!-- cargo-rdme start -->
+
+Library to safely and fallibly initialize pinned `struct`s using in-place constructors.
+
+[Pinning][pinning] is Rust's way of ensuring data does not move.
+
+It also allows in-place initialization of big `struct`s that would otherwise produce a stack
+overflow.
+
+This library's main use-case is in [Rust-for-Linux]. Although this version can be used
+standalone.
+
+There are cases when you want to in-place initialize a struct. For example when it is very big
+and moving it from the stack is not an option, because it is bigger than the stack itself.
+Another reason would be that you need the address of the object to initialize it. This stands
+in direct conflict with Rust's normal process of first initializing an object and then moving
+it into it's final memory location. For more information, see
+<https://rust-for-linux.com/the-safe-pinned-initialization-problem>.
+
+This library allows you to do in-place initialization safely.
+
+### Nightly Needed for `alloc` feature
+
+This library requires the [`allocator_api` unstable feature] when the `alloc` feature is
+enabled and thus this feature can only be used with a nightly compiler. When enabling the
+`alloc` feature, the user will be required to activate `allocator_api` as well.
+
+[`allocator_api` unstable feature]: https://doc.rust-lang.org/nightly/unstable-book/library-features/allocator-api.html
+
+The feature is enabled by default, thus by default `pin-init` will require a nightly compiler.
+However, using the crate on stable compilers is possible by disabling `alloc`. In practice this
+will require the `std` feature, because stable compilers have neither `Box` nor `Arc` in no-std
+mode.
+
+### Nightly needed for `unsafe-pinned` feature
+
+This feature enables the `Wrapper` implementation on the unstable `core::pin::UnsafePinned` type.
+This requires the [`unsafe_pinned` unstable feature](https://github.com/rust-lang/rust/issues/125735)
+and therefore a nightly compiler. Note that this feature is not enabled by default.
+
+## Overview
+
+To initialize a `struct` with an in-place constructor you will need two things:
+- an in-place constructor,
+- a memory location that can hold your `struct` (this can be the [stack], an [`Arc<T>`],
+  [`Box<T>`] or any other smart pointer that supports this library).
+
+To get an in-place constructor there are generally three options:
+- directly creating an in-place constructor using the [`pin_init!`] macro,
+- a custom function/macro returning an in-place constructor provided by someone else,
+- using the unsafe function [`pin_init_from_closure()`] to manually create an initializer.
+
+Aside from pinned initialization, this library also supports in-place construction without
+pinning, the macros/types/functions are generally named like the pinned variants without the
+`pin_` prefix.
+
+## Examples
+
+Throughout the examples we will often make use of the `CMutex` type which can be found in
+`../examples/mutex.rs`. It is essentially a userland rebuild of the `struct mutex` type from
+the Linux kernel. It also uses a wait list and a basic spinlock. Importantly the wait list
+requires it to be pinned to be locked and thus is a prime candidate for using this library.
+
+### Using the [`pin_init!`] macro
+
+If you want to use [`PinInit`], then you will have to annotate your `struct` with
+`#[`[`pin_data`]`]`. It is a macro that uses `#[pin]` as a marker for
+[structurally pinned fields]. After doing this, you can then create an in-place constructor via
+[`pin_init!`]. The syntax is almost the same as normal `struct` initializers. The difference is
+that you need to write `<-` instead of `:` for fields that you want to initialize in-place.
+
+```rust
+use pin_init::{pin_data, pin_init, InPlaceInit};
+
+#[pin_data]
+struct Foo {
+    #[pin]
+    a: CMutex<usize>,
+    b: u32,
+}
+
+let foo = pin_init!(Foo {
+    a <- CMutex::new(42),
+    b: 24,
+});
+```
+
+`foo` now is of the type [`impl PinInit<Foo>`]. We can now use any smart pointer that we like
+(or just the stack) to actually initialize a `Foo`:
+
+```rust
+let foo: Result<Pin<Box<Foo>>, AllocError> = Box::pin_init(foo);
+```
+
+For more information see the [`pin_init!`] macro.
+
+### Using a custom function/macro that returns an initializer
+
+Many types that use this library supply a function/macro that returns an initializer, because
+the above method only works for types where you can access the fields.
+
+```rust
+let mtx: Result<Pin<Arc<CMutex<usize>>>, _> = Arc::pin_init(CMutex::new(42));
+```
+
+To declare an init macro/function you just return an [`impl PinInit<T, E>`]:
+
+```rust
+#[pin_data]
+struct DriverData {
+    #[pin]
+    status: CMutex<i32>,
+    buffer: Box<[u8; 1_000_000]>,
+}
+
+impl DriverData {
+    fn new() -> impl PinInit<Self, Error> {
+        pin_init!(Self {
+            status <- CMutex::new(0),
+            buffer: Box::init(pin_init::init_zeroed())?,
+        }? Error)
+    }
+}
+```
+
+### Manual creation of an initializer
+
+Often when working with primitives the previous approaches are not sufficient. That is where
+[`pin_init_from_closure()`] comes in. This `unsafe` function allows you to create a
+[`impl PinInit<T, E>`] directly from a closure. Of course you have to ensure that the closure
+actually does the initialization in the correct way. Here are the things to look out for
+(we are calling the parameter to the closure `slot`):
+- when the closure returns `Ok(())`, then it has completed the initialization successfully, so
+  `slot` now contains a valid bit pattern for the type `T`,
+- when the closure returns `Err(e)`, then the caller may deallocate the memory at `slot`, so
+  you need to take care to clean up anything if your initialization fails mid-way,
+- you may assume that `slot` will stay pinned even after the closure returns until `drop` of
+  `slot` gets called.
+
+```rust
+use pin_init::{pin_data, pinned_drop, PinInit, PinnedDrop, pin_init_from_closure};
+use core::{
+    marker::PhantomPinned,
+    cell::UnsafeCell,
+    pin::Pin,
+    mem::MaybeUninit,
+};
+mod bindings {
+    #[repr(C)]
+    pub struct foo {
+        /* fields from C ... */
+    }
+    extern "C" {
+        pub fn init_foo(ptr: *mut foo);
+        pub fn destroy_foo(ptr: *mut foo);
+        #[must_use = "you must check the error return code"]
+        pub fn enable_foo(ptr: *mut foo, flags: u32) -> i32;
+    }
+}
+
+/// # Invariants
+///
+/// `foo` is always initialized
+#[pin_data(PinnedDrop)]
+pub struct RawFoo {
+    #[pin]
+    _p: PhantomPinned,
+    #[pin]
+    foo: UnsafeCell<MaybeUninit<bindings::foo>>,
+}
+
+impl RawFoo {
+    pub fn new(flags: u32) -> impl PinInit<Self, i32> {
+        // SAFETY:
+        // - when the closure returns `Ok(())`, then it has successfully initialized and
+        //   enabled `foo`,
+        // - when it returns `Err(e)`, then it has cleaned up before
+        unsafe {
+            pin_init_from_closure(move |slot: *mut Self| {
+                // `slot` contains uninit memory, avoid creating a reference.
+                let foo = &raw mut (*slot).foo;
+                let foo = UnsafeCell::raw_get(foo).cast::<bindings::foo>();
+
+                // Initialize the `foo`
+                bindings::init_foo(foo);
+
+                // Try to enable it.
+                let err = bindings::enable_foo(foo, flags);
+                if err != 0 {
+                    // Enabling has failed, first clean up the foo and then return the error.
+                    bindings::destroy_foo(foo);
+                    Err(err)
+                } else {
+                    // All fields of `RawFoo` have been initialized, since `_p` is a ZST.
+                    Ok(())
+                }
+            })
+        }
+    }
+}
+
+#[pinned_drop]
+impl PinnedDrop for RawFoo {
+    fn drop(self: Pin<&mut Self>) {
+        // SAFETY: Since `foo` is initialized, destroying is safe.
+        unsafe { bindings::destroy_foo(self.foo.get().cast::<bindings::foo>()) };
+    }
+}
+```
+
+For more information on how to use [`pin_init_from_closure()`], take a look at the uses inside
+the `kernel` crate. The [`sync`] module is a good starting point.
+
+[`sync`]: https://rust.docs.kernel.org/kernel/sync/index.html
+[pinning]: https://doc.rust-lang.org/std/pin/index.html
+[structurally pinned fields]: https://doc.rust-lang.org/std/pin/index.html#projections-and-structural-pinning
+[stack]: https://docs.rs/pin-init/latest/pin_init/macro.stack_pin_init.html
+[`impl PinInit<Foo>`]: https://docs.rs/pin-init/latest/pin_init/trait.PinInit.html
+[`impl PinInit<T, E>`]: https://docs.rs/pin-init/latest/pin_init/trait.PinInit.html
+[`impl Init<T, E>`]: https://docs.rs/pin-init/latest/pin_init/trait.Init.html
+[Rust-for-Linux]: https://rust-for-linux.com/
+
+<!-- cargo-rdme end -->
+
+<!-- These links are not picked up by cargo-rdme, since they are behind cfgs... -->
+[`Arc<T>`]: https://doc.rust-lang.org/stable/alloc/sync/struct.Arc.html
+[`Box<T>`]: https://doc.rust-lang.org/stable/alloc/boxed/struct.Box.html
diff --git a/rust/pin-init/examples/big_struct_in_place.rs b/rust/pin-init/examples/big_struct_in_place.rs
new file mode 100644
index 000000000..c05139927
--- /dev/null
+++ b/rust/pin-init/examples/big_struct_in_place.rs
@@ -0,0 +1,43 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
+use pin_init::*;
+
+// Struct with size over 1GiB
+#[derive(Debug)]
+#[allow(dead_code)]
+pub struct BigStruct {
+    buf: [u8; 1024 * 1024 * 1024],
+    a: u64,
+    b: u64,
+    c: u64,
+    d: u64,
+    managed_buf: ManagedBuf,
+}
+
+#[derive(Debug)]
+pub struct ManagedBuf {
+    buf: [u8; 1024 * 1024],
+}
+
+impl ManagedBuf {
+    pub fn new() -> impl Init<Self> {
+        init!(ManagedBuf { buf <- init_zeroed() })
+    }
+}
+
+fn main() {
+    #[cfg(any(feature = "std", feature = "alloc"))]
+    {
+        // we want to initialize the struct in-place, otherwise we would get a stackoverflow
+        let buf: Box<BigStruct> = Box::init(init!(BigStruct {
+            buf <- init_zeroed(),
+            a: 7,
+            b: 186,
+            c: 7789,
+            d: 34,
+            managed_buf <- ManagedBuf::new(),
+        }))
+        .unwrap();
+        println!("{}", core::mem::size_of_val(&*buf));
+    }
+}
diff --git a/rust/pin-init/examples/error.rs b/rust/pin-init/examples/error.rs
new file mode 100644
index 000000000..96f095398
--- /dev/null
+++ b/rust/pin-init/examples/error.rs
@@ -0,0 +1,31 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
+#![cfg_attr(feature = "alloc", feature(allocator_api))]
+
+use core::convert::Infallible;
+
+#[cfg(feature = "alloc")]
+use std::alloc::AllocError;
+
+#[derive(Debug)]
+pub struct Error;
+
+impl From<Infallible> for Error {
+    #[inline]
+    fn from(e: Infallible) -> Self {
+        match e {}
+    }
+}
+
+#[cfg(feature = "alloc")]
+impl From<AllocError> for Error {
+    #[inline]
+    fn from(_: AllocError) -> Self {
+        Self
+    }
+}
+
+#[allow(dead_code)]
+fn main() {
+    let _ = Error;
+}
diff --git a/rust/pin-init/examples/linked_list.rs b/rust/pin-init/examples/linked_list.rs
new file mode 100644
index 000000000..424585fe2
--- /dev/null
+++ b/rust/pin-init/examples/linked_list.rs
@@ -0,0 +1,168 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
+#![allow(clippy::undocumented_unsafe_blocks)]
+#![cfg_attr(feature = "alloc", feature(allocator_api))]
+
+use core::{
+    cell::Cell,
+    marker::PhantomPinned,
+    pin::Pin,
+    ptr::{self, NonNull},
+};
+
+use pin_init::*;
+
+#[allow(unused_attributes)]
+mod error;
+#[allow(unused_imports)]
+use error::Error;
+
+#[pin_data(PinnedDrop)]
+#[repr(C)]
+#[derive(Debug)]
+pub struct ListHead {
+    next: Link,
+    prev: Link,
+    #[pin]
+    pin: PhantomPinned,
+}
+
+impl ListHead {
+    #[inline]
+    pub fn new() -> impl PinInit<Self> {
+        pin_init!(&this in Self {
+            next: unsafe { Link::new_unchecked(this) },
+            prev: unsafe { Link::new_unchecked(this) },
+            pin: PhantomPinned,
+        })
+    }
+
+    #[inline]
+    #[allow(dead_code)]
+    pub fn insert_next(list: &ListHead) -> impl PinInit<Self> + '_ {
+        pin_init!(&this in Self {
+            prev: list.next.prev().replace(unsafe { Link::new_unchecked(this)}),
+            next: list.next.replace(unsafe { Link::new_unchecked(this)}),
+            pin: PhantomPinned,
+        })
+    }
+
+    #[inline]
+    pub fn insert_prev(list: &ListHead) -> impl PinInit<Self> + '_ {
+        pin_init!(&this in Self {
+            next: list.prev.next().replace(unsafe { Link::new_unchecked(this)}),
+            prev: list.prev.replace(unsafe { Link::new_unchecked(this)}),
+            pin: PhantomPinned,
+        })
+    }
+
+    #[inline]
+    pub fn next(&self) -> Option<NonNull<Self>> {
+        if ptr::eq(self.next.as_ptr(), self) {
+            None
+        } else {
+            Some(unsafe { NonNull::new_unchecked(self.next.as_ptr() as *mut Self) })
+        }
+    }
+
+    #[allow(dead_code)]
+    pub fn size(&self) -> usize {
+        let mut size = 1;
+        let mut cur = self.next.clone();
+        while !ptr::eq(self, cur.cur()) {
+            cur = cur.next().clone();
+            size += 1;
+        }
+        size
+    }
+}
+
+#[pinned_drop]
+impl PinnedDrop for ListHead {
+    //#[inline]
+    fn drop(self: Pin<&mut Self>) {
+        if !ptr::eq(self.next.as_ptr(), &*self) {
+            let next = unsafe { &*self.next.as_ptr() };
+            let prev = unsafe { &*self.prev.as_ptr() };
+            next.prev.set(&self.prev);
+            prev.next.set(&self.next);
+        }
+    }
+}
+
+#[repr(transparent)]
+#[derive(Clone, Debug)]
+struct Link(Cell<NonNull<ListHead>>);
+
+impl Link {
+    /// # Safety
+    ///
+    /// The contents of the pointer should form a consistent circular
+    /// linked list; for example, a "next" link should be pointed back
+    /// by the target `ListHead`'s "prev" link and a "prev" link should be
+    /// pointed back by the target `ListHead`'s "next" link.
+    #[inline]
+    unsafe fn new_unchecked(ptr: NonNull<ListHead>) -> Self {
+        Self(Cell::new(ptr))
+    }
+
+    #[inline]
+    fn next(&self) -> &Link {
+        unsafe { &(*self.0.get().as_ptr()).next }
+    }
+
+    #[inline]
+    #[allow(dead_code)]
+    fn prev(&self) -> &Link {
+        unsafe { &(*self.0.get().as_ptr()).prev }
+    }
+
+    #[allow(dead_code)]
+    fn cur(&self) -> &ListHead {
+        unsafe { &*self.0.get().as_ptr() }
+    }
+
+    #[inline]
+    fn replace(&self, other: Link) -> Link {
+        unsafe { Link::new_unchecked(self.0.replace(other.0.get())) }
+    }
+
+    #[inline]
+    fn as_ptr(&self) -> *const ListHead {
+        self.0.get().as_ptr()
+    }
+
+    #[inline]
+    fn set(&self, val: &Link) {
+        self.0.set(val.0.get());
+    }
+}
+
+#[allow(dead_code)]
+#[cfg(not(any(feature = "std", feature = "alloc")))]
+fn main() {}
+
+#[allow(dead_code)]
+#[cfg_attr(test, test)]
+#[cfg(any(feature = "std", feature = "alloc"))]
+fn main() -> Result<(), Error> {
+    let a = Box::pin_init(ListHead::new())?;
+    stack_pin_init!(let b = ListHead::insert_next(&a));
+    stack_pin_init!(let c = ListHead::insert_next(&a));
+    stack_pin_init!(let d = ListHead::insert_next(&b));
+    let e = Box::pin_init(ListHead::insert_next(&b))?;
+    println!("a ({a:p}): {a:?}");
+    println!("b ({b:p}): {b:?}");
+    println!("c ({c:p}): {c:?}");
+    println!("d ({d:p}): {d:?}");
+    println!("e ({e:p}): {e:?}");
+    let mut inspect = &*a;
+    while let Some(next) = inspect.next() {
+        println!("({inspect:p}): {inspect:?}");
+        inspect = unsafe { &*next.as_ptr() };
+        if core::ptr::eq(inspect, &*a) {
+            break;
+        }
+    }
+    Ok(())
+}
diff --git a/rust/pin-init/examples/mutex.rs b/rust/pin-init/examples/mutex.rs
new file mode 100644
index 000000000..e8d4dbb66
--- /dev/null
+++ b/rust/pin-init/examples/mutex.rs
@@ -0,0 +1,217 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
+#![allow(clippy::undocumented_unsafe_blocks)]
+#![cfg_attr(feature = "alloc", feature(allocator_api))]
+#![allow(clippy::missing_safety_doc)]
+
+use core::{
+    cell::{Cell, UnsafeCell},
+    marker::PhantomPinned,
+    ops::{Deref, DerefMut},
+    pin::Pin,
+    sync::atomic::{AtomicBool, Ordering},
+};
+#[cfg(feature = "std")]
+use std::{
+    sync::Arc,
+    thread::{self, sleep, Builder, Thread},
+    time::Duration,
+};
+
+use pin_init::*;
+#[allow(unused_attributes)]
+#[path = "./linked_list.rs"]
+pub mod linked_list;
+use linked_list::*;
+
+pub struct SpinLock {
+    inner: AtomicBool,
+}
+
+impl SpinLock {
+    #[inline]
+    pub fn acquire(&self) -> SpinLockGuard<'_> {
+        while self
+            .inner
+            .compare_exchange(false, true, Ordering::Acquire, Ordering::Relaxed)
+            .is_err()
+        {
+            #[cfg(feature = "std")]
+            while self.inner.load(Ordering::Relaxed) {
+                thread::yield_now();
+            }
+        }
+        SpinLockGuard(self)
+    }
+
+    #[inline]
+    #[allow(clippy::new_without_default)]
+    pub const fn new() -> Self {
+        Self {
+            inner: AtomicBool::new(false),
+        }
+    }
+}
+
+pub struct SpinLockGuard<'a>(&'a SpinLock);
+
+impl Drop for SpinLockGuard<'_> {
+    #[inline]
+    fn drop(&mut self) {
+        self.0.inner.store(false, Ordering::Release);
+    }
+}
+
+#[pin_data]
+pub struct CMutex<T> {
+    #[pin]
+    wait_list: ListHead,
+    spin_lock: SpinLock,
+    locked: Cell<bool>,
+    #[pin]
+    data: UnsafeCell<T>,
+}
+
+impl<T> CMutex<T> {
+    #[inline]
+    pub fn new(val: impl PinInit<T>) -> impl PinInit<Self> {
+        pin_init!(CMutex {
+            wait_list <- ListHead::new(),
+            spin_lock: SpinLock::new(),
+            locked: Cell::new(false),
+            data <- UnsafeCell::pin_init(val),
+        })
+    }
+
+    #[inline]
+    pub fn lock(&self) -> Pin<CMutexGuard<'_, T>> {
+        let mut sguard = self.spin_lock.acquire();
+        if self.locked.get() {
+            stack_pin_init!(let _wait_entry = WaitEntry::insert_new(&self.wait_list));
+            // println!("wait list length: {}", self.wait_list.size());
+            while self.locked.get() {
+                drop(sguard);
+                #[cfg(feature = "std")]
+                thread::park();
+                sguard = self.spin_lock.acquire();
+            }
+        }
+        self.locked.set(true);
+        unsafe {
+            Pin::new_unchecked(CMutexGuard {
+                mtx: self,
+                _pin: PhantomPinned,
+            })
+        }
+    }
+
+    #[allow(dead_code)]
+    pub fn get_data_mut(self: Pin<&mut Self>) -> &mut T {
+        // SAFETY: we have an exclusive reference and thus nobody has access to data.
+        unsafe { &mut *self.data.get() }
+    }
+}
+
+unsafe impl<T: Send> Send for CMutex<T> {}
+unsafe impl<T: Send> Sync for CMutex<T> {}
+
+pub struct CMutexGuard<'a, T> {
+    mtx: &'a CMutex<T>,
+    _pin: PhantomPinned,
+}
+
+impl<T> Drop for CMutexGuard<'_, T> {
+    #[inline]
+    fn drop(&mut self) {
+        let sguard = self.mtx.spin_lock.acquire();
+        self.mtx.locked.set(false);
+        if let Some(list_field) = self.mtx.wait_list.next() {
+            let _wait_entry = list_field.as_ptr().cast::<WaitEntry>();
+            #[cfg(feature = "std")]
+            unsafe {
+                (*_wait_entry).thread.unpark()
+            };
+        }
+        drop(sguard);
+    }
+}
+
+impl<T> Deref for CMutexGuard<'_, T> {
+    type Target = T;
+
+    #[inline]
+    fn deref(&self) -> &Self::Target {
+        unsafe { &*self.mtx.data.get() }
+    }
+}
+
+impl<T> DerefMut for CMutexGuard<'_, T> {
+    #[inline]
+    fn deref_mut(&mut self) -> &mut Self::Target {
+        unsafe { &mut *self.mtx.data.get() }
+    }
+}
+
+#[pin_data]
+#[repr(C)]
+struct WaitEntry {
+    #[pin]
+    wait_list: ListHead,
+    #[cfg(feature = "std")]
+    thread: Thread,
+}
+
+impl WaitEntry {
+    #[inline]
+    fn insert_new(list: &ListHead) -> impl PinInit<Self> + '_ {
+        #[cfg(feature = "std")]
+        {
+            pin_init!(Self {
+                thread: thread::current(),
+                wait_list <- ListHead::insert_prev(list),
+            })
+        }
+        #[cfg(not(feature = "std"))]
+        {
+            pin_init!(Self {
+                wait_list <- ListHead::insert_prev(list),
+            })
+        }
+    }
+}
+
+#[cfg_attr(test, test)]
+#[allow(dead_code)]
+fn main() {
+    #[cfg(feature = "std")]
+    {
+        let mtx: Pin<Arc<CMutex<usize>>> = Arc::pin_init(CMutex::new(0)).unwrap();
+        let mut handles = vec![];
+        let thread_count = 20;
+        let workload = if cfg!(miri) { 100 } else { 1_000 };
+        for i in 0..thread_count {
+            let mtx = mtx.clone();
+            handles.push(
+                Builder::new()
+                    .name(format!("worker #{i}"))
+                    .spawn(move || {
+                        for _ in 0..workload {
+                            *mtx.lock() += 1;
+                        }
+                        println!("{i} halfway");
+                        sleep(Duration::from_millis((i as u64) * 10));
+                        for _ in 0..workload {
+                            *mtx.lock() += 1;
+                        }
+                        println!("{i} finished");
+                    })
+                    .expect("should not fail"),
+            );
+        }
+        for h in handles {
+            h.join().expect("thread panicked");
+        }
+        println!("{:?}", *mtx.lock());
+        assert_eq!(*mtx.lock(), workload * thread_count * 2);
+    }
+}
diff --git a/rust/pin-init/examples/pthread_mutex.rs b/rust/pin-init/examples/pthread_mutex.rs
new file mode 100644
index 000000000..00f457e68
--- /dev/null
+++ b/rust/pin-init/examples/pthread_mutex.rs
@@ -0,0 +1,183 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
+// inspired by <https://github.com/nbdd0121/pin-init/blob/trunk/examples/pthread_mutex.rs>
+#![allow(clippy::undocumented_unsafe_blocks)]
+#![cfg_attr(feature = "alloc", feature(allocator_api))]
+
+#[cfg(not(windows))]
+mod pthread_mtx {
+    #[cfg(feature = "alloc")]
+    use core::alloc::AllocError;
+    use core::{
+        cell::UnsafeCell,
+        marker::PhantomPinned,
+        mem::MaybeUninit,
+        ops::{Deref, DerefMut},
+        pin::Pin,
+    };
+    use pin_init::*;
+    use std::convert::Infallible;
+
+    #[pin_data(PinnedDrop)]
+    pub struct PThreadMutex<T> {
+        #[pin]
+        raw: UnsafeCell<libc::pthread_mutex_t>,
+        data: UnsafeCell<T>,
+        #[pin]
+        pin: PhantomPinned,
+    }
+
+    unsafe impl<T: Send> Send for PThreadMutex<T> {}
+    unsafe impl<T: Send> Sync for PThreadMutex<T> {}
+
+    #[pinned_drop]
+    impl<T> PinnedDrop for PThreadMutex<T> {
+        fn drop(self: Pin<&mut Self>) {
+            unsafe {
+                libc::pthread_mutex_destroy(self.raw.get());
+            }
+        }
+    }
+
+    #[derive(Debug)]
+    pub enum Error {
+        #[allow(dead_code)]
+        IO(std::io::Error),
+        #[allow(dead_code)]
+        Alloc,
+    }
+
+    impl From<Infallible> for Error {
+        fn from(e: Infallible) -> Self {
+            match e {}
+        }
+    }
+
+    #[cfg(feature = "alloc")]
+    impl From<AllocError> for Error {
+        fn from(_: AllocError) -> Self {
+            Self::Alloc
+        }
+    }
+
+    impl<T> PThreadMutex<T> {
+        #[allow(dead_code)]
+        pub fn new(data: T) -> impl PinInit<Self, Error> {
+            fn init_raw() -> impl PinInit<UnsafeCell<libc::pthread_mutex_t>, Error> {
+                let init = |slot: *mut UnsafeCell<libc::pthread_mutex_t>| {
+                    // we can cast, because `UnsafeCell` has the same layout as T.
+                    let slot: *mut libc::pthread_mutex_t = slot.cast();
+                    let mut attr = MaybeUninit::uninit();
+                    let attr = attr.as_mut_ptr();
+                    // SAFETY: ptr is valid
+                    let ret = unsafe { libc::pthread_mutexattr_init(attr) };
+                    if ret != 0 {
+                        return Err(Error::IO(std::io::Error::from_raw_os_error(ret)));
+                    }
+                    // SAFETY: attr is initialized
+                    let ret = unsafe {
+                        libc::pthread_mutexattr_settype(attr, libc::PTHREAD_MUTEX_NORMAL)
+                    };
+                    if ret != 0 {
+                        // SAFETY: attr is initialized
+                        unsafe { libc::pthread_mutexattr_destroy(attr) };
+                        return Err(Error::IO(std::io::Error::from_raw_os_error(ret)));
+                    }
+                    // SAFETY: slot is valid
+                    unsafe { slot.write(libc::PTHREAD_MUTEX_INITIALIZER) };
+                    // SAFETY: attr and slot are valid ptrs and attr is initialized
+                    let ret = unsafe { libc::pthread_mutex_init(slot, attr) };
+                    // SAFETY: attr was initialized
+                    unsafe { libc::pthread_mutexattr_destroy(attr) };
+                    if ret != 0 {
+                        return Err(Error::IO(std::io::Error::from_raw_os_error(ret)));
+                    }
+                    Ok(())
+                };
+                // SAFETY: mutex has been initialized
+                unsafe { pin_init_from_closure(init) }
+            }
+            pin_init!(Self {
+                data: UnsafeCell::new(data),
+                raw <- init_raw(),
+                pin: PhantomPinned,
+            }? Error)
+        }
+
+        #[allow(dead_code)]
+        pub fn lock(&self) -> PThreadMutexGuard<'_, T> {
+            // SAFETY: raw is always initialized
+            unsafe { libc::pthread_mutex_lock(self.raw.get()) };
+            PThreadMutexGuard { mtx: self }
+        }
+    }
+
+    pub struct PThreadMutexGuard<'a, T> {
+        mtx: &'a PThreadMutex<T>,
+    }
+
+    impl<T> Drop for PThreadMutexGuard<'_, T> {
+        fn drop(&mut self) {
+            // SAFETY: raw is always initialized
+            unsafe { libc::pthread_mutex_unlock(self.mtx.raw.get()) };
+        }
+    }
+
+    impl<T> Deref for PThreadMutexGuard<'_, T> {
+        type Target = T;
+
+        fn deref(&self) -> &Self::Target {
+            unsafe { &*self.mtx.data.get() }
+        }
+    }
+
+    impl<T> DerefMut for PThreadMutexGuard<'_, T> {
+        fn deref_mut(&mut self) -> &mut Self::Target {
+            unsafe { &mut *self.mtx.data.get() }
+        }
+    }
+}
+
+#[cfg_attr(test, test)]
+#[cfg_attr(all(test, miri), ignore)]
+fn main() {
+    #[cfg(all(any(feature = "std", feature = "alloc"), not(windows)))]
+    {
+        use core::pin::Pin;
+        use pin_init::*;
+        use pthread_mtx::*;
+        use std::{
+            sync::Arc,
+            thread::{sleep, Builder},
+            time::Duration,
+        };
+        let mtx: Pin<Arc<PThreadMutex<usize>>> = Arc::try_pin_init(PThreadMutex::new(0)).unwrap();
+        let mut handles = vec![];
+        let thread_count = 20;
+        let workload = 1_000_000;
+        for i in 0..thread_count {
+            let mtx = mtx.clone();
+            handles.push(
+                Builder::new()
+                    .name(format!("worker #{i}"))
+                    .spawn(move || {
+                        for _ in 0..workload {
+                            *mtx.lock() += 1;
+                        }
+                        println!("{i} halfway");
+                        sleep(Duration::from_millis((i as u64) * 10));
+                        for _ in 0..workload {
+                            *mtx.lock() += 1;
+                        }
+                        println!("{i} finished");
+                    })
+                    .expect("should not fail"),
+            );
+        }
+        for h in handles {
+            h.join().expect("thread panicked");
+        }
+        println!("{:?}", *mtx.lock());
+        assert_eq!(*mtx.lock(), workload * thread_count * 2);
+    }
+}
diff --git a/rust/pin-init/examples/static_init.rs b/rust/pin-init/examples/static_init.rs
new file mode 100644
index 000000000..8dd52313c
--- /dev/null
+++ b/rust/pin-init/examples/static_init.rs
@@ -0,0 +1,121 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
+#![allow(clippy::undocumented_unsafe_blocks)]
+#![cfg_attr(feature = "alloc", feature(allocator_api))]
+#![allow(unused_imports)]
+
+use core::{
+    cell::{Cell, UnsafeCell},
+    mem::MaybeUninit,
+    ops,
+    pin::Pin,
+    time::Duration,
+};
+use pin_init::*;
+#[cfg(feature = "std")]
+use std::{
+    sync::Arc,
+    thread::{sleep, Builder},
+};
+
+#[allow(unused_attributes)]
+mod mutex;
+use mutex::*;
+
+pub struct StaticInit<T, I> {
+    cell: UnsafeCell<MaybeUninit<T>>,
+    init: Cell<Option<I>>,
+    lock: SpinLock,
+    present: Cell<bool>,
+}
+
+unsafe impl<T: Sync, I> Sync for StaticInit<T, I> {}
+unsafe impl<T: Send, I> Send for StaticInit<T, I> {}
+
+impl<T, I: PinInit<T>> StaticInit<T, I> {
+    pub const fn new(init: I) -> Self {
+        Self {
+            cell: UnsafeCell::new(MaybeUninit::uninit()),
+            init: Cell::new(Some(init)),
+            lock: SpinLock::new(),
+            present: Cell::new(false),
+        }
+    }
+}
+
+impl<T, I: PinInit<T>> ops::Deref for StaticInit<T, I> {
+    type Target = T;
+    fn deref(&self) -> &Self::Target {
+        if self.present.get() {
+            unsafe { (*self.cell.get()).assume_init_ref() }
+        } else {
+            println!("acquire spinlock on static init");
+            let _guard = self.lock.acquire();
+            println!("rechecking present...");
+            std::thread::sleep(std::time::Duration::from_millis(200));
+            if self.present.get() {
+                return unsafe { (*self.cell.get()).assume_init_ref() };
+            }
+            println!("doing init");
+            let ptr = self.cell.get().cast::<T>();
+            match self.init.take() {
+                Some(f) => unsafe { pin_init::raw_init(ptr, f) },
+                None => unsafe { core::hint::unreachable_unchecked() },
+            }
+            self.present.set(true);
+            unsafe { (*self.cell.get()).assume_init_ref() }
+        }
+    }
+}
+
+pub struct CountInit;
+
+unsafe impl PinInit<CMutex<usize>> for CountInit {
+    unsafe fn __init(self, slot: *mut CMutex<usize>) -> Result<(), core::convert::Infallible> {
+        let init = CMutex::new(0);
+        std::thread::sleep(std::time::Duration::from_millis(1000));
+        unsafe { pin_init::raw_init(slot, init) };
+        Ok(())
+    }
+}
+
+pub static COUNT: StaticInit<CMutex<usize>, CountInit> = StaticInit::new(CountInit);
+
+fn main() {
+    #[cfg(feature = "std")]
+    {
+        let mtx: Pin<Arc<CMutex<usize>>> = Arc::pin_init(CMutex::new(0)).unwrap();
+        let mut handles = vec![];
+        let thread_count = 20;
+        let workload = 1_000;
+        for i in 0..thread_count {
+            let mtx = mtx.clone();
+            handles.push(
+                Builder::new()
+                    .name(format!("worker #{i}"))
+                    .spawn(move || {
+                        for _ in 0..workload {
+                            *COUNT.lock() += 1;
+                            std::thread::sleep(std::time::Duration::from_millis(10));
+                            *mtx.lock() += 1;
+                            std::thread::sleep(std::time::Duration::from_millis(10));
+                            *COUNT.lock() += 1;
+                        }
+                        println!("{i} halfway");
+                        sleep(Duration::from_millis((i as u64) * 10));
+                        for _ in 0..workload {
+                            std::thread::sleep(std::time::Duration::from_millis(10));
+                            *mtx.lock() += 1;
+                        }
+                        println!("{i} finished");
+                    })
+                    .expect("should not fail"),
+            );
+        }
+        for h in handles {
+            h.join().expect("thread panicked");
+        }
+        println!("{:?}, {:?}", *mtx.lock(), *COUNT.lock());
+        assert_eq!(*mtx.lock(), workload * thread_count * 2);
+    }
+}
diff --git a/rust/pin-init/internal/src/diagnostics.rs b/rust/pin-init/internal/src/diagnostics.rs
new file mode 100644
index 000000000..c7d9b3e62
--- /dev/null
+++ b/rust/pin-init/internal/src/diagnostics.rs
@@ -0,0 +1,44 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
+use std::fmt::Display;
+
+use proc_macro2::TokenStream;
+use quote::quote_spanned;
+use syn::{spanned::Spanned, Error};
+
+pub(crate) struct DiagCtxt(TokenStream);
+pub(crate) struct ErrorGuaranteed(());
+
+impl DiagCtxt {
+    pub(crate) fn error(&mut self, span: impl Spanned, msg: impl Display) -> ErrorGuaranteed {
+        let error = Error::new(span.span(), msg);
+        self.0.extend(error.into_compile_error());
+        ErrorGuaranteed(())
+    }
+
+    pub(crate) fn warn(&mut self, span: impl Spanned, msg: impl Display) {
+        // Have the message start on a new line for visual clarity.
+        let msg = format!("\n{}", msg);
+        self.0.extend(quote_spanned!(span.span() =>
+            // Approximate using deprecated warning while `proc_macro_diagnostic` is unstable.
+            const _: () = {
+                #[deprecated = #msg]
+                const fn warn() {}
+                warn();
+            };
+        ));
+    }
+
+    pub(crate) fn with(
+        fun: impl FnOnce(&mut DiagCtxt) -> Result<TokenStream, ErrorGuaranteed>,
+    ) -> TokenStream {
+        let mut dcx = Self(TokenStream::new());
+        match fun(&mut dcx) {
+            Ok(mut stream) => {
+                stream.extend(dcx.0);
+                stream
+            }
+            Err(ErrorGuaranteed(())) => dcx.0,
+        }
+    }
+}
diff --git a/rust/pin-init/internal/src/init.rs b/rust/pin-init/internal/src/init.rs
new file mode 100644
index 000000000..fd0b5ea4a
--- /dev/null
+++ b/rust/pin-init/internal/src/init.rs
@@ -0,0 +1,482 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
+use proc_macro2::{Span, TokenStream};
+use quote::{format_ident, quote};
+use syn::{
+    braced,
+    parse::{End, Parse},
+    parse_quote,
+    punctuated::Punctuated,
+    spanned::Spanned,
+    token, Attribute, Block, Expr, ExprCall, ExprPath, Ident, Path, Token, Type,
+};
+
+use crate::diagnostics::{DiagCtxt, ErrorGuaranteed};
+
+pub(crate) struct Initializer {
+    attrs: Vec<InitializerAttribute>,
+    this: Option<This>,
+    path: Path,
+    brace_token: token::Brace,
+    fields: Punctuated<InitializerField, Token![,]>,
+    rest: Option<(Token![..], Expr)>,
+    error: Option<(Token![?], Type)>,
+}
+
+struct This {
+    _and_token: Token![&],
+    ident: Ident,
+    _in_token: Token![in],
+}
+
+struct InitializerField {
+    attrs: Vec<Attribute>,
+    kind: InitializerKind,
+}
+
+enum InitializerKind {
+    Value {
+        ident: Ident,
+        value: Option<(Token![:], Expr)>,
+    },
+    Init {
+        ident: Ident,
+        _left_arrow_token: Token![<-],
+        value: Expr,
+    },
+    Code {
+        _underscore_token: Token![_],
+        _colon_token: Token![:],
+        block: Block,
+    },
+}
+
+impl InitializerKind {
+    fn ident(&self) -> Option<&Ident> {
+        match self {
+            Self::Value { ident, .. } | Self::Init { ident, .. } => Some(ident),
+            Self::Code { .. } => None,
+        }
+    }
+}
+
+enum InitializerAttribute {
+    DefaultError(DefaultErrorAttribute),
+}
+
+struct DefaultErrorAttribute {
+    ty: Box<Type>,
+}
+
+pub(crate) fn expand(
+    Initializer {
+        attrs,
+        this,
+        path,
+        brace_token,
+        fields,
+        rest,
+        error,
+    }: Initializer,
+    default_error: Option<&'static str>,
+    pinned: bool,
+    dcx: &mut DiagCtxt,
+) -> Result<TokenStream, ErrorGuaranteed> {
+    let error = error.map_or_else(
+        || {
+            if let Some(default_error) = attrs.iter().fold(None, |acc, attr| {
+                #[expect(irrefutable_let_patterns)]
+                if let InitializerAttribute::DefaultError(DefaultErrorAttribute { ty }) = attr {
+                    Some(ty.clone())
+                } else {
+                    acc
+                }
+            }) {
+                default_error
+            } else if let Some(default_error) = default_error {
+                syn::parse_str(default_error).unwrap()
+            } else {
+                dcx.error(brace_token.span.close(), "expected `? <type>` after `}`");
+                parse_quote!(::core::convert::Infallible)
+            }
+        },
+        |(_, err)| Box::new(err),
+    );
+    let slot = format_ident!("slot");
+    let (has_data_trait, get_data, init_from_closure) = if pinned {
+        (
+            format_ident!("HasPinData"),
+            format_ident!("__pin_data"),
+            format_ident!("pin_init_from_closure"),
+        )
+    } else {
+        (
+            format_ident!("HasInitData"),
+            format_ident!("__init_data"),
+            format_ident!("init_from_closure"),
+        )
+    };
+    let init_kind = get_init_kind(rest, dcx);
+    let zeroable_check = match init_kind {
+        InitKind::Normal => quote!(),
+        InitKind::Zeroing => quote! {
+            // The user specified `..Zeroable::zeroed()` at the end of the list of fields.
+            // Therefore we check if the struct implements `Zeroable` and then zero the memory.
+            // This allows us to also remove the check that all fields are present (since we
+            // already set the memory to zero and that is a valid bit pattern).
+            fn assert_zeroable<T: ?::core::marker::Sized>(_: *mut T)
+            where T: ::pin_init::Zeroable
+            {}
+            // Ensure that the struct is indeed `Zeroable`.
+            assert_zeroable(#slot);
+            // SAFETY: The type implements `Zeroable` by the check above.
+            unsafe { ::core::ptr::write_bytes(#slot, 0, 1) };
+        },
+    };
+    let this = match this {
+        None => quote!(),
+        Some(This { ident, .. }) => quote! {
+            // Create the `this` so it can be referenced by the user inside of the
+            // expressions creating the individual fields.
+            let #ident = unsafe { ::core::ptr::NonNull::new_unchecked(slot) };
+        },
+    };
+    // `mixed_site` ensures that the data is not accessible to the user-controlled code.
+    let data = Ident::new("__data", Span::mixed_site());
+    let init_fields = init_fields(&fields, pinned, &data, &slot);
+    let field_check = make_field_check(&fields, init_kind, &path);
+    Ok(quote! {{
+        // Get the data about fields from the supplied type.
+        // SAFETY: TODO
+        let #data = unsafe {
+            use ::pin_init::__internal::#has_data_trait;
+            // Can't use `<#path as #has_data_trait>::#get_data`, since the user is able to omit
+            // generics (which need to be present with that syntax).
+            #path::#get_data()
+        };
+        // Ensure that `#data` really is of type `#data` and help with type inference:
+        let init = #data.__make_closure::<_, #error>(
+            move |slot| {
+                #zeroable_check
+                #this
+                #init_fields
+                #field_check
+                // SAFETY: we are the `init!` macro that is allowed to call this.
+                Ok(unsafe { ::pin_init::__internal::InitOk::new() })
+            }
+        );
+        let init = move |slot| -> ::core::result::Result<(), #error> {
+            init(slot).map(|__InitOk| ())
+        };
+        // SAFETY: TODO
+        unsafe { ::pin_init::#init_from_closure::<_, #error>(init) }
+    }})
+}
+
+enum InitKind {
+    Normal,
+    Zeroing,
+}
+
+fn get_init_kind(rest: Option<(Token![..], Expr)>, dcx: &mut DiagCtxt) -> InitKind {
+    let Some((dotdot, expr)) = rest else {
+        return InitKind::Normal;
+    };
+    match &expr {
+        Expr::Call(ExprCall { func, args, .. }) if args.is_empty() => match &**func {
+            Expr::Path(ExprPath {
+                attrs,
+                qself: None,
+                path:
+                    Path {
+                        leading_colon: None,
+                        segments,
+                    },
+            }) if attrs.is_empty()
+                && segments.len() == 2
+                && segments[0].ident == "Zeroable"
+                && segments[0].arguments.is_none()
+                && segments[1].ident == "init_zeroed"
+                && segments[1].arguments.is_none() =>
+            {
+                return InitKind::Zeroing;
+            }
+            _ => {}
+        },
+        _ => {}
+    }
+    dcx.error(
+        dotdot.span().join(expr.span()).unwrap_or(expr.span()),
+        "expected nothing or `..Zeroable::init_zeroed()`.",
+    );
+    InitKind::Normal
+}
+
+/// Generate the code that initializes the fields of the struct using the initializers in `field`.
+fn init_fields(
+    fields: &Punctuated<InitializerField, Token![,]>,
+    pinned: bool,
+    data: &Ident,
+    slot: &Ident,
+) -> TokenStream {
+    let mut guards = vec![];
+    let mut guard_attrs = vec![];
+    let mut res = TokenStream::new();
+    for InitializerField { attrs, kind } in fields {
+        let cfgs = {
+            let mut cfgs = attrs.clone();
+            cfgs.retain(|attr| attr.path().is_ident("cfg"));
+            cfgs
+        };
+
+        let ident = match kind {
+            InitializerKind::Value { ident, .. } => ident,
+            InitializerKind::Init { ident, .. } => ident,
+            InitializerKind::Code { block, .. } => {
+                let stmt = &block.stmts;
+                res.extend(quote! {
+                    #(#attrs)*
+                    {
+                        #(#stmt)*
+                    }
+                });
+                continue;
+            }
+        };
+
+        let slot = if pinned {
+            quote! {
+                // SAFETY:
+                // - `slot` is valid and properly aligned.
+                // - `make_field_check` checks that `&raw mut (*slot).#ident` is properly aligned.
+                // - `make_field_check` prevents `#ident` from being used twice, therefore
+                //   `(*slot).#ident` is exclusively accessed and has not been initialized.
+                (unsafe { #data.#ident(#slot) })
+            }
+        } else {
+            quote! {
+                // For `init!()` macro, everything is unpinned.
+                // SAFETY:
+                // - `&raw mut (*slot).#ident` is valid.
+                // - `make_field_check` checks that `&raw mut (*slot).#ident` is properly aligned.
+                // - `make_field_check` prevents `#ident` from being used twice, therefore
+                //   `(*slot).#ident` is exclusively accessed and has not been initialized.
+                (unsafe {
+                    ::pin_init::__internal::Slot::<::pin_init::__internal::Unpinned, _>::new(
+                        &raw mut (*#slot).#ident
+                    )
+                })
+            }
+        };
+
+        // `mixed_site` ensures that the guard is not accessible to the user-controlled code.
+        let guard = format_ident!("__{ident}_guard", span = Span::mixed_site());
+
+        let init = match kind {
+            InitializerKind::Value { ident, value } => {
+                let value = value
+                    .as_ref()
+                    .map(|(_, value)| quote!(#value))
+                    .unwrap_or_else(|| quote!(#ident));
+
+                quote! {
+                    #(#attrs)*
+                    let mut #guard = #slot.write(#value);
+
+                }
+            }
+            InitializerKind::Init { value, .. } => {
+                quote! {
+                    #(#attrs)*
+                    let mut #guard = #slot.init(#value)?;
+                }
+            }
+            InitializerKind::Code { .. } => unreachable!(),
+        };
+
+        res.extend(quote! {
+            #init
+
+            #(#cfgs)*
+            // Allow `non_snake_case` since the same warning is going to be reported for the struct
+            // field.
+            #[allow(unused_variables, non_snake_case)]
+            let #ident = #guard.let_binding();
+        });
+
+        guards.push(guard);
+        guard_attrs.push(cfgs);
+    }
+    quote! {
+        #res
+        // If execution reaches this point, all fields have been initialized. Therefore we can now
+        // dismiss the guards by forgetting them.
+        #(
+            #(#guard_attrs)*
+            ::core::mem::forget(#guards);
+        )*
+    }
+}
+
+/// Generate the check for ensuring that every field has been initialized and aligned.
+fn make_field_check(
+    fields: &Punctuated<InitializerField, Token![,]>,
+    init_kind: InitKind,
+    path: &Path,
+) -> TokenStream {
+    let field_attrs: Vec<_> = fields
+        .iter()
+        .filter_map(|f| f.kind.ident().map(|_| &f.attrs))
+        .collect();
+    let field_name: Vec<_> = fields.iter().filter_map(|f| f.kind.ident()).collect();
+    let zeroing_trailer = match init_kind {
+        InitKind::Normal => None,
+        InitKind::Zeroing => Some(quote! {
+            ..::core::mem::zeroed()
+        }),
+    };
+    quote! {
+        #[allow(unreachable_code)]
+        // We use unreachable code to perform field checks. They're still checked by the compiler.
+        // SAFETY: this code is never executed.
+        let _ = || unsafe {
+            // Create references to ensure that the initialized field is properly aligned.
+            // Unaligned fields will cause the compiler to emit E0793. We do not support
+            // unaligned fields since `Init::__init` requires an aligned pointer; the call to
+            // `ptr::write` for value-initialization case has the same requirement.
+            #(
+                #(#field_attrs)*
+                let _ = &(*slot).#field_name;
+            )*
+
+            // If the zeroing trailer is not present, this checks that all fields have been
+            // mentioned exactly once. If the zeroing trailer is present, all missing fields will be
+            // zeroed, so this checks that all fields have been mentioned at most once. The use of
+            // struct initializer will still generate very natural error messages for any misuse.
+            ::core::ptr::write(slot, #path {
+                #(
+                    #(#field_attrs)*
+                    #field_name: loop {},
+                )*
+                #zeroing_trailer
+            })
+        };
+    }
+}
+
+impl Parse for Initializer {
+    fn parse(input: syn::parse::ParseStream<'_>) -> syn::Result<Self> {
+        let attrs = input.call(Attribute::parse_outer)?;
+        let this = input.peek(Token![&]).then(|| input.parse()).transpose()?;
+        let path = input.parse()?;
+        let content;
+        let brace_token = braced!(content in input);
+        let mut fields = Punctuated::new();
+        loop {
+            let lh = content.lookahead1();
+            if lh.peek(End) || lh.peek(Token![..]) {
+                break;
+            } else if lh.peek(Ident) || lh.peek(Token![_]) || lh.peek(Token![#]) {
+                fields.push_value(content.parse()?);
+                let lh = content.lookahead1();
+                if lh.peek(End) {
+                    break;
+                } else if lh.peek(Token![,]) {
+                    fields.push_punct(content.parse()?);
+                } else {
+                    return Err(lh.error());
+                }
+            } else {
+                return Err(lh.error());
+            }
+        }
+        let rest = content
+            .peek(Token![..])
+            .then(|| Ok::<_, syn::Error>((content.parse()?, content.parse()?)))
+            .transpose()?;
+        let error = input
+            .peek(Token![?])
+            .then(|| Ok::<_, syn::Error>((input.parse()?, input.parse()?)))
+            .transpose()?;
+        let attrs = attrs
+            .into_iter()
+            .map(|a| {
+                if a.path().is_ident("default_error") {
+                    a.parse_args::<DefaultErrorAttribute>()
+                        .map(InitializerAttribute::DefaultError)
+                } else {
+                    Err(syn::Error::new_spanned(a, "unknown initializer attribute"))
+                }
+            })
+            .collect::<Result<Vec<_>, _>>()?;
+        Ok(Self {
+            attrs,
+            this,
+            path,
+            brace_token,
+            fields,
+            rest,
+            error,
+        })
+    }
+}
+
+impl Parse for DefaultErrorAttribute {
+    fn parse(input: syn::parse::ParseStream<'_>) -> syn::Result<Self> {
+        Ok(Self { ty: input.parse()? })
+    }
+}
+
+impl Parse for This {
+    fn parse(input: syn::parse::ParseStream<'_>) -> syn::Result<Self> {
+        Ok(Self {
+            _and_token: input.parse()?,
+            ident: input.parse()?,
+            _in_token: input.parse()?,
+        })
+    }
+}
+
+impl Parse for InitializerField {
+    fn parse(input: syn::parse::ParseStream<'_>) -> syn::Result<Self> {
+        let attrs = input.call(Attribute::parse_outer)?;
+        Ok(Self {
+            attrs,
+            kind: input.parse()?,
+        })
+    }
+}
+
+impl Parse for InitializerKind {
+    fn parse(input: syn::parse::ParseStream<'_>) -> syn::Result<Self> {
+        let lh = input.lookahead1();
+        if lh.peek(Token![_]) {
+            Ok(Self::Code {
+                _underscore_token: input.parse()?,
+                _colon_token: input.parse()?,
+                block: input.parse()?,
+            })
+        } else if lh.peek(Ident) {
+            let ident = input.parse()?;
+            let lh = input.lookahead1();
+            if lh.peek(Token![<-]) {
+                Ok(Self::Init {
+                    ident,
+                    _left_arrow_token: input.parse()?,
+                    value: input.parse()?,
+                })
+            } else if lh.peek(Token![:]) {
+                Ok(Self::Value {
+                    ident,
+                    value: Some((input.parse()?, input.parse()?)),
+                })
+            } else if lh.peek(Token![,]) || lh.peek(End) {
+                Ok(Self::Value { ident, value: None })
+            } else {
+                Err(lh.error())
+            }
+        } else {
+            Err(lh.error())
+        }
+    }
+}
diff --git a/rust/pin-init/internal/src/lib.rs b/rust/pin-init/internal/src/lib.rs
new file mode 100644
index 000000000..60d5093f3
--- /dev/null
+++ b/rust/pin-init/internal/src/lib.rs
@@ -0,0 +1,59 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
+// When fixdep scans this, it will find this string `CONFIG_RUSTC_VERSION_TEXT`
+// and thus add a dependency on `include/config/RUSTC_VERSION_TEXT`, which is
+// touched by Kconfig when the version string from the compiler changes.
+
+//! `pin-init` proc macros.
+
+// Documentation is done in the pin-init crate instead.
+#![allow(missing_docs)]
+
+use proc_macro::TokenStream;
+use syn::parse_macro_input;
+
+use crate::diagnostics::DiagCtxt;
+
+mod diagnostics;
+mod init;
+mod pin_data;
+mod pinned_drop;
+mod zeroable;
+
+#[proc_macro_attribute]
+pub fn pin_data(args: TokenStream, input: TokenStream) -> TokenStream {
+    let args = parse_macro_input!(args);
+    let input = parse_macro_input!(input);
+    DiagCtxt::with(|dcx| pin_data::pin_data(args, input, dcx)).into()
+}
+
+#[proc_macro_attribute]
+pub fn pinned_drop(args: TokenStream, input: TokenStream) -> TokenStream {
+    let args = parse_macro_input!(args);
+    let input = parse_macro_input!(input);
+    DiagCtxt::with(|dcx| pinned_drop::pinned_drop(args, input, dcx)).into()
+}
+
+#[proc_macro_derive(Zeroable)]
+pub fn derive_zeroable(input: TokenStream) -> TokenStream {
+    let input = parse_macro_input!(input);
+    DiagCtxt::with(|dcx| zeroable::derive(input, dcx)).into()
+}
+
+#[proc_macro_derive(MaybeZeroable)]
+pub fn maybe_derive_zeroable(input: TokenStream) -> TokenStream {
+    let input = parse_macro_input!(input);
+    DiagCtxt::with(|dcx| zeroable::maybe_derive(input, dcx)).into()
+}
+#[proc_macro]
+pub fn init(input: TokenStream) -> TokenStream {
+    let input = parse_macro_input!(input);
+    DiagCtxt::with(|dcx| init::expand(input, Some("::core::convert::Infallible"), false, dcx))
+        .into()
+}
+
+#[proc_macro]
+pub fn pin_init(input: TokenStream) -> TokenStream {
+    let input = parse_macro_input!(input);
+    DiagCtxt::with(|dcx| init::expand(input, Some("::core::convert::Infallible"), true, dcx)).into()
+}
diff --git a/rust/pin-init/internal/src/pin_data.rs b/rust/pin-init/internal/src/pin_data.rs
new file mode 100644
index 000000000..ff194d275
--- /dev/null
+++ b/rust/pin-init/internal/src/pin_data.rs
@@ -0,0 +1,537 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
+use proc_macro2::TokenStream;
+use quote::{format_ident, quote, ToTokens};
+use syn::{
+    parse::{End, Nothing, Parse},
+    parse_quote, parse_quote_spanned,
+    spanned::Spanned,
+    visit_mut::VisitMut,
+    Field, Fields, Generics, Ident, Item, PathSegment, Type, TypePath, Visibility, WhereClause,
+};
+
+use crate::diagnostics::{DiagCtxt, ErrorGuaranteed};
+
+pub(crate) mod kw {
+    syn::custom_keyword!(PinnedDrop);
+}
+
+pub(crate) enum Args {
+    Nothing(Nothing),
+    #[allow(dead_code)]
+    PinnedDrop(kw::PinnedDrop),
+}
+
+impl Parse for Args {
+    fn parse(input: syn::parse::ParseStream<'_>) -> syn::Result<Self> {
+        let lh = input.lookahead1();
+        if lh.peek(End) {
+            input.parse().map(Self::Nothing)
+        } else if lh.peek(kw::PinnedDrop) {
+            input.parse().map(Self::PinnedDrop)
+        } else {
+            Err(lh.error())
+        }
+    }
+}
+
+impl ToTokens for Args {
+    fn to_tokens(&self, tokens: &mut TokenStream) {
+        match self {
+            Self::Nothing(_) => (),
+            Self::PinnedDrop(kw) => kw.to_tokens(tokens),
+        }
+    }
+}
+
+struct FieldInfo<'a> {
+    field: &'a Field,
+    pinned: bool,
+}
+
+pub(crate) fn pin_data(
+    args: Args,
+    input: Item,
+    dcx: &mut DiagCtxt,
+) -> Result<TokenStream, ErrorGuaranteed> {
+    let mut struct_ = match input {
+        Item::Struct(struct_) => struct_,
+        Item::Enum(enum_) => {
+            return Err(dcx.error(
+                enum_.enum_token,
+                "`#[pin_data]` only supports structs for now",
+            ));
+        }
+        Item::Union(union) => {
+            return Err(dcx.error(
+                union.union_token,
+                "`#[pin_data]` only supports structs for now",
+            ));
+        }
+        rest => {
+            return Err(dcx.error(
+                rest,
+                "`#[pin_data]` can only be applied to struct, enum and union definitions",
+            ));
+        }
+    };
+
+    // Handling cfg can gets very complicated, especially for tuple structs. Therefore, resolve all
+    // field cfgs first before continuing.
+    //
+    // We need to perform this after parsing so we can reliably detect field cfgs.
+    for (field_idx, field) in struct_.fields.iter_mut().enumerate() {
+        let cfg: Vec<_> = field
+            .attrs
+            .iter()
+            .filter(|a| a.path().is_ident("cfg"))
+            .map(|a| {
+                a.parse_args::<TokenStream>()
+                    .expect("parse as token stream cannot fail")
+            })
+            .collect();
+
+        if cfg.is_empty() {
+            continue;
+        }
+
+        field.attrs.retain(|a| !a.path().is_ident("cfg"));
+        let cfg_true_struct = quote!(#struct_);
+
+        let punctuated = match &mut struct_.fields {
+            Fields::Named(fields) => &mut fields.named,
+            Fields::Unnamed(fields) => &mut fields.unnamed,
+            Fields::Unit => unreachable!(),
+        };
+        *punctuated = std::mem::take(punctuated)
+            .into_pairs()
+            .enumerate()
+            .filter(|&(i, _)| i != field_idx)
+            .map(|(_, p)| p)
+            .collect();
+        let cfg_false_struct = quote!(#struct_);
+
+        // Resolve one field at a time until we've got no more field cfgs.
+        //
+        // This is linear time because macro invocations with false cfg will not be expanded.
+        return Ok(quote!(
+            #[cfg(all(#(#cfg,)*))]
+            #[::pin_init::pin_data(#args)]
+            #cfg_true_struct
+
+            #[cfg(not(all(#(#cfg,)*)))]
+            #[::pin_init::pin_data(#args)]
+            #cfg_false_struct
+        ));
+    }
+
+    // The generics might contain the `Self` type. Since this macro will define a new type with the
+    // same generics and bounds, this poses a problem: `Self` will refer to the new type as opposed
+    // to this struct definition. Therefore we have to replace `Self` with the concrete name.
+    let mut replacer = {
+        let name = &struct_.ident;
+        let (_, ty_generics, _) = struct_.generics.split_for_impl();
+        SelfReplacer(parse_quote!(#name #ty_generics))
+    };
+    replacer.visit_generics_mut(&mut struct_.generics);
+    replacer.visit_fields_mut(&mut struct_.fields);
+
+    let fields: Vec<FieldInfo<'_>> = struct_
+        .fields
+        .iter_mut()
+        .map(|field| {
+            let len = field.attrs.len();
+            field.attrs.retain(|a| !a.path().is_ident("pin"));
+            let pinned_count = len - field.attrs.len();
+            if pinned_count > 1 {
+                dcx.error(&field, "#[pin] attribute specified more than once");
+            }
+
+            assert!(
+                !field.attrs.iter().any(|a| a.path().is_ident("cfg")),
+                "cfgs should be all resolved at this point"
+            );
+
+            FieldInfo {
+                field: &*field,
+                pinned: pinned_count != 0,
+            }
+        })
+        .collect();
+
+    for field in &fields {
+        let ident = field.field.ident.as_ref().unwrap();
+
+        if !field.pinned && is_phantom_pinned(&field.field.ty) {
+            dcx.warn(
+                field.field,
+                format!(
+                    "The field `{ident}` of type `PhantomPinned` only has an effect \
+                    if it has the `#[pin]` attribute",
+                ),
+            );
+        }
+    }
+
+    let unpin_impl = generate_unpin_impl(&struct_.ident, &struct_.generics, &fields);
+    let drop_impl = generate_drop_impl(&struct_.ident, &struct_.generics, args);
+    let projections =
+        generate_projections(&struct_.vis, &struct_.ident, &struct_.generics, &fields);
+    let the_pin_data =
+        generate_the_pin_data(&struct_.vis, &struct_.ident, &struct_.generics, &fields);
+
+    Ok(quote! {
+        #struct_
+        #projections
+        // We put the rest into this const item, because it then will not be accessible to anything
+        // outside.
+        const _: () = {
+            #the_pin_data
+            #unpin_impl
+            #drop_impl
+        };
+    })
+}
+
+fn is_phantom_pinned(ty: &Type) -> bool {
+    match ty {
+        Type::Path(TypePath { qself: None, path }) => {
+            // Cannot possibly refer to `PhantomPinned` (except alias, but that's on the user).
+            if path.segments.len() > 3 {
+                return false;
+            }
+            // If there is a `::`, then the path needs to be `::core::marker::PhantomPinned` or
+            // `::std::marker::PhantomPinned`.
+            if path.leading_colon.is_some() && path.segments.len() != 3 {
+                return false;
+            }
+            let expected: Vec<&[&str]> = vec![&["PhantomPinned"], &["marker"], &["core", "std"]];
+            for (actual, expected) in path.segments.iter().rev().zip(expected) {
+                if !actual.arguments.is_empty() || expected.iter().all(|e| actual.ident != e) {
+                    return false;
+                }
+            }
+            true
+        }
+        _ => false,
+    }
+}
+
+fn generate_unpin_impl(
+    ident: &Ident,
+    generics: &Generics,
+    fields: &[FieldInfo<'_>],
+) -> TokenStream {
+    let (_, ty_generics, _) = generics.split_for_impl();
+    let mut generics_with_pin_lt = generics.clone();
+    generics_with_pin_lt.params.insert(0, parse_quote!('__pin));
+    generics_with_pin_lt.make_where_clause();
+    let (
+        impl_generics_with_pin_lt,
+        ty_generics_with_pin_lt,
+        Some(WhereClause {
+            where_token,
+            predicates,
+        }),
+    ) = generics_with_pin_lt.split_for_impl()
+    else {
+        unreachable!()
+    };
+    let pinned_fields = fields.iter().filter(|f| f.pinned).map(|f| {
+        let ident = f.field.ident.as_ref().unwrap();
+        let ty = &f.field.ty;
+        quote!(
+            #ident: #ty
+        )
+    });
+    quote! {
+        // This struct will be used for the unpin analysis. It is needed, because only structurally
+        // pinned fields are relevant whether the struct should implement `Unpin`.
+        #[allow(
+            dead_code, // The fields below are never used.
+            non_snake_case // The warning will be emitted on the struct definition.
+        )]
+        struct __Unpin #generics_with_pin_lt
+        #where_token
+            #predicates
+        {
+            __phantom_pin: ::pin_init::__internal::PhantomInvariantLifetime<'__pin>,
+            __phantom: ::pin_init::__internal::PhantomInvariant<#ident #ty_generics>,
+            #(#pinned_fields),*
+        }
+
+        #[doc(hidden)]
+        impl #impl_generics_with_pin_lt ::core::marker::Unpin for #ident #ty_generics
+        #where_token
+            __Unpin #ty_generics_with_pin_lt: ::core::marker::Unpin,
+            #predicates
+        {}
+    }
+}
+
+fn generate_drop_impl(ident: &Ident, generics: &Generics, args: Args) -> TokenStream {
+    let (impl_generics, ty_generics, whr) = generics.split_for_impl();
+    let has_pinned_drop = matches!(args, Args::PinnedDrop(_));
+    // We need to disallow normal `Drop` implementation, the exact behavior depends on whether
+    // `PinnedDrop` was specified in `args`.
+    if has_pinned_drop {
+        // When `PinnedDrop` was specified we just implement `Drop` and delegate.
+        quote! {
+            impl #impl_generics ::core::ops::Drop for #ident #ty_generics
+                #whr
+            {
+                fn drop(&mut self) {
+                    // SAFETY: Since this is a destructor, `self` will not move after this function
+                    // terminates, since it is inaccessible.
+                    let pinned = unsafe { ::core::pin::Pin::new_unchecked(self) };
+                    // SAFETY: Since this is a drop function, we can create this token to call the
+                    // pinned destructor of this type.
+                    let token = unsafe { ::pin_init::__internal::OnlyCallFromDrop::new() };
+                    ::pin_init::PinnedDrop::drop(pinned, token);
+                }
+            }
+        }
+    } else {
+        // When no `PinnedDrop` was specified, then we have to prevent implementing drop.
+        quote! {
+            // We prevent this by creating a trait that will be implemented for all types implementing
+            // `Drop`. Additionally we will implement this trait for the struct leading to a conflict,
+            // if it also implements `Drop`
+            trait MustNotImplDrop {}
+            impl<T: ::core::ops::Drop + ?::core::marker::Sized> MustNotImplDrop for T {}
+            impl #impl_generics MustNotImplDrop for #ident #ty_generics
+                #whr
+            {}
+            // We also take care to prevent users from writing a useless `PinnedDrop` implementation.
+            // They might implement `PinnedDrop` correctly for the struct, but forget to give
+            // `PinnedDrop` as the parameter to `#[pin_data]`.
+            trait UselessPinnedDropImpl_you_need_to_specify_PinnedDrop {}
+            impl<T: ::pin_init::PinnedDrop + ?::core::marker::Sized>
+                UselessPinnedDropImpl_you_need_to_specify_PinnedDrop for T {}
+            impl #impl_generics
+                UselessPinnedDropImpl_you_need_to_specify_PinnedDrop for #ident #ty_generics
+                #whr
+            {}
+        }
+    }
+}
+
+fn generate_projections(
+    vis: &Visibility,
+    ident: &Ident,
+    generics: &Generics,
+    fields: &[FieldInfo<'_>],
+) -> TokenStream {
+    let (impl_generics, ty_generics, _) = generics.split_for_impl();
+    let mut generics_with_pin_lt = generics.clone();
+    generics_with_pin_lt.params.insert(0, parse_quote!('__pin));
+    let (_, ty_generics_with_pin_lt, whr) = generics_with_pin_lt.split_for_impl();
+    let projection = format_ident!("{ident}Projection");
+    let this = format_ident!("this");
+
+    let (fields_decl, fields_proj): (Vec<_>, Vec<_>) = fields
+        .iter()
+        .map(|field| {
+            let Field { vis, ident, ty, .. } = &field.field;
+
+            let ident = ident
+                .as_ref()
+                .expect("only structs with named fields are supported");
+            if field.pinned {
+                (
+                    quote!(
+                        #vis #ident: ::core::pin::Pin<&'__pin mut #ty>,
+                    ),
+                    quote!(
+                        // SAFETY: this field is structurally pinned.
+                        #ident: unsafe { ::core::pin::Pin::new_unchecked(&mut #this.#ident) },
+                    ),
+                )
+            } else {
+                (
+                    quote!(
+                        #vis #ident: &'__pin mut #ty,
+                    ),
+                    quote!(
+                        #ident: &mut #this.#ident,
+                    ),
+                )
+            }
+        })
+        .collect();
+    let structurally_pinned_fields_docs = fields
+        .iter()
+        .filter(|f| f.pinned)
+        .map(|f| format!(" - `{}`", f.field.ident.as_ref().unwrap()));
+    let not_structurally_pinned_fields_docs = fields
+        .iter()
+        .filter(|f| !f.pinned)
+        .map(|f| format!(" - `{}`", f.field.ident.as_ref().unwrap()));
+    let docs = format!(" Pin-projections of [`{ident}`]");
+    quote! {
+        #[doc = #docs]
+        // Allow `non_snake_case` since the same warning will be emitted on
+        // the struct definition.
+        #[allow(dead_code, non_snake_case)]
+        #[doc(hidden)]
+        #vis struct #projection #generics_with_pin_lt
+            #whr
+        {
+            #(#fields_decl)*
+            ___pin_phantom_data: ::core::marker::PhantomData<&'__pin mut ()>,
+        }
+
+        impl #impl_generics #ident #ty_generics
+            #whr
+        {
+            /// Pin-projects all fields of `Self`.
+            ///
+            /// These fields are structurally pinned:
+            #(#[doc = #structurally_pinned_fields_docs])*
+            ///
+            /// These fields are **not** structurally pinned:
+            #(#[doc = #not_structurally_pinned_fields_docs])*
+            #[inline]
+            #vis fn project<'__pin>(
+                self: ::core::pin::Pin<&'__pin mut Self>,
+            ) -> #projection #ty_generics_with_pin_lt {
+                // SAFETY: we only give access to `&mut` for fields not structurally pinned.
+                let #this = unsafe { ::core::pin::Pin::get_unchecked_mut(self) };
+                #projection {
+                    #(#fields_proj)*
+                    ___pin_phantom_data: ::core::marker::PhantomData,
+                }
+            }
+        }
+    }
+}
+
+fn generate_the_pin_data(
+    vis: &Visibility,
+    struct_name: &Ident,
+    generics: &Generics,
+    fields: &[FieldInfo<'_>],
+) -> TokenStream {
+    let (impl_generics, ty_generics, whr) = generics.split_for_impl();
+
+    // For every field, we create an initializing projection function according to its projection
+    // type. If a field is structurally pinned, we create a `Slot` with `Pinned` which must be
+    // initialized via `PinInit`; if it is not structurally pinned, then we create a `Slot` with
+    // `Unpinned` which allows initialization via `Init`.
+    let field_accessors = fields
+        .iter()
+        .map(|f| {
+            let Field { vis, ident, ty, .. } = f.field;
+
+            let field_name = ident
+                .as_ref()
+                .expect("only structs with named fields are supported");
+            let pin_marker = if f.pinned {
+                quote!(Pinned)
+            } else {
+                quote!(Unpinned)
+            };
+            quote! {
+                /// # Safety
+                ///
+                /// - `slot` is valid and properly aligned.
+                /// - `(*slot).#field_name` is properly aligned.
+                /// - `(*slot).#field_name` points to uninitialized and exclusively accessed
+                ///   memory.
+                // Allow `non_snake_case` since the same warning will be emitted on
+                // the struct definition.
+                #[allow(non_snake_case)]
+                #[inline(always)]
+                #vis unsafe fn #field_name(
+                    self,
+                    slot: *mut #struct_name #ty_generics,
+                ) -> ::pin_init::__internal::Slot<::pin_init::__internal::#pin_marker, #ty> {
+                    // SAFETY:
+                    // - If `#pin_marker` is `Pinned`, the corresponding field is structurally
+                    //   pinned.
+                    // - Other safety requirements follows the safety requirement.
+                    unsafe { ::pin_init::__internal::Slot::new(&raw mut (*slot).#field_name) }
+                }
+            }
+        })
+        .collect::<TokenStream>();
+    quote! {
+        // We declare this struct which will host all of the projection function for our type. It
+        // will be invariant over all generic parameters which are inherited from the struct.
+        #[doc(hidden)]
+        #vis struct __ThePinData #generics
+            #whr
+        {
+            __phantom: ::pin_init::__internal::PhantomInvariant<#struct_name #ty_generics>,
+        }
+
+        impl #impl_generics ::core::clone::Clone for __ThePinData #ty_generics
+            #whr
+        {
+            #[inline]
+            fn clone(&self) -> Self { *self }
+        }
+
+        impl #impl_generics ::core::marker::Copy for __ThePinData #ty_generics
+            #whr
+        {}
+
+        #[allow(dead_code)] // Some functions might never be used and private.
+        impl #impl_generics __ThePinData #ty_generics
+            #whr
+        {
+            /// Type inference helper function.
+            #[inline(always)]
+            #vis fn __make_closure<__F, __E>(self, f: __F) -> __F
+            where
+                __F: FnOnce(*mut #struct_name #ty_generics) ->
+                    ::core::result::Result<::pin_init::__internal::InitOk, __E>,
+            {
+                f
+            }
+
+            #field_accessors
+        }
+
+        // SAFETY: We have added the correct projection functions above to `__ThePinData` and
+        // we also use the least restrictive generics possible.
+        unsafe impl #impl_generics ::pin_init::__internal::HasPinData for #struct_name #ty_generics
+            #whr
+        {
+            type PinData = __ThePinData #ty_generics;
+
+            #[inline]
+            unsafe fn __pin_data() -> Self::PinData {
+                __ThePinData { __phantom: ::pin_init::__internal::PhantomInvariant::new() }
+            }
+        }
+    }
+}
+
+struct SelfReplacer(PathSegment);
+
+impl VisitMut for SelfReplacer {
+    fn visit_path_mut(&mut self, i: &mut syn::Path) {
+        if i.is_ident("Self") {
+            let span = i.span();
+            let seg = &self.0;
+            *i = parse_quote_spanned!(span=> #seg);
+        } else {
+            syn::visit_mut::visit_path_mut(self, i);
+        }
+    }
+
+    fn visit_path_segment_mut(&mut self, seg: &mut PathSegment) {
+        if seg.ident == "Self" {
+            let span = seg.span();
+            let this = &self.0;
+            *seg = parse_quote_spanned!(span=> #this);
+        } else {
+            syn::visit_mut::visit_path_segment_mut(self, seg);
+        }
+    }
+
+    fn visit_item_mut(&mut self, _: &mut Item) {
+        // Do not descend into items, since items reset/change what `Self` refers to.
+    }
+}
diff --git a/rust/pin-init/internal/src/pinned_drop.rs b/rust/pin-init/internal/src/pinned_drop.rs
new file mode 100644
index 000000000..a20ac314c
--- /dev/null
+++ b/rust/pin-init/internal/src/pinned_drop.rs
@@ -0,0 +1,61 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
+use proc_macro2::TokenStream;
+use quote::quote;
+use syn::{parse::Nothing, parse_quote, spanned::Spanned, ImplItem, ItemImpl, Token};
+
+use crate::diagnostics::{DiagCtxt, ErrorGuaranteed};
+
+pub(crate) fn pinned_drop(
+    _args: Nothing,
+    mut input: ItemImpl,
+    dcx: &mut DiagCtxt,
+) -> Result<TokenStream, ErrorGuaranteed> {
+    if let Some(unsafety) = input.unsafety {
+        dcx.error(unsafety, "implementing `PinnedDrop` is safe");
+    }
+    input.unsafety = Some(Token![unsafe](input.impl_token.span));
+    match &mut input.trait_ {
+        Some((not, path, _for)) => {
+            if let Some(not) = not {
+                dcx.error(not, "cannot implement `!PinnedDrop`");
+            }
+            for (seg, expected) in path
+                .segments
+                .iter()
+                .rev()
+                .zip(["PinnedDrop", "pin_init", ""])
+            {
+                if expected.is_empty() || seg.ident != expected {
+                    dcx.error(seg, "bad import path for `PinnedDrop`");
+                }
+                if !seg.arguments.is_none() {
+                    dcx.error(&seg.arguments, "unexpected arguments for `PinnedDrop` path");
+                }
+            }
+            *path = parse_quote!(::pin_init::PinnedDrop);
+        }
+        None => {
+            let span = input
+                .impl_token
+                .span
+                .join(input.self_ty.span())
+                .unwrap_or(input.impl_token.span);
+            dcx.error(
+                span,
+                "expected `impl ... PinnedDrop for ...`, got inherent impl",
+            );
+        }
+    }
+    for item in &mut input.items {
+        if let ImplItem::Fn(fn_item) = item {
+            if fn_item.sig.ident == "drop" {
+                fn_item
+                    .sig
+                    .inputs
+                    .push(parse_quote!(_: ::pin_init::__internal::OnlyCallFromDrop));
+            }
+        }
+    }
+    Ok(quote!(#input))
+}
diff --git a/rust/pin-init/internal/src/zeroable.rs b/rust/pin-init/internal/src/zeroable.rs
new file mode 100644
index 000000000..b11feaeb1
--- /dev/null
+++ b/rust/pin-init/internal/src/zeroable.rs
@@ -0,0 +1,78 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
+use proc_macro2::TokenStream;
+use quote::quote;
+use syn::{parse_quote, Data, DeriveInput, Field, Fields};
+
+use crate::{diagnostics::ErrorGuaranteed, DiagCtxt};
+
+pub(crate) fn derive(
+    input: DeriveInput,
+    dcx: &mut DiagCtxt,
+) -> Result<TokenStream, ErrorGuaranteed> {
+    let fields = match input.data {
+        Data::Struct(data_struct) => data_struct.fields,
+        Data::Union(data_union) => Fields::Named(data_union.fields),
+        Data::Enum(data_enum) => {
+            return Err(dcx.error(data_enum.enum_token, "cannot derive `Zeroable` for an enum"));
+        }
+    };
+    let name = input.ident;
+    let mut generics = input.generics;
+    for param in generics.type_params_mut() {
+        param.bounds.insert(0, parse_quote!(::pin_init::Zeroable));
+    }
+    let (impl_gen, ty_gen, whr) = generics.split_for_impl();
+    let field_type = fields.iter().map(|field| &field.ty);
+    Ok(quote! {
+        // SAFETY: Every field type implements `Zeroable` and padding bytes may be zero.
+        #[automatically_derived]
+        unsafe impl #impl_gen ::pin_init::Zeroable for #name #ty_gen
+            #whr
+        {}
+        const _: () = {
+            fn assert_zeroable<T: ?::core::marker::Sized + ::pin_init::Zeroable>() {}
+            fn ensure_zeroable #impl_gen ()
+                #whr
+            {
+                #(
+                    assert_zeroable::<#field_type>();
+                )*
+            }
+        };
+    })
+}
+
+pub(crate) fn maybe_derive(
+    input: DeriveInput,
+    dcx: &mut DiagCtxt,
+) -> Result<TokenStream, ErrorGuaranteed> {
+    let fields = match input.data {
+        Data::Struct(data_struct) => data_struct.fields,
+        Data::Union(data_union) => Fields::Named(data_union.fields),
+        Data::Enum(data_enum) => {
+            return Err(dcx.error(data_enum.enum_token, "cannot derive `Zeroable` for an enum"));
+        }
+    };
+    let name = input.ident;
+    let mut generics = input.generics;
+    for param in generics.type_params_mut() {
+        param.bounds.insert(0, parse_quote!(::pin_init::Zeroable));
+    }
+    for Field { ty, .. } in fields {
+        generics
+            .make_where_clause()
+            .predicates
+            // the `for<'__dummy>` HRTB makes this not error without the `trivial_bounds`
+            // feature <https://github.com/rust-lang/rust/issues/48214#issuecomment-2557829956>.
+            .push(parse_quote!(#ty: for<'__dummy> ::pin_init::Zeroable));
+    }
+    let (impl_gen, ty_gen, whr) = generics.split_for_impl();
+    Ok(quote! {
+        // SAFETY: Every field type implements `Zeroable` and padding bytes may be zero.
+        #[automatically_derived]
+        unsafe impl #impl_gen ::pin_init::Zeroable for #name #ty_gen
+            #whr
+        {}
+    })
+}
diff --git a/rust/pin-init/src/__internal.rs b/rust/pin-init/src/__internal.rs
new file mode 100644
index 000000000..8e9fd18b9
--- /dev/null
+++ b/rust/pin-init/src/__internal.rs
@@ -0,0 +1,409 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
+//! This module contains library internal items.
+//!
+//! These items must not be used outside of this crate and the pin-init-internal crate located at
+//! `../internal`.
+
+use super::*;
+
+/// Zero-sized type used to mark a type as invariant.
+///
+/// This is a polyfill for the [unstable type] in the standard library of the same name.
+///
+/// See the [nomicon] for what subtyping is. See also [this table].
+///
+/// [unstable type]: https://doc.rust-lang.org/nightly/std/marker/struct.PhantomInvariant.html
+/// [nomicon]: https://doc.rust-lang.org/nomicon/subtyping.html
+/// [this table]: https://doc.rust-lang.org/nomicon/phantom-data.html#table-of-phantomdata-patterns
+#[repr(transparent)]
+pub struct PhantomInvariant<T: ?Sized>(PhantomData<fn(T) -> T>);
+
+impl<T: ?Sized> Clone for PhantomInvariant<T> {
+    #[inline(always)]
+    fn clone(&self) -> Self {
+        *self
+    }
+}
+
+impl<T: ?Sized> Copy for PhantomInvariant<T> {}
+
+impl<T: ?Sized> Default for PhantomInvariant<T> {
+    #[inline(always)]
+    fn default() -> Self {
+        Self::new()
+    }
+}
+
+impl<T: ?Sized> PhantomInvariant<T> {
+    #[inline(always)]
+    pub const fn new() -> Self {
+        Self(PhantomData)
+    }
+}
+
+/// Zero-sized type used to mark a lifetime as invariant.
+///
+/// This is a polyfill for the [unstable type] in the standard library of the same name.
+///
+/// [unstable type]: https://doc.rust-lang.org/nightly/std/marker/struct.PhantomInvariantLifetime.html
+#[repr(transparent)]
+#[derive(Clone, Copy, Default)]
+pub struct PhantomInvariantLifetime<'a>(PhantomInvariant<&'a ()>);
+
+impl PhantomInvariantLifetime<'_> {
+    #[inline(always)]
+    pub const fn new() -> Self {
+        Self(PhantomInvariant::new())
+    }
+}
+
+/// Token type to signify successful initialization.
+///
+/// Can only be constructed via the unsafe [`Self::new`] function. The initializer macros use this
+/// token type to prevent returning `Ok` from an initializer without initializing all fields.
+pub struct InitOk(());
+
+impl InitOk {
+    /// Creates a new token.
+    ///
+    /// # Safety
+    ///
+    /// This function may only be called from the `init!` macro in `../internal/src/init.rs`.
+    #[inline(always)]
+    pub unsafe fn new() -> Self {
+        Self(())
+    }
+}
+
+/// This trait is only implemented via the `#[pin_data]` proc-macro. It is used to facilitate
+/// the pin projections within the initializers.
+///
+/// # Safety
+///
+/// Only the `init` module is allowed to use this trait.
+pub unsafe trait HasPinData {
+    type PinData;
+
+    #[expect(clippy::missing_safety_doc)]
+    unsafe fn __pin_data() -> Self::PinData;
+}
+
+/// This trait is automatically implemented for every type. It aims to provide the same type
+/// inference help as `HasPinData`.
+///
+/// # Safety
+///
+/// Only the `init` module is allowed to use this trait.
+pub unsafe trait HasInitData {
+    type InitData;
+
+    #[expect(clippy::missing_safety_doc)]
+    unsafe fn __init_data() -> Self::InitData;
+}
+
+pub struct AllData<T: ?Sized>(PhantomInvariant<T>);
+
+impl<T: ?Sized> Clone for AllData<T> {
+    #[inline]
+    fn clone(&self) -> Self {
+        *self
+    }
+}
+
+impl<T: ?Sized> Copy for AllData<T> {}
+
+impl<T: ?Sized> AllData<T> {
+    /// Type inference helper function.
+    #[inline(always)]
+    pub fn __make_closure<F, E>(self, f: F) -> F
+    where
+        F: FnOnce(*mut T) -> Result<InitOk, E>,
+    {
+        f
+    }
+}
+
+// SAFETY: TODO.
+unsafe impl<T: ?Sized> HasInitData for T {
+    type InitData = AllData<T>;
+
+    #[inline]
+    unsafe fn __init_data() -> Self::InitData {
+        AllData(PhantomInvariant::new())
+    }
+}
+
+/// Stack initializer helper type. Use [`stack_pin_init`] instead of this primitive.
+///
+/// # Invariants
+///
+/// If `self.is_init` is true, then `self.value` is initialized.
+///
+/// [`stack_pin_init`]: crate::stack_pin_init
+pub struct StackInit<T> {
+    value: MaybeUninit<T>,
+    is_init: bool,
+}
+
+impl<T> Drop for StackInit<T> {
+    #[inline]
+    fn drop(&mut self) {
+        if self.is_init {
+            // SAFETY: As we are being dropped, we only call this once. And since `self.is_init` is
+            // true, `self.value` is initialized.
+            unsafe { self.value.assume_init_drop() };
+        }
+    }
+}
+
+impl<T> StackInit<T> {
+    /// Creates a new [`StackInit<T>`] that is uninitialized. Use [`stack_pin_init`] instead of this
+    /// primitive.
+    ///
+    /// [`stack_pin_init`]: crate::stack_pin_init
+    #[inline]
+    pub fn uninit() -> Self {
+        Self {
+            value: MaybeUninit::uninit(),
+            is_init: false,
+        }
+    }
+
+    /// Initializes the contents and returns the result.
+    #[inline]
+    pub fn init<E>(self: Pin<&mut Self>, init: impl PinInit<T, E>) -> Result<Pin<&mut T>, E> {
+        // SAFETY: We never move out of `this`.
+        let this = unsafe { Pin::into_inner_unchecked(self) };
+        // The value is currently initialized, so it needs to be dropped before we can reuse
+        // the memory (this is a safety guarantee of `Pin`).
+        if this.is_init {
+            this.is_init = false;
+            // SAFETY: `this.is_init` was true and therefore `this.value` is initialized.
+            unsafe { this.value.assume_init_drop() };
+        }
+        // SAFETY: The memory slot is valid and this type ensures that it will stay pinned.
+        unsafe { init.__init(this.value.as_mut_ptr())? };
+        // INVARIANT: `this.value` is initialized above.
+        this.is_init = true;
+        // SAFETY: The slot is now pinned, since we will never give access to `&mut T`.
+        Ok(unsafe { Pin::new_unchecked(this.value.assume_init_mut()) })
+    }
+}
+
+#[test]
+#[cfg(feature = "std")]
+fn stack_init_reuse() {
+    use ::std::{borrow::ToOwned, println, string::String};
+    use core::pin::pin;
+
+    #[derive(Debug)]
+    struct Foo {
+        a: usize,
+        b: String,
+    }
+    let mut slot: Pin<&mut StackInit<Foo>> = pin!(StackInit::uninit());
+    let value: Result<Pin<&mut Foo>, core::convert::Infallible> =
+        slot.as_mut().init(crate::init!(Foo {
+            a: 42,
+            b: "Hello".to_owned(),
+        }));
+    let value = value.unwrap();
+    println!("{value:?}");
+    let value: Result<Pin<&mut Foo>, core::convert::Infallible> =
+        slot.as_mut().init(crate::init!(Foo {
+            a: 24,
+            b: "world!".to_owned(),
+        }));
+    let value = value.unwrap();
+    println!("{value:?}");
+}
+
+// Marker types that determines type of `DropGuard`'s let bindings.
+pub struct Pinned;
+pub struct Unpinned;
+
+/// Represent an uninitialized field.
+///
+/// # Invariants
+///
+/// - `ptr` is valid, properly aligned and points to uninitialized and exclusively accessed memory.
+/// - If `P` is `Pinned`, then `ptr` is structurally pinned.
+pub struct Slot<P, T: ?Sized> {
+    ptr: *mut T,
+    _phantom: PhantomData<P>,
+}
+
+impl<P, T: ?Sized> Slot<P, T> {
+    /// # Safety
+    ///
+    /// - `ptr` is valid, properly aligned and points to uninitialized and exclusively accessed
+    ///   memory.
+    /// - If `P` is `Pinned`, then `ptr` is structurally pinned.
+    #[inline(always)]
+    pub unsafe fn new(ptr: *mut T) -> Self {
+        // INVARIANT: Per safety requirement.
+        Self {
+            ptr,
+            _phantom: PhantomData,
+        }
+    }
+
+    /// Initialize the field by value.
+    #[inline(always)]
+    pub fn write(self, value: T) -> DropGuard<P, T>
+    where
+        T: Sized,
+    {
+        // SAFETY: `self.ptr` is a valid and aligned pointer for write.
+        unsafe { self.ptr.write(value) }
+        // SAFETY:
+        // - `self.ptr` is valid and properly aligned per type invariant.
+        // - `*self.ptr` is initialized above and the ownership is transferred to the guard.
+        // - If `P` is `Pinned`, `self.ptr` is pinned.
+        unsafe { DropGuard::new(self.ptr) }
+    }
+}
+
+impl<T: ?Sized> Slot<Unpinned, T> {
+    /// Initialize the field.
+    #[inline(always)]
+    pub fn init<E>(self, init: impl Init<T, E>) -> Result<DropGuard<Unpinned, T>, E> {
+        // SAFETY:
+        // - `self.ptr` is valid and properly aligned.
+        // - when `Err` is returned, we also propagate the error without touching `slot`;
+        //   also `self` is consumed so it cannot be touched further.
+        unsafe { init.__init(self.ptr)? };
+
+        // SAFETY:
+        // - `self.ptr` is valid and properly aligned per type invariant.
+        // - `*self.ptr` is initialized above and the ownership is transferred to the guard.
+        Ok(unsafe { DropGuard::new(self.ptr) })
+    }
+}
+
+impl<T: ?Sized> Slot<Pinned, T> {
+    /// Initialize the field.
+    #[inline(always)]
+    pub fn init<E>(self, init: impl PinInit<T, E>) -> Result<DropGuard<Pinned, T>, E> {
+        // SAFETY:
+        // - `self.ptr` is valid and properly aligned.
+        // - when `Err` is returned, we also propagate the error without touching `ptr`;
+        //   also `self` is consumed so it cannot be touched further.
+        // - the drop guard will not hand out `&mut` (only `Pin<&mut T>`).
+        unsafe { init.__init(self.ptr)? };
+
+        // SAFETY:
+        // - `self.ptr` is valid, properly aligned and pinned per type invariant.
+        // - `*self.ptr` is initialized above and the ownership is transferred to the guard.
+        Ok(unsafe { DropGuard::new(self.ptr) })
+    }
+}
+
+/// When a value of this type is dropped, it drops a `T`.
+///
+/// Can be forgotten to prevent the drop.
+///
+/// # Invariants
+///
+/// - `ptr` is valid and properly aligned.
+/// - `*ptr` is initialized and owned by this guard.
+/// - if `P` is `Pinned`, `ptr` is pinned.
+pub struct DropGuard<P, T: ?Sized> {
+    ptr: *mut T,
+    phantom: PhantomData<P>,
+}
+
+impl<P, T: ?Sized> DropGuard<P, T> {
+    /// Creates a drop guard and transfer the ownership of the pointer content.
+    ///
+    /// The ownership is only relinguished if the guard is forgotten via [`core::mem::forget`].
+    ///
+    /// # Safety
+    ///
+    /// - `ptr` is valid and properly aligned.
+    /// - `*ptr` is initialized, and the ownership is transferred to this guard.
+    /// - if `P` is `Pinned`, `ptr` is pinned.
+    #[inline]
+    pub unsafe fn new(ptr: *mut T) -> Self {
+        // INVARIANT: By safety requirement.
+        Self {
+            ptr,
+            phantom: PhantomData,
+        }
+    }
+}
+
+impl<T: ?Sized> DropGuard<Unpinned, T> {
+    /// Create a let binding for accessor use.
+    #[inline]
+    pub fn let_binding(&mut self) -> &mut T {
+        // SAFETY: Per type invariant.
+        unsafe { &mut *self.ptr }
+    }
+}
+
+impl<T: ?Sized> DropGuard<Pinned, T> {
+    /// Create a let binding for accessor use.
+    #[inline]
+    pub fn let_binding(&mut self) -> Pin<&mut T> {
+        // SAFETY: `self.ptr` is valid, properly aligned, initialized, exclusively accessible and
+        // pinned per type invariant.
+        unsafe { Pin::new_unchecked(&mut *self.ptr) }
+    }
+}
+
+impl<P, T: ?Sized> Drop for DropGuard<P, T> {
+    #[inline]
+    fn drop(&mut self) {
+        // SAFETY: `self.ptr` is valid, properly aligned and `*self.ptr` is owned by this guard.
+        unsafe { ptr::drop_in_place(self.ptr) }
+    }
+}
+
+/// Token used by `PinnedDrop` to prevent calling the function without creating this unsafely
+/// created struct. This is needed, because the `drop` function is safe, but should not be called
+/// manually.
+pub struct OnlyCallFromDrop(());
+
+impl OnlyCallFromDrop {
+    /// # Safety
+    ///
+    /// This function should only be called from the [`Drop::drop`] function and only be used to
+    /// delegate the destruction to the pinned destructor [`PinnedDrop::drop`] of the same type.
+    pub unsafe fn new() -> Self {
+        Self(())
+    }
+}
+
+/// Initializer that always fails.
+///
+/// Used by [`assert_pinned!`].
+///
+/// [`assert_pinned!`]: crate::assert_pinned
+pub struct AlwaysFail<T: ?Sized> {
+    _t: PhantomData<T>,
+}
+
+impl<T: ?Sized> AlwaysFail<T> {
+    /// Creates a new initializer that always fails.
+    #[inline]
+    pub fn new() -> Self {
+        Self { _t: PhantomData }
+    }
+}
+
+impl<T: ?Sized> Default for AlwaysFail<T> {
+    #[inline]
+    fn default() -> Self {
+        Self::new()
+    }
+}
+
+// SAFETY: `__init` always fails, which is always okay.
+unsafe impl<T: ?Sized> PinInit<T, ()> for AlwaysFail<T> {
+    #[inline]
+    unsafe fn __init(self, _slot: *mut T) -> Result<(), ()> {
+        Err(())
+    }
+}
diff --git a/rust/pin-init/src/alloc.rs b/rust/pin-init/src/alloc.rs
new file mode 100644
index 000000000..471652e86
--- /dev/null
+++ b/rust/pin-init/src/alloc.rs
@@ -0,0 +1,160 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
+#[cfg(all(feature = "alloc", not(feature = "std")))]
+use alloc::{boxed::Box, sync::Arc};
+#[cfg(feature = "alloc")]
+use core::alloc::AllocError;
+use core::{mem::MaybeUninit, pin::Pin};
+#[cfg(feature = "std")]
+use std::sync::Arc;
+
+#[cfg(not(feature = "alloc"))]
+type AllocError = core::convert::Infallible;
+
+use crate::{
+    init_from_closure, pin_init_from_closure, InPlaceWrite, Init, PinInit, ZeroableOption,
+};
+
+pub extern crate alloc;
+
+// SAFETY: All zeros is equivalent to `None` (option layout optimization guarantee:
+// <https://doc.rust-lang.org/stable/std/option/index.html#representation>).
+unsafe impl<T> ZeroableOption for Box<T> {}
+
+/// Smart pointer that can initialize memory in-place.
+pub trait InPlaceInit<T>: Sized {
+    /// Use the given pin-initializer to pin-initialize a `T` inside of a new smart pointer of this
+    /// type.
+    ///
+    /// If `T: !Unpin` it will not be able to move afterwards.
+    fn try_pin_init<E>(init: impl PinInit<T, E>) -> Result<Pin<Self>, E>
+    where
+        E: From<AllocError>;
+
+    /// Use the given pin-initializer to pin-initialize a `T` inside of a new smart pointer of this
+    /// type.
+    ///
+    /// If `T: !Unpin` it will not be able to move afterwards.
+    #[inline]
+    fn pin_init(init: impl PinInit<T>) -> Result<Pin<Self>, AllocError> {
+        // SAFETY: We delegate to `init` and only change the error type.
+        let init = unsafe {
+            pin_init_from_closure(|slot| match init.__init(slot) {
+                Ok(()) => Ok(()),
+                Err(i) => match i {},
+            })
+        };
+        Self::try_pin_init(init)
+    }
+
+    /// Use the given initializer to in-place initialize a `T`.
+    fn try_init<E>(init: impl Init<T, E>) -> Result<Self, E>
+    where
+        E: From<AllocError>;
+
+    /// Use the given initializer to in-place initialize a `T`.
+    #[inline]
+    fn init(init: impl Init<T>) -> Result<Self, AllocError> {
+        // SAFETY: We delegate to `init` and only change the error type.
+        let init = unsafe {
+            init_from_closure(|slot| match init.__init(slot) {
+                Ok(()) => Ok(()),
+                Err(i) => match i {},
+            })
+        };
+        Self::try_init(init)
+    }
+}
+
+#[cfg(feature = "alloc")]
+macro_rules! try_new_uninit {
+    ($type:ident) => {
+        $type::try_new_uninit()?
+    };
+}
+#[cfg(all(feature = "std", not(feature = "alloc")))]
+macro_rules! try_new_uninit {
+    ($type:ident) => {
+        $type::new_uninit()
+    };
+}
+
+impl<T> InPlaceInit<T> for Box<T> {
+    #[inline]
+    fn try_pin_init<E>(init: impl PinInit<T, E>) -> Result<Pin<Self>, E>
+    where
+        E: From<AllocError>,
+    {
+        try_new_uninit!(Box).write_pin_init(init)
+    }
+
+    #[inline]
+    fn try_init<E>(init: impl Init<T, E>) -> Result<Self, E>
+    where
+        E: From<AllocError>,
+    {
+        try_new_uninit!(Box).write_init(init)
+    }
+}
+
+impl<T> InPlaceInit<T> for Arc<T> {
+    #[inline]
+    fn try_pin_init<E>(init: impl PinInit<T, E>) -> Result<Pin<Self>, E>
+    where
+        E: From<AllocError>,
+    {
+        let mut this = try_new_uninit!(Arc);
+        let Some(slot) = Arc::get_mut(&mut this) else {
+            // SAFETY: the Arc has just been created and has no external references
+            unsafe { core::hint::unreachable_unchecked() }
+        };
+        let slot = slot.as_mut_ptr();
+        // SAFETY: When init errors/panics, slot will get deallocated but not dropped,
+        // slot is valid and will not be moved, because we pin it later.
+        unsafe { init.__init(slot)? };
+        // SAFETY: All fields have been initialized and this is the only `Arc` to that data.
+        Ok(unsafe { Pin::new_unchecked(this.assume_init()) })
+    }
+
+    #[inline]
+    fn try_init<E>(init: impl Init<T, E>) -> Result<Self, E>
+    where
+        E: From<AllocError>,
+    {
+        let mut this = try_new_uninit!(Arc);
+        let Some(slot) = Arc::get_mut(&mut this) else {
+            // SAFETY: the Arc has just been created and has no external references
+            unsafe { core::hint::unreachable_unchecked() }
+        };
+        let slot = slot.as_mut_ptr();
+        // SAFETY: When init errors/panics, slot will get deallocated but not dropped,
+        // slot is valid.
+        unsafe { init.__init(slot)? };
+        // SAFETY: All fields have been initialized.
+        Ok(unsafe { this.assume_init() })
+    }
+}
+
+impl<T> InPlaceWrite<T> for Box<MaybeUninit<T>> {
+    type Initialized = Box<T>;
+
+    #[inline]
+    fn write_init<E>(mut self, init: impl Init<T, E>) -> Result<Self::Initialized, E> {
+        let slot = self.as_mut_ptr();
+        // SAFETY: When init errors/panics, slot will get deallocated but not dropped,
+        // slot is valid.
+        unsafe { init.__init(slot)? };
+        // SAFETY: All fields have been initialized.
+        Ok(unsafe { self.assume_init() })
+    }
+
+    #[inline]
+    fn write_pin_init<E>(mut self, init: impl PinInit<T, E>) -> Result<Pin<Self::Initialized>, E> {
+        let slot = self.as_mut_ptr();
+        // SAFETY: When init errors/panics, slot will get deallocated but not dropped,
+        // slot is valid and will not be moved, because we pin it later.
+        unsafe { init.__init(slot)? };
+        // SAFETY: All fields have been initialized.
+        Ok(unsafe { self.assume_init() }.into())
+    }
+}
diff --git a/rust/pin-init/src/lib.rs b/rust/pin-init/src/lib.rs
new file mode 100644
index 000000000..7600cdbbb
--- /dev/null
+++ b/rust/pin-init/src/lib.rs
@@ -0,0 +1,1803 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
+//! Library to safely and fallibly initialize pinned `struct`s using in-place constructors.
+//!
+//! [Pinning][pinning] is Rust's way of ensuring data does not move.
+//!
+//! It also allows in-place initialization of big `struct`s that would otherwise produce a stack
+//! overflow.
+//!
+//! This library's main use-case is in [Rust-for-Linux]. Although this version can be used
+//! standalone.
+//!
+//! There are cases when you want to in-place initialize a struct. For example when it is very big
+//! and moving it from the stack is not an option, because it is bigger than the stack itself.
+//! Another reason would be that you need the address of the object to initialize it. This stands
+//! in direct conflict with Rust's normal process of first initializing an object and then moving
+//! it into it's final memory location. For more information, see
+//! <https://rust-for-linux.com/the-safe-pinned-initialization-problem>.
+//!
+//! This library allows you to do in-place initialization safely.
+//!
+//! ## Nightly Needed for `alloc` feature
+//!
+//! This library requires the [`allocator_api` unstable feature] when the `alloc` feature is
+//! enabled and thus this feature can only be used with a nightly compiler. When enabling the
+//! `alloc` feature, the user will be required to activate `allocator_api` as well.
+//!
+//! [`allocator_api` unstable feature]: https://doc.rust-lang.org/nightly/unstable-book/library-features/allocator-api.html
+//!
+//! The feature is enabled by default, thus by default `pin-init` will require a nightly compiler.
+//! However, using the crate on stable compilers is possible by disabling `alloc`. In practice this
+//! will require the `std` feature, because stable compilers have neither `Box` nor `Arc` in no-std
+//! mode.
+//!
+//! ## Nightly needed for `unsafe-pinned` feature
+//!
+//! This feature enables the `Wrapper` implementation on the unstable `core::pin::UnsafePinned` type.
+//! This requires the [`unsafe_pinned` unstable feature](https://github.com/rust-lang/rust/issues/125735)
+//! and therefore a nightly compiler. Note that this feature is not enabled by default.
+//!
+//! # Overview
+//!
+//! To initialize a `struct` with an in-place constructor you will need two things:
+//! - an in-place constructor,
+//! - a memory location that can hold your `struct` (this can be the [stack], an [`Arc<T>`],
+//!   [`Box<T>`] or any other smart pointer that supports this library).
+//!
+//! To get an in-place constructor there are generally three options:
+//! - directly creating an in-place constructor using the [`pin_init!`] macro,
+//! - a custom function/macro returning an in-place constructor provided by someone else,
+//! - using the unsafe function [`pin_init_from_closure()`] to manually create an initializer.
+//!
+//! Aside from pinned initialization, this library also supports in-place construction without
+//! pinning, the macros/types/functions are generally named like the pinned variants without the
+//! `pin_` prefix.
+//!
+//! # Examples
+//!
+//! Throughout the examples we will often make use of the `CMutex` type which can be found in
+//! `../examples/mutex.rs`. It is essentially a userland rebuild of the `struct mutex` type from
+//! the Linux kernel. It also uses a wait list and a basic spinlock. Importantly the wait list
+//! requires it to be pinned to be locked and thus is a prime candidate for using this library.
+//!
+//! ## Using the [`pin_init!`] macro
+//!
+//! If you want to use [`PinInit`], then you will have to annotate your `struct` with
+//! `#[`[`pin_data`]`]`. It is a macro that uses `#[pin]` as a marker for
+//! [structurally pinned fields]. After doing this, you can then create an in-place constructor via
+//! [`pin_init!`]. The syntax is almost the same as normal `struct` initializers. The difference is
+//! that you need to write `<-` instead of `:` for fields that you want to initialize in-place.
+//!
+//! ```rust
+//! # #![feature(allocator_api)]
+//! # #[path = "../examples/mutex.rs"] mod mutex; use mutex::*;
+//! # use core::pin::Pin;
+//! use pin_init::{pin_data, pin_init, InPlaceInit};
+//!
+//! #[pin_data]
+//! struct Foo {
+//!     #[pin]
+//!     a: CMutex<usize>,
+//!     b: u32,
+//! }
+//!
+//! let foo = pin_init!(Foo {
+//!     a <- CMutex::new(42),
+//!     b: 24,
+//! });
+//! # let _ = Box::pin_init(foo);
+//! ```
+//!
+//! `foo` now is of the type [`impl PinInit<Foo>`]. We can now use any smart pointer that we like
+//! (or just the stack) to actually initialize a `Foo`:
+//!
+//! ```rust
+//! # #![feature(allocator_api)]
+//! # #[path = "../examples/mutex.rs"] mod mutex; use mutex::*;
+//! # use core::{alloc::AllocError, pin::Pin};
+//! # use pin_init::*;
+//! #
+//! # #[pin_data]
+//! # struct Foo {
+//! #     #[pin]
+//! #     a: CMutex<usize>,
+//! #     b: u32,
+//! # }
+//! #
+//! # let foo = pin_init!(Foo {
+//! #     a <- CMutex::new(42),
+//! #     b: 24,
+//! # });
+//! let foo: Result<Pin<Box<Foo>>, AllocError> = Box::pin_init(foo);
+//! ```
+//!
+//! For more information see the [`pin_init!`] macro.
+//!
+//! ## Using a custom function/macro that returns an initializer
+//!
+//! Many types that use this library supply a function/macro that returns an initializer, because
+//! the above method only works for types where you can access the fields.
+//!
+//! ```rust
+//! # #![feature(allocator_api)]
+//! # #[path = "../examples/mutex.rs"] mod mutex; use mutex::*;
+//! # use pin_init::*;
+//! # use std::sync::Arc;
+//! # use core::pin::Pin;
+//! let mtx: Result<Pin<Arc<CMutex<usize>>>, _> = Arc::pin_init(CMutex::new(42));
+//! ```
+//!
+//! To declare an init macro/function you just return an [`impl PinInit<T, E>`]:
+//!
+//! ```rust
+//! # #![feature(allocator_api)]
+//! # use pin_init::*;
+//! # #[path = "../examples/error.rs"] mod error; use error::Error;
+//! # #[path = "../examples/mutex.rs"] mod mutex; use mutex::*;
+//! #[pin_data]
+//! struct DriverData {
+//!     #[pin]
+//!     status: CMutex<i32>,
+//!     buffer: Box<[u8; 1_000_000]>,
+//! }
+//!
+//! impl DriverData {
+//!     fn new() -> impl PinInit<Self, Error> {
+//!         pin_init!(Self {
+//!             status <- CMutex::new(0),
+//!             buffer: Box::init(pin_init::init_zeroed())?,
+//!         }? Error)
+//!     }
+//! }
+//! ```
+//!
+//! ## Manual creation of an initializer
+//!
+//! Often when working with primitives the previous approaches are not sufficient. That is where
+//! [`pin_init_from_closure()`] comes in. This `unsafe` function allows you to create a
+//! [`impl PinInit<T, E>`] directly from a closure. Of course you have to ensure that the closure
+//! actually does the initialization in the correct way. Here are the things to look out for
+//! (we are calling the parameter to the closure `slot`):
+//! - when the closure returns `Ok(())`, then it has completed the initialization successfully, so
+//!   `slot` now contains a valid bit pattern for the type `T`,
+//! - when the closure returns `Err(e)`, then the caller may deallocate the memory at `slot`, so
+//!   you need to take care to clean up anything if your initialization fails mid-way,
+//! - you may assume that `slot` will stay pinned even after the closure returns until `drop` of
+//!   `slot` gets called.
+//!
+//! ```rust
+//! # #![feature(extern_types)]
+//! use pin_init::{pin_data, pinned_drop, PinInit, PinnedDrop, pin_init_from_closure};
+//! use core::{
+//!     marker::PhantomPinned,
+//!     cell::UnsafeCell,
+//!     pin::Pin,
+//!     mem::MaybeUninit,
+//! };
+//! mod bindings {
+//!     #[repr(C)]
+//!     pub struct foo {
+//!         /* fields from C ... */
+//!     }
+//!     extern "C" {
+//!         pub fn init_foo(ptr: *mut foo);
+//!         pub fn destroy_foo(ptr: *mut foo);
+//!         #[must_use = "you must check the error return code"]
+//!         pub fn enable_foo(ptr: *mut foo, flags: u32) -> i32;
+//!     }
+//! }
+//!
+//! /// # Invariants
+//! ///
+//! /// `foo` is always initialized
+//! #[pin_data(PinnedDrop)]
+//! pub struct RawFoo {
+//!     #[pin]
+//!     _p: PhantomPinned,
+//!     #[pin]
+//!     foo: UnsafeCell<MaybeUninit<bindings::foo>>,
+//! }
+//!
+//! impl RawFoo {
+//!     pub fn new(flags: u32) -> impl PinInit<Self, i32> {
+//!         // SAFETY:
+//!         // - when the closure returns `Ok(())`, then it has successfully initialized and
+//!         //   enabled `foo`,
+//!         // - when it returns `Err(e)`, then it has cleaned up before
+//!         unsafe {
+//!             pin_init_from_closure(move |slot: *mut Self| {
+//!                 // `slot` contains uninit memory, avoid creating a reference.
+//!                 let foo = &raw mut (*slot).foo;
+//!                 let foo = UnsafeCell::raw_get(foo).cast::<bindings::foo>();
+//!
+//!                 // Initialize the `foo`
+//!                 bindings::init_foo(foo);
+//!
+//!                 // Try to enable it.
+//!                 let err = bindings::enable_foo(foo, flags);
+//!                 if err != 0 {
+//!                     // Enabling has failed, first clean up the foo and then return the error.
+//!                     bindings::destroy_foo(foo);
+//!                     Err(err)
+//!                 } else {
+//!                     // All fields of `RawFoo` have been initialized, since `_p` is a ZST.
+//!                     Ok(())
+//!                 }
+//!             })
+//!         }
+//!     }
+//! }
+//!
+//! #[pinned_drop]
+//! impl PinnedDrop for RawFoo {
+//!     fn drop(self: Pin<&mut Self>) {
+//!         // SAFETY: Since `foo` is initialized, destroying is safe.
+//!         unsafe { bindings::destroy_foo(self.foo.get().cast::<bindings::foo>()) };
+//!     }
+//! }
+//! ```
+//!
+//! For more information on how to use [`pin_init_from_closure()`], take a look at the uses inside
+//! the `kernel` crate. The [`sync`] module is a good starting point.
+//!
+//! [`sync`]: https://rust.docs.kernel.org/kernel/sync/index.html
+//! [pinning]: https://doc.rust-lang.org/std/pin/index.html
+//! [structurally pinned fields]:
+//!     https://doc.rust-lang.org/std/pin/index.html#projections-and-structural-pinning
+//! [stack]: crate::stack_pin_init
+#![cfg_attr(
+    kernel,
+    doc = "[`Arc<T>`]: https://rust.docs.kernel.org/kernel/sync/struct.Arc.html"
+)]
+#![cfg_attr(
+    kernel,
+    doc = "[`Box<T>`]: https://rust.docs.kernel.org/kernel/alloc/kbox/struct.Box.html"
+)]
+#![cfg_attr(not(kernel), doc = "[`Arc<T>`]: alloc::alloc::sync::Arc")]
+#![cfg_attr(not(kernel), doc = "[`Box<T>`]: alloc::alloc::boxed::Box")]
+//! [`impl PinInit<Foo>`]: crate::PinInit
+//! [`impl PinInit<T, E>`]: crate::PinInit
+//! [`impl Init<T, E>`]: crate::Init
+//! [Rust-for-Linux]: https://rust-for-linux.com/
+
+#![forbid(missing_docs, unsafe_op_in_unsafe_fn)]
+#![cfg_attr(not(feature = "std"), no_std)]
+#![cfg_attr(feature = "alloc", feature(allocator_api))]
+#![cfg_attr(
+    all(feature = "unsafe-pinned", CONFIG_RUSTC_HAS_UNSAFE_PINNED),
+    feature(unsafe_pinned)
+)]
+#![cfg_attr(all(USE_RUSTC_FEATURES, doc), allow(internal_features))]
+#![cfg_attr(all(USE_RUSTC_FEATURES, doc), feature(rustdoc_internals))]
+
+use core::{
+    cell::UnsafeCell,
+    convert::Infallible,
+    marker::PhantomData,
+    mem::MaybeUninit,
+    num::*,
+    pin::Pin,
+    ptr::{self, NonNull},
+};
+
+// This is used by doc-tests -- the proc-macros expand to `::pin_init::...` and without this the
+// doc-tests wouldn't have an extern crate named `pin_init`.
+#[allow(unused_extern_crates)]
+extern crate self as pin_init;
+
+#[doc(hidden)]
+pub mod __internal;
+
+#[cfg(any(feature = "std", feature = "alloc"))]
+mod alloc;
+#[cfg(any(feature = "std", feature = "alloc"))]
+pub use alloc::InPlaceInit;
+
+/// Used to specify the pinning information of the fields of a struct.
+///
+/// This is somewhat similar in purpose as
+/// [pin-project-lite](https://crates.io/crates/pin-project-lite).
+/// Place this macro on a struct definition and then `#[pin]` in front of the attributes of each
+/// field you want to structurally pin.
+///
+/// This macro enables the use of the [`pin_init!`] macro. When pin-initializing a `struct`,
+/// then `#[pin]` directs the type of initializer that is required.
+///
+/// If your `struct` implements `Drop`, then you need to add `PinnedDrop` as arguments to this
+/// macro, and change your `Drop` implementation to `PinnedDrop` annotated with
+/// `#[`[`macro@pinned_drop`]`]`, since dropping pinned values requires extra care.
+///
+/// # Examples
+///
+/// ```
+/// # #![feature(allocator_api)]
+/// # #[path = "../examples/mutex.rs"] mod mutex; use mutex::*;
+/// use pin_init::pin_data;
+///
+/// enum Command {
+///     /* ... */
+/// }
+///
+/// #[pin_data]
+/// struct DriverData {
+///     #[pin]
+///     queue: CMutex<Vec<Command>>,
+///     buf: Box<[u8; 1024 * 1024]>,
+/// }
+/// ```
+///
+/// ```
+/// # #![feature(allocator_api)]
+/// # #[path = "../examples/mutex.rs"] mod mutex; use mutex::*;
+/// # mod bindings { pub struct info; pub unsafe fn destroy_info(_: *mut info) {} }
+/// use core::pin::Pin;
+/// use pin_init::{pin_data, pinned_drop, PinnedDrop};
+///
+/// enum Command {
+///     /* ... */
+/// }
+///
+/// #[pin_data(PinnedDrop)]
+/// struct DriverData {
+///     #[pin]
+///     queue: CMutex<Vec<Command>>,
+///     buf: Box<[u8; 1024 * 1024]>,
+///     raw_info: *mut bindings::info,
+/// }
+///
+/// #[pinned_drop]
+/// impl PinnedDrop for DriverData {
+///     fn drop(self: Pin<&mut Self>) {
+///         unsafe { bindings::destroy_info(self.raw_info) };
+///     }
+/// }
+/// ```
+pub use ::pin_init_internal::pin_data;
+
+/// Used to implement `PinnedDrop` safely.
+///
+/// Only works on structs that are annotated via `#[`[`macro@pin_data`]`]`.
+///
+/// # Examples
+///
+/// ```
+/// # #![feature(allocator_api)]
+/// # #[path = "../examples/mutex.rs"] mod mutex; use mutex::*;
+/// # mod bindings { pub struct info; pub unsafe fn destroy_info(_: *mut info) {} }
+/// use core::pin::Pin;
+/// use pin_init::{pin_data, pinned_drop, PinnedDrop};
+///
+/// enum Command {
+///     /* ... */
+/// }
+///
+/// #[pin_data(PinnedDrop)]
+/// struct DriverData {
+///     #[pin]
+///     queue: CMutex<Vec<Command>>,
+///     buf: Box<[u8; 1024 * 1024]>,
+///     raw_info: *mut bindings::info,
+/// }
+///
+/// #[pinned_drop]
+/// impl PinnedDrop for DriverData {
+///     fn drop(self: Pin<&mut Self>) {
+///         unsafe { bindings::destroy_info(self.raw_info) };
+///     }
+/// }
+/// ```
+pub use ::pin_init_internal::pinned_drop;
+
+/// Derives the [`Zeroable`] trait for the given `struct` or `union`.
+///
+/// This can only be used for `struct`s/`union`s where every field implements the [`Zeroable`]
+/// trait.
+///
+/// # Examples
+///
+/// ```
+/// use pin_init::Zeroable;
+///
+/// #[derive(Zeroable)]
+/// pub struct DriverData {
+///     pub(crate) id: i64,
+///     buf_ptr: *mut u8,
+///     len: usize,
+/// }
+/// ```
+///
+/// ```
+/// use pin_init::Zeroable;
+///
+/// #[derive(Zeroable)]
+/// pub union SignCast {
+///     signed: i64,
+///     unsigned: u64,
+/// }
+/// ```
+pub use ::pin_init_internal::Zeroable;
+
+/// Derives the [`Zeroable`] trait for the given `struct` or `union` if all fields implement
+/// [`Zeroable`].
+///
+/// Contrary to the derive macro named [`macro@Zeroable`], this one silently fails when a field
+/// doesn't implement [`Zeroable`].
+///
+/// # Examples
+///
+/// ```
+/// use pin_init::MaybeZeroable;
+///
+/// // implements `Zeroable`
+/// #[derive(MaybeZeroable)]
+/// pub struct DriverData {
+///     pub(crate) id: i64,
+///     buf_ptr: *mut u8,
+///     len: usize,
+/// }
+///
+/// // does not implement `Zeroable`
+/// #[derive(MaybeZeroable)]
+/// pub struct DriverData2 {
+///     pub(crate) id: i64,
+///     buf_ptr: *mut u8,
+///     len: usize,
+///     // this field doesn't implement `Zeroable`
+///     other_data: &'static i32,
+/// }
+/// ```
+pub use ::pin_init_internal::MaybeZeroable;
+
+/// Initialize and pin a type directly on the stack.
+///
+/// # Examples
+///
+/// ```rust
+/// # #![feature(allocator_api)]
+/// # #[path = "../examples/mutex.rs"] mod mutex; use mutex::*;
+/// # use pin_init::*;
+/// # use core::pin::Pin;
+/// #[pin_data]
+/// struct Foo {
+///     #[pin]
+///     a: CMutex<usize>,
+///     b: Bar,
+/// }
+///
+/// #[pin_data]
+/// struct Bar {
+///     x: u32,
+/// }
+///
+/// stack_pin_init!(let foo = pin_init!(Foo {
+///     a <- CMutex::new(42),
+///     b: Bar {
+///         x: 64,
+///     },
+/// }));
+/// let foo: Pin<&mut Foo> = foo;
+/// println!("a: {}", &*foo.a.lock());
+/// ```
+///
+/// # Syntax
+///
+/// A normal `let` binding with optional type annotation. The expression is expected to implement
+/// [`PinInit`]/[`Init`] with the error type [`Infallible`]. If you want to use a different error
+/// type, then use [`stack_try_pin_init!`].
+#[macro_export]
+macro_rules! stack_pin_init {
+    (let $var:ident $(: $t:ty)? = $val:expr) => {
+        let val = $val;
+        let mut $var = ::core::pin::pin!($crate::__internal::StackInit$(::<$t>)?::uninit());
+        let mut $var = match $crate::__internal::StackInit::init($var, val) {
+            Ok(res) => res,
+            Err(x) => {
+                let x: ::core::convert::Infallible = x;
+                match x {}
+            }
+        };
+    };
+}
+
+/// Initialize and pin a type directly on the stack.
+///
+/// # Examples
+///
+/// ```rust
+/// # #![feature(allocator_api)]
+/// # #[path = "../examples/error.rs"] mod error; use error::Error;
+/// # #[path = "../examples/mutex.rs"] mod mutex; use mutex::*;
+/// # use pin_init::*;
+/// #[pin_data]
+/// struct Foo {
+///     #[pin]
+///     a: CMutex<usize>,
+///     b: Box<Bar>,
+/// }
+///
+/// struct Bar {
+///     x: u32,
+/// }
+///
+/// stack_try_pin_init!(let foo: Foo = pin_init!(Foo {
+///     a <- CMutex::new(42),
+///     b: Box::try_new(Bar {
+///         x: 64,
+///     })?,
+/// }? Error));
+/// let foo = foo.unwrap();
+/// println!("a: {}", &*foo.a.lock());
+/// ```
+///
+/// ```rust
+/// # #![feature(allocator_api)]
+/// # #[path = "../examples/error.rs"] mod error; use error::Error;
+/// # #[path = "../examples/mutex.rs"] mod mutex; use mutex::*;
+/// # use pin_init::*;
+/// #[pin_data]
+/// struct Foo {
+///     #[pin]
+///     a: CMutex<usize>,
+///     b: Box<Bar>,
+/// }
+///
+/// struct Bar {
+///     x: u32,
+/// }
+///
+/// stack_try_pin_init!(let foo: Foo =? pin_init!(Foo {
+///     a <- CMutex::new(42),
+///     b: Box::try_new(Bar {
+///         x: 64,
+///     })?,
+/// }? Error));
+/// println!("a: {}", &*foo.a.lock());
+/// # Ok::<_, Error>(())
+/// ```
+///
+/// # Syntax
+///
+/// A normal `let` binding with optional type annotation. The expression is expected to implement
+/// [`PinInit`]/[`Init`]. This macro assigns a result to the given variable, adding a `?` after the
+/// `=` will propagate this error.
+#[macro_export]
+macro_rules! stack_try_pin_init {
+    (let $var:ident $(: $t:ty)? = $val:expr) => {
+        let val = $val;
+        let mut $var = ::core::pin::pin!($crate::__internal::StackInit$(::<$t>)?::uninit());
+        let mut $var = $crate::__internal::StackInit::init($var, val);
+    };
+    (let $var:ident $(: $t:ty)? =? $val:expr) => {
+        let val = $val;
+        let mut $var = ::core::pin::pin!($crate::__internal::StackInit$(::<$t>)?::uninit());
+        let mut $var = $crate::__internal::StackInit::init($var, val)?;
+    };
+}
+
+/// Construct an in-place, fallible pinned initializer for `struct`s.
+///
+/// The error type defaults to [`Infallible`]; if you need a different one, write `? Error` at the
+/// end, after the struct initializer.
+///
+/// The syntax is almost identical to that of a normal `struct` initializer:
+///
+/// ```rust
+/// # use pin_init::*;
+/// # use core::pin::Pin;
+/// #[pin_data]
+/// struct Foo {
+///     a: usize,
+///     b: Bar,
+/// }
+///
+/// #[pin_data]
+/// struct Bar {
+///     x: u32,
+/// }
+///
+/// # fn demo() -> impl PinInit<Foo> {
+/// let a = 42;
+///
+/// let initializer = pin_init!(Foo {
+///     a,
+///     b: Bar {
+///         x: 64,
+///     },
+/// });
+/// # initializer }
+/// # Box::pin_init(demo()).unwrap();
+/// ```
+///
+/// Arbitrary Rust expressions can be used to set the value of a variable.
+///
+/// The fields are initialized in the order that they appear in the initializer. So it is possible
+/// to read already initialized fields using raw pointers.
+///
+/// IMPORTANT: You are not allowed to create references to fields of the struct inside of the
+/// initializer.
+///
+/// # Init-functions
+///
+/// When working with this library it is often desired to let others construct your types without
+/// giving access to all fields. This is where you would normally write a plain function `new` that
+/// would return a new instance of your type. With this library that is also possible. However,
+/// there are a few extra things to keep in mind.
+///
+/// To create an initializer function, simply declare it like this:
+///
+/// ```rust
+/// # use pin_init::*;
+/// # use core::pin::Pin;
+/// # #[pin_data]
+/// # struct Foo {
+/// #     a: usize,
+/// #     b: Bar,
+/// # }
+/// # #[pin_data]
+/// # struct Bar {
+/// #     x: u32,
+/// # }
+/// impl Foo {
+///     fn new() -> impl PinInit<Self> {
+///         pin_init!(Self {
+///             a: 42,
+///             b: Bar {
+///                 x: 64,
+///             },
+///         })
+///     }
+/// }
+/// ```
+///
+/// Users of `Foo` can now create it like this:
+///
+/// ```rust
+/// # use pin_init::*;
+/// # use core::pin::Pin;
+/// # #[pin_data]
+/// # struct Foo {
+/// #     a: usize,
+/// #     b: Bar,
+/// # }
+/// # #[pin_data]
+/// # struct Bar {
+/// #     x: u32,
+/// # }
+/// # impl Foo {
+/// #     fn new() -> impl PinInit<Self> {
+/// #         pin_init!(Self {
+/// #             a: 42,
+/// #             b: Bar {
+/// #                 x: 64,
+/// #             },
+/// #         })
+/// #     }
+/// # }
+/// let foo = Box::pin_init(Foo::new());
+/// ```
+///
+/// They can also easily embed it into their own `struct`s:
+///
+/// ```rust
+/// # use pin_init::*;
+/// # use core::pin::Pin;
+/// # #[pin_data]
+/// # struct Foo {
+/// #     a: usize,
+/// #     b: Bar,
+/// # }
+/// # #[pin_data]
+/// # struct Bar {
+/// #     x: u32,
+/// # }
+/// # impl Foo {
+/// #     fn new() -> impl PinInit<Self> {
+/// #         pin_init!(Self {
+/// #             a: 42,
+/// #             b: Bar {
+/// #                 x: 64,
+/// #             },
+/// #         })
+/// #     }
+/// # }
+/// #[pin_data]
+/// struct FooContainer {
+///     #[pin]
+///     foo1: Foo,
+///     #[pin]
+///     foo2: Foo,
+///     other: u32,
+/// }
+///
+/// impl FooContainer {
+///     fn new(other: u32) -> impl PinInit<Self> {
+///         pin_init!(Self {
+///             foo1 <- Foo::new(),
+///             foo2 <- Foo::new(),
+///             other,
+///         })
+///     }
+/// }
+/// ```
+///
+/// Here we see that when using `pin_init!` with `PinInit`, one needs to write `<-` instead of `:`.
+/// This signifies that the given field is initialized in-place. As with `struct` initializers, just
+/// writing the field (in this case `other`) without `:` or `<-` means `other: other,`.
+///
+/// # Syntax
+///
+/// As already mentioned in the examples above, inside of `pin_init!` a `struct` initializer with
+/// the following modifications is expected:
+/// - Fields that you want to initialize in-place have to use `<-` instead of `:`.
+/// - You can use `_: { /* run any user-code here */ },` anywhere where you can place fields in
+///   order to run arbitrary code.
+/// - In front of the initializer you can write `&this in` to have access to a [`NonNull<Self>`]
+///   pointer named `this` inside of the initializer.
+/// - Using struct update syntax one can place `..Zeroable::init_zeroed()` at the very end of the
+///   struct, this initializes every field with 0 and then runs all initializers specified in the
+///   body. This can only be done if [`Zeroable`] is implemented for the struct.
+///
+/// For instance:
+///
+/// ```rust
+/// # use pin_init::*;
+/// # use core::marker::PhantomPinned;
+/// #[pin_data]
+/// #[derive(Zeroable)]
+/// struct Buf {
+///     // `ptr` points into `buf`.
+///     ptr: *mut u8,
+///     buf: [u8; 64],
+///     #[pin]
+///     pin: PhantomPinned,
+/// }
+///
+/// let init = pin_init!(&this in Buf {
+///     buf: [0; 64],
+///     // SAFETY: TODO.
+///     ptr: unsafe { (&raw mut (*this.as_ptr()).buf).cast() },
+///     pin: PhantomPinned,
+/// });
+/// let init = pin_init!(Buf {
+///     buf: [1; 64],
+///     ..Zeroable::init_zeroed()
+/// });
+/// ```
+///
+/// [`NonNull<Self>`]: core::ptr::NonNull
+pub use pin_init_internal::pin_init;
+
+/// Construct an in-place, fallible initializer for `struct`s.
+///
+/// This macro defaults the error to [`Infallible`]; if you need a different one, write `? Error`
+/// at the end, after the struct initializer.
+///
+/// The syntax is identical to [`pin_init!`] and its safety caveats also apply:
+/// - `unsafe` code must guarantee either full initialization or return an error and allow
+///   deallocation of the memory.
+/// - the fields are initialized in the order given in the initializer.
+/// - no references to fields are allowed to be created inside of the initializer.
+///
+/// This initializer is for initializing data in-place that might later be moved. If you want to
+/// pin-initialize, use [`pin_init!`].
+///
+/// # Examples
+///
+/// ```rust
+/// # #![feature(allocator_api)]
+/// # #[path = "../examples/error.rs"] mod error; use error::Error;
+/// # #[path = "../examples/mutex.rs"] mod mutex; use mutex::*;
+/// # use pin_init::InPlaceInit;
+/// use pin_init::{init, Init, init_zeroed};
+///
+/// struct BigBuf {
+///     small: [u8; 1024 * 1024],
+/// }
+///
+/// impl BigBuf {
+///     fn new() -> impl Init<Self> {
+///         init!(Self {
+///             small <- init_zeroed(),
+///         })
+///     }
+/// }
+/// # let _ = Box::init(BigBuf::new());
+/// ```
+pub use pin_init_internal::init;
+
+/// Asserts that a field on a struct using `#[pin_data]` is marked with `#[pin]` ie. that it is
+/// structurally pinned.
+///
+/// # Examples
+///
+/// This will succeed:
+/// ```
+/// use pin_init::{pin_data, assert_pinned};
+///
+/// #[pin_data]
+/// struct MyStruct {
+///     #[pin]
+///     some_field: u64,
+/// }
+///
+/// assert_pinned!(MyStruct, some_field, u64);
+/// ```
+///
+/// This will fail:
+/// ```compile_fail
+/// use pin_init::{pin_data, assert_pinned};
+///
+/// #[pin_data]
+/// struct MyStruct {
+///     some_field: u64,
+/// }
+///
+/// assert_pinned!(MyStruct, some_field, u64);
+/// ```
+///
+/// Some uses of the macro may trigger the `can't use generic parameters from outer item` error. To
+/// work around this, you may pass the `inline` parameter to the macro. The `inline` parameter can
+/// only be used when the macro is invoked from a function body.
+/// ```
+/// # use core::pin::Pin;
+/// use pin_init::{pin_data, assert_pinned};
+///
+/// #[pin_data]
+/// struct Foo<T> {
+///     #[pin]
+///     elem: T,
+/// }
+///
+/// impl<T> Foo<T> {
+///     fn project_this(self: Pin<&mut Self>) -> Pin<&mut T> {
+///         assert_pinned!(Foo<T>, elem, T, inline);
+///
+///         // SAFETY: The field is structurally pinned.
+///         unsafe { self.map_unchecked_mut(|me| &mut me.elem) }
+///     }
+/// }
+/// ```
+#[macro_export]
+macro_rules! assert_pinned {
+    ($ty:ty, $field:ident, $field_ty:ty, inline) => {
+        // SAFETY: This code is unreachable.
+        let _ = move |ptr: *mut $ty| unsafe {
+            let data = <$ty as $crate::__internal::HasPinData>::__pin_data();
+            _ = data
+                .$field(ptr)
+                .init($crate::__internal::AlwaysFail::<$field_ty>::new());
+        };
+    };
+
+    ($ty:ty, $field:ident, $field_ty:ty) => {
+        const _: () = {
+            $crate::assert_pinned!($ty, $field, $field_ty, inline);
+        };
+    };
+}
+
+/// A pin-initializer for the type `T`.
+///
+/// To use this initializer, you will need a suitable memory location that can hold a `T`. This can
+/// be [`Box<T>`], [`Arc<T>`] or even the stack (see [`stack_pin_init!`]).
+///
+/// Also see the [module description](self).
+///
+/// # Safety
+///
+/// When implementing this trait you will need to take great care. Also there are probably very few
+/// cases where a manual implementation is necessary. Use [`pin_init_from_closure`] where possible.
+///
+/// The [`PinInit::__init`] function:
+/// - returns `Ok(())` if it initialized every field of `slot`,
+/// - returns `Err(err)` if it encountered an error and then cleaned `slot`, this means:
+///     - `slot` can be deallocated without UB occurring,
+///     - `slot` does not need to be dropped,
+///     - `slot` is not partially initialized.
+/// - while constructing the `T` at `slot` it upholds the pinning invariants of `T`.
+///
+#[cfg_attr(
+    kernel,
+    doc = "[`Arc<T>`]: https://rust.docs.kernel.org/kernel/sync/struct.Arc.html"
+)]
+#[cfg_attr(
+    kernel,
+    doc = "[`Box<T>`]: https://rust.docs.kernel.org/kernel/alloc/kbox/struct.Box.html"
+)]
+#[cfg_attr(not(kernel), doc = "[`Arc<T>`]: alloc::alloc::sync::Arc")]
+#[cfg_attr(not(kernel), doc = "[`Box<T>`]: alloc::alloc::boxed::Box")]
+#[must_use = "An initializer must be used in order to create its value."]
+pub unsafe trait PinInit<T: ?Sized, E = Infallible>: Sized {
+    /// Alias of [`PinInit::__init`].
+    ///
+    /// New code should use `__init` instead.
+    ///
+    /// # Safety
+    ///
+    /// Same as `__init`.
+    #[inline(always)]
+    #[cfg(not(kernel))]
+    #[deprecated = "use `raw_try_init` instead"]
+    unsafe fn __pinned_init(self, slot: *mut T) -> Result<(), E> {
+        // SAFETY: Per safety requirement.
+        unsafe { self.__init(slot) }
+    }
+
+    /// Initializes `slot`.
+    ///
+    /// It is not recommended to call this directly. Use [`raw_init`] or [`raw_try_init`].
+    ///
+    /// # Safety
+    ///
+    /// - `slot` is a valid pointer to uninitialized memory.
+    /// - the caller does not touch `slot` when `Err` is returned, they are only permitted to
+    ///   deallocate.
+    /// - `slot` will not move until it is dropped, i.e. it will be pinned.
+    ///   If `Self: Init<T, E>`, this requirement is cancelled and it may be moved.
+    unsafe fn __init(self, slot: *mut T) -> Result<(), E>;
+
+    /// First initializes the value using `self` then calls the function `f` with the initialized
+    /// value.
+    ///
+    /// If `f` returns an error the value is dropped and the initializer will forward the error.
+    ///
+    /// # Examples
+    ///
+    /// ```rust
+    /// # #![feature(allocator_api)]
+    /// # #[path = "../examples/mutex.rs"] mod mutex; use mutex::*;
+    /// # use pin_init::*;
+    /// let mtx_init = CMutex::new(42);
+    /// // Make the initializer print the value.
+    /// let mtx_init = mtx_init.pin_chain(|mtx| {
+    ///     println!("{:?}", mtx.get_data_mut());
+    ///     Ok(())
+    /// });
+    /// ```
+    #[inline]
+    fn pin_chain<F>(self, f: F) -> ChainPinInit<Self, F, T, E>
+    where
+        F: FnOnce(Pin<&mut T>) -> Result<(), E>,
+    {
+        ChainPinInit(self, f, __internal::PhantomInvariant::new())
+    }
+}
+
+/// Initializes `slot` with an initializer.
+///
+/// # Safety
+///
+/// - `slot` is a valid pointer to uninitialized memory.
+/// - `slot` will not move until it is dropped, i.e. it will be pinned.
+///   If `init` implements `Init<T, E>`, this requirement is cancelled and it may be moved.
+#[inline(always)]
+pub unsafe fn raw_init<T>(slot: *mut T, init: impl PinInit<T>) {
+    // SAFETY: Per safety requirement.
+    unsafe { init.__init(slot).unwrap_or_else(|e| match e {}) }
+}
+
+/// Fallibly initializes `slot` with an initializer.
+///
+/// # Safety
+///
+/// - `slot` is a valid pointer to uninitialized memory.
+/// - the caller does not touch `slot` when `Err` is returned, they are only permitted to
+///   deallocate.
+/// - `slot` will not move until it is dropped, i.e. it will be pinned.
+///   If `init` implements `Init<T, E>`, this requirement is cancelled and it may be moved.
+#[inline(always)]
+pub unsafe fn raw_try_init<T, E>(slot: *mut T, init: impl PinInit<T, E>) -> Result<(), E> {
+    // SAFETY: Per safety requirement.
+    unsafe { init.__init(slot) }
+}
+
+/// An initializer returned by [`PinInit::pin_chain`].
+pub struct ChainPinInit<I, F, T: ?Sized, E>(I, F, __internal::PhantomInvariant<(E, T)>);
+
+// SAFETY: The `__init` function is implemented such that it
+// - returns `Ok(())` on successful initialization,
+// - returns `Err(err)` on error and in this case `slot` will be dropped.
+// - considers `slot` pinned.
+unsafe impl<T: ?Sized, E, I, F> PinInit<T, E> for ChainPinInit<I, F, T, E>
+where
+    I: PinInit<T, E>,
+    F: FnOnce(Pin<&mut T>) -> Result<(), E>,
+{
+    #[inline]
+    unsafe fn __init(self, slot: *mut T) -> Result<(), E> {
+        // SAFETY: All requirements fulfilled since this function is `__init`.
+        let slot = unsafe { __internal::Slot::<__internal::Pinned, _>::new(slot) };
+        let mut guard = slot.init(self.0)?;
+        (self.1)(guard.let_binding())?;
+        core::mem::forget(guard);
+        Ok(())
+    }
+}
+
+/// An initializer for `T`.
+///
+/// To use this initializer, you will need a suitable memory location that can hold a `T`. This can
+/// be [`Box<T>`], [`Arc<T>`] or even the stack (see [`stack_pin_init!`]). Because
+/// [`PinInit<T, E>`] is a super trait, you can use every function that takes it as well.
+///
+/// Also see the [module description](self).
+///
+/// # Safety
+///
+/// When implementing this trait you will need to take great care. Also there are probably very few
+/// cases where a manual implementation is necessary. Use [`init_from_closure`] where possible.
+///
+/// The [`PinInit::__init`] function must work without the pinning requirement; the caller is
+/// allowed to move the pointee after initialization.
+///
+#[cfg_attr(
+    kernel,
+    doc = "[`Arc<T>`]: https://rust.docs.kernel.org/kernel/sync/struct.Arc.html"
+)]
+#[cfg_attr(
+    kernel,
+    doc = "[`Box<T>`]: https://rust.docs.kernel.org/kernel/alloc/kbox/struct.Box.html"
+)]
+#[cfg_attr(not(kernel), doc = "[`Arc<T>`]: alloc::alloc::sync::Arc")]
+#[cfg_attr(not(kernel), doc = "[`Box<T>`]: alloc::alloc::boxed::Box")]
+#[must_use = "An initializer must be used in order to create its value."]
+pub unsafe trait Init<T: ?Sized, E = Infallible>: PinInit<T, E> {
+    /// First initializes the value using `self` then calls the function `f` with the initialized
+    /// value.
+    ///
+    /// If `f` returns an error the value is dropped and the initializer will forward the error.
+    ///
+    /// # Examples
+    ///
+    /// ```rust
+    /// use pin_init::{init, init_zeroed, Init};
+    ///
+    /// struct Foo {
+    ///     buf: [u8; 1_000_000],
+    /// }
+    ///
+    /// impl Foo {
+    ///     fn setup(&mut self) {
+    ///         println!("Setting up foo");
+    ///     }
+    /// }
+    ///
+    /// let foo = init!(Foo {
+    ///     buf <- init_zeroed()
+    /// }).chain(|foo| {
+    ///     foo.setup();
+    ///     Ok(())
+    /// });
+    /// ```
+    #[inline]
+    fn chain<F>(self, f: F) -> ChainInit<Self, F, T, E>
+    where
+        F: FnOnce(&mut T) -> Result<(), E>,
+    {
+        ChainInit(self, f, __internal::PhantomInvariant::new())
+    }
+}
+
+/// An initializer returned by [`Init::chain`].
+pub struct ChainInit<I, F, T: ?Sized, E>(I, F, __internal::PhantomInvariant<(E, T)>);
+
+// SAFETY: The `__init` function does not rely on the pinning requirement.
+unsafe impl<T: ?Sized, E, I, F> Init<T, E> for ChainInit<I, F, T, E>
+where
+    I: Init<T, E>,
+    F: FnOnce(&mut T) -> Result<(), E>,
+{
+}
+
+// SAFETY: The `__init` function is implemented such that it
+// - returns `Ok(())` on successful initialization,
+// - returns `Err(err)` on error and in this case `slot` will be dropped.
+unsafe impl<T: ?Sized, E, I, F> PinInit<T, E> for ChainInit<I, F, T, E>
+where
+    I: Init<T, E>,
+    F: FnOnce(&mut T) -> Result<(), E>,
+{
+    #[inline]
+    unsafe fn __init(self, slot: *mut T) -> Result<(), E> {
+        // SAFETY: All requirements fulfilled since this function is `__init`.
+        let slot = unsafe { __internal::Slot::<__internal::Unpinned, _>::new(slot) };
+        let mut guard = slot.init(self.0)?;
+        (self.1)(guard.let_binding())?;
+        core::mem::forget(guard);
+        Ok(())
+    }
+}
+
+/// Implement `PinInit` and `Init` for closures.
+///
+/// It is unsafe to create this type, since the closure needs to fulfill the same safety
+/// requirement as the `__init` functions.
+struct InitClosure<F, T: ?Sized>(F, __internal::PhantomInvariant<T>);
+
+// SAFETY: When constructing via `init_from_closure`, the `__init` function does not rely on the
+// pinning requirement. When constructing via `pin_init_from_closure`, the opaque type prevents this
+// implementation from being visible.
+unsafe impl<T: ?Sized, F, E> Init<T, E> for InitClosure<F, T> where
+    F: FnOnce(*mut T) -> Result<(), E>
+{
+}
+
+// SAFETY: While constructing the `InitClosure`, the user promised that it upholds the
+// `__init` invariants.
+unsafe impl<T: ?Sized, F, E> PinInit<T, E> for InitClosure<F, T>
+where
+    F: FnOnce(*mut T) -> Result<(), E>,
+{
+    #[inline]
+    unsafe fn __init(self, slot: *mut T) -> Result<(), E> {
+        (self.0)(slot)
+    }
+}
+
+/// Creates a new [`PinInit<T, E>`] from the given closure.
+///
+/// # Safety
+///
+/// The closure:
+/// - returns `Ok(())` if it initialized every field of `slot`,
+/// - returns `Err(err)` if it encountered an error and then cleaned `slot`, this means:
+///     - `slot` can be deallocated without UB occurring,
+///     - `slot` does not need to be dropped,
+///     - `slot` is not partially initialized.
+/// - may assume that the `slot` does not move if `T: !Unpin`,
+/// - while constructing the `T` at `slot` it upholds the pinning invariants of `T`.
+#[inline]
+pub const unsafe fn pin_init_from_closure<T: ?Sized, E>(
+    f: impl FnOnce(*mut T) -> Result<(), E>,
+) -> impl PinInit<T, E> {
+    InitClosure(f, __internal::PhantomInvariant::new())
+}
+
+/// Creates a new [`Init<T, E>`] from the given closure.
+///
+/// # Safety
+///
+/// The closure:
+/// - returns `Ok(())` if it initialized every field of `slot`,
+/// - returns `Err(err)` if it encountered an error and then cleaned `slot`, this means:
+///     - `slot` can be deallocated without UB occurring,
+///     - `slot` does not need to be dropped,
+///     - `slot` is not partially initialized.
+/// - the `slot` may move after initialization.
+/// - while constructing the `T` at `slot` it upholds the pinning invariants of `T`.
+#[inline]
+pub const unsafe fn init_from_closure<T: ?Sized, E>(
+    f: impl FnOnce(*mut T) -> Result<(), E>,
+) -> impl Init<T, E> {
+    InitClosure(f, __internal::PhantomInvariant::new())
+}
+
+/// Changes the to be initialized type.
+///
+/// # Safety
+///
+/// - `*mut U` must be castable to `*mut T` and any value of type `T` written through such a
+///   pointer must result in a valid `U`.
+#[inline]
+pub const unsafe fn cast_pin_init<T, U, E>(init: impl PinInit<T, E>) -> impl PinInit<U, E> {
+    // SAFETY: initialization delegated to a valid initializer. Cast is valid by function safety
+    // requirements.
+    unsafe { pin_init_from_closure(|ptr: *mut U| init.__init(ptr.cast::<T>())) }
+}
+
+/// Changes the to be initialized type.
+///
+/// # Safety
+///
+/// - `*mut U` must be castable to `*mut T` and any value of type `T` written through such a
+///   pointer must result in a valid `U`.
+#[inline]
+pub const unsafe fn cast_init<T, U, E>(init: impl Init<T, E>) -> impl Init<U, E> {
+    // SAFETY: initialization delegated to a valid initializer. Cast is valid by function safety
+    // requirements.
+    unsafe { init_from_closure(|ptr: *mut U| init.__init(ptr.cast::<T>())) }
+}
+
+/// An initializer that leaves the memory uninitialized.
+///
+/// The initializer is a no-op. The `slot` memory is not changed.
+#[inline]
+pub fn uninit<T, E>() -> impl Init<MaybeUninit<T>, E> {
+    // SAFETY: The memory is allowed to be uninitialized.
+    unsafe { init_from_closure(|_| Ok(())) }
+}
+
+/// Array initializer from element initializer.
+struct ArrayInit<T: ?Sized, F>(F, __internal::PhantomInvariant<T>);
+
+// SAFETY: On success, all `N` elements of the array have been initialized. On error or panic, the
+// elements that have been initialized so far are dropped, thus leaving the array uninitialized and
+// ready to deallocate.
+unsafe impl<T, F, I, E, const N: usize> PinInit<[T; N], E> for ArrayInit<T, F>
+where
+    F: FnMut(usize) -> I,
+    I: PinInit<T, E>,
+{
+    unsafe fn __init(mut self, slot: *mut [T; N]) -> Result<(), E> {
+        /// # Invariants
+        ///
+        /// - `ptr[..num_init]` contains initialized elements of type `T`
+        /// - `ptr[num_init..N]` (where N is the size of the array) contains uninitialized memory
+        struct ArrayInitGuard<T> {
+            /// A pointer to the first element of the array.
+            ptr: *mut T,
+            /// The number of initialized elements in the array.
+            num_init: usize,
+        }
+
+        impl<T> Drop for ArrayInitGuard<T> {
+            #[inline]
+            fn drop(&mut self) {
+                // SAFETY: Per type invariant, `self.ptr[..self.num_init]` are initialized.
+                unsafe {
+                    core::ptr::drop_in_place(core::ptr::slice_from_raw_parts_mut(
+                        self.ptr,
+                        self.num_init,
+                    ))
+                };
+            }
+        }
+
+        // INVARIANT: nothing is initialized yet.
+        let mut guard = ArrayInitGuard {
+            ptr: slot.cast::<T>(),
+            num_init: 0,
+        };
+
+        for i in 0..N {
+            // INVARIANT: Elements `self.ptr[..self.num_init]` have been initialized
+            // thus far. This holds true for every `self.num_init = i`.
+            guard.num_init = i;
+
+            let init = (self.0)(i);
+            // SAFETY:
+            // - The subslot is derived from `slot` with a valid offset.
+            // - If `Err` is touched, the subslot is not touched further, the guard will drop
+            //   previously initialized elements only.
+            // - `slot` is pinned so is the subslot.
+            unsafe { init.__init(&raw mut (*slot)[i]) }?;
+        }
+
+        // Dismiss the drop guard now that all elements are initialized.
+        core::mem::forget(guard);
+        Ok(())
+    }
+}
+
+// SAFETY: `I: Init` cancels out the pinning requirement on subslots, which is the only place in the
+// `__init` function that relies on `slot` being pinned.
+unsafe impl<T, F, I, E, const N: usize> Init<[T; N], E> for ArrayInit<T, F>
+where
+    F: FnMut(usize) -> I,
+    I: Init<T, E>,
+{
+}
+
+/// Initializes an array by initializing each element via the provided initializer.
+///
+/// # Examples
+///
+/// ```rust
+/// # use pin_init::*;
+/// use pin_init::init_array_from_fn;
+/// let array: Box<[usize; 1_000]> = Box::init(init_array_from_fn(|i| i)).unwrap();
+/// assert_eq!(array.len(), 1_000);
+/// ```
+#[inline]
+pub fn init_array_from_fn<I, const N: usize, T, E>(
+    make_init: impl FnMut(usize) -> I,
+) -> impl Init<[T; N], E>
+where
+    I: Init<T, E>,
+{
+    ArrayInit(make_init, __internal::PhantomInvariant::new())
+}
+
+/// Initializes an array by initializing each element via the provided initializer.
+///
+/// # Examples
+///
+/// ```rust
+/// # #![feature(allocator_api)]
+/// # #[path = "../examples/mutex.rs"] mod mutex; use mutex::*;
+/// # use pin_init::*;
+/// # use core::pin::Pin;
+/// use pin_init::pin_init_array_from_fn;
+/// use std::sync::Arc;
+/// let array: Pin<Arc<[CMutex<usize>; 1_000]>> =
+///     Arc::pin_init(pin_init_array_from_fn(|i| CMutex::new(i))).unwrap();
+/// assert_eq!(array.len(), 1_000);
+/// ```
+#[inline]
+pub fn pin_init_array_from_fn<I, const N: usize, T, E>(
+    make_init: impl FnMut(usize) -> I,
+) -> impl PinInit<[T; N], E>
+where
+    I: PinInit<T, E>,
+{
+    ArrayInit(make_init, __internal::PhantomInvariant::new())
+}
+
+/// Construct an initializer in a closure and run it.
+///
+/// Returns an initializer that first runs the closure and then the initializer returned by it.
+///
+/// See also [`init_scope`].
+///
+/// # Examples
+///
+/// ```
+/// # use pin_init::*;
+/// # #[pin_data]
+/// # struct Foo { a: u64, b: isize }
+/// # struct Bar { a: u32, b: isize }
+/// # fn lookup_bar() -> Result<Bar, Error> { todo!() }
+/// # struct Error;
+/// fn init_foo() -> impl PinInit<Foo, Error> {
+///     pin_init_scope(|| {
+///         let bar = lookup_bar()?;
+///         Ok(pin_init!(Foo { a: bar.a.into(), b: bar.b }? Error))
+///     })
+/// }
+/// ```
+///
+/// This initializer will first execute `lookup_bar()`, match on it, if it returned an error, the
+/// initializer itself will fail with that error. If it returned `Ok`, then it will run the
+/// initializer returned by the [`pin_init!`] invocation.
+#[inline]
+pub fn pin_init_scope<T, E, F, I>(make_init: F) -> impl PinInit<T, E>
+where
+    F: FnOnce() -> Result<I, E>,
+    I: PinInit<T, E>,
+{
+    // SAFETY:
+    // - If `make_init` returns `Err`, `Err` is returned and `slot` is completely uninitialized,
+    // - If `make_init` returns `Ok`, safety requirement are fulfilled by `init.__init`.
+    // - The safety requirements of `init.__init` are fulfilled, since it's being called from an
+    //   initializer.
+    unsafe {
+        pin_init_from_closure(move |slot: *mut T| -> Result<(), E> {
+            let init = make_init()?;
+            init.__init(slot)
+        })
+    }
+}
+
+/// Construct an initializer in a closure and run it.
+///
+/// Returns an initializer that first runs the closure and then the initializer returned by it.
+///
+/// See also [`pin_init_scope`].
+///
+/// # Examples
+///
+/// ```
+/// # use pin_init::*;
+/// # struct Foo { a: u64, b: isize }
+/// # struct Bar { a: u32, b: isize }
+/// # fn lookup_bar() -> Result<Bar, Error> { todo!() }
+/// # struct Error;
+/// fn init_foo() -> impl Init<Foo, Error> {
+///     init_scope(|| {
+///         let bar = lookup_bar()?;
+///         Ok(init!(Foo { a: bar.a.into(), b: bar.b }? Error))
+///     })
+/// }
+/// ```
+///
+/// This initializer will first execute `lookup_bar()`, match on it, if it returned an error, the
+/// initializer itself will fail with that error. If it returned `Ok`, then it will run the
+/// initializer returned by the [`init!`] invocation.
+#[inline]
+pub fn init_scope<T, E, F, I>(make_init: F) -> impl Init<T, E>
+where
+    F: FnOnce() -> Result<I, E>,
+    I: Init<T, E>,
+{
+    // SAFETY:
+    // - If `make_init` returns `Err`, `Err` is returned and `slot` is completely uninitialized,
+    // - If `make_init` returns `Ok`, safety requirement are fulfilled by `init.__init`.
+    // - The safety requirements of `init.__init` are fulfilled, since it's being called from an
+    //   initializer.
+    unsafe {
+        init_from_closure(move |slot: *mut T| -> Result<(), E> {
+            let init = make_init()?;
+            init.__init(slot)
+        })
+    }
+}
+
+// SAFETY: The `__init` function does not rely on slot being pinned after it returns.
+unsafe impl<T> Init<T> for T {}
+
+// SAFETY: the `__init` function always returns `Ok(())` and initializes every field of
+// `slot`. Additionally, all pinning invariants of `T` are upheld.
+unsafe impl<T> PinInit<T> for T {
+    #[inline]
+    unsafe fn __init(self, slot: *mut T) -> Result<(), Infallible> {
+        // SAFETY: `slot` is valid for writes by the safety requirements of this function.
+        unsafe { slot.write(self) };
+        Ok(())
+    }
+}
+
+// SAFETY: The `__init` function does not rely on slot being pinned after it returns.
+unsafe impl<T, E> Init<T, E> for Result<T, E> {}
+
+// SAFETY: when the `__init` function returns with
+// - `Ok(())`, `slot` was initialized and all pinned invariants of `T` are upheld.
+// - `Err(err)`, slot was not written to.
+unsafe impl<T, E> PinInit<T, E> for Result<T, E> {
+    #[inline]
+    unsafe fn __init(self, slot: *mut T) -> Result<(), E> {
+        // SAFETY: `slot` is valid for writes by the safety requirements of this function.
+        unsafe { slot.write(self?) };
+        Ok(())
+    }
+}
+
+/// Smart pointer containing uninitialized memory and that can write a value.
+pub trait InPlaceWrite<T> {
+    /// The type `Self` turns into when the contents are initialized.
+    type Initialized;
+
+    /// Use the given initializer to write a value into `self`.
+    ///
+    /// Does not drop the current value and considers it as uninitialized memory.
+    fn write_init<E>(self, init: impl Init<T, E>) -> Result<Self::Initialized, E>;
+
+    /// Use the given pin-initializer to write a value into `self`.
+    ///
+    /// Does not drop the current value and considers it as uninitialized memory.
+    fn write_pin_init<E>(self, init: impl PinInit<T, E>) -> Result<Pin<Self::Initialized>, E>;
+}
+
+impl<T> InPlaceWrite<T> for &'static mut MaybeUninit<T> {
+    type Initialized = &'static mut T;
+
+    #[inline]
+    fn write_init<E>(self, init: impl Init<T, E>) -> Result<Self::Initialized, E> {
+        let slot = self.as_mut_ptr();
+
+        // SAFETY: `slot` is a valid pointer to uninitialized memory.
+        unsafe { init.__init(slot)? };
+
+        // SAFETY: The above call initialized the memory.
+        unsafe { Ok(self.assume_init_mut()) }
+    }
+
+    #[inline]
+    fn write_pin_init<E>(self, init: impl PinInit<T, E>) -> Result<Pin<Self::Initialized>, E> {
+        let slot = self.as_mut_ptr();
+
+        // SAFETY: `slot` is a valid pointer to uninitialized memory.
+        //
+        // The `'static` borrow guarantees the data will not be
+        // moved/invalidated until it gets dropped (which is never).
+        unsafe { init.__init(slot)? };
+
+        // SAFETY: The above call initialized the memory.
+        Ok(Pin::static_mut(unsafe { self.assume_init_mut() }))
+    }
+}
+
+/// Trait facilitating pinned destruction.
+///
+/// Use [`pinned_drop`] to implement this trait safely:
+///
+/// ```rust
+/// # #![feature(allocator_api)]
+/// # #[path = "../examples/mutex.rs"] mod mutex; use mutex::*;
+/// # use pin_init::*;
+/// use core::pin::Pin;
+/// #[pin_data(PinnedDrop)]
+/// struct Foo {
+///     #[pin]
+///     mtx: CMutex<usize>,
+/// }
+///
+/// #[pinned_drop]
+/// impl PinnedDrop for Foo {
+///     fn drop(self: Pin<&mut Self>) {
+///         println!("Foo is being dropped!");
+///     }
+/// }
+/// ```
+///
+/// # Safety
+///
+/// This trait must be implemented via the [`pinned_drop`] proc-macro attribute on the impl.
+pub unsafe trait PinnedDrop: __internal::HasPinData {
+    /// Executes the pinned destructor of this type.
+    ///
+    /// While this function is marked safe, it is actually unsafe to call it manually. For this
+    /// reason it takes an additional parameter. This type can only be constructed by `unsafe` code
+    /// and thus prevents this function from being called where it should not.
+    ///
+    /// This extra parameter will be generated by the `#[pinned_drop]` proc-macro attribute
+    /// automatically.
+    fn drop(self: Pin<&mut Self>, only_call_from_drop: __internal::OnlyCallFromDrop);
+}
+
+/// Marker trait for types that can be initialized by writing just zeroes.
+///
+/// # Safety
+///
+/// The bit pattern consisting of only zeroes is a valid bit pattern for this type. In other words,
+/// this is not UB:
+///
+/// ```rust,ignore
+/// let val: Self = unsafe { core::mem::zeroed() };
+/// ```
+pub unsafe trait Zeroable {
+    /// Create a new zeroed `Self`.
+    ///
+    /// The returned initializer will write `0x00` to every byte of the given `slot`.
+    #[inline]
+    fn init_zeroed() -> impl Init<Self>
+    where
+        Self: Sized,
+    {
+        init_zeroed()
+    }
+
+    /// Create a `Self` consisting of all zeroes.
+    ///
+    /// Whenever a type implements [`Zeroable`], this function should be preferred over
+    /// [`core::mem::zeroed()`] or using `MaybeUninit<T>::zeroed().assume_init()`.
+    ///
+    /// As const traits are not yet stable, [`pin_init::zeroed()`] can be used instead
+    /// when initialization is required in a `const` context.
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// use pin_init::Zeroable;
+    ///
+    /// #[derive(Zeroable)]
+    /// struct Point {
+    ///     x: u32,
+    ///     y: u32,
+    /// }
+    ///
+    /// let point: Point = Zeroable::zeroed();
+    /// assert_eq!(point.x, 0);
+    /// assert_eq!(point.y, 0);
+    /// ```
+    #[inline]
+    fn zeroed() -> Self
+    where
+        Self: Sized,
+    {
+        zeroed()
+    }
+}
+
+/// Create an initializer for a zeroed `T`.
+///
+/// The returned initializer will write `0x00` to every byte of the given `slot`.
+#[inline]
+pub fn init_zeroed<T: Zeroable>() -> impl Init<T> {
+    // SAFETY: Because `T: Zeroable`, all bytes zero is a valid bit pattern for `T`
+    // and because we write all zeroes, the memory is initialized.
+    unsafe {
+        init_from_closure(|slot: *mut T| {
+            slot.write_bytes(0, 1);
+            Ok(())
+        })
+    }
+}
+
+/// Create a `T` consisting of all zeroes.
+///
+/// Whenever a type implements [`Zeroable`], this function should be preferred over
+/// [`core::mem::zeroed()`] or using `MaybeUninit<T>::zeroed().assume_init()`.
+///
+/// While const traits remain unstable, this function serves as the `const` version of
+/// [`Zeroable::zeroed()`].
+///
+/// # Examples
+///
+/// ```
+/// use pin_init::{Zeroable, zeroed};
+///
+/// #[derive(Zeroable)]
+/// struct Point {
+///     x: u32,
+///     y: u32,
+/// }
+///
+/// let point: Point = zeroed();
+/// assert_eq!(point.x, 0);
+/// assert_eq!(point.y, 0);
+/// ```
+#[inline]
+pub const fn zeroed<T: Zeroable>() -> T {
+    // SAFETY:By the type invariants of `Zeroable`, all zeroes is a valid bit pattern for `T`.
+    unsafe { core::mem::zeroed() }
+}
+
+macro_rules! impl_zeroable {
+    ($($({$($generics:tt)*})? $t:ty, )*) => {
+        // SAFETY: Safety comments written in the macro invocation.
+        $(unsafe impl$($($generics)*)? Zeroable for $t {})*
+    };
+}
+
+impl_zeroable! {
+    // SAFETY: All primitives that are allowed to be zero.
+    bool,
+    char,
+    u8, u16, u32, u64, u128, usize,
+    i8, i16, i32, i64, i128, isize,
+    f32, f64,
+
+    // Note: do not add uninhabited types (such as `!` or `core::convert::Infallible`) to this list;
+    // creating an instance of an uninhabited type is immediate undefined behavior. For more on
+    // uninhabited/empty types, consult The Rustonomicon:
+    // <https://doc.rust-lang.org/stable/nomicon/exotic-sizes.html#empty-types>. The Rust Reference
+    // also has information on undefined behavior:
+    // <https://doc.rust-lang.org/stable/reference/behavior-considered-undefined.html>.
+    //
+    // SAFETY: These are inhabited ZSTs; there is nothing to zero and a valid value exists.
+    {<T: ?Sized>} PhantomData<T>, core::marker::PhantomPinned, (),
+
+    // SAFETY: Type is allowed to take any value, including all zeros.
+    {<T>} MaybeUninit<T>,
+
+    // SAFETY: `T: Zeroable` and `UnsafeCell` is `repr(transparent)`.
+    {<T: ?Sized + Zeroable>} UnsafeCell<T>,
+
+    // SAFETY: `null` pointer is valid.
+    //
+    // We cannot use `T: ?Sized`, since the VTABLE pointer part of fat pointers is not allowed to be
+    // null.
+    //
+    // When `Pointee` gets stabilized, we could use
+    // `T: ?Sized where <T as Pointee>::Metadata: Zeroable`
+    {<T>} *mut T, {<T>} *const T,
+
+    // SAFETY: `null` pointer is valid and the metadata part of these fat pointers is allowed to be
+    // zero.
+    {<T>} *mut [T], {<T>} *const [T], *mut str, *const str,
+
+    // SAFETY: `T` is `Zeroable`.
+    {<const N: usize, T: Zeroable>} [T; N], {<T: Zeroable>} Wrapping<T>,
+}
+
+macro_rules! impl_tuple_zeroable {
+    ($first:ident, $(,)?) => {
+        #[cfg_attr(all(USE_RUSTC_FEATURES, doc), doc(fake_variadic))]
+        /// Implemented for tuples up to 10 items long.
+        // SAFETY: All elements are zeroable and padding can be zero.
+        unsafe impl<$first: Zeroable> Zeroable for ($first,) {}
+    };
+    ($first:ident, $($t:ident),* $(,)?) => {
+        #[cfg_attr(doc, doc(hidden))]
+        // SAFETY: All elements are zeroable and padding can be zero.
+        unsafe impl<$first: Zeroable, $($t: Zeroable),*> Zeroable for ($first, $($t),*) {}
+        impl_tuple_zeroable!($($t),* ,);
+    }
+}
+
+impl_tuple_zeroable!(A, B, C, D, E, F, G, H, I, J);
+
+/// Marker trait for types that allow `Option<Self>` to be set to all zeroes in order to write
+/// `None` to that location.
+///
+/// # Safety
+///
+/// The implementer needs to ensure that `unsafe impl Zeroable for Option<Self> {}` is sound.
+pub unsafe trait ZeroableOption {}
+
+// SAFETY: by the safety requirement of `ZeroableOption`, this is valid.
+unsafe impl<T: ZeroableOption> Zeroable for Option<T> {}
+
+macro_rules! impl_fn_zeroable_option {
+    ([$($abi:literal),* $(,)?] $args:tt) => {
+        $(impl_fn_zeroable_option!({extern $abi} $args);)*
+        $(impl_fn_zeroable_option!({unsafe extern $abi} $args);)*
+    };
+    ({$($prefix:tt)*} {$(,)?}) => {};
+    ({$($prefix:tt)*} {$ret:ident, $arg:ident $(,)?}) => {
+        #[cfg_attr(all(USE_RUSTC_FEATURES, doc), doc(fake_variadic))]
+        /// Implemented for function pointers with up to 20 arity.
+        // SAFETY: function pointers are part of the option layout optimization:
+        // <https://doc.rust-lang.org/stable/std/option/index.html#representation>.
+        unsafe impl<$ret, $arg> ZeroableOption for $($prefix)* fn($arg) -> $ret {}
+        impl_fn_zeroable_option!({$($prefix)*} {$arg,});
+    };
+    ({$($prefix:tt)*} {$ret:ident, $($rest:ident),* $(,)?}) => {
+        #[cfg_attr(doc, doc(hidden))]
+        // SAFETY: function pointers are part of the option layout optimization:
+        // <https://doc.rust-lang.org/stable/std/option/index.html#representation>.
+        unsafe impl<$ret, $($rest),*> ZeroableOption for $($prefix)* fn($($rest),*) -> $ret {}
+        impl_fn_zeroable_option!({$($prefix)*} {$($rest),*,});
+    };
+}
+
+impl_fn_zeroable_option!(["Rust", "C"] { A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U });
+
+macro_rules! impl_zeroable_option {
+    ($($({$($generics:tt)*})? $t:ty, )*) => {
+        // SAFETY: Safety comments written in the macro invocation.
+        $(unsafe impl$($($generics)*)? ZeroableOption for $t {})*
+    };
+}
+
+impl_zeroable_option! {
+    // SAFETY: `Option<&T>` is part of the option layout optimization guarantee:
+    // <https://doc.rust-lang.org/stable/std/option/index.html#representation>.
+    {<T: ?Sized>} &T,
+    // SAFETY: `Option<&mut T>` is part of the option layout optimization guarantee:
+    // <https://doc.rust-lang.org/stable/std/option/index.html#representation>.
+    {<T: ?Sized>} &mut T,
+    // SAFETY: `Option<NonNull<T>>` is part of the option layout optimization guarantee:
+    // <https://doc.rust-lang.org/stable/std/option/index.html#representation>.
+    {<T: ?Sized>} NonNull<T>,
+    // SAFETY: All zeros is equivalent to `None` (option layout optimization guarantee:
+    // <https://doc.rust-lang.org/stable/std/option/index.html#representation>).
+    NonZero<u8>, NonZero<u16>, NonZero<u32>, NonZero<u64>, NonZero<u128>, NonZero<usize>,
+    NonZero<i8>, NonZero<i16>, NonZero<i32>, NonZero<i64>, NonZero<i128>, NonZero<isize>,
+}
+
+/// This trait allows creating an instance of `Self` which contains exactly one
+/// [structurally pinned value](https://doc.rust-lang.org/std/pin/index.html#projections-and-structural-pinning).
+///
+/// This is useful when using wrapper `struct`s like [`UnsafeCell`] or with new-type `struct`s.
+///
+/// # Examples
+///
+/// ```
+/// # use core::cell::UnsafeCell;
+/// # use pin_init::{pin_data, pin_init, Wrapper};
+///
+/// #[pin_data]
+/// struct Foo {}
+///
+/// #[pin_data]
+/// struct Bar {
+///     #[pin]
+///     content: UnsafeCell<Foo>
+/// };
+///
+/// let foo_initializer = pin_init!(Foo{});
+/// let initializer = pin_init!(Bar {
+///     content <- UnsafeCell::pin_init(foo_initializer)
+/// });
+/// ```
+pub trait Wrapper<T> {
+    /// Creates an pin-initializer for a [`Self`] containing `T` from the `value_init` initializer.
+    fn pin_init<E>(value_init: impl PinInit<T, E>) -> impl PinInit<Self, E>;
+}
+
+impl<T> Wrapper<T> for UnsafeCell<T> {
+    #[inline]
+    fn pin_init<E>(value_init: impl PinInit<T, E>) -> impl PinInit<Self, E> {
+        // SAFETY: `UnsafeCell<T>` has a compatible layout to `T`.
+        unsafe { cast_pin_init(value_init) }
+    }
+}
+
+impl<T> Wrapper<T> for MaybeUninit<T> {
+    #[inline]
+    fn pin_init<E>(value_init: impl PinInit<T, E>) -> impl PinInit<Self, E> {
+        // SAFETY: `MaybeUninit<T>` has a compatible layout to `T`.
+        unsafe { cast_pin_init(value_init) }
+    }
+}
+
+#[cfg(all(feature = "unsafe-pinned", CONFIG_RUSTC_HAS_UNSAFE_PINNED))]
+impl<T> Wrapper<T> for core::pin::UnsafePinned<T> {
+    #[inline]
+    fn pin_init<E>(init: impl PinInit<T, E>) -> impl PinInit<Self, E> {
+        // SAFETY: `UnsafePinned<T>` has a compatible layout to `T`.
+        unsafe { cast_pin_init(init) }
+    }
+}