diff options
| author | Paul E. McKenney <paulmck@kernel.org> | 2026-08-14 15:43:51 -0700 |
|---|---|---|
| committer | Paul E. McKenney <paulmck@kernel.org> | 2026-08-14 15:43:51 -0700 |
| commit | 9cc63f8bcd560c760d0b12e15bba8f81c86237cf (patch) | |
| tree | 3b5269b1bcf6714171754946367c51e0590f64da /io_uring/alloc_cache.h | |
| download | linux-stable-9cc63f8bcd560c760d0b12e15bba8f81c86237cf.tar.gz linux-stable-9cc63f8bcd560c760d0b12e15bba8f81c86237cf.zip | |
Merge branches 'expcb.2026.07.24a', 'misc.2026.07.30a', 'rcu-tasks.2026.07.30a', 'srcu.2026.08.11a' and 'torture.2026.08.14a' into HEADgrafted
Changes:
Make expedited grace periods expedite normal RCU callbacks
Miscellaneous fixes:
* Improve diagnostic output with character task states.
* Mark accesses to inform KCSAN of concurrency design.
* Move from kmalloc() to kmalloc_obj().
* Documentation updates.
* Improve handling of RCU deferred quiescent states.
* Clean up unused function arguments and structure fields.
* Reduce show_rcu_gp_kthreads() stack space.
Tasks RCU updates:
* Clean up after SRCU re-implementation of Tasks Trace RCU.
* Mark accesses to inform KCSAN of concurrency design.
* Add ->lazy_timer status to diagnostic output.
* Remove an unnecessary memory barrier.
* Fix a data race, courtesy of KCSAN.
* Documentation updates.
* Convert cond_resched_tasks_rcu_qs() from macro to static inline
function.
SRCU updates:
* Add Rust helpers for SRCU.
* Avoid losing queued work at cleanup_srcu_struct() time.
Torture-test updates:
* Preparation work for immediate RCU priority deboosting.
* Test RCU readers from real interrupt handlers (as opposed to softirq).
* Simplify code through use of cpumask_next_wrap().
* Improve diagnostic output with character task states.
* Add rcutorture.nwriters parameter to allow lightweight stall testing,
and rcutorture.stall_only to make doing so easier.
* Test an RCU Tasks Trace grace period implying an RCU grace period.
* Make RCU Tasks Trace torturing track reader batches.
* Fix a data race, courtesy of KCSAN.
* Plug a shuffle_tmp_mask memory leak on kthread spawn failure.
Diffstat (limited to 'io_uring/alloc_cache.h')
| -rw-r--r-- | io_uring/alloc_cache.h | 70 |
1 files changed, 70 insertions, 0 deletions
diff --git a/io_uring/alloc_cache.h b/io_uring/alloc_cache.h new file mode 100644 index 000000000..962b6e2d0 --- /dev/null +++ b/io_uring/alloc_cache.h @@ -0,0 +1,70 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef IOU_ALLOC_CACHE_H +#define IOU_ALLOC_CACHE_H + +#include <linux/io_uring_types.h> +#include <linux/kasan.h> + +/* + * Don't allow the cache to grow beyond this size. + */ +#define IO_ALLOC_CACHE_MAX 128 + +void io_alloc_cache_free(struct io_alloc_cache *cache, + void (*free)(const void *)); +bool io_alloc_cache_init(struct io_alloc_cache *cache, + unsigned max_nr, unsigned int size, + unsigned int init_bytes); + +void *io_cache_alloc_new(struct io_alloc_cache *cache, gfp_t gfp); + +static inline bool io_alloc_cache_put(struct io_alloc_cache *cache, + void *entry) +{ + if (cache->nr_cached < cache->max_cached) { + if (!kasan_mempool_poison_object(entry)) + return false; + cache->entries[cache->nr_cached++] = entry; + return true; + } + return false; +} + +static inline void *io_alloc_cache_get(struct io_alloc_cache *cache) +{ + if (cache->nr_cached) { + void *entry = cache->entries[--cache->nr_cached]; + + /* + * If KASAN is enabled, always clear the initial bytes that + * must be zeroed post alloc, in case any of them overlap + * with KASAN storage. + */ +#if defined(CONFIG_KASAN) + kasan_mempool_unpoison_object(entry, cache->elem_size); + if (cache->init_clear) + memset(entry, 0, cache->init_clear); +#endif + return entry; + } + + return NULL; +} + +static inline void *io_cache_alloc(struct io_alloc_cache *cache, gfp_t gfp) +{ + void *obj; + + obj = io_alloc_cache_get(cache); + if (obj) + return obj; + return io_cache_alloc_new(cache, gfp); +} + +static inline void io_cache_free(struct io_alloc_cache *cache, void *obj) +{ + if (!io_alloc_cache_put(cache, obj)) + kvfree(obj); +} + +#endif |
