A comprehensive deep-dive into Rust’s ownership model for developers ready to move beyond the fundamentals.
Prerequisites: This guide builds on the foundational concepts covered in Mastering Variables, Constants and Lifetimes in Rust. You should be comfortable with basic ownership rules, borrowing, move vs. copy semantics, and lifetime annotations before proceeding.
Table of Contents
Part I: Deep Ownership Mechanics
- Drop Semantics and RAII Patterns
- Memory Layout Internals
- Ownership Transfer Patterns
- Zero-Sized Types and Phantom Data
Part II: Advanced Move Semantics
- Partial Moves Mastery
- Move and Panic Interactions
- Closure Ownership (FnOnce/Fn/FnMut)
- Iterator Ownership Patterns
Part III: Advanced Borrowing
- Splitting Borrows and Field Sensitivity
- Interior Mutability Deep Dive
- Coercion and Deref Magic
- Variance and Lifetime Subtyping
Part IV: Lifetime Mastery
- Higher-Ranked Trait Bounds (HRTB)
- Variance Rules and Implications
- Self-Referential Structs and Pin
- Generic Associated Types (GATs)
Part V: Ownership in Practice
- Graph Structures and Arena Allocation
- Observer Patterns Without Cycles
- Plugin Architectures
- Real-World Case Studies
Part VI: Unsafe and Ownership
- Raw Pointer Ownership Conventions
- Building Safe Abstractions
- FFI Ownership Patterns
- ManuallyDrop and mem::forget
Part VII: Async Ownership
- Send and Sync Deep Dive
- Lifetime Challenges in Async
- Scoped Tasks and Non-Static Borrows
- Stream Ownership Patterns
Part VIII: Performance and Optimization
- Copy-on-Write Patterns (Cow)
- Small String Optimization
- Memory Locality Strategies
- Cache-Conscious Design
Part IX: Anti-Patterns and Debugging
- Common Ownership Mistakes
- Refactoring Strategies
- Debugging the Borrow Checker
- Tool Ecosystem
Part X: Rust 2024 Advanced Topics
- RPIT Capture Rules and use<>
- Async Closures and Ownership
- Advanced Temporary Scopes
- Future Ownership Directions
Part I: Deep Ownership Mechanics
Drop Semantics and RAII Patterns
The Drop trait is Rust’s mechanism for deterministic resource cleanup, implementing the RAII (Resource Acquisition Is Initialization) pattern.
Custom Drop Implementation:
struct FileGuard {
path: String,
handle: File,
}
impl Drop for FileGuard {
fn drop(&mut self) {
println!("Closing file: {}", self.path);
// File::drop() called automatically after this
}
}
fn main() {
let _guard = FileGuard {
path: "data.txt".into(),
handle: File::create("data.txt").unwrap(),
};
// Prints "Closing file: data.txt" when _guard goes out of scope
}
Drop Order Guarantees:
Rust guarantees specific drop order to prevent use-after-free:
- Fields: Dropped in declaration order (top to bottom)
- Tuples/Arrays: Dropped in index order (first to last)
- Variables: Dropped in reverse declaration order (LIFO)
struct Outer {
first: Inner,
second: Inner,
}
struct Inner(&'static str);
impl Drop for Inner {
fn drop(&mut self) {
println!("Dropping {}", self.0);
}
}
fn main() {
let _container = Outer {
first: Inner("first"),
second: Inner("second"),
};
// Output:
// Dropping first
// Dropping second
}
Drop and Panic Interactions:
If drop panics during unwinding from another panic, the program aborts immediately:
struct PanicDrop;
impl Drop for PanicDrop {
fn drop(&mut self) {
panic!("Drop panic!"); // ABORT if already unwinding
}
}
fn main() {
let _guard = PanicDrop;
panic!("First panic"); // Second panic in drop causes abort
}
The Drop Bomb Pattern:
Ensures cleanup operations are explicitly confirmed:
struct Transaction {
committed: bool,
}
impl Transaction {
fn new() -> Self {
Self { committed: false }
}
fn commit(mut self) {
self.committed = true;
println!("Transaction committed");
}
}
impl Drop for Transaction {
fn drop(&mut self) {
if !self.committed {
panic!("Transaction dropped without commit!");
}
}
}
fn main() {
let tx = Transaction::new();
tx.commit(); // Must explicitly commit
}
Why Drop and Copy Are Mutually Exclusive:
Copy types are duplicated bitwise without ownership transfer. If they implemented Drop, the same resource would be freed multiple times:
// This doesn't compile:
#[derive(Copy, Clone)]
struct Invalid;
impl Drop for Invalid {
fn drop(&mut self) {
// ERROR: Copy types cannot implement Drop
}
}
Memory Layout Internals
Understanding memory layout is crucial for performance optimization and FFI.
Struct Layout and Padding:
Rust automatically adds padding for alignment:
use std::mem::{size_of, align_of};
#[derive(Debug)]
struct Unoptimized {
a: u8, // 1 byte + 7 padding
b: u64, // 8 bytes (requires 8-byte alignment)
c: u16, // 2 bytes + 6 padding
}
#[derive(Debug)]
struct Optimized {
b: u64, // 8 bytes
c: u16, // 2 bytes
a: u8, // 1 byte + 5 padding
}
fn main() {
println!("Unoptimized: {} bytes", size_of::<Unoptimized>()); // 24 bytes
println!("Optimized: {} bytes", size_of::<Optimized>()); // 16 bytes
// Reorder fields from largest to smallest for optimal packing
}
Fat Pointers (Trait Objects and Slices):
Some pointers carry extra metadata:
use std::mem::size_of;
fn main() {
// Regular pointer: 8 bytes (on 64-bit)
let ptr: *const i32 = &42;
println!("Regular pointer: {} bytes", size_of::<*const i32>()); // 8
// Slice: pointer + length = 16 bytes
let slice: &[i32] = &[1, 2, 3];
println!("Slice ref: {} bytes", size_of::<&[i32]>()); // 16
// Trait object: pointer + vtable = 16 bytes
let trait_obj: &dyn std::fmt::Debug = &42;
println!("Trait object: {} bytes", size_of::<&dyn std::fmt::Debug>()); // 16
// String: pointer + length + capacity = 24 bytes
println!("String: {} bytes", size_of::<String>()); // 24
}
Representation Attributes:
Control struct layout for FFI and optimization:
// Default Rust layout (optimized, unspecified order)
#[repr(Rust)]
struct Default {
a: u8,
b: u32,
}
// C-compatible layout (stable ordering)
#[repr(C)]
struct CCompat {
a: u8,
// 3 bytes padding
b: u32,
}
// Packed layout (no padding, slower access)
#[repr(packed)]
struct Packed {
a: u8,
b: u32, // Only 1 byte after a, misaligned!
}
// Transparent (single-field wrapper, same layout as inner type)
#[repr(transparent)]
struct NewType(u32);
fn main() {
use std::mem::size_of;
println!("Default: {} bytes", size_of::<Default>()); // 8 (optimized)
println!("C: {} bytes", size_of::<CCompat>()); // 8 (explicit padding)
println!("Packed: {} bytes", size_of::<Packed>()); // 5 (no padding)
println!("NewType: {} bytes", size_of::<NewType>()); // 4 (same as u32)
}
Ownership Transfer Patterns
Ownership in Closures:
Closures capture variables differently based on the Fn trait they implement:
fn main() {
let data = vec![1, 2, 3];
// FnOnce: Takes ownership, can only be called once
let consume = || {
drop(data); // Consumes data
};
consume();
// consume(); // ERROR: cannot call twice
// Fn: Borrows immutably, can be called multiple times
let data2 = vec![4, 5, 6];
let borrow = || {
println!("{:?}", data2); // Immutable borrow
};
borrow();
borrow(); // OK: can call multiple times
// FnMut: Borrows mutably, can be called multiple times
let mut data3 = vec![7, 8, 9];
let mut mutate = || {
data3.push(10); // Mutable borrow
};
mutate();
mutate(); // OK: can call multiple times
println!("{:?}", data3); // [7, 8, 9, 10, 10]
}
Forcing Move Capture:
fn main() {
let data = vec![1, 2, 3];
// Force move even if only immutable access needed
let closure = move || {
println!("{:?}", data); // data moved into closure
};
// println!("{:?}", data); // ERROR: data was moved
closure();
}
Ownership with Async/Await:
use tokio;
async fn process_data(data: Vec<i32>) -> i32 {
// Ownership transferred into async block
data.iter().sum()
}
#[tokio::main]
async fn main() {
let numbers = vec![1, 2, 3, 4, 5];
let result = process_data(numbers).await;
// numbers moved, cannot use here
println!("Sum: {}", result);
}
Cross-Reference: The
movekeyword’s behavior with closures has important nuances, especially for types that areCopy. While ownership is fully transferred for non-Copytypes,Copytypes are duplicated. See the detailed note in the Closure Ownership section for a full explanation.
Zero-Sized Types and Phantom Data
Zero-Sized Types (ZSTs) occupy no memory and have zero-cost abstractions:
use std::mem::size_of;
struct ZeroSized;
struct Unit;
enum Never {}
fn main() {
println!("ZeroSized: {} bytes", size_of::<ZeroSized>()); // 0
println!("Unit: {} bytes", size_of::<Unit>()); // 0
println!("(): {} bytes", size_of::<()>()); // 0
println!("Never: {} bytes", size_of::<Never>()); // 0
// Moves of ZSTs compile to nothing
let zst1 = ZeroSized;
let zst2 = zst1; // No actual copy or move in assembly
}
PhantomData for Type-Level Programming:
PhantomData marks ownership without storing data:
use std::marker::PhantomData;
struct OwnedData<T> {
_marker: PhantomData<T>,
// Pretends to own T for drop checker
}
impl<T> OwnedData<T> {
fn new() -> Self {
Self {
_marker: PhantomData,
}
}
}
// Drop checker knows this type "owns" T
unsafe impl<T: Send> Send for OwnedData<T> {}
unsafe impl<T: Sync> Sync for OwnedData<T> {}
fn main() {
let _owned: OwnedData<String> = OwnedData::new();
// Behaves as if it owns String for trait bounds
}
Lifetime Variance Markers:
use std::marker::PhantomData;
struct Invariant<'a, T> {
_marker: PhantomData<&'a mut T>,
// Invariant in both 'a and T
}
struct Covariant<'a, T> {
_marker: PhantomData<&'a T>,
// Covariant in both 'a and T
}
struct Contravariant<T> {
_marker: PhantomData<fn(T)>,
// Contravariant in T
}
Part II: Advanced Move Semantics
Partial Moves Mastery
Partial moves allow extracting specific fields while leaving others accessible:
Match Ergonomics with Partial Moves:
#[derive(Debug)]
struct Config {
version: u32, // Copy
database_url: String, // Move
api_key: String, // Move
}
fn main() {
let config = Config {
version: 1,
database_url: String::from("postgres://localhost"),
api_key: String::from("secret_key"),
};
// Pattern matching with partial move
match config {
Config { version, ref database_url, .. } => {
println!("Version: {}", version); // version copied
println!("DB: {}", database_url); // database_url borrowed
}
}
// config.version still accessible (Copy)
println!("Version: {}", config.version);
// config.database_url still accessible (was borrowed, not moved)
println!("DB: {}", config.database_url);
// Can move api_key now
let key = config.api_key;
// println!("{:?}", config); // ERROR: config partially moved
}
Moving Out of Arrays:
fn main() {
let array = [
String::from("a"),
String::from("b"),
String::from("c"),
];
// Cannot move out of array by indexing
// let first = array; // ERROR
// Use pattern matching or into_iter
let [first, second, third] = array;
println!("{} {} {}", first, second, third);
// Alternative: into_iter
let array2 = [String::from("x"), String::from("y")];
for item in array2 {
println!("{}", item); // Moves each item
}
}
Workarounds for Move Restrictions:
fn main() {
let mut data = Some(String::from("hello"));
// Option::take moves out while leaving None
let value = data.take();
println!("{:?}", value); // Some("hello")
println!("{:?}", data); // None
// mem::replace swaps with a new value
use std::mem;
let mut data2 = String::from("world");
let old_value = mem::replace(&mut data2, String::from("new"));
println!("Old: {}, New: {}", old_value, data2);
// mem::take uses Default
let mut data3 = vec![1, 2, 3];
let taken = mem::take(&mut data3);
println!("Taken: {:?}, Remaining: {:?}", taken, data3); "Taken: [1, 2, 3], Remaining: []"
}
Move and Panic Interactions
Understanding move behavior during panics prevents resource leaks:
Partially Moved Structs on Panic:
struct Resource(&'static str);
impl Drop for Resource {
fn drop(&mut self) {
println!("Dropping {}", self.0);
}
}
fn main() {
let data = (Resource("first"), Resource("second"));
let result = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
let first = data.0; // Partial move
panic!("Oops!");
drop(first); // Never reached
}));
// Output during unwinding:
// "Dropping first" - first was moved to local, dropped during unwind
// Caught panic
// "Dropping second" - second remains in partially-moved data, dropped during unwind
match result {
Ok(_) => println!("No panic"),
Err(_) => println!("Caught panic"),
}
}
Panic Safety with Moves:
fn risky_operation(mut vec: Vec<String>) -> Vec<String>{
// Take ownership
let item = vec.pop().unwrap();
// If this panics, vec is moved but item is dropped
process_item(item);
// vec ownership transferred back through return
vec
}
fn process_item(s: String) {
// Might panic
println!("{}", s);
}
Closure Ownership (FnOnce/Fn/FnMut)
Deep dive into closure trait hierarchy:
fn demonstrate_closure_traits() {
let data = vec![1, 2, 3];
// Closure that only borrows
let print = || println!("{:?}", data);
call_fn(print); // Implements Fn
call_fn(print); // Can call multiple times
// Closure that mutates
let mut counter = 0;
let mut increment = || {
counter += 1;
println!("Count: {}", counter);
};
call_fn_mut(&mut increment); // Implements FnMut
call_fn_mut(&mut increment); // Can call multiple times
// Closure that consumes
let consume = || drop(data);
call_fn_once(consume); // Implements FnOnce
// call_fn_once(consume); // ERROR: can only call once
}
fn call_fn<F: Fn()>(f: F) {
f();
}
fn call_fn_mut<F: FnMut()>(f: &mut F) {
f();
}
fn call_fn_once<F: FnOnce()>(f: F) {
f();
}
fn main() {
demonstrate_closure_traits();
}
Trait Hierarchy:
FnOnce (base trait)
↑
FnMut (can be called multiple times with mutable access)
↑
Fn (can be called multiple times with shared access)
#### Note: Unexpected
moveClosure Behavior withCopyTypesA subtle but critical behavior exists in the Rust 2021 and 2024 editions (notably with
rustc 1.90.0) regardingmoveclosures and types that implement theCopytrait.
For
Copytypes (e.g.,i32,bool, simple structs with#[derive(Copy)]): When amoveclosure captures aCopytype, it captures a bitwise copy of the value. The original variable is not moved and remains fully accessible in its scope. The compiler will not issue a “use of moved value” error, which can be misleading.For non-
Copytypes (e.g.,String,Vec): Themovekeyword works as expected, transferring ownership to the closure and making the original variable inaccessible.This behavior is intentional, stemming from the semantics of the
Copytrait itself, but it is a known point of confusion. While no compiler error is generated, be aware that modifications inside the closure will only affect the copy, not the original variable. There is ongoing community discussion about adding a compiler lint to warn about this potentially surprising behavior in the future.
Iterator Ownership Patterns
Iterators have three forms with different ownership semantics:
fn main() {
let data = vec![String::from("a"), String::from("b")];
// iter(): borrows elements (&T)
for item in data.iter() {
println!("{}", item); // item: &String
}
println!("data still valid: {:?}", data);
// iter_mut(): mutably borrows elements (&mut T)
let mut data_mut = vec![String::from("x"), String::from("y")];
for item in data_mut.iter_mut() {
item.push_str("!"); // item: &mut String
}
println!("modified: {:?}", data_mut);
// into_iter(): takes ownership (T)
for item in data {
println!("{}", item); // item: String
} // data moved, no longer valid
// println!("{:?}", data); // ERROR
}
Custom Iterator Ownership:
struct DataIterator {
data: Vec<String>,
index: usize,
}
impl Iterator for DataIterator {
type Item = String; // Owns items
fn next(&mut self) -> Option<Self::Item> {
if self.index < self.data.len() {
let item = self.data.swap_remove(self.index);
Some(item)
} else {
None
}
}
}
fn main() {
let iter = DataIterator {
data: vec![String::from("a"), String::from("b")],
index: 0,
};
for item in iter {
println!("{}", item); // Owns each item
}
}
Part III: Advanced Borrowing
Splitting Borrows and Field Sensitivity
The borrow checker understands field-level granularity:
Splitting Slices:
fn main() {
let mut data = vec![1, 2, 3, 4, 5, 6];
// split_at_mut creates two non-overlapping mutable slices
let (left, right) = data.split_at_mut(3);
left[0] = 10;
right[0] = 40;
println!("{:?}", data); // [10, 2, 3, 40, 5, 6]
}
Splitting Struct Fields:
struct Point {
x: i32,
y: i32,
}
fn main() {
let mut p = Point { x: 0, y: 0 };
// Can borrow different fields mutably
let x = &mut p.x;
let y = &mut p.y;
*x = 10;
*y = 20;
println!("Point: ({}, {})", p.x, p.y);
}
Limitations and Workarounds:
fn main() {
let mut data = (String::from("a"), String::from("b"));
// Conservative: compiler can't prove disjoint access
let first = &mut data.0;
// let second = &mut data.1; // ERROR in some contexts
// Workaround: destructure
let (ref mut first, ref mut second) = data;
first.push_str("pple");
second.push_str("anana");
println!("{}, {}", first, second);
}
Interior Mutability Deep Dive
Interior mutability allows mutation through shared references:
Cell vs RefCell Comparison:
use std::cell::{Cell, RefCell};
fn main() {
// Cell: Copy types only, no runtime checking
let counter = Cell::new(0);
let ref1 = &counter;
let ref2 = &counter;
ref1.set(ref1.get() + 1);
ref2.set(ref2.get() + 1);
println!("Counter: {}", counter.get()); // 2
// RefCell: Any type, runtime borrow checking
let data = RefCell::new(vec![1, 2, 3]);
{
let mut borrowed = data.borrow_mut();
borrowed.push(4);
} // Mutable borrow ends
let borrowed = data.borrow();
println!("{:?}", *borrowed); // [1, 2, 3, 4]
}
Thread-Safe Interior Mutability:
use std::sync::{Arc, Mutex, RwLock};
fn main() {
// Mutex: Exclusive access, blocks threads
let counter = Arc::new(Mutex::new(0));
let handles: Vec<_> = (0..10).map(|_| {
let counter = Arc::clone(&counter);
std::thread::spawn(move || {
let mut num = counter.lock().unwrap();
*num += 1;
})
}).collect();
for handle in handles {
handle.join().unwrap();
}
println!("Counter: {}", *counter.lock().unwrap()); // 10
// RwLock: Multiple readers or one writer
let data = Arc::new(RwLock::new(vec![1, 2, 3]));
// Multiple readers
let data1 = Arc::clone(&data);
let data2 = Arc::clone(&data);
std::thread::scope(|s| {
s.spawn(|| {
let read = data1.read().unwrap();
println!("Read 1: {:?}", *read);
});
s.spawn(|| {
let read = data2.read().unwrap();
println!("Read 2: {:?}", *read);
});
});
}
Building Custom Interior Mutability:
use std::cell::UnsafeCell;
struct MyCell<T> {
value: UnsafeCell<T>,
}
impl<T> MyCell<T> {
fn new(value: T) -> Self {
Self {
value: UnsafeCell::new(value),
}
}
fn get(&self) -> &T {
unsafe { &*self.value.get() }
}
fn set(&self, value: T) {
unsafe {
*self.value.get() = value;
}
}
}
// SAFETY: This is a simplified example. Real implementation needs proper synchronization
Coercion and Deref Magic
Rust performs automatic coercions in specific contexts:
Deref Coercion Chains:
use std::rc::Rc;
fn print_str(s: &str) {
println!("{}", s);
}
fn main() {
let owned = String::from("hello");
print_str(&owned); // &String -> &str
let boxed = Box::new(String::from("world"));
print_str(&boxed); // &Box -> &String -> &str
let rc = Rc::new(String::from("Rust"));
print_str(&rc); // &Rc -> &String -> &str
}
Custom Deref Implementation:
use std::ops::Deref;
struct MyBox<T>(T);
impl<T> MyBox<T> {
fn new(value: T) -> Self {
MyBox(value)
}
}
impl<T> Deref for MyBox<T> {
type Target = T;
fn deref(&self) -> &T {
&self.0
}
}
fn main() {
let boxed = MyBox::new(String::from("Rust"));
// Automatic deref coercion
let len = boxed.len(); // MyBox -> String -> str
println!("Length: {}", len);
}
Variance and Lifetime Subtyping
Variance determines how lifetime relationships propagate through types:
Variance Rules:
| Type | Variance in 'a
|
Variance in T
|
|---|---|---|
&'a T |
Covariant | Covariant |
&'a mut T |
Covariant | Invariant |
*const T |
– | Covariant |
*mut T |
– | Invariant |
fn(T) -> U |
– |
Contravariant in T, Covariant in U
|
Cell |
– | Invariant |
UnsafeCell |
– | Invariant |
Covariance Example (&'a T):
fn assign<'a, 'b: 'a>(long: &'a str, short: &'b str) -> &'a str {
// 'b: 'a means 'b outlives 'a
// &'b str is a subtype of &'a str (covariant)
short // OK: can return shorter lifetime as longer lifetime
}
fn main() {
let long_lived = String::from("long");
let result;
{
let short_lived = String::from("short");
result = assign(&long_lived, &short_lived);
// short_lived dropped here
}
// println!("{}", result); // ERROR: result references short_lived
}
Invariance Example (&'a mut T):
fn attempt_shorten<'a, 'b>(long: &'a mut &'static str, short: &'b mut &'b str) {
// This would be unsound if &mut was covariant:
// *long = *short; // ERROR: Cannot assign &'b str to &'static str
// Invariance prevents this unsound operation
}
fn main() {
let mut static_ref: &'static str = "static";
let local = String::from("local");
let mut local_ref: &str = &local;
// attempt_shorten(&mut static_ref, &mut local_ref);
// If allowed, static_ref would point to dropped local!
}
Contravariance Example (fn(T)):
fn example() {
// Function that accepts &'static str
let f: fn(&'static str) = |s| println!("{}", s);
// Can use it where fn(&'a str) is expected (contravariant)
// A function accepting longer lifetimes works for shorter ones
call_with_local(f);
}
fn call_with_local<'a>(f: fn(&'a str)) {
let local = String::from("local");
f(&local);
}
fn main() {
example();
}
Practical Implications:
use std::cell::Cell;
fn main() {
// Covariant: Can substitute longer lifetime with shorter
let _covariant: &'static str = "hello";
let _shorter: &str = _covariant; // OK
// Invariant: Cannot substitute lifetimes
let cell: Cell<&'static str> = Cell::new("hello");
// let cell2: Cell<&str> = cell; // ERROR: invariant
// Why invariance matters:
let mut data = Some(&0);
{
let local = 42;
// If Cell<&'static str> was covariant:
// cell.set(&local); // Would compile but be unsound!
}
}
Part IV: Lifetime Mastery
Higher-Ranked Trait Bounds (HRTB)
HRTBs allow functions to work with any lifetime:
Basic HRTB Syntax:
// Function that works for any lifetime 'a
fn call_with_ref<F>(f: F)
where
F: for<'a> Fn(&'a str) -> &'a str, // HRTB
{
let data = String::from("hello");
let result = f(&data);
println!("{}", result);
}
fn main() {
call_with_ref(|s| s);
}
Why HRTBs Are Needed:
// Without HRTB (doesn't work):
trait Processor<'a> {
fn process(&self, input: &'a str) -> &'a str;
}
// With HRTB (works):
trait ProcessorHRTB {
fn process(&self, input: &str) -> &str; // Elided lifetime
}
fn use_processor<P>(processor: P)
where
P: for<'a> Fn(&'a str) -> &'a str,
{
let data1 = String::from("first");
let result1 = processor(&data1);
println!("{}", result1);
let data2 = String::from("second");
let result2 = processor(&data2);
println!("{}", result2);
}
fn main() {
use_processor(|s| s);
}
Common HRTB Patterns:
// Pattern 1: Closures that work with any lifetime
fn map_ref<F, T, U>(data: &[T], f: F) -> Vec<U>
where
F: for<'a> Fn(&'a T) -> U,
{
data.iter().map(|x| f(x)).collect()
}
// Pattern 2: Trait bounds with lifetime parameters
trait Apply {
fn apply<F>(&self, f: F)
where
F: for<'a> Fn(&'a Self);
}
impl<T> Apply for T {
fn apply<F>(&self, f: F)
where
F: for<'a> Fn(&'a Self),
{
f(self);
}
}
fn main() {
let numbers = vec![1, 2, 3];
let strings = map_ref(&numbers, |n| n.to_string());
println!("{:?}", strings);
42.apply(|x| println!("Value: {}", x));
}
Variance Rules and Implications
Understanding variance prevents lifetime errors:
Covariance: “If 'a outlives 'b, then Type<'a> is a subtype of Type<'b>“
fn covariant_example() {
let static_str: &'static str = "hello";
// &'static str is a subtype of &'a str for any 'a
fn takes_any<'a>(s: &'a str) {
println!("{}", s);
}
takes_any(static_str); // OK: covariant
}
Invariance: “No lifetime substitution allowed”
fn invariant_example() {
use std::cell::Cell;
let static_cell: Cell<&'static str> = Cell::new("hello");
// Cannot treat Cell<&'static str> as Cell<&'a str>
fn takes_cell<'a>(_cell: Cell<&'a str>) {}
// takes_cell(static_cell); // ERROR: invariant
}
Contravariance: “If 'a outlives 'b, then Type<'b> is a subtype of Type<'a>“
fn contravariant_example() {
// fn(&'static str) is a subtype of fn(&'a str)
let f: fn(&'static str) = |s| println!("{}", s);
fn call_with_short<'a>(f: fn(&'a str)) {
let local = String::from("local");
f(&local);
}
call_with_short(f); // OK: contravariant in argument position
}
Self-Referential Structs and Pin
Self-referential structs require special handling:
The Problem:
// This doesn't compile:
struct SelfReferential {
data: String,
pointer: *const String, // Points to data field
}
// Moving invalidates the pointer!
Solution: Pin:
use std::pin::Pin;
use std::marker::PhantomPinned;
struct SelfReferential {
data: String,
pointer: Option<*const String>,
_pin: PhantomPinned, // Makes this !Unpin
}
impl SelfReferential {
fn new(data: String) -> Pin<Box<Self>> {
let mut boxed = Box::pin(SelfReferential {
data,
pointer: None,
_pin: PhantomPinned,
});
// SAFETY: We never move out of Pin>
unsafe {
let mut_ref = Pin::as_mut(&mut boxed);
let ptr = &mut_ref.get_unchecked_mut().data as *const String;
mut_ref.get_unchecked_mut().pointer = Some(ptr);
}
boxed
}
fn get_pointer(&self) -> Option<&String> {
self.pointer.map(|p| unsafe { &*p })
}
}
fn main() {
let data = SelfReferential::new(String::from("hello"));
println!("Data: {:?}", data.get_pointer());
}
Pin in Async:
use std::future::Future;
use std::pin::Pin;
use std::task::{Context, Poll};
struct MyFuture {
data: String,
}
impl Future for MyFuture {
type Output = ();
fn poll(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll<Self::Output> {
// self is pinned, cannot move
println!("Polling with data: {}", self.data);
Poll::Ready(())
}
}
async fn use_future() {
let future = MyFuture {
data: String::from("async"),
};
future.await;
}
#[tokio::main]
async fn main() {
use_future().await;
}
Generic Associated Types (GATs)
GATs enable advanced trait designs with lifetime parameters:
Lending Iterator Pattern:
trait LendingIterator {
type Item<'a> where Self: 'a;
fn next<'a>(&'a mut self) -> Option<Self::Item<'a>>;
}
struct WindowsMut<'data, T> {
slice: &'data mut [T],
window_size: usize,
position: usize,
}
impl<'data, T> LendingIterator for WindowsMut<'data, T> {
type Item<'a> = &'a mut [T] where Self: 'a;
fn next<'a>(&'a mut self) -> Option<Self::Item<'a>> {
if self.position + self.window_size > self.slice.len() {
return None;
}
let start = self.position;
let end = start + self.window_size;
self.position += 1;
// SAFETY: We return non-overlapping windows
unsafe {
let ptr = self.slice.as_mut_ptr();
Some(std::slice::from_raw_parts_mut(
ptr.add(start),
self.window_size,
))
}
}
}
fn main() {
let mut data = vec![1, 2, 3, 4, 5];
let mut windows = WindowsMut {
slice: &mut data,
window_size: 3,
position: 0,
};
while let Some(window) = windows.next() {
println!("{:?}", window);
}
}
Higher-Kinded Types Emulation:
trait Container {
type Inner<T>;
fn wrap<T>(value: T) -> Self::Inner<T>;
fn unwrap<T>(container: Self::Inner<T>) -> T;
}
struct VecContainer;
impl Container for VecContainer {
type Inner<T> = Vec<T>;
fn wrap<T>(value: T) -> Vec<T> {
vec![value]
}
fn unwrap<T>(mut container: Vec<T>) -> T {
container.pop().expect("Empty vector")
}
}
fn main() {
let wrapped = VecContainer::wrap(42);
let unwrapped = VecContainer::unwrap(wrapped);
println!("{}", unwrapped);
}
Part V: Ownership in Practice
Graph Structures and Arena Allocation
Graphs are notoriously difficult in Rust due to cyclic references. Arena allocation provides an elegant solution.
The Graph Problem:
// This doesn't work - cannot have cyclic references
struct Node {
value: i32,
neighbors: Vec<Box<Node>>, // Each node owns its neighbors
// But neighbors also need to point back!
}
Solution: Arena Allocation with Typed Arenas:
use typed_arena::Arena;
use std::cell::Cell;
struct Node<'a> {
value: i32,
neighbors: Cell<Vec<&'a Node<'a>>>,
}
impl<'a> Node<'a> {
fn new(value: i32, arena: &'a Arena<Node<'a>>) -> &'a Node<'a> {
arena.alloc(Node {
value,
neighbors: Cell::new(Vec::new()),
})
}
fn add_edge(&self, neighbor: &'a Node<'a>) {
let mut neighbors = self.neighbors.take();
neighbors.push(neighbor);
self.neighbors.set(neighbors);
}
}
fn main() {
let arena = Arena::new();
// Create nodes
let node1 = Node::new(1, &arena);
let node2 = Node::new(2, &arena);
let node3 = Node::new(3, &arena);
// Build graph with cycles
node1.add_edge(node2);
node2.add_edge(node3);
node3.add_edge(node1); // Cycle!
println!("Graph built successfully");
// Arena deallocates everything at once
}
Generational Indices Pattern (Type-Safe Alternative):
use std::collections::HashMap;
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
struct NodeId {
index: usize,
generation: u64,
}
struct Node {
value: i32,
neighbors: Vec<NodeId>,
}
struct Graph {
nodes: HashMap<NodeId, Node>,
next_index: usize,
generation: u64,
}
impl Graph {
fn new() -> Self {
Self {
nodes: HashMap::new(),
next_index: 0,
generation: 0,
}
}
fn add_node(&mut self, value: i32) -> NodeId {
let id = NodeId {
index: self.next_index,
generation: self.generation,
};
self.nodes.insert(id, Node {
value,
neighbors: Vec::new(),
});
self.next_index += 1;
id
}
fn add_edge(&mut self, from: NodeId, to: NodeId) {
if let Some(node) = self.nodes.get_mut(&from) {
node.neighbors.push(to);
}
}
fn remove_node(&mut self, id: NodeId) {
self.nodes.remove(&id);
self.generation += 1; // Invalidate old references
}
fn get_node(&self, id: NodeId) -> Option<&Node> {
self.nodes.get(&id)
}
}
fn main() {
let mut graph = Graph::new();
let n1 = graph.add_node(1);
let n2 = graph.add_node(2);
let n3 = graph.add_node(3);
graph.add_edge(n1, n2);
graph.add_edge(n2, n3);
graph.add_edge(n3, n1); // Cycle is fine!
println!("Node 1: {:?}", graph.get_node(n1));
}
SlotMap Pattern (Production-Ready):
use slotmap::{SlotMap, DefaultKey};
struct Node {
value: i32,
neighbors: Vec<DefaultKey>,
}
struct Graph {
nodes: SlotMap<DefaultKey, Node>,
}
impl Graph {
fn new() -> Self {
Self {
nodes: SlotMap::new(),
}
}
fn add_node(&mut self, value: i32) -> DefaultKey {
self.nodes.insert(Node {
value,
neighbors: Vec::new(),
})
}
fn add_edge(&mut self, from: DefaultKey, to: DefaultKey) {
if let Some(node) = self.nodes.get_mut(from) {
node.neighbors.push(to);
}
}
fn traverse(&self, start: DefaultKey) {
let mut visited = std::collections::HashSet::new();
let mut stack = vec![start];
while let Some(id) = stack.pop() {
if !visited.insert(id) {
continue;
}
if let Some(node) = self.nodes.get(id) {
println!("Visiting: {}", node.value);
stack.extend(&node.neighbors);
}
}
}
}
fn main() {
let mut graph = Graph::new();
let n1 = graph.add_node(1);
let n2 = graph.add_node(2);
let n3 = graph.add_node(3);
graph.add_edge(n1, n2);
graph.add_edge(n2, n3);
graph.add_edge(n3, n1);
graph.traverse(n1);
}
Observer Patterns Without Cycles
Traditional observer patterns create reference cycles. Here are Rust-idiomatic alternatives:
Weak References Pattern:
use std::rc::{Rc, Weak};
use std::cell::RefCell;
trait Observer {
fn notify(&self, message: &str);
}
struct Subject {
observers: RefCell<Vec<Weak<dyn Observer>>>,
}
impl Subject {
fn new() -> Self {
Self {
observers: RefCell::new(Vec::new()),
}
}
fn subscribe(&self, observer: Weak<dyn Observer>) {
self.observers.borrow_mut().push(observer);
}
fn notify_all(&self, message: &str) {
// Clean up dead references while notifying
self.observers.borrow_mut().retain(|weak| {
if let Some(observer) = weak.upgrade() {
observer.notify(message);
true
} else {
false // Remove dead weak reference
}
});
}
}
struct ConcreteObserver {
id: u32,
}
impl Observer for ConcreteObserver {
fn notify(&self, message: &str) {
println!("Observer {} received: {}", self.id, message);
}
}
fn main() {
let subject = Subject::new();
{
let observer1 = Rc::new(ConcreteObserver { id: 1 });
let observer2 = Rc::new(ConcreteObserver { id: 2 });
subject.subscribe(Rc::downgrade(&observer1));
subject.subscribe(Rc::downgrade(&observer2));
subject.notify_all("Event 1");
// observer1 and observer2 alive here
}
// Observers dropped, weak references now dead
subject.notify_all("Event 2"); // No output
}
Channel-Based Observer (Async-Friendly):
use tokio::sync::broadcast;
struct EventBus {
sender: broadcast::Sender<String>,
}
impl EventBus {
fn new() -> Self {
let (sender, _) = broadcast::channel(100);
Self { sender }
}
fn subscribe(&self) -> broadcast::Receiver<String> {
self.sender.subscribe()
}
fn publish(&self, message: String) {
let _ = self.sender.send(message);
}
}
#[tokio::main]
async fn main() {
let bus = EventBus::new();
let mut rx1 = bus.subscribe();
let mut rx2 = bus.subscribe();
tokio::spawn(async move {
while let Ok(msg) = rx1.recv().await {
println!("Observer 1: {}", msg);
}
});
tokio::spawn(async move {
while let Ok(msg) = rx2.recv().await {
println!("Observer 2: {}", msg);
}
});
bus.publish("Event 1".to_string());
bus.publish("Event 2".to_string());
tokio::time::sleep(tokio::time::Duration::from_millis(100)).await;
}
Plugin Architectures
Building extensible systems with ownership guarantees:
Trait Object Plugin System:
trait Plugin: Send + Sync {
fn name(&self) -> &str;
fn execute(&self, input: &str) -> String;
}
struct PluginRegistry {
plugins: Vec<Box<dyn Plugin>>,
}
impl PluginRegistry {
fn new() -> Self {
Self {
plugins: Vec::new(),
}
}
fn register(&mut self, plugin: Box<dyn Plugin>) {
println!("Registered plugin: {}", plugin.name());
self.plugins.push(plugin);
}
fn execute_all(&self, input: &str) {
for plugin in &self.plugins {
let result = plugin.execute(input);
println!("{}: {}", plugin.name(), result);
}
}
}
struct UppercasePlugin;
impl Plugin for UppercasePlugin {
fn name(&self) -> &str {
"Uppercase"
}
fn execute(&self, input: &str) -> String {
input.to_uppercase()
}
}
struct ReversePlugin;
impl Plugin for ReversePlugin {
fn name(&self) -> &str {
"Reverse"
}
fn execute(&self, input: &str) -> String {
input.chars().rev().collect()
}
}
fn main() {
let mut registry = PluginRegistry::new();
registry.register(Box::new(UppercasePlugin));
registry.register(Box::new(ReversePlugin));
registry.execute_all("hello");
}
Dynamic Loading with Type-Erased Ownership:
use std::any::Any;
use std::collections::HashMap;
trait PluginFactory: Send + Sync {
fn create(&self) -> Box<dyn Any + Send>;
}
struct PluginManager {
factories: HashMap<String, Box<dyn PluginFactory>>,
}
impl PluginManager {
fn new() -> Self {
Self {
factories: HashMap::new(),
}
}
fn register<F>(&mut self, name: String, factory: F)
where
F: PluginFactory + 'static,
{
self.factories.insert(name, Box::new(factory));
}
fn create_plugin(&self, name: &str) -> Option<Box<dyn Any + Send>> {
self.factories.get(name).map(|f| f.create())
}
}
struct StringProcessorFactory;
impl PluginFactory for StringProcessorFactory {
fn create(&self) -> Box<dyn Any + Send> {
Box::new("StringProcessor".to_string())
}
}
fn main() {
let mut manager = PluginManager::new();
manager.register("processor".to_string(), StringProcessorFactory);
if let Some(plugin) = manager.create_plugin("processor") {
if let Some(processor) = plugin.downcast_ref::<String>() {
println!("Created: {}", processor);
}
}
}
Real-World Case Study: HTTP Router
Demonstrating ownership in a practical HTTP routing scenario:
use std::collections::HashMap;
type Handler = Box<dyn Fn(&Request) -> Response + Send + Sync>;
struct Request {
path: String,
method: String,
}
struct Response {
status: u16,
body: String,
}
struct Router {
routes: HashMap<(String, String), Handler>,
}
impl Router {
fn new() -> Self {
Self {
routes: HashMap::new(),
}
}
fn route<F>(&mut self, method: &str, path: &str, handler: F)
where
F: Fn(&Request) -> Response + Send + Sync + 'static,
{
self.routes.insert(
(method.to_string(), path.to_string()),
Box::new(handler),
);
}
fn handle(&self, request: &Request) -> Response {
let key = (request.method.clone(), request.path.clone());
if let Some(handler) = self.routes.get(&key) {
handler(request)
} else {
Response {
status: 404,
body: "Not Found".to_string(),
}
}
}
}
fn main() {
let mut router = Router::new();
// Closures capture their environment
let greeting = "Hello".to_string();
router.route("GET", "/hello", move |_req| Response {
status: 200,
body: format!("{}, World!", greeting), // Moved into closure
});
router.route("POST", "/echo", |req| Response {
status: 200,
body: req.path.clone(),
});
let req = Request {
path: "/hello".to_string(),
method: "GET".to_string(),
};
let res = router.handle(&req);
println!("Status: {}, Body: {}", res.status, res.body);
}
Part VI: Unsafe and Ownership
Raw Pointer Ownership Conventions
Raw pointers bypass Rust’s ownership system, requiring manual safety guarantees:
Ownership Conventions:
fn main() {
let data = vec![1, 2, 3];
let ptr = data.as_ptr();
// Convention 1: Pointer does not own data
// data still owns the Vec
unsafe {
println!("First element: {}", *ptr);
}
// Convention 2: Transfer ownership via raw pointer
let boxed = Box::new(42);
let raw = Box::into_raw(boxed); // Ownership transferred
// Must manually free
unsafe {
let _reclaimed = Box::from_raw(raw); // Ownership restored
} // Dropped here
}
Building a Safe Linked List:
use std::ptr;
struct Node<T> {
value: T,
next: Option<Box<Node<T>>>,
}
struct LinkedList<T> {
head: Option<Box<Node<T>>>,
}
impl<T> LinkedList<T> {
fn new() -> Self {
Self { head: None }
}
fn push(&mut self, value: T) {
let new_node = Box::new(Node {
value,
next: self.head.take(),
});
self.head = Some(new_node);
}
fn pop(&mut self) -> Option<T> {
self.head.take().map(|node| {
self.head = node.next;
node.value
})
}
// Unsafe peek implementation
fn peek_raw(&self) -> Option<*const T> {
self.head.as_ref().map(|node| {
&node.value as *const T
})
}
}
fn main() {
let mut list = LinkedList::new();
list.push(1);
list.push(2);
list.push(3);
if let Some(ptr) = list.peek_raw() {
unsafe {
println!("Peeked: {}", *ptr);
}
}
while let Some(value) = list.pop() {
println!("{}", value);
}
}
Building Safe Abstractions
Encapsulating unsafe code with safe APIs:
Safe Vec-Like Container:
use std::alloc::{alloc, dealloc, Layout};
use std::ptr;
struct MyVec<T> {
ptr: *mut T,
len: usize,
capacity: usize,
}
impl<T> MyVec<T> {
fn new() -> Self {
Self {
ptr: ptr::null_mut(),
len: 0,
capacity: 0,
}
}
fn push(&mut self, value: T) {
if self.len == self.capacity {
self.grow();
}
unsafe {
ptr::write(self.ptr.add(self.len), value);
}
self.len += 1;
}
fn pop(&mut self) -> Option<T> {
if self.len == 0 {
None
} else {
self.len -= 1;
unsafe {
Some(ptr::read(self.ptr.add(self.len)))
}
}
}
fn grow(&mut self) {
let new_capacity = if self.capacity == 0 {
1
} else {
self.capacity * 2
};
let new_layout = Layout::array::<T>(new_capacity).unwrap();
let new_ptr = if self.capacity == 0 {
unsafe { alloc(new_layout) as *mut T }
} else {
let old_layout = Layout::array::<T>(self.capacity).unwrap();
unsafe {
std::alloc::realloc(
self.ptr as *mut u8,
old_layout,
new_layout.size(),
) as *mut T
}
};
self.ptr = new_ptr;
self.capacity = new_capacity;
}
}
impl<T> Drop for MyVec<T> {
fn drop(&mut self) {
if self.capacity != 0 {
// Drop elements
while let Some(_) = self.pop() {}
// Deallocate memory
let layout = Layout::array::<T>(self.capacity).unwrap();
unsafe {
dealloc(self.ptr as *mut u8, layout);
}
}
}
}
fn main() {
let mut vec = MyVec::new();
vec.push(1);
vec.push(2);
vec.push(3);
println!("{:?}", vec.pop()); // Some(3)
println!("{:?}", vec.pop()); // Some(2)
}
FFI Ownership Patterns
Managing ownership across language boundaries:
C-Compatible Structs:
#[repr(C)]
struct Point {
x: f64,
y: f64,
}
// Ownership transfer to C
#[no_mangle]
pub extern "C" fn create_point(x: f64, y: f64) -> *mut Point {
let point = Box::new(Point { x, y });
Box::into_raw(point) // Transfer ownership
}
// Ownership transfer from C
#[no_mangle]
pub extern "C" fn destroy_point(ptr: *mut Point) {
if !ptr.is_null() {
unsafe {
let _ = Box::from_raw(ptr); // Reclaim ownership and drop
}
}
}
// Borrow from C
#[no_mangle]
pub extern "C" fn point_distance(ptr: *const Point) -> f64 {
if ptr.is_null() {
return 0.0;
}
unsafe {
let point = &*ptr; // Borrow, no ownership transfer
(point.x * point.x + point.y * point.y).sqrt()
}
}
fn main() {
let ptr = create_point(3.0, 4.0);
let distance = point_distance(ptr);
println!("Distance: {}", distance);
destroy_point(ptr);
}
ManuallyDrop and mem::forget
Preventing automatic drops when needed:
ManuallyDrop Usage:
use std::mem::ManuallyDrop;
struct Resource {
id: u32,
}
impl Drop for Resource {
fn drop(&mut self) {
println!("Dropping resource {}", self.id);
}
}
fn main() {
// Normal drop
{
let r = Resource { id: 1 };
} // Prints "Dropping resource 1"
// Prevent automatic drop
{
let mut r = ManuallyDrop::new(Resource { id: 2 });
// Must manually drop
unsafe {
ManuallyDrop::drop(&mut r);
}
} // No automatic drop
// mem::forget alternative (leaks memory)
{
let r = Resource { id: 3 };
std::mem::forget(r); // Never dropped, leaked!
}
}
Two-Phase Initialization:
use std::mem::ManuallyDrop;
struct Complex {
data: ManuallyDrop<Vec<u8>>,
initialized: bool,
}
impl Complex {
fn new() -> Self {
Self {
data: ManuallyDrop::new(Vec::new()),
initialized: false,
}
}
fn initialize(&mut self, size: usize) {
let mut vec = Vec::with_capacity(size);
vec.resize(size, 0);
self.data = ManuallyDrop::new(vec);
self.initialized = true;
}
}
impl Drop for Complex {
fn drop(&mut self) {
if self.initialized {
unsafe {
ManuallyDrop::drop(&mut self.data);
}
}
}
}
fn main() {
let mut complex = Complex::new();
complex.initialize(100);
}
Part VII: Async Ownership
Send and Sync Deep Dive
Understanding thread-safety requirements in async code:
Send vs Sync Explained:
use std::rc::Rc;
use std::sync::Arc;
fn main() {
// Send: Can be moved between threads
let arc = Arc::new(42);
std::thread::spawn(move || {
println!("{}", *arc); // OK: Arc is Send
});
// Rc is NOT Send
let rc = Rc::new(42);
// std::thread::spawn(move || {
// println!("{}", *rc); // ERROR: Rc is not Send
// });
// Sync: Can be shared between threads (&T is Send)
// &Arc is Send because Arc is Sync
let shared = Arc::new(42);
let shared_ref = &shared;
// If T: Sync, then &T: Send
// This means multiple threads can hold &T simultaneously
}
Auto Traits and Negative Impls:
use std::marker::PhantomData;
// Explicitly NOT Send
struct NotSend {
_marker: PhantomData<*const ()>,
}
// Explicitly NOT Sync
struct NotSync {
_marker: PhantomData<std::cell::Cell<()>>,
}
fn assert_send<T: Send>() {}
fn assert_sync<T: Sync>() {}
fn main() {
assert_send::<String>(); // OK
// assert_send::(); // ERROR
assert_sync::<i32>(); // OK
// assert_sync::(); // ERROR
}
Lifetime Challenges in Async
The 'static requirement and workarounds:
Problem: ‘static Requirement:
use tokio;
async fn process_data(data: &str) -> usize {
data.len()
}
#[tokio::main]
async fn main() {
let data = String::from("hello");
// This works: spawned task doesn't need 'static bound
tokio::spawn(async move {
let result = process_data(&data).await;
println!("Length: {}", result);
}).await.unwrap();
// This doesn't work: data doesn't live long enough
// let handle = tokio::spawn(process_data(&data));
// ERROR: `data` doesn't live 'static
}
Solution 1: Move Ownership:
use tokio;
#[tokio::main]
async fn main() {
let data = String::from("hello");
// Move data into task
tokio::spawn(async move {
let len = data.len(); // data moved into closure
println!("Length: {}", len);
}).await.unwrap();
// data no longer available here
}
Solution 2: Scoped Tasks (Rust 2024+):
use tokio;
#[tokio::main]
async fn main() {
let data = String::from("hello");
// Use tokio::task::scope for non-'static borrows
tokio::task::scope(|scope| {
scope.spawn(async {
println!("Length: {}", data.len()); // Borrow is fine!
});
}).await;
println!("Data still accessible: {}", data);
}
Solution 3: Arc for Sharing:
use std::sync::Arc;
use tokio;
#[tokio::main]
async fn main() {
let data = Arc::new(String::from("hello"));
let mut handles = vec![];
for i in 0..3 {
let data_clone = Arc::clone(&data);
let handle = tokio::spawn(async move {
println!("Task {}: {}", i, data_clone);
});
handles.push(handle);
}
for handle in handles {
handle.await.unwrap();
}
}
Scoped Tasks and Non-Static Borrows
Structured concurrency patterns:
use tokio;
async fn parallel_processing(items: &[String]) -> Vec<usize> {
let mut handles = Vec::with_capacity(items.len());
for item in items {
let item = item.clone(); // Clone to avoid lifetime issues
let handle = tokio::spawn(async move {
item.len()
});
handles.push(handle);
}
let mut results = Vec::with_capacity(items.len());
for handle in handles {
results.push(handle.await.unwrap());
}
results
}
#[tokio::main]
async fn main() {
let items = vec![
String::from("one"),
String::from("two"),
String::from("three"),
];
let results = parallel_processing(&items).await;
println!("Results: {:?}", results);
println!("Items: {:?}", items);
}
Stream Ownership Patterns
Handling ownership in async streams:
use tokio_stream::{self as stream, StreamExt};
async fn process_stream() {
let data = vec![1, 2, 3, 4, 5];
// Owned stream
let mut stream = stream::iter(data);
while let Some(item) = stream.next().await {
println!("Item: {}", item); // Owns each item
}
}
async fn borrowed_stream(data: &[i32]) {
let mut stream = stream::iter(data.iter());
while let Some(item) = stream.next().await {
println!("Item: {}", item); // Borrows each item
}
}
#[tokio::main]
async fn main() {
process_stream().await;
let data = vec![10, 20, 30];
borrowed_stream(&data).await;
println!("Data still valid: {:?}", data);
}
Part VIII: Performance and Optimization
Copy-on-Write Patterns (Cow)
Deferring clones until mutation is needed:
use std::borrow::Cow;
fn process_data(input: Cow<str>) -> Cow<str> {
if input.contains("bad") {
// Only clone if modification needed
Cow::Owned(input.replace("bad", "good"))
} else {
// Return borrowed data as-is
input
}
}
fn main() {
// Case 1: No modification needed (zero-copy)
let original = "hello world";
let result = process_data(Cow::Borrowed(original));
println!("{}", result); // Borrowed
// Case 2: Modification needed (cloned)
let original2 = "bad data";
let result2 = process_data(Cow::Borrowed(original2));
println!("{}", result2); // Owned: "good data"
}
Cow with Collections:
use std::borrow::Cow;
fn deduplicate<'a>(data: &'a [i32]) -> Cow<'a, [i32]> {
let mut seen = std::collections::HashSet::new();
let mut has_duplicates = false;
for &item in data {
if !seen.insert(item) {
has_duplicates = true;
break;
}
}
if has_duplicates {
// Clone only if duplicates found
let mut unique: Vec<i32> = data.to_vec();
unique.dedup();
Cow::Owned(unique)
} else {
// Return borrowed slice
Cow::Borrowed(data)
}
}
fn main() {
let no_dupes = vec![1, 2, 3, 4];
let result1 = deduplicate(&no_dupes);
println!("No dupes: {:?}", result1); // Borrowed
let has_dupes = vec![1, 2, 2, 3];
let result2 = deduplicate(&has_dupes);
println!("Has dupes: {:?}", result2); // Owned: [1, 2, 3]
}
Small String Optimization
Understanding inline storage patterns:
use std::mem::size_of;
fn main() {
// String always uses heap (24 bytes on 64-bit)
println!("String size: {}", size_of::<String>()); // 24
// Small strings could use inline storage with smartstring
use smartstring::alias::String as SmartString;
let small: SmartString = "hello".into(); // Inline (no heap)
let large: SmartString = "a".repeat(100).into(); // Heap
println!("SmartString size: {}", size_of::<SmartString>()); // 24
// Same size, but small strings avoid heap allocation
}
Custom Inline Storage:
use std::mem::ManuallyDrop;
use std::ptr;
const INLINE_CAPACITY: usize = 23;
enum SmallString {
Inline {
len: u8,
data: [u8; INLINE_CAPACITY],
},
Heap(String),
}
impl SmallString {
fn new(s: &str) -> Self {
if s.len() <= INLINE_CAPACITY {
let mut data = [0u8; INLINE_CAPACITY];
data[..s.len()].copy_from_slice(s.as_bytes());
SmallString::Inline {
len: s.len() as u8,
data,
}
} else {
SmallString::Heap(s.to_string())
}
}
fn as_str(&self) -> &str {
match self {
SmallString::Inline { len, data } => {
std::str::from_utf8(&data[..*len as usize]).unwrap()
}
SmallString::Heap(s) => s.as_str(),
}
}
}
fn main() {
let small = SmallString::new("hello");
let large = SmallString::new(&"x".repeat(100));
println!("Small: {}", small.as_str());
println!("Large: {}", large.as_str());
println!("SmallString size: {}", std::mem::size_of::<SmallString>());
}
Memory Locality Strategies
Optimizing cache performance through ownership choices:
Vec vs Vec:
use std::time::Instant;
#[derive(Clone)]
struct Data {
values: [u64; 8], // 64 bytes
}
fn benchmark_contiguous() {
let data: Vec<Data> = (0..10000)
.map(|i| Data { values: [i; 8] })
.collect();
let start = Instant::now();
let sum: u64 = data.iter().map(|d| d.values).sum();
let elapsed = start.elapsed();
println!("Contiguous: {:?}, sum: {}", elapsed, sum);
}
fn benchmark_boxed() {
let data: Vec<Box<Data>> = (0..10000)
.map(|i| Box::new(Data { values: [i; 8] }))
.collect();
let start = Instant::now();
let sum: u64 = data.iter().map(|d| d.values).sum();
let elapsed = start.elapsed();
println!("Boxed: {:?}, sum: {}", elapsed, sum);
}
fn main() {
benchmark_contiguous(); // Faster: better cache locality
benchmark_boxed(); // Slower: pointer chasing
}
Arena vs Individual Allocation:
use typed_arena::Arena;
use std::time::Instant;
struct Node {
value: i32,
}
fn benchmark_arena() {
let arena = Arena::new();
let start = Instant::now();
let nodes: Vec<&Node> = (0..10000)
.map(|i| arena.alloc(Node { value: i }))
.collect();
let elapsed = start.elapsed();
println!("Arena allocation: {:?}", elapsed);
}
fn benchmark_individual() {
let start = Instant::now();
let nodes: Vec<Box<Node>> = (0..10000)
.map(|i| Box::new(Node { value: i }))
.collect();
let elapsed = start.elapsed();
println!("Individual allocation: {:?}", elapsed);
}
fn main() {
benchmark_arena(); // Faster: single allocation
benchmark_individual(); // Slower: many allocations
}
Cache-Conscious Design
Structuring data for CPU cache efficiency:
// Bad: Struct of Arrays (scattered in memory)
struct ParticlesSoA {
positions_x: Vec<f32>,
positions_y: Vec<f32>,
velocities_x: Vec<f32>,
velocities_y: Vec<f32>,
}
// Good: Array of Structs (contiguous)
#[derive(Clone, Copy)]
struct Particle {
position_x: f32,
position_y: f32,
velocity_x: f32,
velocity_y: f32,
}
struct ParticlesAoS {
particles: Vec<Particle>,
}
fn update_aos(particles: &mut ParticlesAoS) {
// All particle data is contiguous
for particle in &mut particles.particles {
particle.position_x += particle.velocity_x;
particle.position_y += particle.velocity_y;
}
}
fn main() {
let mut particles = ParticlesAoS {
particles: vec![Particle {
position_x: 0.0,
position_y: 0.0,
velocity_x: 1.0,
velocity_y: 1.0,
}; 10000],
};
update_aos(&mut particles);
}
Part IX: Anti-Patterns and Debugging
Common Ownership Mistakes
Anti-Pattern 1: Clone Addiction:
// Bad: Unnecessary clones
fn bad_example(data: &str) -> String {
let owned = data.to_string(); // Clone 1
let copy = owned.clone(); // Clone 2
copy.to_uppercase() // Clone 3 (uppercase creates new String)
}
// Good: Minimal cloning
fn good_example(data: &str) -> String {
data.to_uppercase() // Only one allocation
}
fn main() {
let input = "hello";
println!("{}", bad_example(input));
println!("{}", good_example(input));
}
Anti-Pattern 2: Premature Arc>:
use std::sync::{Arc, Mutex};
// Bad: Arc> for everything
struct BadDesign {
data: Arc<Mutex<Vec<i32>>>,
}
// Good: Own when possible
struct GoodDesign {
data: Vec<i32>, // Simple ownership
}
// Only use Arc> when actually sharing
fn actually_need_sharing() {
let shared = Arc::new(Mutex::new(vec![1, 2, 3]));
let handles: Vec<_> = (0..3).map(|_| {
let data = Arc::clone(&shared);
std::thread::spawn(move || {
let mut guard = data.lock().unwrap();
guard.push(42);
})
}).collect();
for h in handles {
h.join().unwrap();
}
}
fn main() {
actually_need_sharing();
}
Anti-Pattern 3: Fighting the Borrow Checker:
// Bad: Trying to hold reference and mutate
fn bad_pattern(vec: &mut Vec<String>) {
let first = &vec; // Immutable borrow
// vec.push(String::from("new")); // ERROR: can't mutate while borrowed
// println!("{}", first);
}
// Good: Use indices or split operations
fn good_pattern(vec: &mut Vec<String>) {
let first_index = 0;
let first_value = vec[first_index].clone(); // Clone if needed
vec.push(String::from("new")); // Now we can mutate
println!("{}", first_value);
}
fn main() {
let mut data = vec![String::from("hello")];
good_pattern(&mut data);
}
Refactoring Strategies
Strategy 1: Moving from Owned to Borrowed:
// Before: Takes ownership
fn process_v1(data: Vec<i32>) -> i32 {
data.iter().sum()
} // data dropped here
// After: Borrows instead
fn process_v2(data: &[i32]) -> i32 {
data.iter().sum()
} // data still valid in caller
fn main() {
let numbers = vec![1, 2, 3];
// v1 requires move
// let sum = process_v1(numbers);
// println!("{:?}", numbers); // ERROR: moved
// v2 allows continued use
let sum = process_v2(&numbers);
println!("Sum: {}, Data: {:?}", sum, numbers); // OK
}
Strategy 2: Restructuring for Better Borrowing:
// Bad: God object resists borrowing
struct BadDesign {
users: Vec<String>,
scores: Vec<i32>,
metadata: String,
}
impl BadDesign {
fn update_score(&mut self, user: &str, score: i32) {
// Can't borrow users and scores separately
if let Some(pos) = self.users.iter().position(|u| u == user) {
self.scores[pos] = score;
}
}
}
// Good: Split into borrowable components
struct GoodDesign {
users: UserManager,
scores: ScoreManager,
}
struct UserManager {
users: Vec<String>,
}
struct ScoreManager {
scores: Vec<i32>,
}
impl GoodDesign {
fn update_score(&mut self, user: &str, score: i32) {
// Can borrow users and scores independently
if let Some(pos) = self.users.users.iter().position(|u| u == user) {
self.scores.scores[pos] = score;
}
}
}
fn main() {
let mut design = GoodDesign {
users: UserManager { users: vec![String::from("Alice")] },
scores: ScoreManager { scores: vec! },
};
design.update_score("Alice", 200);
}
Debugging the Borrow Checker
Understanding Error Messages:
fn demonstrate_errors() {
let mut data = vec![1, 2, 3];
// Error 1: Cannot borrow as mutable while immutably borrowed
let first = &data;
// data.push(4); // ERROR: cannot borrow `data` as mutable
println!("{}", first);
// Error 2: Cannot move out of borrowed content
let borrowed = &data;
// let moved = data; // ERROR: cannot move out of `data`
println!("{:?}", borrowed);
// Error 3: Value used after move
let vec1 = vec![1, 2, 3];
let vec2 = vec1; // Move
// println!("{:?}", vec1); // ERROR: value used after move
println!("{:?}", vec2);
}
fn main() {
demonstrate_errors();
}
Using rustc –explain:
# Get detailed explanation of error code
$ rustc --explain E0502
# Example output explains:
# "cannot borrow as mutable because it is also borrowed as immutable"
Tool Ecosystem
Miri for Detecting Undefined Behavior:
# Install miri
$ rustup +nightly component add miri
# Run program with miri
$ cargo +nightly miri run
Rust Analyzer Features:
- Inline borrow checker hints
- Lifetime annotations on hover
- Move/copy indicators
- Ownership flow visualization
Part X: Rust 2024 Advanced Topics
RPIT Capture Rules and use<>
Rust 2024 changes how lifetimes are captured in return position impl Trait:
New Capture Behavior:
// Rust 2021: Captures all lifetimes in scope
fn old_behavior<'a, 'b>(x: 'a str, y: 'b str) -> impl Iterator<Item = char> {
// Implicitly captures both 'a and 'b
x.chars()
}
// Rust 2024: Explicit capture with use<>
fn new_behavior<'a, 'b>(x: 'a str, y: 'b str) -> impl Iterator<Item = char> + use<'a> {
// Explicitly captures only 'a
x.chars()
}
fn main() {
let x = String::from("hello");
let y = String::from("world");
let iter = new_behavior(&x, &y);
drop(y); // OK: y not captured
for ch in iter {
print!("{}", ch);
}
}
Precise Lifetime Control:
trait Process {
fn process<'a>('a self, input: 'a str) -> impl Iterator<Item = char> + use<'a>;
}
struct Processor;
impl Process for Processor {
fn process<'a>('a self, input: 'a str) -> impl Iterator<Item = char> + use<'a> {
input.chars()
}
}
fn main() {
let processor = Processor;
let input = String::from("test");
let iter = processor.process(input);
for ch in iter {
print!("{}", ch);
}
}
Async Closures and Ownership
Rust 2024 stabilizes async closures with proper ownership semantics:
use tokio;
async fn process_async<F, Fut>(f: F)
where
F: Fn(String) -> Fut,
Fut: std::future::Future<Output = usize>,
{
let result = f(String::from("test")).await;
println!("Result: {}", result);
}
\#[tokio::main]
async fn main() {
// Async closure (Rust 2024)
let closure = async |s: String| {
s.len()
};
process_async(closure).await;
}
Tail Expression Temporary Scope (Breaking Change)
What Changed: In Rust 2024, temporaries in tail expressions (block return values) now drop before local variables, not after. This fixes a class of borrow checker limitations.
The Classic RefCell Example:
use std::cell::RefCell;
// ❌ Rust 2021: ERROR - borrow doesn't live long enough
// ✅ Rust 2024: Compiles successfully
fn get_length_2021_breaks() -> usize {
let c = RefCell::new(String::from("hello"));
c.borrow().len()
// Rust 2021: temporary Ref from borrow() lives until after c drops
// Rust 2024: temporary drops BEFORE c, so no conflict
}
fn main() {
println!("Length: {}", get_length_2021_breaks()); // Works in 2024!
}
Why This Matters:
use std::cell::RefCell;
fn tail_expression_example() -> String {
let data = RefCell::new(vec![String::from("a"), String::from("b")]);
// Rust 2024: temporary borrow drops before data
data.borrow().clone() // ✅ Works in 2024
}
// Contrast with non-tail position (works in both editions)
fn non_tail_example() -> String {
let data = RefCell::new(vec![String::from("a"), String::from("b")]);
let result = data.borrow().clone(); // Not a tail expression
result // ✅ Works in both 2021 and 2024
}
fn main() {
println!("{}", tail_expression_example());
println!("{}", non_tail_example());
}
Migration Note: If code relied on temporaries living longer (e.g., RAII guards protecting a return value), explicitly bind them:
use std::sync::Mutex;
fn needs_guard_lifetime() -> i32 {
let data = Mutex::new(42);
// Rust 2021: guard lives until after return (implicit)
// Rust 2024: guard drops immediately, need explicit binding
let guard = data.lock().unwrap(); // Explicit binding
*guard
}
fn main() {
let result = needs_guard_lifetime();
println!("Result: {}", result);
}
if let Temporary Scope (Breaking Change)
What Changed: In Rust 2024, temporaries created in the if let scrutinee drop before the else block executes, not after.
The Deadlock Fix:
use std::sync::RwLock;
// ❌ Rust 2021: DEADLOCK
// ✅ Rust 2024: Works correctly
fn check_and_update(value: &RwLock<Option<bool>>) {
if let Some(x) = *value.read().unwrap() {
println!("Value is {}", x);
}
// <-- Rust 2024: read lock drops HERE
else {
// Rust 2021: read lock still held, this deadlocks
// Rust 2024: read lock already dropped, this succeeds
let mut v = value.write().unwrap(); // ✅ No deadlock in 2024!
if v.is_none() {
*v = Some(true);
}
}
}
fn main() {
let data = RwLock::new(None);
check_and_update(data);
println!("Final value: {:?}", *data.read().unwrap());
}
Another Example with RefCell:
use std::cell::RefCell;
fn process_option(data: &RefCell<Vec<i32>>) {
if let Some(first) = data.borrow().first() {
println!("First element: {}", first);
} // <-- Rust 2024: borrow drops HERE
else {
// Rust 2021: immutable borrow still held, this panics
// Rust 2024: borrow already dropped, this succeeds
data.borrow_mut().push(42); // ✅ Works in 2024!
}
}
fn main() {
let data = RefCell::new(vec![]);
process_option(data); // Empty vec, enters else branch
println!("{:?}", data.borrow()); //
}
if let Chains Also Affected:
use std::sync::Mutex;
fn if_let_chain_example(a: &Mutex<Option<i32>>, b: &Mutex<Option<i32>>) {
// Rust 2024: Both locks drop before else
if let Some(x) = *a.lock().unwrap() &&
let Some(y) = *b.lock().unwrap() {
println!("Both values: {}, {}", x, y);
} else {
// Can safely acquire locks again in 2024
let mut guard_a = a.lock().unwrap();
*guard_a = Some(100);
}
}
fn main() {
let a = Mutex::new(Some(1));
let b = Mutex::new(None);
if_let_chain_example(&a, &b);
}
Migration Lints: Enable rust_2024_temporary_if_let_scope to detect code affected by this change.
Prelude Additions
Rust 2024 adds new items to the prelude that may cause method ambiguity:
New Prelude Items:
-
Futuretrait -
IntoFuturetrait
Potential Conflict:
// Custom type with poll method
struct MyType;
impl MyType {
fn poll(self) -> bool {
true
}
}
fn main() {
let obj = MyType;
// Rust 2021: calls MyType::poll
// Rust 2024: may conflict with Future::poll if type inference unclear
// Use explicit UFCS to disambiguate:
MyType::poll(&obj);
}
Reserved Keywords
Rust 2024 reserves gen as a keyword for future generator syntax:
// ❌ Rust 2024: ERROR - 'gen' is a reserved keyword
// fn create_gen() {
// let gen = 42;
// }
// ✅ Rename to something else
fn create_generator() {
let generator = 42;
println!("{}", generator);
}
fn main() {
create_generator();
}
Future Ownership Directions
Polonius Borrow Checker:
The next-generation borrow checker with more precise analysis:
// Currently requires workarounds
fn current_limitation() {
let mut data = vec!;
let first = data;
if *first > 0 {
// Currently: ERROR even though first not used after
// data.push(4);
}
// Polonius will understand first isn't used here
}
// With Polonius: More flexible borrowing
fn polonius_enables() {
let mut map = std::collections::HashMap::new();
map.insert("key", vec!);
// Will work with Polonius
let value = map.get_mut("key").unwrap();
value.push(4);
}
fn main() {
polonius_enables();
}
