Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

English Original

Rust 设计模式与工程实践 (Rust Patterns & Engineering How-Tos)

讲师简介

  • Microsoft SCHIE(硅与云硬件基础设施工程)团队首席固件架构师
  • 资深行业专家,在安全、系统编程(固件、操作系统、管理程序)、CPU 与平台架构以及 C++ 系统领域拥有深厚背景
  • 自 2017 年(于 AWS EC2)起开始使用 Rust 编程,从此便深深爱上了这门语言

这是一本针对中高级 Rust 模式的实战指南,涵盖了真实代码库中出现的各种案例。这不是一本语言入门教程 —— 它默认你已经掌握了 Rust 基础,并希望进一步提升。每一章节都剥离出一个核心概念,解释“何时”以及“为何”使用它,并提供可编译的示例及随堂练习。

适用对象

  • 读完《Rust 原理(The Rust Programming Language)》但仍苦恼于“我该如何设计这个系统?”的开发者
  • 正在将生产系统从 C++/C# 迁移至 Rust 的工程师
  • 在泛型、Trait 约束或生命周期错误面前碰壁,希望获得系统性工具集的任何人

先决条件

在开始之前,你应该已经熟悉:

  • 所有权(Ownership)、借用(Borrowing)和生命周期(Lifetimes)的基础知识
  • 枚举(Enums)、模式匹配以及 Option/Result
  • 结构体、方法以及基础 Trait(Display, Debug, Clone)
  • Cargo 基础操作:cargo build, cargo test, cargo run

本书阅读指南

难度等级说明

每一章都标记了难度等级:

符号等级含义
🟢基础篇每个 Rust 开发者都必须掌握的核心概念
🟡进阶篇生产级代码库中常用的模式
🔴高级篇语言底层的深刻机制 —— 可根据需要反复研读

学习路线图

章节主题建议耗时重点核对
第一部分:类型级模式
1. 泛型 🟢单态化、常量泛型、const fn1–2 小时能解释何时 dyn Trait 优于泛型
2. Trait 🟡关联类型、GATs、覆盖实现、虚表3–4 小时能设计带有关联类型的 Trait
3. Newtype 与类型状态模式 🟡零成本安全、编译时状态机2–3 小时能构建类型状态构造者(Builder)模式
4. PhantomData 🔴生命周期烙印、型变(Variance)、Drop 检查2–3 小时能解释为何 PhantomData<fn(T)> 与 PhantomData<T> 不同
第二部分:并发与运行时
5. 通道(Channels) 🟢mpsc, crossbeam, select!, Actor1–2 小时能实现基于通道的工作线程池
6. 并发 🟡线程、rayon、Mutex、RwLock、原子操作2–3 小时能根据场景选择正确的同步原语
7. 闭包 🟢Fn/FnMut/FnOnce, 组合器1–2 小时能编写接受闭包的高阶函数
8. 函数式 vs 命题式 🟡组合器、迭代器适配器、函数式模式2–3 小时能解释何时函数式风格优于命题式
9. 智能指针 🟡Box, Rc, Arc, RefCell, Cow, Pin2–3 小时能解释每种智能指针的适用场景
第三部分:系统与生产
10. 错误处理 🟢thiserror, anyhow, ? 操作符1–2 小时能设计错误类型层次结构
11. 序列化 🟡serde, 零拷贝, 二进制数据2–3 小 byte能编写自定义的 serde 反序列化器
12. Unsafe 🔴超能力、FFI、内存不安全陷阱、分配器2–3 小时能将 Unsafe 代码包装在健全的安全 API 中
13. 宏 🟡macro_rules!, 过程宏, syn/quote2–3 小时能编写带有 tt 匹配的声明式宏
14. 测试 🟢单元/集成/文档测试, proptest, criterion1–2 小时能设置基于属性的测试(Property-based testing)
15. API 设计 🟡模块布局、人体工学 API、功能标志(Feature flags)2–3 小时能应用“解析而非校验(parse, don’t validate)”模式
16. 异步 🔴Future, Tokio, 常见陷阱1–2 小时能识别异步反模式(Anti-patterns)
附录
总结与速查表快速查看 Trait 约束、生命周期及模式随时查阅—
终极实战项目类型安全的任务调度器4–6 小时提交一份可运行的实现版本

建议总耗时:完成所有学习与练习约需 30–45 小时。


目录

第一部分:类型级模式

1. 泛型:全景概览 🟢 单态化、代码膨胀的权衡、泛型 vs 枚举 vs Trait 对象、常量泛型、const fn。

2. 深入理解 Trait 🟡 关联类型、GATs、覆盖实现(Blanket impls)、标记 Trait、虚表、HRTBs、扩展 Trait、枚举分发。

3. Newtype 与类型状态(Type-State)模式 🟡 零成本类型安全、编译时状态机、构造者模式、配置 Trait。

4. PhantomData:不携带数据的类型 🔴 生命周期烙印、单位度量模式、Drop 检查、型变(Variance)。

第二部分:并发与运行时

5. 通道与消息传递 🟢 std::sync::mpsc, crossbeam, select!, 背压控制, Actor 模式。

6. 并发 vs 并行 vs 线程 🟡 OS 线程、作用域线程、rayon、Mutex/RwLock/原子操作、条件变量、OnceLock、无锁模式。

7. 闭包与高阶函数 🟢 Fn/FnMut/FnOnce, 闭包作为参数/返回值, 组合器, 高阶 API。

8. 函数式 vs 命题式:何为优雅? 🟡 组合器、迭代器适配器、函数式模式。

9. 智能指针与内部可变性 🟡 Box, Rc, Arc, Weak, Cell/RefCell, Cow, Pin, ManuallyDrop。

第三部分:系统与生产

10. 错误处理模式 🟢 thiserror vs anyhow, #[from], .context(), ? 操作符, Panics。

11. 序列化、零拷贝与二进制数据 🟡 serde 基础, 枚举表示, 零拷贝反序列化, repr(C), bytes::Bytes。

12. Unsafe Rust:受控的危险 🔴 五大超能力、健全的抽象、FFI、内存不安全陷阱、Arena/Slab 分配器。

13. 宏:编写代码的代码 🟡 macro_rules!, 何时(及何时不)使用宏, 过程宏, Derive 宏, syn/quote。

14. 测试与基准测试模式 🟢 单元/集成/文档测试, proptest, criterion, Mock 策略。

15. Crate 架构与 API 设计 🟡 模块布局、API 设计清单、符合人体工学的参数、功能标志(Feature flags)、工作区(Workspaces)。

16. 异步基础 🔴 Future, Tokio 快速入门, 常见陷阱。(关于更深度的异步覆盖,请参阅我们的《异步 Rust 训练》。)

附录

总结与速查表 模式决策指南、Trait 约束速查、生命周期洗去规则、延伸阅读。

实战项目:类型安全的任务调度器 将泛型、Trait、类型状态、通道、错误处理和测试整合到一个完整的系统中。


English Original

1. 泛型:全景概览 🟢

你将学到:

  • 单态化(Monomorphization)如何实现零成本泛型 —— 以及它何时会导致代码膨胀
  • 决策框架:泛型 vs 枚举 vs Trait 对象
  • 常量泛型(Const generics)用于编译时数组大小,以及 const fn 用于编译时计算
  • 何时在冷路径上将静态分发换成动态分发

单态化与零成本 (Monomorphization and Zero Cost)

Rust 中的泛型是单态化的 —— 编译器会为泛型函数所使用的每种具体类型生成一个专用的特殊副本。这与 Java/C# 相反,后者的泛型在运行时会被擦除(erased)。

fn max_of<T: PartialOrd>(a: T, b: T) -> T {
    if a >= b { a } else { b }
}

fn main() {
    max_of(3_i32, 5_i32);     // 编译器生成 max_of_i32
    max_of(2.0_f64, 7.0_f64); // 编译器生成 max_of_f64
    max_of("a", "z");         // 编译器生成 max_of_str
}

编译器实际生成的代码(概略性):

#![allow(unused)]
fn main() {
// 三个独立的函数 —— 无运行时分发,无虚表:
fn max_of_i32(a: i32, b: i32) -> i32 { if a >= b { a } else { b } }
fn max_of_f64(a: f64, b: f64) -> f64 { if a >= b { a } else { b } }
fn max_of_str<'a>(a: &'a str, b: &'a str) -> &'a str { if a >= b { a } else { b } }
}

为什么 max_of_str 需要 <'a> 而 max_of_i32 不需要? i32 和 f64 是 Copy 类型 —— 函数返回一个拥有的(owned)值。但 &str 是引用,因此编译器必须知道返回引用的生命周期。<'a> 注解表示“返回的 &str 的生命周期至少与两个输入一样长”。

优势:零运行时成本 —— 与手写的专用代码完全相同。优化器可以对每个副本独立进行内联、向量化和专门化。

与 C++ 的对比:Rust 的泛型工作方式类似于 C++ 模板,但有一个关键区别 —— 边界检查(bounds checking)发生在定义处,而非实例化处。在 C++ 中,模板只有在配合具体类型使用时才会编译,这往往导致报错信息深藏在库代码深处且晦涩难懂。在 Rust 中,当你定义函数时,编译器就会检查 T: PartialOrd,因此错误能被及早捕捉且信息清晰。

#![allow(unused)]
fn main() {
// Rust:在定义处报错 —— “T 未实现 Display”
fn broken<T>(val: T) {
    println!("{val}"); // ❌ 错误:T 未实现 Display
}
}
#![allow(unused)]
fn main() {
// 修复:添加边界
fn fixed<T: std::fmt::Display>(val: T) {
    println!("{val}"); // ✅
}
}

泛型的代价:代码膨胀 (When Generics Hurt: Code Bloat)

单态化是有代价的 —— 二进制文件的大小。每一种唯一的实例化都会复制一遍函数体:

// 这个看似无害的函数...
fn serialize<T: serde::Serialize>(value: &T) -> Vec<u8> {
    serde_json::to_vec(value).unwrap()
}

// ...配合 50 种不同的类型使用 → 二进制文件中会有 50 个副本。

缓解策略:

// 1. 提取非泛型核心(“outline” 模式)
fn serialize<T: serde::Serialize>(value: &T) -> Result<Vec<u8>, serde_json::Error> {
    // 泛型部分:仅包含序列化调用
    let json_value = serde_json::to_value(value)?;
    // 非泛型部分:提取到单独的函数中
    serialize_value(json_value)
}

fn serialize_value(value: serde_json::Value) -> Result<Vec<u8>, serde_json::Error> {
    // 该函数在二进制文件中仅存在一次
    serde_json::to_vec(&value)
}

// 2. 当内联不是关键时,使用 Trait 对象(动态分发)
fn log_item(item: &dyn std::fmt::Display) {
    // 只有一个副本 —— 使用虚表(vtable)进行分发
    println!("[LOG] {item}");
}

经验法则:在内联至关重要的热路径(hot paths)上使用泛型。在虚表调用开销可以忽略不计的冷路径(错误处理、日志记录、配置加载)上使用 dyn Trait。

泛型 vs 枚举 vs Trait 对象 —— 决策指南 (Generics vs Enums vs Trait Objects — Decision Guide)

在 Rust 中处理“不同类型,统一接口”的三种方法:

方案分发方式确定时机是否可扩展?开销
泛型 (impl Trait / <T: Trait>)静态 (单态化)编译阶段✅ (开放集合)零 —— 已内联
枚举 (Enum)Match 分支编译阶段❌ (封闭集合)零 —— 无虚表
Trait 对象 (dyn Trait)动态 (虚表)运行阶段✅ (开放集合)虚表指针 + 间接调用
// --- 泛型:开放集合,零成本,编译时确定 ---
fn process<H: Handler>(handler: H, request: Request) -> Response {
    handler.handle(request) // 单态化 —— 每种 H 生成一份副本
}

// --- 枚举:封闭集合,零成本,详尽匹配 ---
enum Shape {
    Circle(f64),
    Rect(f64, f64),
    Triangle(f64, f64, f64),
}

impl Shape {
    fn area(&self) -> f64 {
        match self {
            Shape::Circle(r) => std::f64::consts::PI * r * r,
            Shape::Rect(w, h) => w * h,
            Shape::Triangle(a, b, c) => {
                let s = (a + b + c) / 2.0;
                (s * (s - a) * (s - b) * (s - c)).sqrt()
            }
        }
    }
}
// 添加新变体将强制更新所有 match 分支 —— 编译器强制执行详尽性。
// 适用于“我能控制所有变体”的场景。

// --- Trait 对象:开放集合,运行时成本,可扩展 ---
fn log_all(items: &[Box<dyn std::fmt::Display>]) {
    for item in items {
        println!("{item}"); // 通过虚表分发
    }
}

决策流程图:

flowchart TD
    A["是否在编译时已知<br>所有可能的类型?"]
    A -->|"是,且是小型<br>封闭集合"| B["枚举 (Enum)"]
    A -->|"是,但集合<br>是开放的"| C["泛型<br>(单态化)"]
    A -->|"否 —— 类型在<br>运行时确定"| D["dyn Trait"]

    C --> E{"是否为性能关键的<br>热路径?(数百万次调用)"}
    E -->|是| F["泛型<br>(可内联)"]
    E -->|否| G["dyn Trait<br>即可"]

    D --> H{"是否需要在同一个集合中<br>混合不同类型?"}
    H -->|是| I["Vec&lt;Box&lt;dyn Trait&gt;&gt;"]
    H -->|No| C

    style A fill:#e8f4f8,stroke:#2980b9,color:#000
    style B fill:#d4efdf,stroke:#27ae60,color:#000
    style C fill:#d4efdf,stroke:#27ae60,color:#000
    style D fill:#fdebd0,stroke:#e67e22,color:#000
    style F fill:#d4efdf,stroke:#27ae60,color:#000
    style G fill:#fdebd0,stroke:#e67e22,color:#000
    style I fill:#fdebd0,stroke:#e67e22,color:#000
    style E fill:#fef9e7,stroke:#f1c40f,color:#000
    style H fill:#fef9e7,stroke:#f1c40f,color:#000

常量泛型 (Const Generics)

自 Rust 1.51 起,你可以根据常量值(而不仅仅是类型)来参数化类型和函数:

#![allow(unused)]
fn main() {
// 根据大小参数化的数组包装器
struct Matrix<const ROWS: usize, const COLS: usize> {
    data: [[f64; COLS]; ROWS],
}

impl<const ROWS: usize, const COLS: usize> Matrix<ROWS, COLS> {
    fn new() -> Self {
        Matrix { data: [[0.0; COLS]; ROWS] }
    }

    fn transpose(&self) -> Matrix<COLS, ROWS> {
        let mut result = Matrix::<COLS, ROWS>::new();
        for r in 0..ROWS {
            for c in 0..COLS {
                result.data[c][r] = self.data[r][c];
            }
        }
        result
    }
}

// 编译器强制执行维度正确性:
fn multiply<const M: usize, const N: usize, const P: usize>(
    a: &Matrix<M, N>,
    b: &Matrix<N, P>, // N 必须匹配!
) -> Matrix<M, P> {
    let mut result = Matrix::<M, P>::new();
    for i in 0..M {
        for j in 0..P {
            for k in 0..N {
                result.data[i][j] += a.data[i][k] * b.data[k][j];
            }
        }
    }
    result
}

// 使用:
let a = Matrix::<2, 3>::new(); // 2×3
let b = Matrix::<3, 4>::new(); // 3×4
let c = multiply(&a, &b);      // 2×4 ✅

// let d = Matrix::<5, 5>::new();
// multiply(&a, &d); // ❌ 编译错误:期待 Matrix<3, _>,得到 Matrix<5, 5>
}

与 C++ 对比:这类似于 C++ 中的 template<int N>,但 Rust 的常量泛型会进行及早的类型检查(eagerly type-checked),且不受 SFINAE 复杂性的困扰。

常量函数 (const fn)

const fn 将函数标记为可以在编译时求值的函数 —— 相当于 C++ 的 constexpr。其结果可用于 const 和 static 上下文:

#![allow(unused)]
fn main() {
// 基础常量函数 —— 在 const 上下文中使用时在编译时计算
const fn celsius_to_fahrenheit(c: f64) -> f64 {
    c * 9.0 / 5.0 + 32.0
}

const BOILING_F: f64 = celsius_to_fahrenheit(100.0); // 在编译时计算
const FREEZING_F: f64 = celsius_to_fahrenheit(0.0);  // 32.0

// 常量构造函数 —— 无需 lazy_static! 即可创建静态变量
struct BitMask(u32);

impl BitMask {
    const fn new(bit: u32) -> Self {
        BitMask(1 << bit)
    }

    const fn or(self, other: BitMask) -> Self {
        BitMask(self.0 | other.0)
    }

    const fn contains(&self, bit: u32) -> bool {
        self.0 & (1 << bit) != 0
    }
}

// 静态查找表 —— 零运行时成本,无延迟初始化
const GPIO_INPUT:  BitMask = BitMask::new(0);
const GPIO_OUTPUT: BitMask = BitMask::new(1);
const GPIO_IRQ:    BitMask = BitMask::new(2);
const GPIO_IO:     BitMask = GPIO_INPUT.or(GPIO_OUTPUT);

// 寄存器映射为常量数组:
const SENSOR_THRESHOLDS: [u16; 4] = {
    let mut table = [0u16; 4];
    table[0] = 50;   // 警告
    table[1] = 70;   // 高危
    table[2] = 85;   // 暴击
    table[3] = 100;  // 熔断
    table
};
// 整个表直接存在于二进制文件中 —— 零堆内存,零运行时初始化。
}

你在 const fn 中“可以”做的事(截至 Rust 1.79+):

  • 算术运算、位运算、比较运算
  • if/else, match, loop, while (控制流)
  • 创建和修改局部变量 (let mut)
  • 调用其他 const fn
  • 引用 (&, &mut —— 在常量上下文内)
  • panic!()(如果在编译时被触及,会触发编译错误)
  • 基础浮点运算(+, -, *, /;复杂运算如 sqrt/sin 尚不可行)

“不能”做的事(目前):

  • 堆分配(Box, Vec, String)
  • Trait 方法调用(仅限内部固有方法)
  • I/O 或产生副作用
#![allow(unused)]
fn main() {
// 带有 panic 的 const fn —— 会触发编译时错误:
const fn checked_div(a: u32, b: u32) -> u32 {
    if b == 0 {
        panic!("division by zero"); // 若在 const 时刻 b 为 0,则引发编译错误
    }
    a / b
}

const RESULT: u32 = checked_div(100, 4);  // ✅ 25
// const BAD: u32 = checked_div(100, 0);  // ❌ 编译错误:"division by zero"
}

与 C++ 对比:const fn 是 Rust 版的 constexpr。关键区别:Rust 版是显式开启(opt-in)的,且编译器会严密验证仅使用了常量兼容的操作。在 C++ 中,constexpr 函数可能会静默回退到运行时求值 —— 在 Rust 中,const 上下文必须要求编译时求值,否则将引发严重错误。

实用建议:尽可能让构造函数和简单的工具函数变为 const fn —— 这没有任何成本,且能允许调用方在常量上下文中使用。对于硬件诊断代码,const fn 是定义寄存器、构建位掩码和阈值表的理想选择。

核心要点 —— 泛型

  • 单态化提供了零成本抽象,但可能导致代码膨胀 —— 在冷路径上应使用 dyn Trait。
  • 常量泛型 ([T; N]) 取代了 C++ 的模板技巧,提供了经编译时检查的数组大小。
  • const fn 针对编译时可计算值消除了对 lazy_static! 的需求。

扩展阅读: 参阅 第 2 章 —— 深入理解 Trait 了解 Trait 边界、关联类型和 Trait 对象。参阅 第 4 章 —— PhantomData 了解零大小泛型标记。


练习:带淘汰机制的泛型缓存 ★★ (~30 分钟)

构建一个泛型 Cache<K, V> 结构体,用于存储键值对,并具有可配置的最大容量。当容量占满时,淘汰最旧的条目(FIFO)。要求:

  • fn new(capacity: usize) -> Self
  • fn insert(&mut self, key: K, value: V) —— 如果达到容量限制,则淘汰最旧的条目
  • fn get(&self, key: &K) -> Option<&V>
  • fn len(&self) -> usize
  • 对 K 使用 Eq + Hash + Clone 约束
🔑 参考方案
use std::collections::{HashMap, VecDeque};
use std::hash::Hash;

struct Cache<K, V> {
    map: HashMap<K, V>,
    order: VecDeque<K>,
    capacity: usize,
}

impl<K: Eq + Hash + Clone, V> Cache<K, V> {
    fn new(capacity: usize) -> Self {
        Cache {
            map: HashMap::with_capacity(capacity),
            order: VecDeque::with_capacity(capacity),
            capacity,
        }
    }

    fn insert(&mut self, key: K, value: V) {
        if self.capacity == 0 {
            // 无容量!
            return;
        }
        if self.map.contains_key(&key) {
            self.map.insert(key, value);
            return;
        }
        if self.map.len() >= self.capacity {
            if let Some(oldest) = self.order.pop_front() {
                self.map.remove(&oldest);
            }
        }
        self.order.push_back(key.clone());
        self.map.insert(key, value);
    }

    fn get(&self, key: &K) -> Option<&V> {
        self.map.get(key)
    }

    fn len(&self) -> usize {
        self.map.len()
    }
}

fn main() {
    // 基础缓存测试
    let mut cache = Cache::new(3);
    cache.insert("a", 1);
    cache.insert("b", 2);
    cache.insert("c", 3);
    assert_eq!(cache.len(), 3);

    cache.insert("d", 4); // 淘汰 "a"
    assert_eq!(cache.get(&"a"), None);
    assert_eq!(cache.get(&"d"), Some(&4));

    // 留给读者思考:`capacity` 属性应该用什么类型,
    // 以确保不能定义这种无效的空缓存?
    let mut empty_cache = Cache::new(0);
    empty_cache.insert("0", 0);
    assert_eq!(empty_cache.get(&"0"), None);
    assert_eq!(empty_cache.len(), 0);

    println!("缓存运行正常!len = {}", cache.len());
}

English Original

2. 深入 Trait 🟡

你将学到:

  • 关联类型与泛型参数的区别——以及何时使用它们
  • GATs、全包含实现(blanket impls)、标记 trait(marker traits)以及 trait 对象安全规则
  • vtable 和虚指针(fat pointers)在底层的运作机制
  • 扩展 trait(Extension traits)、枚举分发(enum dispatch)以及有类型命令模式

关联类型 vs 泛型参数

两者都允许 trait 与不同的类型协同工作,但它们的用途各不相同:

#![allow(unused)]
fn main() {
// --- 关联类型:每个类型只能有一个实现 ---
trait Iterator {
    type Item; // 每个迭代器产生且仅产生一种类型的项

    fn next(&mut self) -> Option<Self::Item>;
}

// 一个始终返回 i32 的自定义迭代器——别无选择
struct Counter { max: i32, current: i32 }

impl Iterator for Counter {
    type Item = i32; // 每个实现只能有一个 Item 类型
    fn next(&mut self) -> Option<i32> {
        if self.current < self.max {
            self.current += 1;
            Some(self.current)
        } else {
            None
        }
    }
}

// --- 泛型参数:每个类型可以有多个实现 ---
trait Convert<T> {
    fn convert(&self) -> T;
}

// 一个类型可以为多种目标类型实现 Convert:
impl Convert<f64> for i32 {
    fn convert(&self) -> f64 { *self as f64 }
}
impl Convert<String> for i32 {
    fn convert(&self) -> String { self.to_string() }
}
}

何时使用哪种:

使用何时使用
关联类型每个实现类型正好有一个自然的输出/结果。例如 Iterator::Item、Deref::Target、Add::Output
泛型参数一个类型可以有意义地为许多不同的类型实现该 trait。例如 From<T>、AsRef<T>、PartialEq<Rhs>

直觉判断:如果问“这个迭代器的 Item 是什么?”是有意义的,请使用关联类型。如果问“这个类型能转换成 f64 吗?转换成 String 吗?转换成 bool 吗?”是有意义的,请使用泛型参数。

#![allow(unused)]
fn main() {
// 现实世界的例子:std::ops::Add
trait Add<Rhs = Self> {
    type Output; // 关联类型——加法运算只有一个结果类型
    fn add(self, rhs: Rhs) -> Self::Output;
}

// Rhs 是泛型参数——你可以给 Meters 加不同类型:
struct Meters(f64);
struct Centimeters(f64);

impl Add<Meters> for Meters {
    type Output = Meters;
    fn add(self, rhs: Meters) -> Meters { Meters(self.0 + rhs.0) }
}
impl Add<Centimeters> for Meters {
    type Output = Meters;
    fn add(self, rhs: Centimeters) -> Meters { Meters(self.0 + rhs.0 / 100.0) }
}
}

泛型关联类型 (GATs)

自 Rust 1.65 起,关联类型可以拥有自己的泛型参数。这使得 借用迭代器 (lending iterators) 成为可能——这种迭代器返回的引用绑定到迭代器本身,而不是底层的集合:

#![allow(unused)]
fn main() {
// 没有 GATs——无法表达借用迭代器:
// trait LendingIterator {
//     type Item<'a>;  // ← 在 1.65 之前会被拒绝
// }

// 使用 GATs (Rust 1.65+):
trait LendingIterator {
    type Item<'a> where Self: 'a;

    fn next(&mut self) -> Option<Self::Item<'_>>;
}

// 示例:一个产生重叠窗口的迭代器
struct WindowIter<'data> {
    data: &'data [u8],
    pos: usize,
    window_size: usize,
}

impl<'data> LendingIterator for WindowIter<'data> {
    type Item<'a> = &'a [u8] where Self: 'a;

    fn next(&mut self) -> Option<&[u8]> {
        if self.pos + self.window_size <= self.data.len() {
            let window = &self.data[self.pos..self.pos + self.window_size];
            self.pos += 1;
            Some(window)
        } else {
            None
        }
    }
}
}

何时需要 GATs:借用迭代器、流式解析器,或者任何关联类型的生命周期依赖于 &self 借用的 trait。对于大多数代码,普通的关联类型就足够了。

父 trait 与 Trait 层级 (Supertraits and Trait Hierarchies)

Trait 可以要求其他 Trait 作为先决条件,从而形成层级结构:

graph BT
    Display["Display"]
    Debug["Debug"]
    Error["Error"]
    Clone["Clone"]
    Copy["Copy"]
    PartialEq["PartialEq"]
    Eq["Eq"]
    PartialOrd["PartialOrd"]
    Ord["Ord"]

    Error --> Display
    Error --> Debug
    Copy --> Clone
    Eq --> PartialEq
    Ord --> Eq
    Ord --> PartialOrd
    PartialOrd --> PartialEq

    style Display fill:#e8f4f8,stroke:#2980b9,color:#000
    style Debug fill:#e8f4f8,stroke:#2980b9,color:#000
    style Error fill:#fdebd0,stroke:#e67e22,color:#000
    style Clone fill:#d4efdf,stroke:#27ae60,color:#000
    style Copy fill:#d4efdf,stroke:#27ae60,color:#000
    style PartialEq fill:#fef9e7,stroke:#f1c40f,color:#000
    style Eq fill:#fef9e7,stroke:#f1c40f,color:#000
    style PartialOrd fill:#fef9e7,stroke:#f1c40f,color:#000
    style Ord fill:#fef9e7,stroke:#f1c40f,color:#000

箭头从子 trait 指向父 trait:实现 Error 要求同时实现 Display + Debug。

一个 Trait 可以要求实现者也必须实现其他 Trait:

#![allow(unused)]
fn main() {
use std::fmt;

// Display 是 Error 的父 trait
trait Error: fmt::Display + fmt::Debug {
    fn source(&self) -> Option<&(dyn Error + 'static)> { None }
}
// 任何实现 Error 的类型也必须实现 Display 和 Debug

// 构建你自己的层级:
trait Identifiable {
    fn id(&self) -> u64;
}

trait Timestamped {
    fn created_at(&self) -> chrono::DateTime<chrono::Utc>;
}

// Entity 需要同时满足上述两者:
trait Entity: Identifiable + Timestamped {
    fn is_active(&self) -> bool;
}

// 实现 Entity 强制要求你实现这三者:
struct User { id: u64, name: String, created: chrono::DateTime<chrono::Utc> }

impl Identifiable for User {
    fn id(&self) -> u64 { self.id }
}
impl Timestamped for User {
    fn created_at(&self) -> chrono::DateTime<chrono::Utc> { self.created }
}
impl Entity for User {
    fn is_active(&self) -> bool { true }
}
}

全包含实现 (Blanket Implementations)

为所有满足某些约束的类型实现一个 Trait:

#![allow(unused)]
fn main() {
// 标准库就是这么做的:任何实现 Display 的类型都会自动获得 ToString
impl<T: fmt::Display> ToString for T {
    fn to_string(&self) -> String {
        format!("{self}")
    }
}
// 现在 i32、&str、以及你的自定义类型——只要有 Display,就能免费获得 to_string()。

// 你自己的全包含实现:
trait Loggable {
    fn log(&self);
}

// 每个实现了 Debug 的类型自动成为 Loggable:
impl<T: std::fmt::Debug> Loggable for T {
    fn log(&self) {
        eprintln!("[LOG] {self:?}");
    }
}

// 现在任何 Debug 类型都有了 .log() 方法:
// 42.log();              // [LOG] 42
// "hello".log();         // [LOG] "hello"
// vec![1, 2, 3].log();   // [LOG] [1, 2, 3]
}

注意:全包含实现功能强大但也具有不可逆性——你不能再为已经在这个范围内的类型添加更具体的实现(受孤儿规则和一致性限制)。请谨慎设计。

标记 Trait (Marker Traits)

没有方法的 Trait——它们仅将某种类型标记为具有某种属性:

#![allow(unused)]
fn main() {
// 标准库中的标记 trait:
// Send    — 可在线程间安全地传递
// Sync    — 可在线程间安全地共享 (&T)
// Unpin   — 被固化(pinning)后可安全移动
// Sized   — 在编译时具有已知大小
// Copy    — 可通过 memcpy 直接复制

// 你自己的标记 trait:
/// 标记:此传感器已经过出厂校准
trait Calibrated {}

struct RawSensor { reading: f64 }
struct CalibratedSensor { reading: f64 }

impl Calibrated for CalibratedSensor {}

// 只有经过校准的传感器才能在生产中使用:
fn record_measurement<S: Calibrated>(sensor: &S) {
    // ...
}
// record_measurement(&RawSensor { reading: 0.0 }); // ❌ 编译错误
// record_measurement(&CalibratedSensor { reading: 0.0 }); // ✅
}

这与第三章中的 类型状态模式 (type-state pattern) 直接联系在一起。

Trait 对象安全规则 (Trait Object Safety Rules)

并非每个 Trait 都能被用作 dyn Trait。一个 Trait 只有在满足以下条件时才是 对象安全 (object-safe) 的:

  1. 没有 Self: Sized 约束:Trait 本身没有这个约束。
  2. 方法没有泛型参数:所有方法都不能有泛型。
  3. 返回位置没有 Self:除非通过间接引用(如 Box<Self>)。
  4. 没有关联函数:所有方法必须带有 &self、&mut self 或 self 参数。
#![allow(unused)]
fn main() {
// ✅ 对象安全——可以被用作 dyn Drawable
trait Drawable {
    fn draw(&self);
    fn bounding_box(&self) -> (f64, f64, f64, f64);
}

let shapes: Vec<Box<dyn Drawable>> = vec![/* ... */]; // ✅ 正常工作

// ❌ 非对象安全——在返回位置使用了 Self
trait Cloneable {
    fn clone_self(&self) -> Self;
    //                       ^^^^ 运行时无法确定具体的类型大小
}
// let items: Vec<Box<dyn Cloneable>> = ...; // ❌ 编译错误

// ❌ 非对象安全——带有泛型的方法
trait Converter {
    fn convert<T>(&self) -> T;
    //        ^^^ vtable 无法包含无限多的单态化函数指针
}

// ❌ 非对象安全——关联函数(没有 self)
trait Factory {
    fn create() -> Self;
    // 没有 &self——你该如何通过 trait 对象来调用它?
}
}

解决方法:

#![allow(unused)]
fn main() {
// 通过添加 `where Self: Sized` 约束将某个方法从 vtable 中排除:
trait MyTrait {
    fn regular_method(&self); // 包含在 vtable 中

    fn generic_method<T>(&self) -> T
    where
        Self: Sized; // 从 vtable 中排除——无法通过 dyn MyTrait 调用
}

// 现在 dyn MyTrait 是合法的,但 generic_method 只能在具体类型已知时调用。
}

经验法则:如果你打算使用 dyn Trait,请保持方法简单——不要使用泛型、不要在返回类型中使用 Self、不要使用 Sized 约束。如果不确定,尝试定义 let _: Box<dyn YourTrait>;,让编译器告诉你答案。

Trait 对象底层原理——虚函数表 (vtables) 与胖指针 (Fat Pointers)

一个 &dyn Trait(或 Box<dyn Trait>)是一个 胖指针 (fat pointer) ——由两个机器字(machine words)组成:

┌──────────────────────────────────────────────────┐
│  &dyn Drawable (在 64 位系统上共 16 字节)          │
├──────────────┬───────────────────────────────────┤
│  data_ptr    │  vtable_ptr                       │
│  (8 字节)    │  (8 字节)                         │
│  ↓           │  ↓                                │
│  ┌─────────┐ │  ┌──────────────────────────────┐ │
│  │ Circle  │ │  │ <Circle as Drawable> 的 vtable │ │
│  │ {       │ │  │                              │ │
│  │  r: 5.0 │ │  │                              │ │
│  │ }       │ │  │  drop_in_place: 0x7f...a0    │ │
│  └─────────┘ │  │  size:           8           │ │
│              │  │  align:          8           │ │
│              │  │  draw:          0x7f...b4    │ │
│              │  │  bounding_box:  0x7f...c8    │ │
│              │  └──────────────────────────────┘ │
└──────────────┴───────────────────────────────────┘

vtable 调用是如何运作的(例如 shape.draw()):

  1. 从胖指针(第二个字)中加载 vtable_ptr。
  2. 在 vtable 中索引,找到 draw 函数指针。
  3. 调用该指针,并将 data_ptr 作为 self 参数传递。

这在开销上与 C++ 的虚函数分发(每次调用一次指针间接跳转)类似,但 Rust 将 vtable 指针存储在胖指针中,而不是对象内部——因此,栈上的普通 Circle 类型完全不携带任何 vtable 指针。

trait Drawable {
    fn draw(&self);
    fn area(&self) -> f64;
}

struct Circle { radius: f64 }

impl Drawable for Circle {
    fn draw(&self) { println!("正在绘制圆 r={}", self.radius); }
    fn area(&self) -> f64 { std::f64::consts::PI * self.radius * self.radius }
}

struct Square { side: f64 }

impl Drawable for Square {
    fn draw(&self) { println!("正在绘制正方形 s={}", self.side); }
    fn area(&self) -> f64 { self.side * self.side }
}

fn main() {
    let shapes: Vec<Box<dyn Drawable>> = vec![
        Box::new(Circle { radius: 5.0 }),
        Box::new(Square { side: 3.0 }),
    ];

    // 每个元素都是一个胖指针:(data_ptr, vtable_ptr)
    // Circle 和 Square 的 vtable 是不同的
    for shape in &shapes {
        shape.draw();  // vtable 分发 → Circle::draw 或 Square::draw
        println!("  面积 = {:.2}", shape.area());
    }

    // 大小对比:
    println!("size_of::<&Circle>()        = {}", size_of::<&Circle>());
    // → 8 字节(普通指针——编译器已知具体类型)
    println!("size_of::<&dyn Drawable>()  = {}", size_of::<&dyn Drawable>());
    // → 16 字节(data_ptr + vtable_ptr)
}

性能开销模型:

特性静态分发 (impl Trait / 泛型)动态分发 (dyn Trait)
调用开销零——被 LLVM 内联每次调用一次指针间接寻址
内联✅ 编译器可内联❌ 不透明的函数指针
二进制大小较大(每种类型一份副本)较小(一份共享函数)
指针大小薄(Thin,1 个机器字)胖(Fat,2 个机器字)
异构集合❌✅ Vec<Box<dyn Trait>>

何时 vtable 开销很重要:在循环调用数百万次 trait 方法的紧凑循环中,间接寻址和无法内联可能导致显著影响(慢 2-10 倍)。对于非热点路径、配置或插件架构,dyn Trait 带来的灵活性完全抵得过这点微小的开销。

高阶 Trait 约束 (Higher-Ranked Trait Bounds, HRTBs)

有时你需要一个函数能处理 任何 生命周期的引用,而不是某个特定的生命周期。这就是 for<'a> 语法出现的地方:

// 问题:这个函数需要一个闭包,该闭包能处理具有 任何 生命周期的引用,
// 而不仅仅是一个特定的生命周期。

// ❌ 这太严格了——'a 由调用者固定:
// fn apply<'a, F: Fn(&'a str) -> &'a str>(f: F, data: &'a str) -> &'a str

// ✅ HRTB:F 必须适用于所有可能的生命周期:
fn apply<F>(f: F, data: &str) -> &str
where
    F: for<'a> Fn(&'a str) -> &'a str,
{
    f(data)
}

fn main() {
    let result = apply(|s| s.trim(), "  hello  ");
    println!("{result}"); // "hello"
}

何时会遇到 HRTBs:

  • Fn(&T) -> &U trait——在大多数情况下,编译器会自动推导 for<'a>。
  • 必须跨不同借用工作的自定义 trait 实现。
  • 使用 serde 进行反序列化时:for<'de> Deserialize<'de>。
// serde 的 DeserializeOwned 定义为:
// trait DeserializeOwned: for<'de> Deserialize<'de> {}
// 意思是:“可以从具有 任何 生命周期的数据中反序列化”
// (即结果不借用输入数据)

use serde::de::DeserializeOwned;

fn parse_json<T: DeserializeOwned>(input: &str) -> T {
    serde_json::from_str(input).unwrap()
}

实用建议:你很少需要亲自编写 for<'a>。它主要出现在闭包参数的 trait 约束中,编译器通常会自动处理。但在报错信息中识别出它(例如 “expected a for<'a> Fn(&'a ...) bound”)能帮你理解编译器的要求。

impl Trait —— 参数位置 vs 返回位置

impl Trait 出现在两个位置,其 语义完全不同:

#![allow(unused)]
fn main() {
// --- 参数位置的 impl Trait (APIT) ---
// “调用者选择类型”——泛型参数的语法糖
fn print_all(items: impl Iterator<Item = i32>) {
    for item in items { println!("{item}"); }
}
// 等同于:
fn print_all_verbose<I: Iterator<Iitem = i32>>(items: I) {
    for item in items { println!("{item}"); }
}
// 调用者决定具体类型:print_all(vec![1,2,3].into_iter())
//                     print_all(0..10)

// --- 返回位置的 impl Trait (RPIT) ---
// “被调用者选择类型”——函数体选择一个具体的类型
fn evens(limit: i32) -> impl Iterator<Item = i32> {
    (0..limit).filter(|x| x % 2 == 0)
    // 具体类型是 Filter<Range<i32>, Closure>
    // 但调用者只能看到“某个 Iterator<Item = i32>”
}
}

核心区别:

特性APIT (fn foo(x: impl T))RPIT (fn foo() -> impl T)
谁来选类型?调用者被调用者(函数体)
是否单态化?是——每种类型一份副本是——一个具体的类型
支持 Turbofish 语法?否(不允许 foo::<X>())不适用
等价于fn foo<X: T>(x: X)存在类型 (Existential type)

Trait 定义中的 RPIT (RPITIT)

自 Rust 1.75 起,你可以在 trait 定义中直接使用 -> impl Trait:

#![allow(unused)]
fn main() {
trait Container {
    fn items(&self) -> impl Iterator<Item = &str>;
    //                 ^^^^ 每个实现者返回其自己的具体类型
}

struct CsvRow {
    fields: Vec<String>,
}

impl Container for CsvRow {
    fn items(&self) -> impl Iterator<Item = &str> {
        self.fields.iter().map(String::as_str)
    }
}

struct FixedFields;

impl Container for FixedFields {
    fn items(&self) -> impl Iterator<Item = &str> {
        ["host", "port", "timeout"].into_iter()
    }
}
}

在 Rust 1.75 之前,你必须使用 Box<dyn Iterator> 或关联类型在 trait 中实现此功能。RPITIT 消除了堆内存分配。

impl Trait vs dyn Trait —— 决策指南

你在编译时知道具体的类型吗?
├── 是 → 使用 impl Trait 或泛型(零开销,可内联)
└── 否 → 你需要异构集合(heterogeneous collection)吗?
     ├── 是 → 使用 dyn Trait (Box<dyn T>, &dyn T)
     └── 否 → 你是否需要跨 API 边界使用同一个 trait 对象?
          ├── 是 → 使用 dyn Trait
          └── 否 → 使用泛型 / impl Trait
特性impl Traitdyn Trait
分发方式静态分发 (单态化)动态分发 (vtable)
性能最佳——可内联每次调用一次间接寻址
异构集合❌✅
每种类型的二进制大小每种类型一份副本代码共享
Trait 必须对象安全?否是
可在 Trait 定义中使用?✅ (Rust 1.75+)始终支持

使用 Any 和 TypeId 进行类型擦除

有时你需要存储 未知 类型的值并在稍后进行向下转换(downcast)——这种模式类似于 C 语言中的 void* 或 C# 中的 object。Rust 通过 std::any::Any 提供了这一功能:

use std::any::Any;

// 存储不同类型的值:
fn log_value(value: &dyn Any) {
    if let Some(s) = value.downcast_ref::<String>() {
        println!("String: {s}");
    } else if let Some(n) = value.downcast_ref::<i32>() {
        println!("i32: {n}");
    } else {
        // TypeId 允许你在运行时检查类型:
        println!("未知类型: {:?}", value.type_id());
    }
}

// 适用于插件系统、事件总线或 ECS 风格的架构:
struct AnyMap(std::collections::HashMap<std::any::TypeId, Box<dyn Any + Send>>);

impl AnyMap {
    fn new() -> Self { AnyMap(std::collections::HashMap::new()) }

    fn insert<T: Any + Send + 'static>(&mut self, value: T) {
        self.0.insert(std::any::TypeId::of::<T>(), Box::new(value));
    }

    fn get<T: Any + Send + 'static>(&self) -> Option<&T> {
        self.0.get(&std::any::TypeId::of::<T>())?
            .downcast_ref()
    }
}

fn main() {
    let mut map = AnyMap::new();
    map.insert(42_i32);
    map.insert(String::from("hello"));

    assert_eq!(map.get::<i32>(), Some(&42));
    assert_eq!(map.get::<String>().map(|s| s.as_str()), Some("hello"));
    assert_eq!(map.get::<f64>(), None); // 从未插入过
}

何时使用 Any:插件/扩展系统、类型索引映射 (typemap)、错误向下转换 (anyhow::Error::downcast_ref)。如果在编译时已知类型集,请优先使用泛型或 trait 对象——Any 是牺牲编译时安全性以换取灵活性的最后手段。


扩展 Trait (Extension Traits) —— 为不属于你的类型添加方法

Rust 的孤儿规则(orphan rule)阻止你为外部类型实现外部 Trait。扩展 Trait 是标准的解决方法:在你的 crate 中定义一个 新 Trait,并为任何满足约束的类型提供全包含实现。调用者只需导入该 Trait,新方法就会出现在现有类型上。

这种模式在 Rust 生态系统中随处可见:itertools::Itertools、futures::StreamExt、tokio::io::AsyncReadExt、tower::ServiceExt。

问题场景

#![allow(unused)]
fn main() {
// 我们想为所有产生 f64 的迭代器添加一个 .mean() 方法。
// 但 Iterator 定义在 std 中,而 f64 是原始类型——孤儿规则阻止了这种做法:
//
// impl<I: Iterator<Item = f64>> I {   // ❌ 无法为外部类型添加固有方法(inherent methods)
//     fn mean(self) -> f64 { ... }
// }
}

解决方案:扩展 Trait

#![allow(unused)]
fn main() {
/// 为数值类型的迭代器提供的扩展方法。
pub trait IteratorExt: Iterator {
    /// 计算算术平均值。对于空迭代器返回 `None`。
    fn mean(self) -> Option<f64>
    where
        Self: Sized,
        Self::Item: Into<f64>;
}

// 全包含实现——自动应用于所有迭代器
impl<I: Iterator> IteratorExt for I {
    fn mean(self) -> Option<f64>
    where
        Self: Sized,
        Self::Item: Into<f64>,
    {
        let mut sum: f64 = 0.0;
        let mut count: u64 = 0;
        for item in self {
            sum += item.into();
            count += 1;
        }
        if count == 0 { None } else { Some(sum / count as f64) }
    }
}

// 使用方法——只需导入该 trait:
use crate::IteratorExt;  // 一次导入,所有迭代器都拥有了该方法

fn analyze_temperatures(readings: &[f64]) -> Option<f64> {
    readings.iter().copied().mean()  // .mean() 现在可用了!
}

fn analyze_sensor_data(data: &[i32]) -> Option<f64> {
    data.iter().copied().mean()  // 同样适用于 i32 (因为 i32: Into<f64>)
}
}

现实案例:诊断结果扩展

#![allow(unused)]
fn main() {
use std::collections::HashMap;

struct DiagResult {
    component: String,
    passed: bool,
    message: String,
}

/// Vec<DiagResult> 的扩展 trait —— 添加领域特定的分析方法。
pub trait DiagResultsExt {
    fn passed_count(&self) -> usize;
    fn failed_count(&self) -> usize;
    fn overall_pass(&self) -> bool;
    fn failures_by_component(&self) -> HashMap<String, Vec<&DiagResult>>;
}

impl DiagResultsExt for Vec<DiagResult> {
    fn passed_count(&self) -> usize {
        self.iter().filter(|r| r.passed).count()
    }

    fn failed_count(&self) -> usize {
        self.iter().filter(|r| !r.passed).count()
    }

    fn overall_pass(&self) -> bool {
        self.iter().all(|r| r.passed)
    }

    fn failures_by_component(&self) -> HashMap<String, Vec<&DiagResult>> {
        let mut map = HashMap::new();
        for r in self.iter().filter(|r| !r.passed) {
            map.entry(r.component.clone()).or_default().push(r);
        }
        map
    }
}

// 现在任何 Vec<DiagResult> 都拥有了这些方法:
fn report(results: Vec<DiagResult>) {
    if !results.overall_pass() {
        let failures = results.failures_by_component();
        for (component, fails) in &failures {
            eprintln!("{component}: {} 次失败", fails.len());
        }
    }
}
}

命名约定

Rust 生态系统使用一致的 Ext 后缀:

Crate扩展 Trait扩展对象
itertoolsItertoolsIterator
futuresStreamExt, FutureExtStream, Future
tokioAsyncReadExt, AsyncWriteExtAsyncRead, AsyncWrite
towerServiceExtService
bytesBufMut (部分)&mut [u8]
你的 CrateDiagResultsExtVec<DiagResult>

何时使用

情况是否使用扩展 Trait?
为外部类型添加便捷方法✅
在泛型集合上对领域特定逻辑进行分组✅
方法需要访问私有字段❌ (请使用包装器/newtype)
方法逻辑上属于你控制的新类型❌ (直接添加到你的类型中)
希望方法在不导入的情况下可用❌ (仅限固有方法)

枚举分发 (Enum Dispatch) —— 无需 dyn 的静态多态

当你拥有一组 闭置 (closed set) 的类型实现了某个 Trait 时,你可以使用枚举来代替 dyn Trait。枚举的每个变体(variant)持有具体的类型。这种方式消除了 vtable 间接寻址和堆内存分配,同时保持了相同的调用接口。

dyn Trait 的局限性

#![allow(unused)]
fn main() {
trait Sensor {
    fn read(&self) -> f64;
    fn name(&self) -> &str;
}

struct Gps { lat: f64, lon: f64 }
struct Thermometer { temp_c: f64 }
struct Accelerometer { g_force: f64 }

impl Sensor for Gps {
    fn read(&self) -> f64 { self.lat }
    fn name(&self) -> &str { "GPS" }
}
impl Sensor for Thermometer {
    fn read(&self) -> f64 { self.temp_c }
    fn name(&self) -> &str { "温度计" }
}
impl Sensor for Accelerometer {
    fn read(&self) -> f64 { self.g_force }
    fn name(&self) -> &str { "加速度计" }
}

// 使用 dyn 的异构集合——虽然可行,但有额外开销:
fn read_all_dyn(sensors: &[Box<dyn Sensor>]) -> Vec<f64> {
    sensors.iter().map(|s| s.read()).collect()
    // 每次调用 .read() 都要通过 vtable 间接跳转
    // 每个 Box 都要在堆上分配内存
}
}

枚举分发解决方案

// 用枚举代替 trait 对象:
enum AnySensor {
    Gps(Gps),
    Thermometer(Thermometer),
    Accelerometer(Accelerometer),
}

impl AnySensor {
    fn read(&self) -> f64 {
        match self {
            AnySensor::Gps(s) => s.read(),
            AnySensor::Thermometer(s) => s.read(),
            AnySensor::Accelerometer(s) => s.read(),
        }
    }

    fn name(&self) -> &str {
        match self {
            AnySensor::Gps(s) => s.name(),
            AnySensor::Thermometer(s) => s.name(),
            AnySensor::Accelerometer(s) => s.name(),
        }
    }
}

// 现在:无需堆分配,无需 vtable,数据内联存储
fn read_all(sensors: &[AnySensor]) -> Vec<f64> {
    sensors.iter().map(|s| s.read()).collect()
    // 每次调用 .read() 都是一个 match 分支——编译器可以内联所有内容
}

fn main() {
    let sensors = vec![
        AnySensor::Gps(Gps { lat: 47.6, lon: -122.3 }),
        AnySensor::Thermometer(Thermometer { temp_c: 72.5 }),
        AnySensor::Accelerometer(Accelerometer { g_force: 1.02 }),
    ];

    for sensor in &sensors {
        println!("{}: {:.2}", sensor.name(), sensor.read());
    }
}

在枚举上实现 Trait

为了保持互操作性,你可以在枚举本身上实现原始 Trait:

#![allow(unused)]
fn main() {
impl Sensor for AnySensor {
    fn read(&self) -> f64 {
        match self {
            AnySensor::Gps(s) => s.read(),
            AnySensor::Thermometer(s) => s.read(),
            AnySensor::Accelerometer(s) => s.read(),
        }
    }

    fn name(&self) -> &str {
        match self {
            AnySensor::Gps(s) => s.name(),
            AnySensor::Thermometer(s) => s.name(),
            AnySensor::Accelerometer(s) => s.name(),
        }
    }
}

// 现在 AnySensor 可以在任何通过泛型约束要求 Sensor 的地方工作:
fn report<S: Sensor>(s: &S) {
    println!("{}: {:.2}", s.name(), s.read());
}
}

使用宏减少模板代码

match 分支的委托调用非常重复。可以使用宏来消除它:

#![allow(unused)]
fn main() {
macro_rules! dispatch_sensor {
    ($self:expr, $method:ident $(, $arg:expr)*) => {
        match $self {
            AnySensor::Gps(s) => s.$method($($arg),*),
            AnySensor::Thermometer(s) => s.$method($($arg),*),
            AnySensor::Accelerometer(s) => s.$method($($arg),*),
        }
    };
}

impl Sensor for AnySensor {
    fn read(&self) -> f64     { dispatch_sensor!(self, read) }
    fn name(&self) -> &str    { dispatch_sensor!(self, name) }
}
}

对于大型项目,enum_dispatch crate 可以使这一切自动化:

#![allow(unused)]
fn main() {
use enum_dispatch::enum_dispatch;

#[enum_dispatch]
trait Sensor {
    fn read(&self) -> f64;
    fn name(&self) -> &str;
}

#[enum_dispatch(Sensor)]
enum AnySensor {
    Gps(Gps),
    Thermometer(Thermometer),
    Accelerometer(Accelerometer),
}
// 所有委托代码都会自动生成。
}

dyn Trait vs 枚举分发 —— 决策指南

类型集是闭置的吗(即在编译时已知)?
├── 是 → 优先使用枚举分发(更快速,无堆分配)
│         ├── 变体较少 (< ~20)?     → 手动定义枚举
│         └── 变体较多或不断增加? → 使用 enum_dispatch crate
└── 否 → 必须使用 dyn Trait (用于插件系统、用户自定义类型)
属性dyn Trait枚举分发 (Enum Dispatch)
分发开销Vtable 间接跳转 (~2ns)分支预测 (~0.3ns)
堆内存分配通常有 (Box)无 (内联)
缓存友好否 (指针追踪)是 (连续存储)
对新类型开放✅ (任何人均可实现)❌ (闭置集合)
代码体积共享每个变体各一份副本
Trait 必须对象安全是否
添加新变体无需修改代码需要更新枚举和 match 分支

何时使用枚举分发

场景建议
诊断测试类型 (CPU, GPU, NIC, 内存, …)✅ 枚举分发——闭置集合,编译时已知
总线协议 (SPI, I2C, UART, …)✅ 枚举分发或 Config trait
插件系统 (用户在运行时加载 .so)❌ 使用 dyn Trait
2-3 个变体✅ 手动枚举分发
10 个以上变体且有许多方法✅ enum_dispatch crate
性能关键的内部循环✅ 枚举分发 (消除 vtable 开销)

职能混入 (Capability Mixins) —— 作为零成本组合的关联类型

Ruby 开发者通过 混入 (mixins) 来组合行为——include SomeModule 会将方法注入到类中。Rust 通过 关联类型 + 默认方法 + 全包含实现 的组合可以达到同样的效果,但具有以下优势:

  • 所有内容都在 编译时 解析——不会出现 “method-missing” 的惊喜。
  • 每个关联类型都是一个 旋钮 (knob),可以改变默认方法产生的结果。
  • 编译器对每种组合进行 单态化 (monomorphises) ——零 vtable 开销。

问题场景:交叉引用的总线依赖

硬件诊断程序通常共享一些通用操作——读取 IPMI 传感器、切换 GPIO 引脚、过 SPI 采样温度——但不同的诊断程序需要不同的组合。Rust 中不存在继承体系。将每个总线句柄(bus handle)作为函数参数传递会导致函数签名过于冗长。我们需要一种方式来按需 混入 (mix in) 总线职能。

第一步:定义“原料”Trait (Ingredient Traits)

每种原料通过一个关联类型提供一种硬件职能:

#![allow(unused)]
fn main() {
use std::io;

// ── 总线抽象 (由硬件团队提供的 trait) ──────────────────────────
pub trait SpiBus {
    fn spi_transfer(&self, tx: &[u8], rx: &mut [u8]) -> io::Result<()>;
}

pub trait I2cBus {
    fn i2c_read(&self, addr: u8, reg: u8, buf: &mut [u8]) -> io::Result<()>;
    fn i2c_write(&self, addr: u8, reg: u8, data: &[u8]) -> io::Result<()>;
}

pub trait GpioPin {
    fn set_high(&self) -> io::Result<()>;
    fn set_low(&self) -> io::Result<()>;
    fn read_level(&self) -> io::Result<bool>;
}

pub trait IpmiBmc {
    fn raw_command(&self, net_fn: u8, cmd: u8, data: &[u8]) -> io::Result<Vec<u8>>;
    fn read_sensor(&self, sensor_id: u8) -> io::Result<f64>;
}

// ── 原料 trait:每个总线对应一个,带有一个关联类型 ───
pub trait HasSpi {
    type Spi: SpiBus;
    fn spi(&self) -> &Self::Spi;
}

pub trait HasI2c {
    type I2c: I2cBus;
    fn i2c(&self) -> &Self::I2c;
}

pub trait HasGpio {
    type Gpio: GpioPin;
    fn gpio(&self) -> &Self::Gpio;
}

pub trait HasIpmi {
    type Ipmi: IpmiBmc;
    fn ipmi(&self) -> &Self::Ipmi;
}
}

每种原料都非常精简、通用,并且可以独立测试。

第二步:定义“混入”Trait (Mixin Traits)

混入 trait 将其所需的原料声明为父 trait,然后通过 默认实现 提供所有方法——实现者可以免费获得这些方法:

#![allow(unused)]
fn main() {
/// 混入:风扇诊断——需要 I2C (转速计) + GPIO (PWM 使能)
pub trait FanDiagMixin: HasI2c + HasGpio {
    /// 通过 I2C 从转速计 IC 读取风扇 RPM。
    fn read_fan_rpm(&self, fan_id: u8) -> io::Result<u32> {
        let mut buf = [0u8; 2];
        self.i2c().i2c_read(0x48 + fan_id, 0x00, &mut buf)?;
        Ok(u16::from_be_bytes(buf) as u32 * 60) // 转速计数 → RPM
    }

    /// 通过 GPIO 使能或禁用风扇 PWM 输出。
    fn set_fan_pwm(&self, enable: bool) -> io::Result<()> {
        if enable { self.gpio().set_high() }
        else      { self.gpio().set_low() }
    }

    /// 全面的风扇健康检查——读取 RPM 并验证是否在阈值范围内。
    fn check_fan_health(&self, fan_id: u8, min_rpm: u32) -> io::Result<bool> {
        let rpm = self.read_fan_rpm(fan_id)?;
        Ok(rpm >= min_rpm)
    }
}

/// 混入:温度监控——需要 SPI (热电偶 ADC) + IPMI (BMC 传感器)
pub trait TempMonitorMixin: HasSpi + HasIpmi {
    /// 通过 SPI ADC (例如 MAX31855) 读取热电偶数值。
    fn read_thermocouple(&self) -> io::Result<f64> {
        let mut rx = [0u8; 4];
        self.spi().spi_transfer(&[0x00; 4], &mut rx)?;
        let raw = i32::from_be_bytes(rx) >> 18; // 14位有符号数
        Ok(raw as f64 * 0.25)
    }

    /// 通过 IPMI 读取 BMC 管理的温度传感器。
    fn read_bmc_temp(&self, sensor_id: u8) -> io::Result<f64> {
        self.ipmi().read_sensor(sensor_id)
    }

    /// 交叉验证:热电偶与 BMC 的读数差值必须在范围内。
    fn validate_temps(&self, sensor_id: u8, max_delta: f64) -> io::Result<bool> {
        let tc = self.read_thermocouple()?;
        let bmc = self.read_bmc_temp(sensor_id)?;
        Ok((tc - bmc).abs() <= max_delta)
    }
}

/// 混入:电源顺序控制——需要 GPIO (电源轨使能) + IPMI (事件日志)
pub trait PowerSeqMixin: HasGpio + HasIpmi {
    /// 断言 power-good GPIO 引脚并通过 IPMI 传感器验证。
    fn enable_power_rail(&self, sensor_id: u8) -> io::Result<bool> {
        self.gpio().set_high()?;
        std::thread::sleep(std::time::Duration::from_millis(50));
        let voltage = self.ipmi().read_sensor(sensor_id)?;
        Ok(voltage > 0.8) // 高于 80% 标称值 = 正常
    }

    /// 撤销电源使能并通过 IPMI OEM 命令记录关机日志。
    fn disable_power_rail(&self) -> io::Result<()> {
        self.gpio().set_low()?;
        // 向 BMC 记录 OEM "电源轨已禁用" 事件
        self.ipmi().raw_command(0x2E, 0x01, &[0x00, 0x01])?;
        Ok(())
    }
}
}

第三步:全包含实现使其真正成为“混入”

神奇的代码行——提供原料,即可获得方法:

#![allow(unused)]
fn main() {
impl<T: HasI2c + HasGpio>  FanDiagMixin    for T {}
impl<T: HasSpi  + HasIpmi>  TempMonitorMixin for T {}
impl<T: HasGpio + HasIpmi>  PowerSeqMixin   for T {}
}

任何实现了正确原料 Trait 的结构体会 自动 获得每个混入方法——无需重复代码、无需转发、无需继承。

第四步:接入生产环境

#![allow(unused)]
fn main() {
// ── 具体的总线实现 (Linux 平台) ─────────────────────────────────
struct LinuxSpi  { dev: String }
struct LinuxI2c  { dev: String }
struct SysfsGpio { pin: u32 }
struct IpmiTool  { timeout_secs: u32 }

impl SpiBus for LinuxSpi {
    fn spi_transfer(&self, _tx: &[u8], _rx: &mut [u8]) -> io::Result<()> {
        // spidev ioctl —— 为简洁起见省略实现
        Ok(())
    }
}
impl I2cBus for LinuxI2c {
    fn i2c_read(&self, _addr: u8, _reg: u8, _buf: &mut [u8]) -> io::Result<()> {
        // i2c-dev ioctl —— 省略实现
        Ok(())
    }
    fn i2c_write(&self, _addr: u8, _reg: u8, _data: &[u8]) -> io::Result<()> { Ok(()) }
}
impl GpioPin for SysfsGpio {
    fn set_high(&self) -> io::Result<()>  { /* /sys/class/gpio */ Ok(()) }
    fn set_low(&self) -> io::Result<()>   { Ok(()) }
    fn read_level(&self) -> io::Result<bool> { Ok(true) }
}
impl IpmiBmc for IpmiTool {
    fn raw_command(&self, _nf: u8, _cmd: u8, _data: &[u8]) -> io::Result<Vec<u8>> {
        // 调用 ipmitool 命令行 —— 省略实现
        Ok(vec![])
    }
    fn read_sensor(&self, _id: u8) -> io::Result<f64> { Ok(25.0) }
}

// ── 生产平台 —— 拥有所有四种总线 ──────────────────────────────
struct DiagPlatform {
    spi:  LinuxSpi,
    i2c:  LinuxI2c,
    gpio: SysfsGpio,
    ipmi: IpmiTool,
}

impl HasSpi  for DiagPlatform { type Spi  = LinuxSpi;  fn spi(&self)  -> &LinuxSpi  { &self.spi  } }
impl HasI2c  for DiagPlatform { type I2c  = LinuxI2c;  fn i2c(&self)  -> &LinuxI2c  { &self.i2c  } }
impl HasGpio for DiagPlatform { type Gpio = SysfsGpio; fn gpio(&self) -> &SysfsGpio { &self.gpio } }
impl HasIpmi for DiagPlatform { type Ipmi = IpmiTool;  fn ipmi(&self) -> &IpmiTool  { &self.ipmi } }

// DiagPlatform 现在拥有了所有混入方法:
fn production_diagnostics(platform: &DiagPlatform) -> io::Result<()> {
    let rpm = platform.read_fan_rpm(0)?;       // 来自 FanDiagMixin
    let tc  = platform.read_thermocouple()?;   // 来自 TempMonitorMixin
    let ok  = platform.enable_power_rail(42)?;  // 来自 PowerSeqMixin
    println!("风扇: {rpm} RPM, 温度: {tc}°C, 电源状态: {ok}");
    Ok(())
}
}

第五步:使用 Mock 进行测试 (无需硬件)

#![allow(unused)]
fn main() {
#[cfg(test)]
mod tests {
    use super::*;
    use std::cell::Cell;

    struct MockSpi  { temp: Cell<f64> }
    struct MockI2c  { rpm: Cell<u32> }
    struct MockGpio { level: Cell<bool> }
    struct MockIpmi { sensor_val: Cell<f64> }

    impl SpiBus for MockSpi {
        fn spi_transfer(&self, _tx: &[u8], rx: &mut [u8]) -> io::Result<()> {
            // 将 mock 温度编码为 MAX31855 格式
            let raw = ((self.temp.get() / 0.25) as i32) << 18;
            rx.copy_from_slice(&raw.to_be_bytes());
            Ok(())
        }
    }
    impl I2cBus for MockI2c {
        fn i2c_read(&self, _addr: u8, _reg: u8, buf: &mut [u8]) -> io::Result<()> {
            let tach = (self.rpm.get() / 60) as u16;
            buf.copy_from_slice(&tach.to_be_bytes());
            Ok(())
        }
        fn i2c_write(&self, _: u8, _: u8, _: &[u8]) -> io::Result<()> { Ok(()) }
    }
    impl GpioPin for MockGpio {
        fn set_high(&self)  -> io::Result<()>   { self.level.set(true);  Ok(()) }
        fn set_low(&self)   -> io::Result<()>   { self.level.set(false); Ok(()) }
        fn read_level(&self) -> io::Result<bool> { Ok(self.level.get()) }
    }
    impl IpmiBmc for MockIpmi {
        fn raw_command(&self, _: u8, _: u8, _: &[u8]) -> io::Result<Vec<u8>> { Ok(vec![]) }
        fn read_sensor(&self, _: u8) -> io::Result<f64> { Ok(self.sensor_val.get()) }
    }

    // ── 部分平台:仅包含风扇相关的总线 ─────────────────
    struct FanTestRig {
        i2c:  MockI2c,
        gpio: MockGpio,
    }
    impl HasI2c  for FanTestRig { type I2c  = MockI2c;  fn i2c(&self)  -> &MockI2c  { &self.i2c  } }
    impl HasGpio for FanTestRig { type Gpio = MockGpio; fn gpio(&self) -> &MockGpio { &self.gpio } }
    // FanTestRig 获得了 FanDiagMixin,但没有 TempMonitorMixin 或 PowerSeqMixin

    #[test]
    fn fan_health_check_passes_above_threshold() {
        let rig = FanTestRig {
            i2c:  MockI2c  { rpm: Cell::new(6000) },
            gpio: MockGpio { level: Cell::new(false) },
        };
        assert!(rig.check_fan_health(0, 4000).unwrap());
    }

    #[test]
    fn fan_health_check_fails_below_threshold() {
        let rig = FanTestRig {
            i2c:  MockI2c  { rpm: Cell::new(2000) },
            gpio: MockGpio { level: Cell::new(false) },
        };
        assert!(!rig.check_fan_health(0, 4000).unwrap());
    }
}
}

请注意,FanTestRig 仅实现了 HasI2c + HasGpio —— 它会自动获得 FanDiagMixin,但编译器会 拒绝 调用 rig.read_thermocouple(),因为 HasSpi 未被满足。这就是在编译时强制执行的混入作用域。

条件方法 —— 超越 Ruby 的能力

可以给单个默认方法添加 where 约束。该方法只有在关联类型满足额外约束时才 存在:

#![allow(unused)]
fn main() {
/// 具有 DMA 能力的 SPI 控制器的标记 trait
pub trait DmaCapable: SpiBus {
    fn dma_transfer(&self, tx: &[u8], rx: &mut [u8]) -> io::Result<()>;
}

/// 具有中断能力的 GPIO 引脚的标记 trait
pub trait InterruptCapable: GpioPin {
    fn wait_for_edge(&self, timeout_ms: u32) -> io::Result<bool>;
}

pub trait AdvancedDiagMixin: HasSpi + HasGpio {
    // 始终可用
    fn basic_probe(&self) -> io::Result<bool> {
        let mut rx = [0u8; 1];
        self.spi().spi_transfer(&[0xFF], &mut rx)?;
        Ok(rx[0] != 0x00)
    }

    // 只有当 SPI 控制器支持 DMA 时才存在
    fn bulk_sensor_read(&self, buf: &mut [u8]) -> io::Result<()>
    where
        Self::Spi: DmaCapable,
    {
        self.spi().dma_transfer(&vec![0x00; buf.len()], buf)
    }

    // 只有当 GPIO 引脚支持中断时才存在
    fn wait_for_fault_signal(&self, timeout_ms: u32) -> io::Result<bool>
    where
        Self::Gpio: InterruptCapable,
    {
        self.gpio().wait_for_edge(timeout_ms)
    }
}

impl<T: HasSpi + HasGpio> AdvancedDiagMixin for T {}
}

如果你的平台的 SPI 不支持 DMA,调用 bulk_sensor_read() 将会导致 编译错误,而不是运行时崩溃。Ruby 的 respond_to? 检查是最接近的等效功能——但它发生在部署阶段,而不是编译阶段。

可组合性:堆叠混入

多个混入可以共享同一个原料——不存在菱形继承问题:

┌─────────────┐    ┌───────────┐    ┌──────────────┐
│ FanDiagMixin│    │TempMonitor│    │ PowerSeqMixin│
│  (I2C+GPIO) │    │ (SPI+IPMI)│    │  (GPIO+IPMI) │
└──────┬──────┘    └─────┬─────┘    └──────┬───────┘
       │                 │                 │
       │   ┌─────────────┴─────────────┐   │
       └──►│      DiagPlatform         │◄──┘
           │ HasSpi+HasI2c+HasGpio     │
           │        +HasIpmi           │
           └───────────────────────────┘

DiagPlatform 只需 一次 实现 HasGpio,而 FanDiagMixin 和 PowerSeqMixin 都会使用同一个 self.gpio()。在 Ruby 中,这将是两个模块都调用 self.gpio_pin ——但如果它们期望不同的引脚编号,你只能在运行时才发现冲突。在 Rust 中,你可以在类型层面上消除歧义。

对比:Ruby Mixins vs Rust 职能混入

维度Ruby MixinsRust 职能混入 (Capability Mixins)
分发方式运行时 (方法表查找)编译时 (单态化)
安全组合MRO 线性化隐藏了冲突编译器拒绝歧义
条件方法运行时的 respond_to?编译时的 where 约束
开销方法分发 + GC 开销零成本 (内联)
可测试性通过元编程进行 Stub/Mock针对 Mock 类型进行泛型化
添加新总线运行时 include添加原料 trait,重新编译
运行时灵活性extend, prepend, 开放类无 (全静态)

何时使用职能混入

场景是否使用混入?
多个诊断程序共享读取总线的逻辑✅
测试夹具需要不同的总线子集✅ (部分原料结构体)
仅对某些总线能力 (DMA, IRQ) 有效的方法✅ (条件性的 where 约束)
需要运行时模块加载 (插件)❌ (使用 dyn Trait 或枚举分发)
只有一个总线的单一结构体——无需共享❌ (保持简单)
存在一致性问题的跨 crate 原料⚠️ (使用 newtype 包装器)

关键要点 —— 职能混入

  1. 原料 trait (Ingredient trait) = 关联类型 + 访问方法 (例如 HasSpi)
  2. 混入 trait (Mixin trait) = 对原料的父 trait 约束 + 默认方法体
  3. 全包含实现 (Blanket impl) = impl<T: HasX + HasY> Mixin for T {} —— 自动注入方法
  4. 条件方法 (Conditional methods) = 在单个默认方法上添加 where Self::Spi: DmaCapable
  5. 部分平台 (Partial platforms) = 仅实现了所需原料的测试结构体
  6. 零运行时成本 —— 编译器为每个平台类型生成专门的代码

有类型命令 (Typed Commands) —— GADT 风格的返回类型安全

在 Haskell 中,广义代数数据类型 (GADTs) 允许每个数据类型的构造函数细化(refine)类型参数——因此 Expr Int 和 Expr Bool 会受到类型检查器的强制约束。Rust 虽然没有直接的 GADT 语法,但 带有关联类型的 Trait 可以实现同样的保障:命令类型 决定了 响应类型,将两者混淆会导致编译错误。

这种模式对于硬件诊断特别强大,因为 IPMI 命令、寄存器读取和传感器查询各自返回不同的物理量,这些物理量永远不应该被混淆。

问题场景:未类型化的 Vec<u8> 泥潭

大多数 C/C++ 的 IPMI 协议栈——以及一些幼稚的 Rust 移植版本——在任何地方都使用原始字节:

#![allow(unused)]
fn main() {
use std::io;

struct BmcConnectionUntyped { timeout_secs: u32 }

impl BmcConnectionUntyped {
    fn raw_command(&self, net_fn: u8, cmd: u8, data: &[u8]) -> io::Result<Vec<u8>> {
        // ... 调用 ipmitool ...
        Ok(vec![0x00, 0x19, 0x00]) // 桩代码
    }
}

fn diagnose_thermal_untyped(bmc: &BmcConnectionUntyped) -> io::Result<()> {
    // 读取 CPU 温度 —— 传感器 ID 0x20
    let raw = bmc.raw_command(0x04, 0x2D, &[0x20])?;
    let cpu_temp = raw[0] as f64;  // 🤞 祈祷第 0 个字节就是读数

    // 读取风扇转速 —— 传感器 ID 0x30
    let raw = bmc.raw_command(0x04, 0x2D, &[0x30])?;
    let fan_rpm = raw[0] as u32;  // 🐛 错误:风扇速度是 2 字节小端序 (LE)

    // 读取入口电压 —— 传感器 ID 0x40
    let raw = bmc.raw_command(0x04, 0x2D, &[0x40])?;
    let voltage = raw[0] as f64;  // 🐛 错误:需要除以 1000

    // 🐛 将摄氏度与 RPM 进行比较 —— 可以编译,但毫无意义
    if cpu_temp > fan_rpm as f64 {
        println!("糟糕");
    }

    // 🐛 将电压作为温度传递 —— 编译完全正常
    log_temp_untyped(voltage);
    log_volts_untyped(cpu_temp);

    Ok(())
}

fn log_temp_untyped(t: f64)  { println!("温度: {t}°C"); }
fn log_volts_untyped(v: f64) { println!("电压: {v}V"); }
}

每个读数都是 f64 —— 编译器完全不知道一个是温度,另一个是 RPM,还有一个是电压。四个截然不同的错误在没有任何警告的情况下传递通过:

#错误后果发现时机
1风扇 RPM 被解析为 1 字节而非 2 字节读数为 25 RPM 而非 6400凌晨 3 点,风扇故障报警淹没生产环境
2电压没有除以 1000读数为 12000V 而非 12.0V阈值检查标记每个 PSU 都有故障
3将摄氏度与 RPM 比较毫无意义的布尔值可能永远发现不了
4电压被传递给 log_temp_untyped()日志中出现静默数据损坏6 个月后,查看历史记录时

解决方案:通过关联类型实现有类型命令

第一步:领域特定的 Newtype

#![allow(unused)]
fn main() {
#[derive(Debug, Clone, Copy, PartialEq, PartialOrd)]
struct Celsius(f64);

#[derive(Debug, Clone, Copy, PartialEq, PartialOrd)]
struct Rpm(u32);

#[derive(Debug, Clone, Copy, PartialEq, PartialOrd)]
struct Volts(f64);

#[derive(Debug, Clone, Copy, PartialEq, PartialOrd)]
struct Watts(f64);
}

第二步:命令 Trait (等效于 GADT)

关联类型 Response 是关键——它将每个命令与其返回类型绑定在一起:

#![allow(unused)]
fn main() {
trait IpmiCmd {
    /// GADT “索引”——决定了 execute() 的返回类型。
    type Response;

    fn net_fn(&self) -> u8;
    fn cmd_byte(&self) -> u8;
    fn payload(&self) -> Vec<u8>;

    /// 解析逻辑封装在此处 —— 每个命令都知道自己的字节布局。
    fn parse_response(&self, raw: &[u8]) -> io::Result<Self::Response>;
}
}

第三步:每个命令对应一个结构体,解析逻辑编写一次

#![allow(unused)]
fn main() {
struct ReadTemp { sensor_id: u8 }
impl IpmiCmd for ReadTemp {
    type Response = Celsius;  // ← “此命令返回温度”
    fn net_fn(&self) -> u8 { 0x04 }
    fn cmd_byte(&self) -> u8 { 0x2D }
    fn payload(&self) -> Vec<u8> { vec![self.sensor_id] }
    fn parse_response(&self, raw: &[u8]) -> io::Result<Celsius> {
        // 根据 IPMI SDR 定义的有符号字节 —— 编写一次,测试一次
        Ok(Celsius(raw[0] as i8 as f64))
    }
}

struct ReadFanSpeed { fan_id: u8 }
impl IpmiCmd for ReadFanSpeed {
    type Response = Rpm;     // ← “此命令返回 RPM”
    fn net_fn(&self) -> u8 { 0x04 }
    fn cmd_byte(&self) -> u8 { 0x2D }
    fn payload(&self) -> Vec<u8> { vec![self.fan_id] }
    fn parse_response(&self, raw: &[u8]) -> io::Result<Rpm> {
        // 2 字节小端序 (LE) —— 正确的布局,编码一次
        Ok(Rpm(u16::from_le_bytes([raw[0], raw[1]]) as u32))
    }
}

struct ReadVoltage { rail: u8 }
impl IpmiCmd for ReadVoltage {
    type Response = Volts;   // ← “此命令返回电压”
    fn net_fn(&self) -> u8 { 0x04 }
    fn cmd_byte(&self) -> u8 { 0x2D }
    fn payload(&self) -> Vec<u8> { vec![self.rail] }
    fn parse_response(&self, raw: &[u8]) -> io::Result<Volts> {
        // 毫伏转为伏特,始终正确
        Ok(Volts(u16::from_le_bytes([raw[0], raw[1]]) as f64 / 1000.0))
    }
}

struct ReadFru { fru_id: u8 }
impl IpmiCmd for ReadFru {
    type Response = String;
    fn net_fn(&self) -> u8 { 0x0A }
    fn cmd_byte(&self) -> u8 { 0x11 }
    fn payload(&self) -> Vec<u8> { vec![self.fru_id, 0x00, 0x00, 0xFF] }
    fn parse_response(&self, raw: &[u8]) -> io::Result<String> {
        Ok(String::from_utf8_lossy(raw).to_string())
    }
}
}

第四步:执行器 (零 dyn,单态化)

#![allow(unused)]
fn main() {
struct BmcConnection { timeout_secs: u32 }

impl BmcConnection {
    /// 对任何命令泛型化 —— 编译器为每个命令类型生成一个版本。
    fn execute<C: IpmiCmd>(&self, cmd: &C) -> io::Result<C::Response> {
        let raw = self.raw_send(cmd.net_fn(), cmd.cmd_byte(), &cmd.payload())?;
        cmd.parse_response(&raw)
    }

    fn raw_send(&self, _nf: u8, _cmd: u8, _data: &[u8]) -> io::Result<Vec<u8>> {
        Ok(vec![0x19, 0x00]) // 桩代码 —— 实际实现调用 ipmitool
    }
}
}

第五步:调用方代码:所有四个错误都变成了编译错误

#![allow(unused)]
fn main() {
fn diagnose_thermal(bmc: &BmcConnection) -> io::Result<()> {
    let cpu_temp: Celsius = bmc.execute(&ReadTemp { sensor_id: 0x20 })?;
    let fan_rpm:  Rpm     = bmc.execute(&ReadFanSpeed { fan_id: 0x30 })?;
    let voltage:  Volts   = bmc.execute(&ReadVoltage { rail: 0x40 })?;

    // 错误 #1 —— 不可能发生:解析逻辑存在于 ReadFanSpeed::parse_response 中
    // 错误 #2 —— 不可能发生:缩放逻辑存在于 ReadVoltage::parse_response 中

    // 错误 #3 —— 编译错误:
    // if cpu_temp > fan_rpm { }
    //    ^^^^^^^^   ^^^^^^^
    //    Celsius    Rpm      → “类型不匹配 (mismatched types)” ❌

    // 错误 #4 —— 编译错误:
    // log_temperature(voltage);
    //                 ^^^^^^^  预期 Celsius,得到 Volts ❌

    // 只有正确类型的比较才能通过编译:
    if cpu_temp > Celsius(85.0) {
        println!("CPU 过热: {:?}", cpu_temp);
    }
    if fan_rpm < Rpm(4000) {
        println!("风扇转速过慢: {:?}", fan_rpm);
    }

    Ok(())
}

fn log_temperature(t: Celsius) { println!("温度: {:?}", t); }
fn log_voltage(v: Volts)       { println!("电压: {:?}", v); }
}

诊断脚本的宏 DSL

对于需要按顺序运行许多命令的大型诊断程序,宏可以提供简练的声明式语法,同时保持完全的类型安全:

#![allow(unused)]
fn main() {
/// 执行一系列有类型的 IPMI 命令,返回一个结果元组。
/// 元组的每个元素都有其命令对应的 Response 类型。
macro_rules! diag_script {
    ($bmc:expr; $($cmd:expr),+ $(,)?) => {{
        ( $( $bmc.execute(&$cmd)?, )+ )
    }};
}

fn full_pre_flight(bmc: &BmcConnection) -> io::Result<()> {
    // 展开为: (Celsius, Rpm, Volts, String) —— 每个类型都被追踪
    let (temp, rpm, volts, board_pn) = diag_script!(bmc;
        ReadTemp     { sensor_id: 0x20 },
        ReadFanSpeed { fan_id:    0x30 },
        ReadVoltage  { rail:      0x40 },
        ReadFru      { fru_id:    0x00 },
    );

    println!("主板: {:?}", board_pn);
    println!("CPU: {:?}, 风扇: {:?}, 12V: {:?}", temp, rpm, volts);

    // 类型安全的阈值检查:
    assert!(temp  < Celsius(95.0), "CPU 太烫了");
    assert!(rpm   > Rpm(3000),     "风扇太慢了");
    assert!(volts > Volts(11.4),   "12V 电压下降了");

    Ok(())
}
}

这个宏仅仅是语法糖 —— 元组类型 (Celsius, Rpm, Volts, String) 完全由编译器推导出来。交换两个命令,解构就会在编译时报错,而不是在运行时。

用于异构命令列表的枚举分发

当你需要一个混合命令的 Vec(例如从 JSON 加载的可配置脚本)时,请使用枚举分发来保持“非 dyn”状态:

#![allow(unused)]
fn main() {
enum AnyReading {
    Temp(Celsius),
    Rpm(Rpm),
    Volt(Volts),
    Text(String),
}

enum AnyCmd {
    Temp(ReadTemp),
    Fan(ReadFanSpeed),
    Voltage(ReadVoltage),
    Fru(ReadFru),
}

impl AnyCmd {
    fn execute(&self, bmc: &BmcConnection) -> io::Result<AnyReading> {
        match self {
            AnyCmd::Temp(c)    => Ok(AnyReading::Temp(bmc.execute(c)?)),
            AnyCmd::Fan(c)     => Ok(AnyReading::Rpm(bmc.execute(c)?)),
            AnyCmd::Voltage(c) => Ok(AnyReading::Volt(bmc.execute(c)?)),
            AnyCmd::Fru(c)     => Ok(AnyReading::Text(bmc.execute(c)?)),
        }
    }
}

```rust
/// 动态诊断脚本 —— 在运行时加载命令
fn run_script(bmc: &BmcConnection, script: &[AnyCmd]) -> io::Result<Vec<AnyReading>> {
    script.iter().map(|cmd| cmd.execute(bmc)).collect()
}
}

虽然你失去了对每个元素的类型追踪(所有内容都是 AnyReading 类型),但你获得了运行时的灵活性 —— 并且解析逻辑仍然封装在每个 IpmiCmd 的实现中。

测试有类型命令

#![allow(unused)]
fn main() {
#[cfg(test)]
mod tests {
    use super::*;

    struct StubBmc {
        responses: std::collections::HashMap<u8, Vec<u8>>,
    }

    impl StubBmc {
        fn execute<C: IpmiCmd>(&self, cmd: &C) -> io::Result<C::Response> {
            let key = cmd.payload()[0]; // 使用传感器 ID 作为键
            let raw = self.responses.get(&key)
                .ok_or_else(|| io::Error::new(io::ErrorKind::NotFound, "未找到桩数据"))?;
            cmd.parse_response(raw)
        }
    }

    #[test]
    fn read_temp_parses_signed_byte() {
        let bmc = StubBmc {
            responses: [( 0x20, vec![0xE7] )].into() // -25 (i8) = 0xE7
        };
        let temp = bmc.execute(&ReadTemp { sensor_id: 0x20 }).unwrap();
        assert_eq!(temp, Celsius(-25.0));
    }

    #[test]
    fn read_fan_parses_two_byte_le() {
        let bmc = StubBmc {
            responses: [( 0x30, vec![0x00, 0x19] )].into() // 0x1900 = 6400
        };
        let rpm = bmc.execute(&ReadFanSpeed { fan_id: 0x30 }).unwrap();
        assert_eq!(rpm, Rpm(6400));
    }

    #[test]
    fn read_voltage_scales_millivolts() {
        let bmc = StubBmc {
            responses: [( 0x40, vec![0xE8, 0x2E] )].into() // 0x2EE8 = 12008 mV
        };
        let v = bmc.execute(&ReadVoltage { rail: 0x40 }).unwrap();
        assert!((v.0 - 12.008).abs() < 0.001);
    }
}
}

每个命令的解析逻辑都可以独立测试。如果在新的 IPMI 规范修订版本中,ReadFanSpeed 从 2 字节小端序变为了 4 字节大端序,你只需更新 一处 parse_response 的实现,测试就会捕捉到任何回归风险。

这如何映射到 Haskell 的 GADT

Haskell GADT                         Rust 等效实现
────────────────                     ───────────────────────
data Cmd a where                     trait IpmiCmd {
  ReadTemp :: SensorId -> Cmd Temp       type Response;
  ReadFan  :: FanId    -> Cmd Rpm        ...
                                     }

eval :: Cmd a -> IO a                fn execute<C: IpmiCmd>(&self, cmd: &C)
                                         -> io::Result<C::Response>

类型分支中的类型细化             单态化:编译器生成 
                                     execute::<ReadTemp>() → 返回 Celsius
                                     execute::<ReadFanSpeed>() → 返回 Rpm

两者都能保证:命令决定了返回类型。Rust 通过泛型单态化而非类型层面的分支分析实现了这一点 —— 拥有相同的安全性,且具有零运行时开销。

前后对比总结

维度未类型化 (Vec<u8>)有类型命令 (Typed Commands)
每个传感器的代码行数~3 行 (在每个调用处都要重复)~15 行 (编写并测试一次)
可能出现解析错误在每个调用处都可能发生仅在 parse_response 实现中
单位混淆 Bug无数零 (编译时报错)
添加新传感器需要改动 N 个文件,复制解析代码添加 1 个结构体 + 1 个实现
运行时开销—完全相同 (单态化)
IDE 自动补全到处都是 f64Celsius, Rpm, Volts —— 自解释
代码审查负担必须验证每个原始字节的解析验证每个传感器的 parse_response 实现
宏 DSL不适用diag_script!(bmc; ReadTemp{..}, ReadFan{..}) → (Celsius, Rpm)
动态脚本手动分发行为AnyCmd 枚举分发 —— 依然不含 dyn

何时使用有类型命令

场景建议
具有不同物理单位的 IPMI 传感器读取✅ 使用有类型命令
具有不同宽度字段的寄存器图谱✅ 使用有类型命令
网络协议消息 (请求 → 响应)✅ 使用有类型命令
只有一种返回格式的单一命令类型❌ 过度设计 —— 直接返回对应类型即可
对未知设备进行原型设计 / 探索❌ 先使用原始字节,稍后再引入类型
命令在编译时未知的插件系统⚠️ 使用 AnyCmd 枚举分发

关键要点 —— Trait

  • 关联类型 = 每个类型一个实现;泛型参数 = 每个类型多个实现
  • GATs 解锁了借用迭代器(lending iterators)和 Trait 中的 Async 模式
  • 对闭置集合使用枚举分发(快速);对开置集合使用 dyn Trait(灵活)
  • 当编译时类型未知时,Any + TypeId 是逃生窗口

参见: 第一章 —— 泛型 了解单态化以及泛型何时会导致代码膨胀。第三章 —— Newtype 与类型状态 了解如何将 Trait 与配置 Trait 模式结合使用。


练习:带有关联类型的 Repository ★★★ (~40 分钟)

设计一个带有 Error、Id 和 Item 关联类型的 Repository trait。为一个内存存储实现该 trait,并演示编译时的类型安全性。

🔑 答案
use std::collections::HashMap;

trait Repository {
    type Item;
    type Id;
    type Error;

    fn get(&self, id: &Self::Id) -> Result<Option<&Self::Item>, Self::Error>;
    fn insert(&mut self, item: Self::Item) -> Result<Self::Id, Self::Error>;
    fn delete(&mut self, id: &Self::Id) -> Result<bool, Self::Error>;
}

#[derive(Debug, Clone)]
struct User {
    name: String,
    email: String,
}

struct InMemoryUserRepo {
    data: HashMap<u64, User>,
    next_id: u64,
}

impl InMemoryUserRepo {
    fn new() -> Self {
        InMemoryUserRepo { data: HashMap::new(), next_id: 1 }
    }
}

impl Repository for InMemoryUserRepo {
    type Item = User;
    type Id = u64;
    type Error = std::convert::Infallible;

    fn get(&self, id: &u64) -> Result<Option<&User>, Self::Error> {
        Ok(self.data.get(id))
    }

    fn insert(&mut self, item: User) -> Result<u64, Self::Error> {
        let id = self.next_id;
        self.next_id += 1;
        self.data.insert(id, item);
        Ok(id)
    }

    fn delete(&mut self, id: &u64) -> Result<bool, Self::Error> {
        Ok(self.data.remove(id).is_some())
    }
}

fn create_and_fetch<R: Repository>(repo: &mut R, item: R::Item) -> Result<(), R::Error>
where
    R::Item: std::fmt::Debug,
    R::Id: std::fmt::Debug,
{
    let id = repo.insert(item)?;
    println!("插入成功,ID: {id:?}");
    let retrieved = repo.get(&id)?;
    println!("检索结果: {retrieved:?}");
    Ok(())
}

fn main() {
    let mut repo = InMemoryUserRepo::new();
    create_and_fetch(&mut repo, User {
        name: "Alice".into(),
        email: "[email protected]".into(),
    }).unwrap();
}

English Original

第 3 章:Newtype 与类型状态 (Type-State) 模式 🟡

你将学到:

  • Newtype 模式:实现零成本编译时类型安全
  • 类型状态 (Type-state) 模式:使非法状态转移不可表示 (Unrepresentable)
  • 结合类型状态的建造者 (Builder) 模式:用于编译时强制构建
  • 配置 Trait (Config trait) 模式:驯服泛型参数爆炸

Newtype:零成本类型安全

Newtype 模式将一个已有类型封装在单字段元组结构体中,以创建一个独特的新类型,且运行时开销为零:

#![allow(unused)]
fn main() {
// 不使用 newtype —— 很容易混淆:
fn create_user(name: String, email: String, age: u32, employee_id: u32) { }
// create_user(name, email, age, id);  — 但如果我们交换了 age 和 id 呢?
// create_user(name, email, id, age);  — 编译正常,但有 Bug
 
// 使用 newtype —— 编译器会捕捉错误:
struct UserName(String);
struct Email(String);
struct Age(u32);
struct EmployeeId(u32);

fn create_user(name: UserName, email: Email, age: Age, id: EmployeeId) { }
// create_user(name, email, EmployeeId(42), Age(30));
// ❌ 编译错误:expected Age, got EmployeeId
}

为 Newtype 实现 impl Deref —— 强大但有陷阱

为 Newtype 实现 Deref 可以让它自动强制转换为内部类型的引用,让你“免费”获得内部类型的所有方法:

#![allow(unused)]
fn main() {
use std::ops::Deref;

struct Email(String);

impl Email {
    fn new(raw: &str) -> Result<Self, &'static str> {
        if raw.contains('@') {
            Ok(Email(raw.to_string()))
        } else {
            Err("invalid email: missing @")
        }
    }
}

impl Deref for Email {
    type Target = str;
    fn deref(&self) -> &str { &self.0 }
}

// 现在 Email 自动解引用为 &str:
let email = Email::new("[email protected]").unwrap();
println!("Length: {}", email.len()); // 通过 Deref 使用 str::len
}

这很方便 —— 但它实际上在你的 Newtype 抽象边界上 打了一个洞,因为目标类型上的 每一个 方法都变得对你的包装器可见。

何时 Deref 是合适的

场景示例为什么没问题
智能指针包装器Box<T>, Arc<T>, MutexGuard<T>包装器的主要目的就是表现得像 T
透明的“薄”包装器String → str, PathBuf → Path, Vec<T> → [T]包装器本身就是目标类型的超集 (IS-A)
Newtype 确实等同于内部类型struct Hostname(String) 且你始终想要完整的字符串操作限制 API 不会带来任何价值

何时 Deref 是反模式

场景问题
带有不变量的领域类型Email 解引用为 &str,因此调用者可以调用 .split_at()、.trim() 等方法 —— 这些都无法保证“必须包含 @”的不变量。如果有人存储了修剪后的 &str 并重新解构,不变量就丢失了。
想要限制 API 的类型带有 Deref<Target = str> 的 struct Password(String) 会泄露 .as_bytes()、.chars() 以及 Debug 输出 —— 而这正是你试图隐藏的。
伪继承 (Fake inheritance)使用 Deref 让 ManagerWidget 自动解引用为 Widget 来模拟 OOP 继承。这是明确反对的 —— 参见 Rust API 指南 (C-DEREF)。

经验法则:如果你的 Newtype 存在是为了 增加类型安全 或 限制 API,请不要实现 Deref。如果它是为了在保持内部类型完整接口的同时 增加功能(例如智能指针),那么 Deref 是正确的选择。

DerefMut —— 双重风险

如果你还实现了 DerefMut,调用者可以直接 修改 内部值,从而绕过构造函数中的任何验证:

#![allow(unused)]
fn main() {
use std::ops::{Deref, DerefMut};

struct PortNumber(u16);

impl Deref for PortNumber {
    type Target = u16;
    fn deref(&self) -> &u16 { &self.0 }
}

impl DerefMut for PortNumber {
    fn deref_mut(&mut self) -> &mut u16 { &mut self.0 }
}

let mut port = PortNumber(443);
*port = 0; // 绕过了任何验证 —— 现在是一个无效的端口
}

只有当内部类型没有需要保护的不变量时,才实现 DerefMut。

优先选择显式委托 (Explicit Delegation)

当你只需要内部类型的 部分 方法时,请进行显式委托:

#![allow(unused)]
fn main() {
struct Email(String);

impl Email {
    fn new(raw: &str) -> Result<Self, &'static str> {
        if raw.contains('@') { Ok(Email(raw.to_string())) }
        else { Err("missing @") }
    }

    // 仅暴露合理的部分:
    pub fn as_str(&self) -> &str { &self.0 }
    pub fn len(&self) -> usize { self.0.len() }
    pub fn domain(&self) -> &str {
        self.0.split('@').nth(1).unwrap_or("")
    }
    // .split_at(), .trim(), .replace() — 不暴露
}
}

Clippy 与生态系统

  • clippy::wrong_self_convention 可能会在 Deref 强制转换导致方法解析结果出乎意料时触发(例如,is_empty() 解析到了内部类型的版本,而不是你打算遮蔽的版本)。
  • Rust API 指南 (C-DEREF) 指出:“只有智能指针应该实现 Deref。” 请将其作为强有力的默认规则;只有在有明确理由时才偏离。
  • 如果你需要 Trait 兼容性(例如将 Email 传递给预期 &str 的函数),请考虑实现 AsRef<str> 和 Borrow<str> —— 它们是显式转换,没有自动类型转换带来的意外。

决策矩阵

你是否希望内部类型的所有方法都可调用?
  ├─ 是 → 你的类型是否强制执行不变量或限制了 API?
  │    ├─ 否  → 实现 Deref ✅ (智能指针 / 透明包装器)
  │    └─ 是 → 不要实现 Deref ❌ (不变量泄露)
  └─ 否  → 不要实现 Deref ❌ (使用 AsRef / 显式委托)

类型状态 (Type-State):编译时协议强制执行

类型状态模式利用类型系统来强制要求操作按正确顺序发生。使非法状态变得 不可表示 (Unrepresentable)。

stateDiagram-v2
    [*] --> Disconnected: new()
    Disconnected --> Connected: connect()
    Connected --> Authenticated: authenticate()
    Authenticated --> Authenticated: request()
    Authenticated --> [*]: drop

    Disconnected --> Disconnected: ❌ request() 无法通过编译
    Connected --> Connected: ❌ request() 无法通过编译

每次转换都会 消耗 (consume) self 并返回一个新类型 —— 编译器以此强制执行有效的顺序。

// 问题:一个网络连接必须按以下步骤:
// 1. 创建 (Created)
// 2. 连接 (Connected)
// 3. 身份验证 (Authenticated)
// 4. 然后用于请求 (Request)
// 在 authenticate() 之前调用 request() 应该是一个编译错误。

// --- 类型状态标记 (零大小类型/ZSTs) ---
struct Disconnected;
struct Connected;
struct Authenticated;

// --- 由状态参数化的连接结构体 ---
struct Connection<State> {
    address: String,
    _state: std::marker::PhantomData<State>,
}

// 只有已断开 (Disconnected) 的连接可以进行 connect:
impl Connection<Disconnected> {
    fn new(address: &str) -> Self {
        Connection {
            address: address.to_string(),
            _state: std::marker::PhantomData,
        }
    }

    fn connect(self) -> Connection<Connected> {
        println!("Connecting to {}...", self.address);
        Connection {
            address: self.address,
            _state: std::marker::PhantomData,
        }
    }
}

// 只有已连接 (Connected) 的连接可以进行 authenticate:
impl Connection<Connected> {
    fn authenticate(self, _token: &str) -> Connection<Authenticated> {
        println!("Authenticating...");
        Connection {
            address: self.address,
            _state: std::marker::PhantomData,
        }
    }
}

// 只有已授权 (Authenticated) 的连接可以发起 request:
impl Connection<Authenticated> {
    fn request(&self, path: &str) -> String {
        format!("GET {} from {}", path, self.address)
    }
}

fn main() {
    let conn = Connection::new("api.example.com");
    // conn.request("/data"); // ❌ 编译错误:Connection<Disconnected> 没有 `request` 方法

    let conn = conn.connect();
    // conn.request("/data"); // ❌ 编译错误:Connection<Connected> 没有 `request` 方法

    let conn = conn.authenticate("secret-token");
    let response = conn.request("/data"); // ✅ 仅在身份验证后起作用
    println!("{response}");
}

关键洞察:每次状态转换都会 消耗 self 并返回一个新类型。 你在状态转换后无法使用旧状态 —— 编译器强制执行了这一点。 零运行时开销 —— PhantomData 是零大小的,状态信息在编译时被擦除。

对比 C++/C#:在 C++ 或 C# 中,你会通过运行时检查(例如 if (!authenticated) throw ...)来强制执行此操作。Rust 的类型状态模式将这些检查移至编译时 —— 非法状态在类型系统中确实无法表示。

结合类型状态的建造者 (Builder) 模式

一个实际应用 —— 一个强制要求提供必要字段的建造者:

use std::marker::PhantomData;

// 必要字段的标记类型
struct NeedsName;
struct NeedsPort;
struct Ready;

struct ServerConfig<State> {
    name: Option<String>,
    port: Option<u16>,
    max_connections: usize, // 可选,有默认值
    _state: PhantomData<State>,
}

impl ServerConfig<NeedsName> {
    fn new() -> Self {
        ServerConfig {
            name: None,
            port: None,
            max_connections: 100,
            _state: PhantomData,
        }
    }

    fn name(self, name: &str) -> ServerConfig<NeedsPort> {
        ServerConfig {
            name: Some(name.to_string()),
            port: self.port,
            max_connections: self.max_connections,
            _state: PhantomData,
        }
    }
}

impl ServerConfig<NeedsPort> {
    fn port(self, port: u16) -> ServerConfig<Ready> {
        ServerConfig {
            name: self.name,
            port: Some(port),
            max_connections: self.max_connections,
            _state: PhantomData,
        }
    }
}

impl ServerConfig<Ready> {
    fn max_connections(mut self, n: usize) -> Self {
        self.max_connections = n;
        self
    }

    fn build(self) -> Server {
        Server {
            name: self.name.unwrap(),
            port: self.port.unwrap(),
            max_connections: self.max_connections,
        }
    }
}

struct Server {
    name: String,
    port: u16,
    max_connections: usize,
}

fn main() {
    // 必须提供名称,然后是端口,最后才能进行 build:
    let server = ServerConfig::new()
        .name("my-server")
        .port(8080)
        .max_connections(500)
        .build();

    // ServerConfig::new().port(8080); // ❌ 编译错误:NeedsName 没有 `port` 方法
    // ServerConfig::new().name("x").build(); // ❌ 编译错误:NeedsPort 没有 `build` 方法
}

案例研究:类型安全的连接池

现实世界的系统需要连接池,其中连接在定义良好的状态之间移动。以下是类型状态模式如何在生产级连接池中强制执行正确性的:

stateDiagram-v2
    [*] --> Idle: pool.acquire()
    Idle --> Active: conn.begin_transaction()
    Active --> Active: conn.execute(query)
    Active --> Idle: conn.commit() / conn.rollback()
    Idle --> [*]: pool.release(conn)

    Active --> [*]: ❌ 无法在事务中释放
use std::marker::PhantomData;

// 状态
struct Idle;
struct InTransaction;

struct PooledConnection<State> {
    id: u32,
    _state: PhantomData<State>,
}

struct Pool {
    next_id: u32,
}

impl Pool {
    fn new() -> Self { Pool { next_id: 0 } }

    fn acquire(&mut self) -> PooledConnection<Idle> {
        self.next_id += 1;
        println!("[pool] Acquired connection #{}", self.next_id);
        PooledConnection { id: self.next_id, _state: PhantomData }
    }

    // 只有空闲连接可以被释放 —— 防止事务中途泄露
    fn release(&self, conn: PooledConnection<Idle>) {
        println!("[pool] Released connection #{}", conn.id);
    }
}

impl PooledConnection<Idle> {
    fn begin_transaction(self) -> PooledConnection<InTransaction> {
        println!("[conn #{}] BEGIN", self.id);
        PooledConnection { id: self.id, _state: PhantomData }
    }
}

impl PooledConnection<InTransaction> {
    fn execute(&self, query: &str) {
        println!("[conn #{}] EXEC: {}", self.id, query);
    }

    fn commit(self) -> PooledConnection<Idle> {
        println!("[conn #{}] COMMIT", self.id);
        PooledConnection { id: self.id, _state: PhantomData }
    }

    fn rollback(self) -> PooledConnection<Idle> {
        println!("[conn #{}] ROLLBACK", self.id);
        PooledConnection { id: self.id, _state: PhantomData }
    }
}

fn main() {
    let mut pool = Pool::new();

    let conn = pool.acquire();
    let conn = conn.begin_transaction();
    conn.execute("INSERT INTO users VALUES ('Alice')");
    conn.execute("INSERT INTO orders VALUES (1, 42)");
    let conn = conn.commit(); // 回到空闲 (Idle)
    pool.release(conn);       // ✅ 仅对空闲连接有效

    // pool.release(conn_active); // ❌ 编译错误:无法在 InTransaction 状态下释放
}

为什么这在生产环境中重要:在事务中途泄露的连接会无限期地持有数据库锁。类型状态模式使这种情况变得不可能实现 —— 在事务被提交或回滚之前,你确实无法将连接归还给连接池。


配置 Trait (Config Trait) 模式 —— 驯服泛型参数爆炸

问题描述

随着一个结构体承担更多的职责,且由于每一个职责都对应一个受 Trait 约束的泛型,其类型签名会变得极其臃肿:

#![allow(unused)]
fn main() {
trait SpiBus   { fn spi_transfer(&self, tx: &[u8], rx: &mut [u8]) -> Result<(), BusError>; }
trait ComPort  { fn com_send(&self, data: &[u8]) -> Result<usize, BusError>; }
trait I3cBus   { fn i3c_read(&self, addr: u8, buf: &mut [u8]) -> Result<(), BusError>; }
trait SmBus    { fn smbus_read_byte(&self, addr: u8, cmd: u8) -> Result<u8, BusError>; }
trait GpioBus  { fn gpio_set(&self, pin: u32, high: bool); }

// ❌ 每一个新增的总线 Trait 都会增加一个泛型参数
struct DiagController<S: SpiBus, C: ComPort, I: I3cBus, M: SmBus, G: GpioBus> {
    spi: S,
    com: C,
    i3c: I,
    smbus: M,
    gpio: G,
}
// impl 块、函数签名和调用方都必须重复这一长串列表。
// 增加第 6 个总线意味着要修改 DiagController<S, C, I, M, G> 的每一处引用。
}

这通常被称为 “泛型参数爆炸”。由于每一个环节都必须重复完整的参数列表,其影响在 impl 块、函数参数和下游消费者中会不断叠加。

解决方案:配置 Trait

将所有关联类型打包到一个单一的 Trait 中。这样,无论包含多少个组件类型,结构体都只需要 一个 泛型参数:

#![allow(unused)]
fn main() {
#[derive(Debug)]
enum BusError {
    Timeout,
    NakReceived,
    HardwareFault(String),
}

// --- 总线 Trait (保持不变) ---
trait SpiBus {
    fn spi_transfer(&self, tx: &[u8], rx: &mut [u8]) -> Result<(), BusError>;
    fn spi_write(&self, data: &[u8]) -> Result<(), BusError>;
}

trait ComPort {
    fn com_send(&self, data: &[u8]) -> Result<usize, BusError>;
    fn com_recv(&self, buf: &mut [u8], timeout_ms: u32) -> Result<usize, BusError>;
}

trait I3cBus {
    fn i3c_read(&self, addr: u8, buf: &mut [u8]) -> Result<(), BusError>;
    fn i3c_write(&self, addr: u8, data: &[u8]) -> Result<(), BusError>;
}

// --- 配置 Trait:每个组件对应一个关联类型 ---
trait BoardConfig {
    type Spi: SpiBus;
    type Com: ComPort;
    type I3c: I3cBus;
}

// --- DiagController 现在只有唯一一个泛型参数 ---
struct DiagController<Cfg: BoardConfig> {
    spi: Cfg::Spi,
    com: Cfg::Com,
    i3c: Cfg::I3c,
}
}

DiagController<Cfg> 永远不会再增加多余的泛型参数。 增加第 4 个总线只需向 BoardConfig 添加一个关联类型,并向 DiagController 添加一个字段 —— 下游的所有签名均无需修改。

实现控制器

#![allow(unused)]
fn main() {
impl<Cfg: BoardConfig> DiagController<Cfg> {
    fn new(spi: Cfg::Spi, com: Cfg::Com, i3c: Cfg::I3c) -> Self {
        DiagController { spi, com, i3c }
    }

    fn read_flash_id(&self) -> Result<u32, BusError> {
        let cmd = [0x9F]; // JEDEC Read ID
        let mut id = [0u8; 4];
        self.spi.spi_transfer(&cmd, &mut id)?;
        Ok(u32::from_be_bytes(id))
    }

    fn send_bmc_command(&self, cmd: &[u8]) -> Result<Vec<u8>, BusError> {
        self.com.com_send(cmd)?;
        let mut resp = vec![0u8; 256];
        let n = self.com.com_recv(&mut resp, 1000)?;
        resp.truncate(n);
        Ok(resp)
    }

    fn read_sensor_temp(&self, sensor_addr: u8) -> Result<i16, BusError> {
        let mut buf = [0u8; 2];
        self.i3c.i3c_read(sensor_addr, &mut buf)?;
        Ok(i16::from_be_bytes(buf))
    }

    fn run_full_diag(&self) -> Result<DiagReport, BusError> {
        let flash_id = self.read_flash_id()?;
        let bmc_resp = self.send_bmc_command(b"VERSION\n")?;
        let cpu_temp = self.read_sensor_temp(0x48)?;
        let gpu_temp = self.read_sensor_temp(0x49)?;

        Ok(DiagReport {
            flash_id,
            bmc_version: String::from_utf8_lossy(&bmc_resp).to_string(),
            cpu_temp_c: cpu_temp,
            gpu_temp_c: gpu_temp,
        })
    }
}

#[derive(Debug)]
struct DiagReport {
    flash_id: u32,
    bmc_version: String,
    cpu_temp_c: i16,
    gpu_temp_c: i16,
}
}

生产环境接入

通过一个 impl BoardConfig 来选择具体的硬件驱动:

struct PlatformSpi  { dev: String, speed_hz: u32 }
struct UartCom      { dev: String, baud: u32 }
struct LinuxI3c     { dev: String }

impl SpiBus for PlatformSpi {
    fn spi_transfer(&self, tx: &[u8], rx: &mut [u8]) -> Result<(), BusError> {
        // 在生产环境中使用 ioctl(SPI_IOC_MESSAGE)
        rx[0..4].copy_from_slice(&[0xEF, 0x40, 0x18, 0x00]);
        Ok(())
    }
    fn spi_write(&self, _data: &[u8]) -> Result<(), BusError> { Ok(()) }
}

impl ComPort for UartCom {
    fn com_send(&self, _data: &[u8]) -> Result<usize, BusError> { Ok(0) }
    fn com_recv(&self, buf: &mut [u8], _timeout: u32) -> Result<usize, BusError> {
        let resp = b"BMC v2.4.1\n";
        buf[..resp.len()].copy_from_slice(resp);
        Ok(resp.len())
    }
}

impl I3cBus for LinuxI3c {
    fn i3c_read(&self, _addr: u8, buf: &mut [u8]) -> Result<(), BusError> {
        buf[0] = 0x00; buf[1] = 0x2D; // 45°C
        Ok(())
    }
    fn i3c_write(&self, _addr: u8, _data: &[u8]) -> Result<(), BusError> { Ok(()) }
}

// ✅ 一个结构体,一个实现 —— 所有的具体类型都在这里确定
struct ProductionBoard;
impl BoardConfig for ProductionBoard {
    type Spi = PlatformSpi;
    type Com = UartCom;
    type I3c = LinuxI3c;
}

fn main() {
    let ctrl = DiagController::<ProductionBoard>::new(
        PlatformSpi { dev: "/dev/spidev0.0".into(), speed_hz: 10_000_000 },
        UartCom     { dev: "/dev/ttyS0".into(),     baud: 115200 },
        LinuxI3c    { dev: "/dev/i3c-0".into() },
    );
    let report = ctrl.run_full_diag().unwrap();
    println!("{report:#?}");
}

使用 Mock 进行测试接入

通过定义不同的 BoardConfig 来直接切换整个硬件层:

#![allow(unused)]
fn main() {
struct MockSpi  { flash_id: [u8; 4] }
struct MockCom  { response: Vec<u8> }
struct MockI3c  { temps: std::collections::HashMap<u8, i16> }

impl SpiBus for MockSpi {
    fn spi_transfer(&self, _tx: &[u8], rx: &mut [u8]) -> Result<(), BusError> {
        rx[..4].copy_from_slice(&self.flash_id);
        Ok(())
    }
    fn spi_write(&self, _data: &[u8]) -> Result<(), BusError> { Ok(()) }
}

impl ComPort for MockCom {
    fn com_send(&self, _data: &[u8]) -> Result<usize, BusError> { Ok(0) }
    fn com_recv(&self, buf: &mut [u8], _timeout: u32) -> Result<usize, BusError> {
        let n = self.response.len().min(buf.len());
        buf[..n].copy_from_slice(&self.response[..n]);
        Ok(n)
    }
}

impl I3cBus for MockI3c {
    fn i3c_read(&self, addr: u8, buf: &mut [u8]) -> Result<(), BusError> {
        let temp = self.temps.get(&addr).copied().unwrap_or(0);
        buf[..2].copy_from_slice(&temp.to_be_bytes());
        Ok(())
    }
    fn i3c_write(&self, _addr: u8, _data: &[u8]) -> Result<(), BusError> { Ok(()) }
}

struct TestBoard;
impl BoardConfig for TestBoard {
    type Spi = MockSpi;
    type Com = MockCom;
    type I3c = MockI3c;
}

#[cfg(test)]
mod tests {
    use super::*;

    fn make_test_controller() -> DiagController<TestBoard> {
        let mut temps = std::collections::HashMap::new();
        temps.insert(0x48, 45i16);
        temps.insert(0x49, 72i16);

        DiagController::<TestBoard>::new(
            MockSpi  { flash_id: [0xEF, 0x40, 0x18, 0x00] },
            MockCom  { response: b"BMC v2.4.1\n".to_vec() },
            MockI3c  { temps },
        )
    }

    #[test]
    fn test_flash_id() {
        let ctrl = make_test_controller();
        assert_eq!(ctrl.read_flash_id().unwrap(), 0xEF401800);
    }

    #[test]
    fn test_sensor_temps() {
        let ctrl = make_test_controller();
        assert_eq!(ctrl.read_sensor_temp(0x48).unwrap(), 45);
        assert_eq!(ctrl.read_sensor_temp(0x49).unwrap(), 72);
    }

    #[test]
    fn test_full_diag() {
        let ctrl = make_test_controller();
        let report = ctrl.run_full_diag().unwrap();
        assert_eq!(report.flash_id, 0xEF401800);
        assert_eq!(report.cpu_temp_c, 45);
        assert_eq!(report.gpu_temp_c, 72);
        assert!(report.bmc_version.contains("2.4.1"));
    }
}
}

未来添加新总线

当你需要第 4 个总线时,只有两个地方会变动 —— BoardConfig 和 DiagController。 下游签名无需变动。 泛型参数的数量依然保持为一:

#![allow(unused)]
fn main() {
trait SmBus {
    fn smbus_read_byte(&self, addr: u8, cmd: u8) -> Result<u8, BusError>;
}

// 1. 添加一个关联类型:
trait BoardConfig {
    type Spi: SpiBus;
    type Com: ComPort;
    type I3c: I3cBus;
    type Smb: SmBus;     // ← 新增
}

// 2. 添加一个字段:
struct DiagController<Cfg: BoardConfig> {
    spi: Cfg::Spi,
    com: Cfg::Com,
    i3c: Cfg::I3c,
    smb: Cfg::Smb,       // ← 新增
}

// 3. 在每一个配置实现中提供具体类型:
impl BoardConfig for ProductionBoard {
    type Spi = PlatformSpi;
    type Com = UartCom;
    type I3c = LinuxI3c;
    type Smb = LinuxSmbus; // ← 新增
}
}

何时使用此模式

场景是否使用配置 Trait?替代方案
一个结构体上有 3 个以上受 Trait 约束的泛型✅ 是—
需要切换整个硬件/平台层✅ 是—
仅有 1-2 个泛型❌ 过度设计直接使用泛型
需要运行时多态❌dyn Trait 对象
开放式插件系统❌类型映射 / Any
组件 Trait 构成了一个自然的逻辑组 (board, platform)✅ 是—

关键特性

  • 永远只有一个泛型参数 —— DiagController<Cfg> 永远不会获得更多 <A, B, C, ...> 泛型。
  • 完全静态分发 —— 没有虚函数表 (vtables),没有 dyn,没有 Trait 对象的堆分配。
  • 整洁的测试切换 —— 定义带有 mock 实现的 TestBoard,且零条件编译。
  • 编译时安全 —— 遗漏关联类型会导致编译错误,而不是运行时崩溃。
  • 经过实战检验 —— 这是 Substrate/Polkadot 的 FRAME 系统用来通过一个单一 Config Trait 管理 20 多个关联类型的模式。

关键要点 —— Newtype 与类型状态

  • Newtype 以零运行时成本提供编译时类型 safety。
  • 类型状态模式使非法状态转移成为编译错误,而非运行时 Bug。
  • 配置 Trait 可以在大型系统中驯服泛型参数爆炸。

另请参阅: 第 4 章 —— PhantomData 了解驱动类型状态模式的零大小标记。第 2 章 —— 深入 Trait 了解配置 Trait 模式中使用的关联类型。


案例研究:双轴类型状态 (Dual-Axis Typestate) —— 厂商 × 协议状态

上述模式每次处理一个轴线:类型状态强制执行 协议顺序,而 Trait 抽象处理 多个厂商。现实系统往往需要 同时处理两者:一个包装器 Handle<Vendor, State>,其可用方法取决于 插入了哪个厂商 且 该句柄处于哪个状态。

本节展示了 双轴条件实现 (Dual-Axis Conditional impl) 模式 —— 其中 impl 块同时受厂商 Trait 约束和状态标记 Trait 约束。

二维问题描述

考虑一个调试探针接口 (JTAG/SWD)。多个厂商制造探针,每个探针在寄存器可访问之前都必须解锁。某些厂商额外支持直接内存读取 —— 但仅在执行了 扩展解锁 之后:

graph LR
    subgraph "State Axis (What stage?)"
        L["🔒 Locked"] -- "unlock()" --> U["🔓 Unlocked"]
        U -- "extended_unlock()" --> E["🔓🧠 ExtendedUnlocked"]
    end

    subgraph "Vendor Axis (Who provides it?)"
        V1["ProbeVendor (Basic)"]
        V2["MemoryVendor (Adv)"]
    end

    U -. "read_reg()" .-> U
    E -. "read_memory()" .-> E
    
    style V1 fill:#f9f,stroke:#333
    style V2 fill:#bbf,stroke:#333

其挑战在于:完全在编译时 表达此矩阵,并使用静态分发,使得在基础探针上调用 extended_unlock() 或在未执行扩展解锁的句柄上调用 read_memory() 都会产生编译错误。

解决方案:带有标记 Trait 的 Jtag<V, S>

第一步 —— 状态令牌与能力标记:

#![allow(unused)]
fn main() {
use std::marker::PhantomData;

// 零大小的状态令牌 —— 无运行时开销
struct Locked;
struct Unlocked;
struct ExtendedUnlocked;

// 标记 Trait 表达每个状态具备哪些能力
trait HasRegAccess {}
impl HasRegAccess for Unlocked {}
impl HasRegAccess for ExtendedUnlocked {}

trait HasMemAccess {}
impl HasMemAccess for ExtendedUnlocked {}
}

为什么使用标记 Trait 而非具体状态? 编写 impl<V, S: HasRegAccess> Jtag<V, S> 意味着 read_reg() 可以在 任何 具备寄存器访问能力的状态下工作 —— 现在是 Unlocked 和 ExtendedUnlocked,但如果你明天增加了 DebugHalted 状态,只需添加一行:impl HasRegAccess for DebugHalted {} 即可。

第二步 —— 厂商 Trait (原始操作):

#![allow(unused)]
fn main() {
// 每个探针厂商都实现这些
trait JtagVendor {
    fn raw_unlock(&mut self);
    fn raw_read_reg(&self, addr: u32) -> u32;
    fn raw_write_reg(&mut self, addr: u32, val: u32);
}

// 具备内存访问能力的厂商还需实现此 Super-trait
trait JtagMemoryVendor: JtagVendor {
    fn raw_extended_unlock(&mut self);
    fn raw_read_memory(&self, addr: u64, buf: &mut [u8]);
    fn raw_write_memory(&mut self, addr: u64, data: &[u8]);
}
}

第三步 —— 带有条件 impl 块的包装器:

#![allow(unused)]
fn main() {
struct Jtag<V, S = Locked> {
    vendor: V,
    _state: PhantomData<S>,
}

// 构建 —— 初始状态始终为 Locked
impl<V: JtagVendor> Jtag<V, Locked> {
    fn new(vendor: V) -> Self {
        Jtag { vendor, _state: PhantomData }
    }

    fn unlock(mut self) -> Jtag<V, Unlocked> {
        self.vendor.raw_unlock();
        Jtag { vendor: self.vendor, _state: PhantomData }
    }
}

// 寄存器 I/O —— 任何厂商,在任何具备 HasRegAccess 的状态下
impl<V: JtagVendor, S: HasRegAccess> Jtag<V, S> {
    fn read_reg(&self, addr: u32) -> u32 {
        self.vendor.raw_read_reg(addr)
    }
    fn write_reg(&mut self, addr: u32, val: u32) {
        self.vendor.raw_write_reg(addr, val);
    }
}

// 扩展解锁 —— 仅限具备内存能力的厂商,且仅能从 Unlocked 状态发起
impl<V: JtagMemoryVendor> Jtag<V, Unlocked> {
    fn extended_unlock(mut self) -> Jtag<V, ExtendedUnlocked> {
        self.vendor.raw_extended_unlock();
        Jtag { vendor: self.vendor, _state: PhantomData }
    }
}

// 内存 I/O —— 仅限具备内存能力的厂商,且仅在 ExtendedUnlocked 状态下
impl<V: JtagMemoryVendor, S: HasMemAccess> Jtag<V, S> {
    fn read_memory(&self, addr: u64, buf: &mut [u8]) {
        self.vendor.raw_read_memory(addr, buf);
    }
    fn write_memory(&mut self, addr: u64, data: &[u8]) {
        self.vendor.raw_write_memory(addr, data);
    }
}
}

编译器会阻止什么

尝试操作错误信息原因
Jtag<_, Locked>::read_reg()no method read_regLocked 未实现 HasRegAccess
Jtag<BasicProbe, _>::extended_unlock()no method extended_unlockBasicProbe 未实现 JtagMemoryVendor
Jtag<_, Unlocked>::read_memory()no method read_memoryUnlocked 未实现 HasMemAccess
调用 unlock() 两次value used after moveunlock() 消耗了 self

所有这些错误都会被 编译器捕获。没有运行时恐慌,没有 Option,也没有运行时状态枚举。

编写泛型函数

函数只需绑定它们关心的轴:

#![allow(unused)]
fn main() {
/// 适用于任何厂商、任何授权寄存器访问的状态。
fn read_idcode<V: JtagVendor, S: HasRegAccess>(jtag: &Jtag<V, S>) -> u32 {
    jtag.read_reg(0x00)
}

/// 仅能针对具备内存能力的厂商在 ExtendedUnlocked 状态下编译。
fn dump_firmware<V: JtagMemoryVendor, S: HasMemAccess>(jtag: &Jtag<V, S>) {
    let mut buf = [0u8; 256];
    jtag.read_memory(0x0800_0000, &mut buf);
}
}

跨领域同类模式:存储后端 (Storage Backends)

双轴技术并非硬件专用。以下是存储层的相同结构,其中某些后端支持事务:

#![allow(unused)]
fn main() {
// 状态
struct Closed;
struct Open;
struct InTransaction;

trait HasReadWrite {}
impl HasReadWrite for Open {}
impl HasReadWrite for InTransaction {}

// 厂商 Trait
trait StorageBackend {
    fn raw_open(&mut self);
    fn raw_read(&self, key: &[u8]) -> Option<Vec<u8>>;
    fn raw_write(&mut self, key: &[u8], value: &[u8]);
}

trait TransactionalBackend: StorageBackend {
    fn raw_begin(&mut self);
    fn raw_commit(&mut self);
    fn raw_rollback(&mut self);
}

// 包装器
struct Store<B, S = Closed> { backend: B, _s: PhantomData<S> }

impl<B: StorageBackend> Store<B, Closed> {
    fn open(mut self) -> Store<B, Open> { self.backend.raw_open(); todo!() }
}
impl<B: StorageBackend, S: HasReadWrite> Store<B, S> {
    fn read(&self, key: &[u8]) -> Option<Vec<u8>>  { self.backend.raw_read(key) }
    fn write(&mut self, key: &[u8], val: &[u8])    { self.backend.raw_write(key, val) }
}
impl<B: TransactionalBackend> Store<B, Open> {
    fn begin(mut self) -> Store<B, InTransaction>   { todo!() }
}
}

平面文件 (Flat-file) 后端仅实现 StorageBackend —— begin() 将无法通过编译。数据库后端则增加了 TransactionalBackend —— 于是 Open → InTransaction → Open 的完整周期变得可用。

何时采用此模式

信号为什么双轴模式适用
存在两个独立的轴:“谁提供它”和“它处于什么状态”impl 块矩阵直接对两者进行编码
某些提供者具备比其他提供者更多的能力Super-trait + 条件 impl
误用状态或能力会导致安全/正确性 Bug编译时预防 > 运行时检查
你追求静态分发 (无虚函数表 vtables)PhantomData + 泛型 = 零开销

关键要点:双轴模式是类型状态 (Typestate) 与基于 Trait 抽象的交汇。每个 impl 块映射到 (厂商 × 状态) 矩阵的一个单元格。编译器强制执行整个矩阵 —— 无运行时状态检查,无非法状态恐慌,且零成本。


练习:类型安全状态机 ★★ (~30 分钟)

使用类型状态模式构建一个红绿灯状态机。灯光必须按照 红 -> 绿 -> 黄 -> 红 的顺序转换,且不允许任何其他顺序。

🔑 参考答案
use std::marker::PhantomData;

struct Red;
struct Green;
struct Yellow;

struct TrafficLight<State> {
    _state: PhantomData<State>,
}

impl TrafficLight<Red> {
    fn new() -> Self {
        println!("🔴 红灯 — 停止");
        TrafficLight { _state: PhantomData }
    }

    fn go(self) -> TrafficLight<Green> {
        println!("🟢 绿灯 — 行进");
        TrafficLight { _state: PhantomData }
    }
}

impl TrafficLight<Green> {
    fn caution(self) -> TrafficLight<Yellow> {
        println!("🟡 黄灯 — 注意");
        TrafficLight { _state: PhantomData }
    }
}

impl TrafficLight<Yellow> {
    fn stop(self) -> TrafficLight<Red> {
        println!("🔴 红灯 — 停止");
        TrafficLight { _state: PhantomData }
    }
}

fn main() {
    let light = TrafficLight::new(); // 红灯
    let light = light.go();          // 绿灯
    let light = light.caution();     // 黄灯
    let _light = light.stop();       // 红灯

    // light.caution(); // ❌ 编译错误:Red 状态没有 `caution` 方法
    // TrafficLight::new().stop(); // ❌ 编译错误:Red 状态没有 `stop` 方法
}

关键要点:非法转换是编译错误,而非运行时崩溃。


English Original

第 4 章:PhantomData —— 不携带数据的类型 🔴

你将学到:

  • 为什么 PhantomData<T> 存在以及它解决的三个问题
  • 生命周期烙印 (Lifetime Branding):用于编译时作用域强制执行
  • 单位量纲 (Unit-of-measure) 模式:用于量纲安全的算术运算
  • 型变 (Variance)(协变、逆变、不变)以及 PhantomData 如何控制它

PhantomData 解决了什么

PhantomData<T> 是一种零大小类型,它告诉编译器:“这个结构体在逻辑上与 T 相关联,尽管它并不包含 T。”它会影响型变、Drop 检查以及 Auto-trait 推导 —— 且不占用任何内存。

#![allow(unused)]
fn main() {
use std::marker::PhantomData;

// 不使用 PhantomData:
struct Slice<'a, T> {
    ptr: *const T,
    len: usize,
    // 问题:编译器不知道这个结构体借用了 'a,
    // 也不知道为了 Drop 检查的目的它与 T 相关联。
}

// 使用 PhantomData:
struct Slice<'a, T> {
    ptr: *const T,
    len: usize,
    _marker: PhantomData<&'a T>,
    // 现在编译器知道了:
    // 1. 这个结构体借用了生命周期为 'a 的数据
    // 2. 它对 'a 是协变的(生命周期可以缩小)
    // 3. Drop 检查会考虑 T
}
}

PhantomData 的三项职责:

职责示例作用
生命周期绑定PhantomData<&'a T>结构体被视为借用了 'a
所有权模拟PhantomData<T>Drop 检查假设结构体拥有一个 T
型变控制PhantomData<fn(T)>使结构体对 T 是逆变的

生命周期烙印 (Lifetime Branding)

使用 PhantomData 来防止混用来自不同“会话”或“上下文”的值:

use std::marker::PhantomData;

/// 仅在特定 Arena 的生命周期内有效的句柄
struct ArenaHandle<'arena> {
    index: usize,
    _brand: PhantomData<&'arena ()>,
}

struct Arena {
    data: Vec<String>,
}

impl Arena {
    fn new() -> Self {
        Arena { data: Vec::new() }
    }

    /// 分配一个字符串并返回一个带有烙印的句柄
    fn alloc<'a>(&'a mut self, value: String) -> ArenaHandle<'a> {
        let index = self.data.len();
        self.data.push(value);
        ArenaHandle { index, _brand: PhantomData }
    }

    /// 通过句柄查找 —— 仅接受来自 *此* Arena 的句柄
    fn get<'a>(&'a self, handle: ArenaHandle<'a>) -> &'a str {
        &self.data[handle.index]
    }
}

fn main() {
    let mut arena1 = Arena::new();
    let handle1 = arena1.alloc("hello".to_string());

    // 无法将 handle1 用于不同的 Arena —— 生命周期将不匹配
    // let mut arena2 = Arena::new();
    // arena2.get(handle1); // ❌ 生命周期不匹配

    println!("{}", arena1.get(handle1)); // ✅
}

单位量纲 (Unit-of-Measure) 模式

在编译时防止混用不兼容的单位,且运行时开销为零:

use std::marker::PhantomData;
use std::ops::{Add, Mul};

// 单位标记类型 (零大小)
struct Meters;
struct Seconds;
struct MetersPerSecond;

#[derive(Debug, Clone, Copy)]
struct Quantity<Unit> {
    value: f64,
    _unit: PhantomData<Unit>,
}

impl<U> Quantity<U> {
    fn new(value: f64) -> Self {
        Quantity { value, _unit: PhantomData }
    }
}

// 只能相加相同的单位:
impl<U> Add for Quantity<U> {
    type Output = Quantity<U>;
    fn add(self, rhs: Self) -> Self::Output {
        Quantity::new(self.value + rhs.value)
    }
}

// 米 / 秒 = 米每秒 (自定义 Trait)
impl std::ops::Div<Quantity<Seconds>> for Quantity<Meters> {
    type Output = Quantity<MetersPerSecond>;
    fn div(self, rhs: Quantity<Seconds>) -> Quantity<MetersPerSecond> {
        Quantity::new(self.value / rhs.value)
    }
}

fn main() {
    let dist = Quantity::<Meters>::new(100.0);
    let time = Quantity::<Seconds>::new(9.58);
    let speed = dist / time; // Quantity<MetersPerSecond>
    println!("速度: {:.2} m/s", speed.value); // 10.44 m/s

    // let nonsense = dist + time; // ❌ 编译错误:无法将“米”与“秒”相加
}

这是纯粹的类型系统魔法 —— PhantomData<Meters> 是零大小的,因此 Quantity<Meters> 的内存布局与 f64 完全相同。在运行时没有包装开销,但在编译时具备完全的单位安全性。

PhantomData 与 Drop 检查

当编译器检查结构体的析构函数是否可能访问已过时的数据时,它会使用 PhantomData 来做出决定:

#![allow(unused)]
fn main() {
use std::marker::PhantomData;

// PhantomData<T> — 编译器假设我们 *可能* 会 drop 一个 T
// 这意味着 T 的生命周期必须比我们的结构体更长
struct OwningSemantic<T> {
    ptr: *const T,
    _marker: PhantomData<T>,  // “我在逻辑上拥有一个 T”
}

// PhantomData<*const T> — 编译器假设我们 *不* 拥有 T
// 要求更宽松 —— T 的生命周期不需要比我们更长
struct NonOwningSemantic<T> {
    ptr: *const T,
    _marker: PhantomData<*const T>,  // “我只是指向 T”
}
}

实践规则:在包装原始指针时,请深思熟虑地选择 PhantomData:

  • 编写一个拥有其数据的容器? → PhantomData<T>
  • 编写一个视图/引用类型? → PhantomData<&'a T> 或 PhantomData<*const T>

型变 (Variance) —— 为什么 PhantomData 的类型参数很重要

型变 决定了一个泛型类型是否可以用其子类型或超类型进行代换(在 Rust 中,“子类型”意味着“具有更长的生命周期”)。搞错型变会导致编译器要么拒绝本应安全的代码 (rejected-good-code),要么接受实际上不安全的代码 (unsound-accepted-code)。

graph LR
    subgraph Covariant (协变)
        direction TB
        A1["&'long T"] -->|"可以变为"| A2["&'short T"]
    end

    subgraph Contravariant (逆变)
        direction TB
        B1["fn(&'short T)"] -->|"可以变为"| B2["fn(&'long T)"]
    end

    subgraph Invariant (不变)
        direction TB
        C1["&'a mut T"] ---|"不允许代换"| C2["&'b mut T"]
    end

    style A1 fill:#d4efdf,stroke:#27ae60,color:#000
    style A2 fill:#d4efdf,stroke:#27ae60,color:#000
    style B1 fill:#e8daef,stroke:#8e44ad,color:#000
    style B2 fill:#e8daef,stroke:#8e44ad,color:#000
    style C1 fill:#fadbd8,stroke:#e74c3c,color:#000
    style C2 fill:#fadbd8,stroke:#e74c3c,color:#000

三种型变

型变含义“我能否用……代换?”Rust 示例
协变 (Covariant)子类型关系流向一致在需要 'short 的地方使用 'long ✅&'a T, Vec<T>, Box<T>
逆变 (Contravariant)子类型关系流向相反在需要 'long 的地方使用 'short ✅fn(T) (在参数位置)
不变 (Invariant)不允许代换两个方向都不允许 ✅&mut T, Cell<T>, UnsafeCell<T>

为什么 &'a T 对 'a 是协变的

fn print_str(s: &str) {
    println!("{s}");
}

fn main() {
    let owned = String::from("hello");
    // owned 的生命周期贯穿整个函数 ('long)
    // print_str 预期 &'_ str ('short — 仅在调用期间有效)
    print_str(&owned); // ✅ 协变:'long → 'short 是安全的
    // 长生命周期的引用总是可以在需要短生命周期引用的地方使用。
}

为什么 &mut T 对 T 是不变的

#![allow(unused)]
fn main() {
// 如果 &mut T 对 T 是协变的,那么这段代码就能编译:
fn evil(s: &mut &'static str) {
    // 我们本可以将一个短生命周期的 &str 写入预留给 &'static str 的位置!
    let local = String::from("temporary");
    // *s = &local; // ← 这将创建一个悬垂的 &'static str
}

// “不变性”防止了这种情况:在变动 (mutating) 时,&'static str ≠ &'a str。
// 编译器会完全拒绝这种代换。
}

PhantomData 如何控制型变

PhantomData<X> 会赋予你的结构体与 X 相同的型变:

#![allow(unused)]
fn main() {
use std::marker::PhantomData;

// 对 'a 是协变的 — Ref<'long> 可以作为 Ref<'short> 使用
struct Ref<'a, T> {
    ptr: *const T,
    _marker: PhantomData<&'a T>,  // 对 'a 协变, 对 T 协变
}

// 对 T 是不变的 — 防止对 T 执行不安全的生命周期缩短
struct MutRef<'a, T> {
    ptr: *mut T,
    _marker: PhantomData<&'a mut T>,  // 对 'a 协变, 对 T 不变
}

// 对 T 是逆变的 — 在回调容器中很有用
struct CallbackSlot<T> {
    _marker: PhantomData<fn(T)>,  // 对 T 逆变
}
}

PhantomData 型变速查表:

PhantomData 类型对 T 的型变对 'a 的型变何时使用
PhantomData<T>协变—你在逻辑上拥有一个 T
PhantomData<&'a T>协变协变你借用一个生命周期为 'a 的 T
PhantomData<&'a mut T>不变协变你可变地借用 T
PhantomData<*const T>协变—指向 T 的非拥有指针
PhantomData<*mut T>不变—非拥有且可变的指针
PhantomData<fn(T)>逆变—T 出现在参数位置
PhantomData<fn() -> T>协变—T 出现在返回位置
PhantomData<fn(T) -> T>不变—T 在两个位置上互相抵消

案例分析:为什么这在实践中很重要

use std::marker::PhantomData;

// 一个用会话生命周期“标记”值的令牌。
// 必须对 'a 是协变的 —— 否则调用者在将其传递给需要
// 更短生命周期的函数时,将无法缩短生命周期。
struct SessionToken<'a> {
    id: u64,
    _brand: PhantomData<&'a ()>,  // ✅ 协变 — 调用者可以缩短 'a
    // _brand: PhantomData<fn(&'a ())>,  // ❌ 逆变 — 破坏易用性
}

fn use_token(token: &SessionToken<'_>) {
    println!("使用令牌 {}", token.id);
}

fn main() {
    let token = SessionToken { id: 42, _brand: PhantomData };
    use_token(&token); // ✅ 之所以可行,是因为 SessionToken 对 'a 是协变的
}

决策规则:默认先使用 PhantomData<&'a T>(协变)。只有当你的抽象层会分发对 T 的可变访问权限时,才切换到 PhantomData<&'a mut T>(不变)。几乎 永远不要 使用 PhantomData<fn(T)>(逆变)—— 它仅在回调存储等特定场景下才是正确的。

关键要点 —— PhantomData

  • PhantomData<T> 在不产生运行时开销的情况下携带类型/生命周期信息。
  • 使用它来实现生命周期烙印、型变控制以及单位量纲模式。
  • Drop 检查:PhantomData<T> 告诉编译器你的类型在逻辑上拥有一个 T。

另请参阅: 第 3 章 —— Newtype 与类型状态 了解使用 PhantomData 的类型状态模式。第 11 章 —— 不安全 Rust 了解 PhantomData 如何与原始指针交互。


练习:使用 PhantomData 实现单位量纲 ★★ (~30 分钟)

扩展单位量纲模式,以支持:

  • Meters (米)、Seconds (秒)、Kilograms (千克)
  • 相同单位的加法
  • 乘法:Meters * Meters = SquareMeters
  • 除法:Meters / Seconds = MetersPerSecond
🔑 参考答案
use std::marker::PhantomData;
use std::ops::{Add, Mul, Div};

#[derive(Clone, Copy)]
struct Meters;
#[derive(Clone, Copy)]
struct Seconds;
#[derive(Clone, Copy)]
struct Kilograms;
#[derive(Clone, Copy)]
struct SquareMeters;
#[derive(Clone, Copy)]
struct MetersPerSecond;

#[derive(Debug, Clone, Copy)]
struct Qty<U> {
    value: f64,
    _unit: PhantomData<U>,
}

impl<U> Qty<U> {
    fn new(v: f64) -> Self { Qty { value: v, _unit: PhantomData } }
}

impl<U> Add for Qty<U> {
    type Output = Qty<U>;
    fn add(self, rhs: Self) -> Self::Output { Qty::new(self.value + rhs.value) }
}

impl Mul<Qty<Meters>> for Qty<Meters> {
    type Output = Qty<SquareMeters>;
    fn mul(self, rhs: Qty<Meters>) -> Qty<SquareMeters> {
        Qty::new(self.value * rhs.value)
    }
}

impl Div<Qty<Seconds>> for Qty<Meters> {
    type Output = Qty<MetersPerSecond>;
    fn div(self, rhs: Qty<Seconds>) -> Qty<MetersPerSecond> {
        Qty::new(self.value / rhs.value)
    }
}

fn main() {
    let width = Qty::<Meters>::new(5.0);
    let height = Qty::<Meters>::new(3.0);
    let area = width * height; // Qty<SquareMeters>
    println!("面积: {:.1} m²", area.value);

    let dist = Qty::<Meters>::new(100.0);
    let time = Qty::<Seconds>::new(9.58);
    let speed = dist / time;
    println!("速度: {:.2} m/s", speed.value);

    let sum = width + height; // 相同单位 ✅
    println!("总和: {:.1} m", sum.value);

    // let bad = width + time; // ❌ 编译错误:无法将“米”与“秒”相加
}

English Original

第 5 章:信道 (Channels) 与消息传递 🟢

你将学到:

  • std::sync::mpsc 基础,以及何时升级到 crossbeam-channel
  • 使用 select! 进行多源消息处理的信道选择
  • 有界与无界信道及背压 (Backpressure) 策略
  • 用于封装并发状态的 Actor 模式

std::sync::mpsc —— 标准信道

Rust 标准库提供了一个多生产者、单消费者 (MPSC) 信道:

use std::sync::mpsc;
use std::thread;
use std::time::Duration;

fn main() {
    // 创建一个信道:tx (发送者) 和 rx (接收者)
    let (tx, rx) = mpsc::channel();

    // 派生一个生产者线程
    let tx1 = tx.clone(); // 为多个生产者克隆发送者
    thread::spawn(move || {
        for i in 0..5 {
            tx1.send(format!("生产者-1: 消息 {i}")).unwrap();
            thread::sleep(Duration::from_millis(100));
        }
    });

    // 第二个生产者
    thread::spawn(move || {
        for i in 0..5 {
            tx.send(format!("生产者-2: 消息 {i}")).unwrap();
            thread::sleep(Duration::from_millis(150));
        }
    });

    // 消费者:接收所有消息
    for msg in rx {
        // 当所有发送者都被 drop 时,rx 迭代器结束
        println!("收到: {msg}");
    }
    println!("所有生产者已完成。");
}

注意:为了简明起见,我们在 .send() 上使用了 .unwrap()。如果接收者已被 drop,它会触发 panic。生产环境代码应优雅地处理 SendError。

关键属性:

  • 默认是 无界 (Unbounded) 的(如果消费者速度较慢,可能会填满内存)
  • mpsc::sync_channel(N) 创建一个带有背压的 有界 (Bounded) 信道
  • rx.recv() 会阻塞当前线程,直到有消息到达
  • 如果没有任何消息准备就绪,rx.try_recv() 会立即返回 Err(TryRecvError::Empty)
  • 当所有 Sender 都被 drop 时,信道关闭
#![allow(unused)]
fn main() {
// 带有背压的有界信道:
let (tx, rx) = mpsc::sync_channel(10); // 缓冲区容量为 10 条消息

thread::spawn(move || {
    for i in 0..1000 {
        tx.send(i).unwrap(); // 如果缓冲区已满则阻塞 —— 实现自然的背压
    }
});
}

注意:为了简明起见使用了 .unwrap()。在生产环境中,请处理 SendError(接收者已 drop)而不是直接 panic。

crossbeam-channel —— 工业级利器

crossbeam-channel 是生产环境中信道使用的行业标准。它比 std::sync::mpsc 更快,且支持多消费者 (MPMC):

// Cargo.toml:
//   [dependencies]
//   crossbeam-channel = "0.5"
use crossbeam_channel::{bounded, unbounded, select, Sender, Receiver};
use std::thread;
use std::time::Duration;

fn main() {
    // 有界 MPMC 信道
    let (tx, rx) = bounded::<String>(100);

    // 多个生产者
    for id in 0..4 {
        let tx = tx.clone();
        thread::spawn(move || {
            for i in 0..10 {
                tx.send(format!("工作者-{id}: 条目-{i}")).unwrap();
            }
        });
    }
    drop(tx); // drop 原始发送者,以便信道可以关闭

    // 多个消费者(这在 std::sync::mpsc 中是不可能的!)
    let rx2 = rx.clone();
    let consumer1 = thread::spawn(move || {
        while let Ok(msg) = rx.recv() {
            println!("[消费者-1] {msg}");
        }
    });
    let consumer2 = thread::spawn(move || {
        while let Ok(msg) = rx2.recv() {
            println!("[消费者-2] {msg}");
        }
    });

    consumer1.join().unwrap();
    consumer2.join().unwrap();
}

信道选择 (select!)

同时监听多个信道 —— 类似于 Go 中的 select:

use crossbeam_channel::{bounded, tick, after, select};
use std::time::Duration;

fn main() {
    let (work_tx, work_rx) = bounded::<String>(10);
    let ticker = tick(Duration::from_secs(1));        // 周期性滴答
    let deadline = after(Duration::from_secs(10));     // 一次性超时

    // 生产者
    let tx = work_tx.clone();
    std::thread::spawn(move || {
        for i in 0..100 {
            tx.send(format!("任务-{i}")).unwrap();
            std::thread::sleep(Duration::from_millis(500));
        }
    });
    drop(work_tx);

    loop {
        select! {
            recv(work_rx) -> msg => {
                match msg {
                    Ok(job) => println!("处理中: {job}"),
                    Err(_) => {
                        println!("工作信道已关闭");
                        break;
                    }
                }
            },
            recv(ticker) -> _ => {
                println!("滴答 — 心跳");
            },
            recv(deadline) -> _ => {
                println!("截止时间已到 — 正在关闭");
                break;
            },
        }
    }
}

与 Go 语言对比:这与 Go 语言中跨信道的 select 语句完全一致。crossbeam 的 select! 宏会通过随机化顺序来防止饥饿,这一点也与 Go 相同。

有界 vs 无界与背压 (Backpressure)

类型缓冲区满时的行为内存占用使用场景
无界 (Unbounded)从不阻塞(在堆上增长)无限制 ⚠️罕见 —— 仅当生产者速度明显慢于消费者时使用
有界 (Bounded)send() 会阻塞直到有空位固定生产环境默认选择 —— 防止内存溢出 (OOM)
会合 (Rendezvous) (bounded(0))send() 阻塞直到有接收者就绪无用于同步或移交 (Handoff)
#![allow(unused)]
fn main() {
// 会合信道 —— 零容量,直接移交
let (tx, rx) = crossbeam_channel::bounded(0);
// tx.send(x) 会阻塞直到有人调用 rx.recv(),反之亦然。
// 这能让两个线程实现精确同步。
}

规则:生产环境中应始终使用有界信道,除非你能证明生产者绝对不会超过消费者的处理速度。

使用信道的 Actor 模式

Actor 模式利用信道来串行化对可变状态的访问 —— 这种方式不需要互斥锁:

use std::sync::mpsc;
use std::thread;

// Actor 能够接收的消息类型
enum CounterMsg {
    Increment,
    Decrement,
    Get(mpsc::Sender<i64>), // 响应信道
}

struct CounterActor {
    count: i64,
    rx: mpsc::Receiver<CounterMsg>,
}

impl CounterActor {
    fn new(rx: mpsc::Receiver<CounterMsg>) -> Self {
        CounterActor { count: 0, rx }
    }

    fn run(mut self) {
        while let Ok(msg) = self.rx.recv() {
            match msg {
                CounterMsg::Increment => self.count += 1,
                CounterMsg::Decrement => self.count -= 1,
                CounterMsg::Get(reply) => {
                    let _ = reply.send(self.count);
                }
            }
        }
    }
}

// Actor 句柄 —— 克隆成本低,且支持 Send + Sync
#[derive(Clone)]
struct Counter {
    tx: mpsc::Sender<CounterMsg>,
}

impl Counter {
    fn spawn() -> Self {
        let (tx, rx) = mpsc::channel();
        thread::spawn(move || CounterActor::new(rx).run());
        Counter { tx }
    }

    fn increment(&self) { let _ = self.tx.send(CounterMsg::Increment); }
    fn decrement(&self) { let _ = self.tx.send(CounterMsg::Decrement); }

    fn get(&self) -> i64 {
        let (reply_tx, reply_rx) = mpsc::channel();
        self.tx.send(CounterMsg::Get(reply_tx)).unwrap();
        reply_rx.recv().unwrap()
    }
}

fn main() {
    let counter = Counter::spawn();

    // 多个线程可以安全地使用计数器 —— 无需互斥锁!
    let handles: Vec<_> = (0..10).map(|_| {
        let counter = counter.clone();
        thread::spawn(move || {
            for _ in 0..1000 {
                counter.increment();
            }
        })
    }).collect();

    for h in handles { h.join().unwrap(); }
    println!("最终计数值: {}", counter.get()); // 10000
}

何时使用 Actor vs 互斥锁:当状态具有复杂的不变量、操作耗时较长,或者你希望在不考虑加锁顺序的情况下串行化访问时,Actor 是非常出色的选择。互斥锁则在处理简单的临界区时更为直接。

关键要点 —— 信道

  • crossbeam-channel 是生产环境中的主力 —— 它比 std::sync::mpsc 更快且功能更丰富。
  • select! 取代了复杂的多元轮询,提供声明式的信道选择。
  • 有界信道提供自然的背压;无界信道存在内存溢出 (OOM) 风险。

另请参阅: 第 6 章 —— 并发 了解线程、Mutex 和共享状态。第 15 章 —— 异步 了解异步信道 (tokio::sync::mpsc)。


练习:基于信道的工作者池 (Worker Pool) ★★★ (~45 分钟)

构建一个使用信道的工作者池,要求如下:

  • 调度器通过信道发送 Job 结构体
  • N 个工作者消耗任务并将结果发回
  • 使用 std::sync::mpsc 配合 Arc<Mutex<Receiver>> 来实现共享的工作队列
🔑 参考答案
use std::sync::mpsc;
use std::thread;

struct Job {
    id: u64,
    data: String,
}

struct JobResult {
    job_id: u64,
    output: String,
    worker_id: usize,
}

fn worker_pool(jobs: Vec<Job>, num_workers: usize) -> Vec<JobResult> {
    let (job_tx, job_rx) = mpsc::channel::<Job>();
    let (result_tx, result_rx) = mpsc::channel::<JobResult>();

    let job_rx = std::sync::Arc::new(std::sync::Mutex::new(job_rx));

    let mut handles = Vec::new();
    for worker_id in 0..num_workers {
        let job_rx = job_rx.clone();
        let result_tx = result_tx.clone();
        handles.push(thread::spawn(move || {
            loop {
                let job = {
                    let rx = job_rx.lock().unwrap();
                    rx.recv()
                };
                match job {
                    Ok(job) => {
                        let output = format!("由工作者 {worker_id} 处理了 '{}'", job.data);
                        result_tx.send(JobResult {
                            job_id: job.id, output, worker_id,
                        }).unwrap();
                    }
                    Err(_) => break, // 信道已关闭
                }
            }
        }));
    }
    drop(result_tx); // 必须在发送端全部 drop 之后,rx 的迭代器才会停止

    let num_jobs = jobs.len();
    for job in jobs {
        job_tx.send(job).unwrap();
    }
    drop(job_tx); // 关闭工作队列

    let results: Vec<_> = result_rx.into_iter().collect();
    assert_eq!(results.len(), num_jobs);

    for h in handles { h.join().unwrap(); }
    results
}

fn main() {
    let jobs: Vec<Job> = (0..20).map(|i| Job {
        id: i, data: format!("任务-{i}"),
    }).collect();

    let results = worker_pool(jobs, 4);
    for r in &results {
        println!("[工作者 {}] 任务 {}: {}", r.worker_id, r.job_id, r.output);
    }
}

English Original

第 6 章:并发 (Concurrency) vs 并行 (Parallelism) vs 线程 🟡

你将学到:

  • 并发与并行之间的准确区别
  • OS 线程、作用域线程 (Scoped Threads) 以及用于数据并行的 rayon
  • 共享状态原语:Arc, Mutex, RwLock, Atomics, Condvar
  • 使用 OnceLock/LazyLock 的延迟初始化以及无锁 (Lock-free) 模式

术语:并发 ≠ 并行

这两个术语经常被混淆。以下是它们的准确区别:

并发 (Concurrency)并行 (Parallelism)
定义管理多个可以同时推进的任务同时执行多个任务
硬件要求单核即可需要多个核心
类比一个厨师,同时做几道菜(在它们之间切换)多个厨师,每人负责一道菜
Rust 工具async/await, 信道 (channels), select!rayon, thread::spawn, par_iter()
并发 (单核):                         并行 (多核):
                                      
任务 A: ██░░██░░██                   任务 A: ██████████
任务 B: ░░██░░██░░                   任务 B: ██████████
─────────────────→ 时间              ─────────────────→ 时间
(在单核上交替进行)                   (在双核上同时进行)

std::thread —— OS 线程

Rust 线程与 OS 线程是 1:1 映射的。每个线程都有自己的栈(通常为 2-8 MB):

use std::thread;
use std::time::Duration;

fn main() {
    // 派生一个线程 —— 接收一个闭包
    let handle = thread::spawn(|| {
        for i in 0..5 {
            println!("派生线程: {i}");
            thread::sleep(Duration::from_millis(100));
        }
        42 // 返回值
    });

    // 在主线程上同时进行工作
    for i in 0..3 {
        println!("主线程: {i}");
        thread::sleep(Duration::from_millis(150));
    }

    // 等待线程结束并获取其返回值
    let result = handle.join().unwrap(); // 如果线程 panic,unwrap 也会 panic
    println!("线程返回了: {result}");
}

Thread::spawn 的类型要求:

#![allow(unused)]
fn main() {
// 闭包必须满足:
// 1. Send —— 可以转移到另一个线程
// 2. 'static —— 不能从调用作用域中借用数据
// 3. FnOnce —— 获取捕获变量的所有权

let data = vec![1, 2, 3];

// ❌ 借用了数据 —— 不满足 'static
// thread::spawn(|| println!("{data:?}"));

// ✅ 将所有权移动 (move) 到线程中
thread::spawn(move || println!("{data:?}"));
// 此时在这里无法再访问 data
}

作用域线程 (std::thread::scope)

自 Rust 1.63 起,作用域线程解决了 'static 约束的问题 —— 线程可以从父作用域借用数据:

use std::thread;

fn main() {
    let mut data = vec![1, 2, 3, 4, 5];

    thread::scope(|s| {
        // 线程 1:借用共享引用
        s.spawn(|| {
            let sum: i32 = data.iter().sum();
            println!("总和: {sum}");
        });

        // 线程 2:同样借用共享引用 (多个读者 OK)
        s.spawn(|| {
            let max = data.iter().max().unwrap();
            println!("最大值: {max}");
        });

        // ❌ 当存在共享借用时尚不能进行可变借用:
        // s.spawn(|| data.push(6));
    });
    // 所有作用域线程都在此处 join —— 保证在作用域返回前完成

    // 现在进行修改是安全的 —— 所有线程都已结束
    data.push(6);
    println!("更新后: {data:?}");
}

这意义重大:在作用域线程出现之前,你必须使用 Arc::clone() 克隆所有内容才能与线程共享。现在你可以直接借用,且编译器能够证明所有线程都会在数据超出作用域之前结束。

rayon —— 数据并行

rayon 提供了并行迭代器,能够自动将工作分配到线程池中:

// Cargo.toml: rayon = "1"
use rayon::prelude::*;

fn main() {
    let data: Vec<u64> = (0..1_000_000).collect();

    // 串行执行:
    let sum_seq: u64 = data.iter().map(|x| x * x).sum();

    // 并行执行 —— 只需将 .iter() 改为 .par_iter():
    let sum_par: u64 = data.par_iter().map(|x| x * x).sum();

    assert_eq!(sum_seq, sum_par);

    // 并行排序:
    let mut numbers = vec![5, 2, 8, 1, 9, 3];
    numbers.par_sort();

    // 结合 map/filter/collect 的并行处理:
    let results: Vec<_> = data
        .par_iter()
        .filter(|&&x| x % 2 == 0)
        .map(|&x| expensive_computation(x))
        .collect();
}

fn expensive_computation(x: u64) -> u64 {
    // 模拟重度 CPU 计算
    (0..1000).fold(x, |acc, _| acc.wrapping_mul(7).wrapping_add(13))
}

何时使用 rayon vs 线程:

用途何时使用
async + tokioI/O 密集型并发(网络、文件 I/O)

共享状态:Arc, Mutex, RwLock, Atomics

当线程需要共享可变状态时,Rust 提供了安全的抽象:

注意:在这些示例中,我们为了简明起见在 .lock()、.read() 和 .write() 上使用了 .unwrap()。只有当另一个线程在持有锁的情况下 panic(导致锁被“毒化” (poisoning))时,这些调用才会失败。生产环境代码应决定是尝试从被毒化的锁中恢复,还是传播错误。

#![allow(unused)]
fn main() {
use std::sync::{Arc, Mutex, RwLock};
use std::sync::atomic::{AtomicU64, Ordering};
use std::thread;

// --- Arc<Mutex<T>>: 共享 + 独占访问 ---
fn mutex_example() {
    let counter = Arc::new(Mutex::new(0u64));
    let mut handles = vec![];

    for _ in 0..10 {
        let counter = Arc::clone(&counter);
        handles.push(thread::spawn(move || {
            for _ in 0..1000 {
                let mut guard = counter.lock().unwrap();
                *guard += 1;
            } // guard 被释放 → 锁被释放
        }));
    }

    for h in handles { h.join().unwrap(); }
    println!("计数器: {}", counter.lock().unwrap()); // 10000
}

// --- Arc<RwLock<T>>: 多个读者 或 一个写者 ---
fn rwlock_example() {
    let config = Arc::new(RwLock::new(String::from("初始值")));

    // 多个读者 —— 互不阻塞
    let readers: Vec<_> = (0..5).map(|id| {
        let config = Arc::clone(&config);
        thread::spawn(move || {
            let guard = config.read().unwrap();
            println!("读者 {id}: {guard}");
        })
    }).collect();

    // 写者 —— 阻塞并等待所有读者完成
    {
        let mut guard = config.write().unwrap();
        *guard = "已更新".to_string();
    }

    for r in readers { r.join().unwrap(); }
}

// --- Atomics: 对简单值的无锁操作 ---
fn atomic_example() {
    let counter = Arc::new(AtomicU64::new(0));
    let mut handles = vec![];

    for _ in 0..10 {
        let counter = Arc::clone(&counter);
        handles.push(thread::spawn(move || {
            for _ in 0..1000 {
                counter.fetch_add(1, Ordering::Relaxed);
                // 没有锁,没有互斥锁 —— 纯硬件原子指令
            }
        }));
    }

    for h in handles { h.join().unwrap(); }
    println!("原子计数器: {}", counter.load(Ordering::Relaxed)); // 10000
}
}

简要对比

原语使用场景开销竞争情况
Mutex<T>短临界区加锁 + 解锁线程排队等待
RwLock<T>读多写少读写锁读者并发,写者独占
AtomicU64 等计数器、标志硬件 CAS无锁 —— 无需等待
信道 (Channels)消息传递队列操作生产者/消费者解耦

条件变量 (Condvar)

Condvar 允许线程 等待 (wait) 直到另一个线程发出某个条件为真的信号,而无需忙轮询。它总是与一个 Mutex 配合使用:

#![allow(unused)]
fn main() {
use std::sync::{Arc, Mutex, Condvar};
use std::thread;

let pair = Arc::new((Mutex::new(false), Condvar::new()));
let pair2 = Arc::clone(&pair);

// 派生线程:等待 ready == true
let handle = thread::spawn(move || {
    let (lock, cvar) = &*pair2;
    let mut ready = lock.lock().unwrap();
    while !*ready {
        ready = cvar.wait(ready).unwrap(); // 原子性解锁 + 进入睡眠
    }
    println!("工作线程:条件满足,继续执行");
});

// 主线程:设置 ready = true,然后发信号
{
    let (lock, cvar) = &*pair;
    let mut ready = lock.lock().unwrap();
    *ready = true;
    cvar.notify_one(); // 唤醒一个等待线程 (使用 notify_all 唤醒所有)
}
handle.join().unwrap();
}

模式:始终在 wait() 返回后的 while 循环中重新检查条件 —— OS 允许发生虚假唤醒 (spurious wakeups)。

延迟初始化:OnceLock 与 LazyLock

在 Rust 1.80 之前,初始化需要运行时计算的全局静态变量(例如解析配置、编译正则)需要使用 lazy_static! 宏或 once_cell crate。现在标准库原生提供了两种类型来涵盖这些场景:

#![allow(unused)]
fn main() {
use std::sync::{OnceLock, LazyLock};
use std::collections::HashMap;

// OnceLock —— 在第一次通过 `get_or_init` 使用时初始化。
// 当初始化值依赖于运行时参数时非常有用。
static CONFIG: OnceLock<HashMap<String, String>> = OnceLock::new();

fn get_config() -> &'static HashMap<String, String> {
    CONFIG.get_or_init(|| {
        // 耗时操作:读取并解析配置文件 —— 仅发生一次。
        let mut m = HashMap::new();
        m.insert("log_level".into(), "info".into());
        m
    })
}

// LazyLock —— 在第一次访问时初始化,闭包在定义处提供。
// 相当于 lazy_static! 但无需宏。
static REGEX: LazyLock<regex::Regex> = LazyLock::new(|| {
    regex::Regex::new(r"^[a-zA-Z0-9_]+$").unwrap()
});

fn is_valid_identifier(s: &str) -> bool {
    REGEX.is_match(s) // 第一次调用编译正则;后续调用重用它。
}
}
类型稳定化版本初始化时机何时使用
OnceLock<T>Rust 1.70调用处 (get_or_init)初始化依赖运行时参数
LazyLock<T>Rust 1.80定义处 (闭包)初始化是自包含的
lazy_static!—定义处 (宏)1.80 之前的旧代码库 (建议迁移)
const fn + static始终可用编译时值可以在编译时计算出

迁移提示:可以使用 static X: LazyLock<T> = LazyLock::new(|| expr); 替换 lazy_static! { static ref X: T = expr; } —— 语义相同,无宏,且无外部依赖。

无锁模式 (Lock-Free Patterns)

对于高性能代码,可以尝试完全避免使用锁:

#![allow(unused)]
fn main() {
use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering};
use std::sync::Arc;

// 模式 1:自旋锁 (演示用途 —— 生产环境请优先使用 std::sync::Mutex)
// ⚠️ 警告:这仅作为一个教学示例。真实的自旋锁需要:
//   - RAII 守卫 (以确保持锁时发生 panic 不会造成永久死锁)
//   - 公平性保证 (基础实现可能会导致线程饥饿)
//   - 退避策略 (如指数退避、让出 CPU 给 OS)
// 生产环境中请使用 std::sync::Mutex 或 parking_lot::Mutex。
struct SpinLock {
    locked: AtomicBool,
}

impl SpinLock {
    fn new() -> Self { SpinLock { locked: AtomicBool::new(false) } }

    fn lock(&self) {
        while self.locked
            .compare_exchange_weak(false, true, Ordering::Acquire, Ordering::Relaxed)
            .is_err()
        {
            std::hint::spin_loop(); // CPU 提示:我们正在自旋
        }
    }

    fn unlock(&self) {
        self.locked.store(false, Ordering::Release);
    }
}

// 模式 2:无锁 SPSC (单生产者,单消费者)
// 生产环境中请使用 crossbeam::queue::ArrayQueue 或类似库。

// 模式 3:用于“等待无关读” (Wait-free reads) 的序列计数器
// ⚠️ 最适用于单机器字类型 (u64, f64);较宽的 T 在读取时可能会发生撕裂。
struct SeqLock<T: Copy> {
    seq: AtomicUsize,
    data: std::cell::UnsafeCell<T>,
}

unsafe impl<T: Copy + Send> Sync for SeqLock<T> {}

impl<T: Copy> SeqLock<T> {
    fn new(val: T) -> Self {
        SeqLock {
            seq: AtomicUsize::new(0),
            data: std::cell::UnsafeCell::new(val),
        }
    }

    fn read(&self) -> T {
        loop {
            let s1 = self.seq.load(Ordering::Acquire);
            if s1 & 1 != 0 { continue; } // 写者正在执行,重试

            // 安全性:我们使用 ptr::read_volatile 来防止编译器
            // 对读取操作进行重排序或缓存。SeqLock 协议(读取后检查 s1 == s2)
            // 确保了如果写者当时处于活动状态,我们将执行重试。
            // 这借鉴了 C 语言中 SeqLock 的模式,即数据读取必须使用
            // volatile/relaxed 语义以便在并发下避免数据撕裂。
            let value = unsafe { core::ptr::read_volatile(self.data.get() as *const T) };

            // 获取屏障 (Acquire fence):确保上述数据读取
            // 发生在我们重新检查序列计数器之前。
            std::sync::atomic::fence(Ordering::Acquire);
            let s2 = self.seq.load(Ordering::Relaxed);

            if s1 == s2 { return value; } // 没有写者干扰
            // 否则重试
        }
    }

    /// # 安全性约定
    /// 同一时间只能有一个线程调用 `write()`。如果需要多个写者,
    /// 请在外部使用 `Mutex` 包装 `write()` 调用。
    fn write(&self, val: T) {
        // 递增至奇数 (标志着写入正在进行中)。
        // AcqRel:Acquire 侧防止后续的数据写入被重排序到此次递增之前
        // (读者必须在观察到部分写入之前看到奇数)。
        // Release 侧对于单写者来说在技术上不是必需的,但一致且无害。
        self.seq.fetch_add(1, Ordering::AcqRel);
        // 安全性:单写者不变量由调用者维护(见上方文档)。
        // UnsafeCell 允许内部可变性;序列计数器负责保护读者。
        unsafe { *self.data.get() = val; }
        // 递增至偶数 (标志着写入完成)。
        // Release:确保在读者看到偶数序列号之前,数据写入已经可见。
        self.seq.fetch_add(1, Ordering::Release);
    }
}
}

⚠️ Rust 内存模型注意事项:在 write() 中通过 UnsafeCell 执行的非原子性写入与 read() 中非原子性的 ptr::read_volatile 同时发生,在 Rust 抽象机下技术上属于“数据竞争” —— 尽管 SeqLock 协议能确保读者在读到旧数据时总是会重试。这模拟了 C 内核的 SeqLock 模式,在所有现代硬件上对于能放入单个机器字(如 u64)的类型 T 来说在实践中是安全的。对于更宽的类型,请考虑对数据字段使用 AtomicU64 或使用 Mutex 包装访问。 参阅 Rust 不安全代码指南 了解关于 UnsafeCell 并发的更多动态。

实用建议:无锁代码很难写对。除非性能分析 (profiling) 显示锁竞争是你的瓶颈,否则请优先使用 Mutex 或 RwLock。当你确实需要无锁方案时,请选用经过验证的 crate(如 crossbeam、arc-swap、dashmap),而不是自造轮子。

关键要点 —— 并发

  • 作用域线程 (thread::scope) 允许你在无需 Arc 的情况下借用栈上的数据。
  • rayon::par_iter() 仅需一个方法调用即可实现迭代器的并行化。
  • 优先使用 OnceLock/LazyLock 而非 lazy_static!;在使用原子操作前先考虑 Mutex。
  • 无锁代码极具挑战 —— 优先选用经过验证的 crate,而非手写实现。

另请参阅: 第 5 章 —— 信道 了解消息传递并发。第 8 章 —— 智能指针 了解 Arc/Rc 的细节。

flowchart TD
    A["需要共享的<br>可变状态吗?"] -->|是| B{"竞争压力<br>有多大?"}
    A -->|否| C["使用信道<br>(第 5 章)"]

    B -->|"读多写少"| D["RwLock"]
    B -->|"短临界区"| E["Mutex"]
    B -->|"简单的计数器<br>或标志"| F["Atomics"]
    B -->|"复杂状态"| G["Actor + 信道"]

    H["需要并行计算吗?"] -->|"集合处理"| I["rayon::par_iter"]
    H -->|"后台任务"| J["thread::spawn"]
    H -->|"借用局部数据"| K["thread::scope"]

    style A fill:#e8f4f8,stroke:#2980b9,color:#000
    style B fill:#fef9e7,stroke:#f1c40f,color:#000
    style C fill:#d4efdf,stroke:#27ae60,color:#000
    style D fill:#fdebd0,stroke:#e67e22,color:#000
    style E fill:#fdebd0,stroke:#e67e22,color:#000
    style F fill:#fdebd0,stroke:#e67e22,color:#000
    style G fill:#fdebd0,stroke:#e67e22,color:#000
    style H fill:#e8f4f8,stroke:#2980b9,color:#000
    style I fill:#d4efdf,stroke:#27ae60,color:#000
    style J fill:#d4efdf,stroke:#27ae60,color:#000
    style K fill:#d4efdf,stroke:#27ae60,color:#000

练习:使用作用域线程实现并行 Map ★★ (~25 分钟)

编写一个函数 parallel_map<T, R>(data: &[T], f: fn(&T) -> R, num_threads: usize) -> Vec<R>,将 data 分成 num_threads 个分块,并在作用域线程中处理每个分块。不要使用 rayon —— 请使用 std::thread::scope。

🔑 参考答案
fn parallel_map<T: Sync, R: Send>(data: &[T], f: fn(&T) -> R, num_threads: usize) -> Vec<R> {
    let chunk_size = (data.len() + num_threads - 1) / num_threads;
    let mut results = Vec::with_capacity(data.len());

    std::thread::scope(|s| {
        let mut handles = Vec::new();
        for chunk in data.chunks(chunk_size) {
            handles.push(s.spawn(move || {
                chunk.iter().map(f).collect::<Vec<_>>()
            }));
        }
        for h in handles {
            results.extend(h.join().unwrap());
        }
    });

    results
}

fn main() {
    let data: Vec<u64> = (1..=20).collect();
    let squares = parallel_map(&data, |x| x * x, 4);
    assert_eq!(squares, (1..=20).map(|x: u64| x * x).collect::<Vec<_>>());
    println!("并行计算的平方数: {squares:?}");
}

English Original

第 7 章:闭包 (Closures) 与高阶函数 🟢

你将学到:

  • 三种闭包 Trait (Fn, FnMut, FnOnce) 及其捕获方式
  • 将闭包作为参数传递,以及从函数中返回闭包
  • 函数式编程风格中的组合子链 (Combinator Chains) 与迭代器适配器
  • 如何使用合适的 Trait 约束来设计你自己的高阶 API

Fn, FnMut, FnOnce —— 闭包 Trait

Rust 中的每个闭包都会根据其捕获变量的方式来实现一种 or 多种 Trait:

#![allow(unused)]
fn main() {
// FnOnce —— 消耗捕获的值 (只能被调用一次)
let name = String::from("Alice");
let greet = move || {
    println!("你好, {name}!"); // 获取 `name` 的所有权
    drop(name); // name 被消耗
};
greet(); // ✅ 第一次调用
// greet(); // ❌ 无法再次调用 —— `name` 已被消耗

// FnMut —— 可变借用捕获的值 (可以被调用多次)
let mut count = 0;
let mut increment = || {
    count += 1; // 可变借用 `count`
};
increment(); // count == 1
increment(); // count == 2

// Fn —— 不可变借用捕获的值 (可以被调用多次,且支持并发)
let prefix = "结果";
let display = |x: i32| {
    println!("{prefix}: {x}"); // 不可变借用 `prefix`
};
display(1);
display(2);
}

层级结构:Fn : FnMut : FnOnce —— 每一层都是下一层的子 Trait (Subtrait):

FnOnce  ← 所有闭包至少可以被调用一次
 ↑
FnMut   ← 可以被重复调用 (可能会修改状态)
 ↑
Fn      ← 可以被重复且并发地调用 (不能修改状态)

如果一个闭包实现了 Fn,它同时也实现了 FnMut 和 FnOnce。

将闭包作为参数和返回值

// --- 作为参数 ---

// 静态分发 (静态单态化 —— 速度最快)
fn apply_twice<F: Fn(i32) -> i32>(f: F, x: i32) -> i32 {
    f(f(x))
}

// 也可以使用 impl Trait 写法:
fn apply_twice_v2(f: impl Fn(i32) -> i32, x: i32) -> i32 {
    f(f(x))
}

// 动态分发 (Trait 对象 —— 灵活,但有轻微开销)
fn apply_dyn(f: &dyn Fn(i32) -> i32, x: i32) -> i32 {
    f(x)
}

// --- 作为返回值 ---

// 如果不使用 Box,无法按值返回闭包 (因为它们是匿名类型):
fn make_adder(n: i32) -> Box<dyn Fn(i32) -> i32> {
    Box::new(move |x| x + n)
}

// 使用 impl Trait (更简单,单态化,但不支持动态性):
fn make_adder_v2(n: i32) -> impl Fn(i32) -> i32 {
    move |x| x + n
}

fn main() {
    let double = |x: i32| x * 2;
    println!("{}", apply_twice(double, 3)); // 12

    let add5 = make_adder(5);
    println!("{}", add5(10)); // 15
}

组合子链与迭代器适配器

高阶函数在迭代器中大放异彩 —— 这是最惯用的 Rust 写法:

#![allow(unused)]
fn main() {
// C 风格循环 (指令式):
let data = vec![1, 2, 3, 4, 5, 6, 7, 8, 9, 10];
let mut result = Vec::new();
for x in &data {
    if x % 2 == 0 {
        result.push(x * x);
    }
}

// 惯用的 Rust 写法 (函数式组合子链):
let result: Vec<i32> = data.iter()
    .filter(|&&x| x % 2 == 0)
    .map(|&x| x * x)
    .collect();

// 性能相同 —— 迭代器是惰性的,且会被 LLVM 深度优化
assert_eq!(result, vec![4, 16, 36, 64, 100]);
}

常见组合子速查表:

组合子作用示例
.map(f)转换每个元素`.map(
.filter(p)保留谓词为真的元素`.filter(
.filter_map(f)map + filter 的结合 (返回 Option)`.filter_map(
.flat_map(f)先 map 然后展平嵌套迭代器`.flat_map(
.fold(init, f)归约为单个值`.fold(0,
.any(p) / .all(p)短路布尔检查`.any(
.enumerate()添加索引`.enumerate().map(
.zip(other)与另一个迭代器配对.zip(labels.iter())
.take(n) / .skip(n)取前 N 个 / 跳过前 N 个元素.take(10)
.chain(other)拼接两个迭代器.chain(extra.iter())
.peekable()在不消耗的情况下查看下一个元素.peek()
.collect()聚合到集合中.collect::<Vec<_>>()

实现你自己的高阶 API

设计接受闭包进行自定义的 API:

#![allow(unused)]
fn main() {
/// 根据可配置策略重试操作
fn retry<T, E, F, S>(
    mut operation: F,
    mut should_retry: S,
    max_attempts: usize,
) -> Result<T, E>
where
    F: FnMut() -> Result<T, E>,
    S: FnMut(&E, usize) -> bool, // (错误, 尝试次数) → 是否重试?
{
    for attempt in 1..=max_attempts {
        match operation() {
            Ok(val) => return Ok(val),
            Err(e) if attempt < max_attempts && should_retry(&e, attempt) => {
                continue;
            }
            Err(e) => return Err(e),
        }
    }
    unreachable!()
}

// 用法 —— 调用者控制重试逻辑:
let result = retry(
    || connect_to_database(),
    |err, attempt| {
        eprintln!("第 {attempt} 次尝试失败: {err}");
        true // 总是重试
    },
    3,
);

// 用法 —— 仅针对特定错误进行重试:
let result = retry(
    || http_get(url),
    |err, _| err.is_transient(), // 仅重试瞬时错误 (Transient Errors)
    5,
);
}

with 模式 —— 括号式资源访问 (Bracketed Resource Access)

有时你需要保证资源在操作期间处于特定状态,并在操作结束后恢复 —— 无论调用者的代码如何退出(早期返回、? 运算符、panic)。与其直接暴露资源并寄希望于调用者记得正确进行设置 (Setup) 和拆除 (Teardown),不如 通过闭包借出资源:

建立 (Set up) → 通过资源调用闭包 → 拆除 (Tear down)

调用者永远不会接触设置或拆除过程。他们不会忘记,不会弄错,也无法在闭包作用域之外持有该资源。

示例:GPIO 引脚方向

GPIO 控制器管理支持双向 I/O 的引脚。有的调用者需要将引脚配置为输入,有的则需要配置为输出。与其暴露原始的引脚访问权限并信任调用者会正确设置方向,控制器提供了 with_pin_input 和 with_pin_output 方法:

/// GPIO 引脚方向 —— 非公开,调用者永远无法直接设置它。
#[derive(Debug, Clone, Copy, PartialEq)]
enum Direction { In, Out }

/// 借给闭包的 GPIO 引脚句柄。无法被存储或克隆 ——
/// 它仅在回调期间存在。
pub struct GpioPin<'a> {
    pin_number: u8,
    _controller: &'a GpioController,
}

impl GpioPin<'_> {
    pub fn read(&self) -> bool {
        // 从硬件寄存器读取引脚电平
        println!("  正在读取引脚 {}", self.pin_number);
        true // 桩代码
    }

    pub fn write(&self, high: bool) {
        // 通过硬件寄存器驱动引脚电平
        println!("  正在写入引脚 {} = {high}", self.pin_number);
    }
}

pub struct GpioController {
    current_direction: std::cell::Cell<Option<Direction>>,
}

impl GpioController {
    pub fn new() -> Self {
        GpioController {
            current_direction: std::cell::Cell::new(None),
        }
    }

    /// 将引脚配置为输入,运行闭包,然后恢复状态。
    /// 调用者收到的 `GpioPin` 仅在回调期间存活。
    pub fn with_pin_input<R>(
        &self,
        pin: u8,
        mut f: impl FnMut(&GpioPin<'_>) -> R,
    ) -> R {
        let prev = self.current_direction.get();
        self.set_direction(pin, Direction::In);
        let handle = GpioPin { pin_number: pin, _controller: self };
        let result = f(&handle);
        // 恢复先前的方向 (或者保持原状 —— 取决于策略选择)
        if let Some(dir) = prev {
            self.set_direction(pin, dir);
        }
        result
    }

    /// 将引脚配置为输出,运行闭包,然后恢复状态。
    /// 调用者收到的 `GpioPin` 仅在回调期间存活。
    pub fn with_pin_output<R>(
        &self,
        pin: u8,
        mut f: impl FnMut(&GpioPin<'_>) -> R,
    ) -> R {
        let prev = self.current_direction.get();
        self.set_direction(pin, Direction::Out);
        let handle = GpioPin { pin_number: pin, _controller: self };
        let result = f(&handle);
        if let Some(dir) = prev {
            self.set_direction(pin, dir);
        }
        result
    }

    fn set_direction(&self, pin: u8, dir: Direction) {
        println!("  [hw] 引脚 {pin} → {dir:?}");
        self.current_direction.set(Some(dir));
    }
}

fn main() {
    let gpio = GpioController::new();

    // 调用者 1:需要输入 —— 不知道也不关心方向是如何管理的
    let level = gpio.with_pin_input(4, |pin| {
        pin.read()
    });
    println!("引脚 4 电平: {level}");

    // 调用者 2:需要输出 —— 相同的 API 形式,不同的保证
    gpio.with_pin_output(4, |pin| {
        pin.write(true);
        // 执行更多工作...
        pin.write(false);
    });

    // 无法在闭包外使用引脚句柄:
    // let escaped_pin = gpio.with_pin_input(4, |pin| pin);
    // ❌ 错误:被借用的值存活时间不够长
}

with 模式保证了:

  • 引脚方向 总是在 调用者代码运行前设置好
  • 引脚方向 总是在 之后恢复,即使闭包发生了早期返回
  • GpioPin 句柄 无法逃逸 出闭包 —— 借用检查器通过与控制器引用绑定的生命周期来强制执行这一点
  • 调用者永远不需要导入 Direction,也不需要调用 set_direction —— 该 API 不可能被误用 (Impossible to misuse)

该模式出现在哪里

with 模式遍布 Rust 标准库和生态系统:

API设置 (Setup)回调拆除 (Teardown)
std::thread::scope创建作用域`s
Mutex::lock获取锁使用 MutexGuard (RAII,非闭包,但思路相同)Drop 时释放
tempfile::tempdir创建临时目录使用路径Drop 时删除
std::io::BufWriter::new缓冲写入写入操作Drop 时刷新 (Flush)
GPIO with_pin_* (见上文)设置方向使用引脚句柄恢复方向

在以下情况下,基于闭包的变体最为强力:

  • 设置与拆除成对出现,漏掉任何一个都是 Bug
  • 资源不应超出操作的存活时间 —— 借用检查器会自然地强制执行这一点
  • 存在多种配置 (with_pin_input 与 with_pin_output) —— 每个 with_* 方法都封装了不同的设置,而无需向调用者暴露具体的配置细节

with vs RAII (Drop):两者都保证了清理工作。当调用者需要在多个语句和函数调用中持有资源时,请使用 RAII / Drop。当操作是 括号式 (Bracketed) 的 —— 即:一次设置、一块工作、一次拆除 —— 且你不希望调用者能够打破这个“括号”时,请使用 with 模式。

关键要点 —— 闭包

  • Fn 借用数据,FnMut 可变借用数据,FnOnce 消耗数据 —— 尽量接受你的 API 所需的最弱约束。
  • 在参数中使用 impl Fn,在存储时使用 Box<dyn Fn>,在返回时使用 impl Fn (如果涉及动态分发则使用 Box<dyn Fn>)。
  • 组合子链 (map, filter, and_then) 的写法非常简洁,且能被编译器内联优化为高性能循环。
  • with 模式 (通过闭包实现的扩展访问) 能够保证设置/拆除逻辑的执行并防止资源逃逸 —— 当调用者不应管理配置生命周期时非常适用。

另请参阅: 第 2 章 —— 深入 Trait 了解 Fn/FnMut/FnOnce 与 Trait 对象的关联。第 8 章 —— 函数式 vs 指令式 了解何时选择组合子而非循环。第 15 章 —— API 设计 了解如何编写符合人体工程学 (Ergonomic) 的参数模式。

graph TD
    FnOnce["FnOnce<br>(只能调用一次)"]
    FnMut["FnMut<br>(可以调用多次,<br>可能修改捕获变量)"]
    Fn["Fn<br>(可以调用多次,<br>不可变捕获)"]

    Fn -->|"implements"| FnMut
    FnMut -->|"implements"| FnOnce

    style Fn fill:#d4efdf,stroke:#27ae60,color:#000
    style FnMut fill:#fef9e7,stroke:#f1c40f,color:#000
    style FnOnce fill:#fadbd8,stroke:#e74c3c,color:#000

每个 Fn 同时也是 FnMut,每个 FnMut 同时也是 FnOnce。默认情况下接受 FnMut —— 它是对调用者最灵活的约束。


练习:高阶组合子流水线 (Higher-Order Combinator Pipeline) ★★ (~25 分钟)

创建一个 Pipeline 结构体,用于链接一系列转换操作。它应该支持通过 .pipe(f) 添加转换,并通过 .execute(input) 运行完整的转换链。

🔑 参考答案
struct Pipeline<T> {
    transforms: Vec<Box<dyn Fn(T) -> T>>,
}

impl<T: 'static> Pipeline<T> {
    fn new() -> Self {
        Pipeline { transforms: Vec::new() }
    }

    fn pipe(mut self, f: impl Fn(T) -> T + 'static) -> Self {
        self.transforms.push(Box::new(f));
        self
    }

    fn execute(self, input: T) -> T {
        self.transforms.into_iter().fold(input, |val, f| f(val))
    }
}

fn main() {
    let result = Pipeline::new()
        .pipe(|s: String| s.trim().to_string())
        .pipe(|s| s.to_uppercase())
        .pipe(|s| format!(">>> {s} <<<"))
        .execute("  hello world  ".to_string());

    println!("{result}"); // >>> HELLO WORLD <<<

    let result = Pipeline::new()
        .pipe(|x: i32| x * 2)
        .pipe(|x| x + 10)
        .pipe(|x| x * x)
        .execute(5);

    println!("{result}"); // (5*2 + 10)^2 = 400
}

English Original

第 8 章:函数式 (Functional) vs. 指令式 (Imperative):优雅何时更胜一筹 (以及何时不适用)

难度: 🟡 中级 | 时间: 2–3 小时 | 前置知识: 第 7 章 —— 闭包

Rust 让函数式与指令式风格保持了真正的同等地位。与 Haskell (强制函数式) 或 C (默认指令式) 不同,Rust 允许你自行选择 —— 而正确的选择取决于你想要表达的内容。本章将帮助你建立做出明智选择的判断力。

核心原则: 当你 通过流水线 (Pipeline) 转换数据 时,函数式风格大放异彩。当你 管理带有副作用 (Side effects) 的状态转换 时,指令式风格更胜一筹。大多数现实世界的代码同时包含两者,而真正的技巧在于知道两者的界限在哪里。


8.1 你一直想要却没发现的组合子

许多 Rust 开发者会这样写:

#![allow(unused)]
fn main() {
let value = if let Some(x) = maybe_config() {
    x
} else {
    default_config()
};
process(value);
}

其实他们可以这样写:

#![allow(unused)]
fn main() {
process(maybe_config().unwrap_or_else(default_config));
}

或者这种常见的模式:

#![allow(unused)]
fn main() {
let display_name = if let Some(name) = user.nickname() {
    name.to_uppercase()
} else {
    "ANONYMOUS".to_string()
};
}

可以改写为:

#![allow(unused)]
fn main() {
let display_name = user.nickname()
    .map(|n| n.to_uppercase())
    .unwrap_or_else(|| "ANONYMOUS".to_string());
}

函数式版本不仅更短 —— 它还能直接告诉你 发生了什么 (转换,然后设置默认值),而无需让你追踪控制流。if let 版本则强制你阅读两个分支,才能发现两条路径最终殊途同归。

Option 组合子家族

这里有一个心智模型:Option<T> 是一个“包含一个元素或为空”的集合。Option 上的每个组合子都能在集合操作中找到类比。

你写的…等同于 (指令式)…它传达的语义
opt.unwrap_or(default)if let Some(x) = opt { x } else { default }“使用此值,或者回退到默认值”
`opt.unwrap_or_else(expensive())`
opt.map(f)match opt { Some(x) => Some(f(x)), None => None }“转换内部值,传递‘空’状态”
opt.and_then(f)match opt { Some(x) => f(x), None => None }“链接可能失败的操作” (Flatmap)
`opt.filter(xpred(x))`
opt.zip(other)if let (Some(a), Some(b)) = (opt, other) { Some((a,b)) } else { None }“全有或全无”
opt.or(fallback)if opt.is_some() { opt } else { fallback }“第一个可用的值”
`opt.or_else(try_another())`
opt.map_or(default, f)if let Some(x) = opt { f(x) } else { default }“转换或默认” —— 单行搞定
opt.map_or_else(default_fn, f)if let Some(x) = opt { f(x) } else { default_fn() }同上,两端都是闭包
opt?match opt { Some(x) => x, None => return None }“将缺失状态向上传递”

Result 组合子家族

相同的模式也适用于 Result<T, E>:

你写的…等同于 (指令式)…它传达的语义
res.map(f)match res { Ok(x) => Ok(f(x)), Err(e) => Err(e) }转换成功路径
res.map_err(f)match res { Ok(x) => Ok(x), Err(e) => Err(f(e)) }转换错误
res.and_then(f)match_res { Ok(x) => f(x), Err(e) => Err(e) }链接可能失败的操作
`res.unwrap_or_else(edefault(e))`
res.ok()match res { Ok(x) => Some(x), Err(_) => None }“我不关心具体的错误”
res?match res { Ok(x) => x, Err(e) => return Err(e.into()) }将错误向上传递

什么时候 if let 反而更好

在以下情况下,组合子会逊色:

  • 你需要在 Some 分支中写多条语句。 一个 5 行长的 map 闭包比 5 行长的 if let 糟糕得多。
  • 控制流本身就是重点。 if let Some(connection) = pool.try_get() { /* 使用它 */ } else { /* 日志记录、重试、报警 */ } —— 两个分支是截然不同的代码路径,而不仅仅是“转换或默认”。
  • 副作用占据主导地位。 如果两个分支都在执行带有不同错误处理的 I/O,那么组合子版本会模糊这些重要的区别。

经验法则: 如果 else 分支产生与 Some 分支 相同类型 的结果,且主体是短表达式,请使用组合子。如果两个分支做的事情有本质上的不同,请使用 if let 或 match。


8.2 布尔组合子:.then() 与 .then_some()

另一种比想象中更常用的模式:

#![allow(unused)]
fn main() {
let label = if is_admin {
    Some("ADMIN")
} else {
    None
};
}

Rust 1.62+ 提供了:

#![allow(unused)]
fn main() {
let label = is_admin.then_some("ADMIN");
}

或者使用计算出的值:

#![allow(unused)]
fn main() {
let permissions = is_admin.then(|| compute_admin_permissions());
}

这在链式调用中尤其强大:

#![allow(unused)]
fn main() {
// 指令式 (Imperative)
let mut tags = Vec::new();
if user.is_admin { tags.push("admin"); }
if user.is_verified { tags.push("verified"); }
if user.score > 100 { tags.push("power-user"); }

// 函数式 (Functional)
let tags: Vec<&str> = [
    user.is_admin.then_some("admin"),
    user.is_verified.then_some("verified"),
    (user.score > 100).then_some("power-user"),
]
.into_iter()
.flatten()
.collect();
}

函数式版本明确化了这种模式:“从条件元素构建列表”。指令式版本则要求你阅读每一个 if,以确认它们都在做同一件事(即 push 一个标签)。


8.3 迭代器链 vs. 循环:决策框架

第 7 章介绍了迭代器的机制。本节将培养你做出选择的判断力。

什么时候迭代器胜出

数据流水线 (Data pipelines) —— 通过一系列步骤转换集合:

#![allow(unused)]
fn main() {
// 指令式:8 行代码,2 个可变变量
let mut results = Vec::new();
for item in inventory {
    if item.category == Category::Server {
        if let Some(temp) = item.last_temperature() {
            if temp > 80.0 {
                results.push((item.id, temp));
            }
        }
    }
}

// 函数式:6 行代码,0 个可变变量,一个流水线
let results: Vec<_> = inventory.iter()
    .filter(|item| item.category == Category::Server)
    .filter_map(|item| item.last_temperature().map(|t| (item.id, t)))
    .filter(|(_, temp)| *temp > 80.0)
    .collect();
}

函数式版本胜在:

  • 每个过滤器 (Filter) 都是独立可读的。
  • 无需 mut —— 数据向单一方向流动。
  • 你可以增加、删除或重新排序流水线阶段,而无需重构整体结构。
  • LLVM 会将迭代器适配器内联成与循环完全相同的机器码。

聚合 (Aggregation) —— 从集合中计算出单个值:

#![allow(unused)]
fn main() {
// 指令式
let mut total_power = 0.0;
let mut count = 0;
for server in fleet {
    total_power += server.power_draw();
    count += 1;
}
let avg = total_power / count as f64;

// 函数式
let (total_power, count) = fleet.iter()
    .map(|s| s.power_draw())
    .fold((0.0, 0usize), |(sum, n), p| (sum + p, n + 1));
let avg = total_power / count as f64;
}

如果只需要求和,则更简单:

#![allow(unused)]
fn main() {
let total: f64 = fleet.iter().map(|s| s.power_draw()).sum();
}

什么时候循环胜出

带有复杂状态的早期退出 (Early exit):

#![allow(unused)]
fn main() {
// 这种写法清晰且直观
let mut best_candidate = None;
for server in fleet {
    let score = evaluate(server);
    if score > threshold {
        if server.is_available() {
            best_candidate = Some(server);
            break; // 找到了一个 —— 立即停止
        }
    }
}

// 函数式版本显得有些吃力
let best_candidate = fleet.iter()
    .filter(|s| evaluate(s) > threshold)
    .find(|s| s.is_available());
}

虽然在这里函数式写法还算整洁,但让我们看一个它真正处于劣势的案例:

同时构建多个输出:

#![allow(unused)]
fn main() {
// 指令式:清晰,每个分支做不同的事
let mut warnings = Vec::new();
let mut errors = Vec::new();
let mut stats = Stats::default();

for event in log_stream {
    match event.severity {
        Severity::Warn => {
            warnings.push(event.clone());
            stats.warn_count += 1;
        }
        Severity::Error => {
            errors.push(event.clone());
            stats.error_count += 1;
            if event.is_critical() {
                alert_oncall(&event);
            }
        }
        _ => stats.other_count += 1,
    }
}

// 函数式版本:牵强、笨拙,没人想读这样的代码
let (warnings, errors, stats) = log_stream.iter().fold(
    (Vec::new(), Vec::new(), Stats::default()),
    |(mut w, mut e, mut s), event| {
        match event.severity {
            Severity::Warn => { w.push(event.clone()); s.warn_count += 1; }
            Severity::Error => {
                e.push(event.clone()); s.error_count += 1;
                if event.is_critical() { alert_oncall(event); }
            }
            _ => s.other_count += 1,
        }
        (w, e, s)
    },
);
}

fold 版本 更长、更难读,且依然包含可变性(被解构出的累加器 mut)。循环胜在:

  • 可以并行构建多个输出。
  • 在逻辑中混杂了副作用(如警报)。
  • 分支主体是语句 (Statements) 而非表达式 (Expressions)。

带有 I/O 的状态机:

#![allow(unused)]
fn main() {
// 一个读取 Token 的解析器 —— 循环本身就是算法的体现
let mut state = ParseState::Start;
loop {
    let token = lexer.next_token()?;
    state = match state {
        ParseState::Start => match token {
            Token::Keyword(k) => ParseState::GotKeyword(k),
            Token::Eof => break,
            _ => return Err(ParseError::UnexpectedToken(token)),
        },
        ParseState::GotKeyword(k) => match token {
            Token::Ident(name) => ParseState::GotName(k, name),
            _ => return Err(ParseError::ExpectedIdentifier),
        },
        // ...更多状态
    };
}
}

没有比这更简洁的函数式写法了。带有 match state 的循环是状态机的自然表达方式。

决策流程图

flowchart TB
    START{你正在做什么?}

    START -->|"将一个集合转换<br>为另一个集合"| PIPE[使用迭代器链]
    START -->|"从集合中计算<br>出单个值"| AGG{多复杂?}
    START -->|"单次遍历产生<br>多个输出"| LOOP[使用 for 循环]
    START -->|"带有 I/O 或副作<br>用的状态机"| LOOP
    START -->|"单次 Option/Result<br>转换 + 默认值"| COMB[使用组合子]

    AGG -->|"求和、计数、<br>最大/最小值"| BUILTIN["使用 .sum(), .count(),<br>.min(), .max()"]
    AGG -->|"自定义累加"| FOLD{累加器包含可变性<br>或副作用吗?}
    FOLD -->|"否"| FOLDF["使用 .fold()"]
    FOLD -->|"是"| LOOP

    style PIPE fill:#d4efdf,stroke:#27ae60,color:#000
    style COMB fill:#d4efdf,stroke:#27ae60,color:#000
    style BUILTIN fill:#d4efdf,stroke:#27ae60,color:#000
    style FOLDF fill:#d4efdf,stroke:#27ae60,color:#000
    style LOOP fill:#fef9e7,stroke:#f1c40f,color:#000

边栏:作用域可变性 —— 内部指令式,外部函数式

Rust 的代码块也是表达式。这允许你将可变性限制在构建阶段,并将其结果绑定为不可变:

#![allow(unused)]
fn main() {
use rand::random;

let samples = {
    let mut buf = Vec::with_capacity(10);
    while buf.len() < 10 {
        let reading: f64 = random();
        buf.push(reading);
        if random::<u8>() % 3 == 0 { break; } // 随机提前停止
    }
    buf
};
// samples 是不可变的 —— 包含 1 到 10 个元素
}

内部的 buf 仅在代码块内是可变的。一旦块执行完毕并返回结果,外部绑定 samples 就是不可变的,编译器将拒绝后续任何 samples.push(...) 调用。

为什么不使用迭代器链? 你可能会尝试这样做:

#![allow(unused)]
fn main() {
let samples: Vec<f64> = std::iter::from_fn(|| Some(random()))
    .take(10)
    .take_while(|_| random::<u8>() % 3 != 0)
    .collect();
}

但 take_while 会 排除掉 那个导致谓词失败的元素,导致结果可能包含 0 到 9 个元素,而不是指令式版本所能保证的“至少一个”。你可以通过 scan 或 chain 来规避这个问题,但指令式版本显然更清晰。

作用域可变性真正胜出的场景:

场景为什么迭代器难以处理
先排序后冷冻 (Sort-then-freeze) (sort_unstable() + dedup())这两个方法都返回 () —— 无法链式输出 (Itertools 提供了 .sorted().dedup(),如果可用的话)
有状态终止 (基于与数据无关的代码停止)take_while 会丢失边界元素
多步骤结构体填充 (从不同数据源逐个字段填充)没有天然的单一流水线可用

实际建议: 对于大多数集合构建任务,优先考虑迭代器链或 itertools。当构建逻辑包含分支、早期退出或不适用于单一流水线的就地修改时,请使用作用域可变性。这种模式的真正价值在于:它教会了我们“可变性的作用域可以小于变量的生命周期” —— 这是 Rust 的一项基本原则,往往会让习惯了 C++、C# 和 Python 的开发者感到惊讶。


8.4 ? 运算符:函数式与指令式的交汇点

? 运算符是 Rust 对两种风格最优雅的综合应用。它在本质上是 .and_then() 加上“早期返回”的结合:

#![allow(unused)]
fn main() {
// 这一串 and_then 链...
fn load_config() -> Result<Config, Error> {
    read_file("config.toml")
        .and_then(|contents| parse_toml(&contents))
        .and_then(|table| validate_config(table))
        .and_then(|valid| Config::from_validated(valid))
}

// ...与这个完全等价
fn load_config() -> Result<Config, Error> {
    let contents = read_file("config.toml")?;
    let table = parse_toml(&contents)?;
    let valid = validate_config(table)?;
    Config::from_validated(valid)
}
}

两者在精神上都是函数式的(它们自动传播错误),但 ? 版本为你提供了 命名的中间变量,这在以下情况非常重要:

  • 你稍后需要再次使用 contents。
  • 你想在每一步添加 .context("解析配置时")? (使用 anyhow/eyre 等库)。
  • 你正在调试并希望能观察中间值。

反模式: 在可以使用 ? 的地方强行使用长链条的 .and_then()。如果链条中的每个闭包都只是 |x| next_step(x),你就是在牺牲可读性的情况下重新发明了 ?。

什么时候 .and_then() 优于 ?:

#![allow(unused)]
fn main() {
// 在 Option 内部进行转换,不涉及早期返回
let port: Option<u16> = config.get("port")
    .and_then(|v| v.parse::<u16>().ok())
    .filter(|&p| p > 0 && p < 65535);
}

你不能在这里使用 ?,因为没有外部函数可以返回 —— 你是在构建一个 Option,而不是传播它。


8.5 集合构建:collect() vs. Push 循环

collect() 的强大超出了大多数开发者的想象:

收集 (Collect) 到 Result 中

#![allow(unused)]
fn main() {
// 指令式:解析列表,在遇到第一个错误时失败
let mut numbers = Vec::new();
for s in input_strings {
    let n: i64 = s.parse().map_err(|_| Error::BadInput(s.clone()))?;
    numbers.push(n);
}

// 函数式:收集到 Result<Vec<_>, _> 中
let numbers: Vec<i64> = input_strings.iter()
    .map(|s| s.parse::<i64>().map_err(|_| Error::BadInput(s.clone())))
    .collect::<Result<_, _>>()?;
}

collect::<Result<Vec<_>, _>>() 的技巧之所以有效,是因为 Result 实现了 FromIterator。它会在遇到第一个 Err 时短路,就像带 ? 的循环一样。

收集到 HashMap 中

#![allow(unused)]
fn main() {
// 指令式
let mut index = HashMap::new();
for server in fleet {
    index.insert(server.id.clone(), server);
}

// 函数式
let index: HashMap<_, _> = fleet.into_iter()
    .map(|s| (s.id.clone(), s))
    .collect();
}

收集到 String 中

#![allow(unused)]
fn main() {
// 指令式
let mut csv = String::new();
for (i, field) in fields.iter().enumerate() {
    if i > 0 { csv.push(','); }
    csv.push_str(field);
}

// 函数式
let csv = fields.join(",");

// 或者进行更复杂的格式化:
let csv: String = fields.iter()
    .map(|f| format!("\"{f}\""))
    .collect::<Vec<_>>()
    .join(",");
}

什么时候循环版本胜出

collect() 会分配新的集合。如果你是在 就地 (In-place) 修改,那么循环既更清晰也更高效:

#![allow(unused)]
fn main() {
// 就地更新 —— 没有更好的函数式等价写法
for server in &mut fleet {
    if server.needs_refresh() {
        server.refresh_telemetry()?;
    }
}
}

函数式版本需要写成 .iter_mut().for_each(|s| { ... }),这本质上只是多套了一层语法的循环。


8.6 将模式匹配作为函数分发 (Function Dispatch)

Rust 的 match 往往被开发者以指令式的方式使用,但它其实是一个函数式构建块。以下是函数式的视角:

将 Match 作为查找表

#![allow(unused)]
fn main() {
// 指令式思维:“检查每一种情况”
fn status_message(code: StatusCode) -> &'static str {
    if code == StatusCode::OK { "成功" }
    else if code == StatusCode::NOT_FOUND { "未找到" }
    else if code == StatusCode::INTERNAL { "服务器错误" }
    else { "未知" }
}

// 函数式思维:“从定义域映射到值域”
fn status_message(code: StatusCode) -> &'static str {
    match code {
        StatusCode::OK => "成功",
        StatusCode::NOT_FOUND => "未找到",
        StatusCode::INTERNAL => "服务器错误",
        _ => "未知",
    }
}
}

match 版本不仅是风格问题 —— 编译器还会验证 完备性 (Exhaustiveness)。如果增加了一个新的变体,每个未处理该变体的 match 都会导致编译错误;而 if/else 链则会静默地进入默认分支。

Match + 解构作为流水线

#![allow(unused)]
fn main() {
// 解析命令 —— 每个分支提取并转换数据
fn execute(cmd: Command) -> Result<Response, Error> {
    match cmd {
        Command::Get { key } => db.get(&key).map(Response::Value),
        Command::Set { key, value } => db.set(key, value).map(|_| Response::Ok),
        Command::Delete { key } => db.delete(&key).map(|_| Response::Ok),
        Command::Batch(cmds) => cmds.into_iter()
            .map(execute)
            .collect::<Result<Vec<_>, _>>()
            .map(Response::Batch),
    }
}
}

每个分支都是一个返回相同类型的表达式。这就是作为函数分发的模式匹配 —— match 分支本质上是一个由枚举变体索引的函数表。


8.7 在自定义类型上使用链式方法

函数式风格不仅限于标准库类型。构建者模式 (Builder patterns) 和流式 API (Fluent APIs) 其实都是伪装的函数式编程:

#![allow(unused)]
fn main() {
// 这是一个基于你自定义类型的组合子链
let query = QueryBuilder::new("servers")
    .filter("status", Eq, "active")
    .filter("rack", In, &["A1", "A2", "B1"])
    .order_by("temperature", Desc)
    .limit(50)
    .build();
}

关键洞察: 如果你的类型具有获取 self 并返回 Self (或转换后的类型) 的方法,你就是在构建一个组合子。同样的“函数式 vs 指令式”判断准则依然适用:

#![allow(unused)]
fn main() {
// 好的写法:可以链式调用,因为每一步都是简单的转换
let config = Config::default()
    .with_timeout(Duration::from_secs(30))
    .with_retries(3)
    .with_tls(true);

// 坏的写法:虽然可以链式调用,但链条做了太多不相关的事情
let result = processor
    .load_data(path)?       // I/O
    .validate()             // 纯函数 (Pure)
    .transform(rule_set)    // 纯函数 (Pure)
    .save_to_disk(output)?  // I/O
    .notify_downstream()?;  // 副作用

// 更好的写法:将纯逻辑流水线与 I/O 边界分开
let data = load_data(path)?;
let processed = data.validate().transform(rule_set);
save_to_disk(output, &processed)?;
notify_downstream()?;
}

当链条混合了纯转换与 I/O 时,它就失效了。读者无法区分哪些调用可能失败、哪些有副作用,以及真正的转化逻辑发生在何处。


8.8 性能:两者是等同的

一个常见的误区是:“函数式风格更慢,因为有大量的闭包和内存分配”。

在 Rust 中,迭代器链编译成的机器码与手写循环完全相同。LLVM 会内联闭包调用,消除迭代器适配器结构体,并经常产生完全一致的汇编代码。这被称为 零成本抽象 (Zero-cost abstraction),它不是一种愿景,而是实实在在的测评结果。

#![allow(unused)]
fn main() {
// 在 Release 构建下,这两者产生的汇编代码完全一致:

// 函数式
let sum: i64 = (0..1000).filter(|n| n % 2 == 0).map(|n| n * n).sum();

// 指令式
let mut sum: i64 = 0;
for n in 0..1000 {
    if n % 2 == 0 {
        sum += n * n;
    }
}
}

唯一一个例外: .collect() 会分配内存。如果你链式调用 .map().collect().iter().map().collect() 并产生了中间集合,你就在为循环版本中不需要的内存分配买单。解决方法:通过直接链接适配器来消除中间的 collect;或者如果确实因为某些原因需要中间集合,则使用循环。


8.9 品味测试:转换目录

以下是针对最常见的“我写了 6 行但其实单行就能搞定”模式的参考表:

指令式模式函数式等价写法何时优先选择函数式
if let Some(x) = opt { f(x) } else { default }opt.map_or(default, f)两端都是短表达式时
if let Some(x) = opt { Some(g(x)) } else { None }opt.map(g)始终如此 —— 这就是 map 的用途
if condition { Some(x) } else { None }condition.then_some(x)始终如此
if condition { Some(compute()) } else { None }condition.then(compute)始终如此
match opt { Some(x) if pred(x) => Some(x), _ => None }opt.filter(pred)始终如此
for x in iter { if pred(x) { result.push(f(x)); } }iter.filter(pred).map(f).collect()当流水线能在一屏内读完时
if a.is_some() && b.is_some() { Some((a?, b?)) }a.zip(b)始终如此 —— .zip() 正是为此设计的
match (a, b) { (Some(x), Some(y)) => x + y, _ => 0 }`a.zip(b).map((x,y)
iter.map(f).collect::<Vec<_>>()[0]iter.map(f).next().unwrap()不要为了一个元素分配 Vec
let mut v = vec; v.sort(); v{ let mut v = vec; v.sort(); v }Rust std 中没有 .sorted() (请使用 itertools)

8.10 反模式 (Anti-Patterns)

过度函数化:没人能读懂的 5 层深度链条

#![allow(unused)]
fn main() {
// 这并不优雅。这是一个谜题。
let result = data.iter()
    .filter_map(|x| x.metadata.as_ref())
    .flat_map(|m| m.tags.iter())
    .filter(|t| t.starts_with("env:"))
    .map(|t| t.strip_prefix("env:").unwrap())
    .filter(|env| allowed_envs.contains(env))
    .map(|env| env.to_uppercase())
    .collect::<HashSet<_>>()
    .into_iter()
    .sorted()
    .collect::<Vec<_>>();
}

当链条超过 ~4 个适配器时,请使用有命名的中间变量将其拆分,或者提取一个辅助函数:

#![allow(unused)]
fn main() {
let env_tags = data.iter()
    .filter_map(|x| x.metadata.as_ref())
    .flat_map(|m| m.tags.iter());

let allowed: Vec<_> = env_tags
    .filter_map(|t| t.strip_prefix("env:"))
    .filter(|env| allowed_envs.contains(env))
    .map(|env| env.to_uppercase())
    .sorted()
    .collect();
}

函数化不足:明明有现成工具却在写 C 风格循环

#![allow(unused)]
fn main() {
// 这其实就是 .any()
let mut found = false;
for item in &list {
    if item.is_expired() {
        found = true;
        break;
    }
}

// 应该写成这样
let found = list.iter().any(|item| item.is_expired());
}
#![allow(unused)]
fn main() {
// 这其实就是 .find()
let mut target = None;
for server in &fleet {
    if server.id == target_id {
        target = Some(server);
        break;
    }
}

// 应该写成这样
let target = fleet.iter().find(|s| s.id == target_id);
}
#![allow(unused)]
fn main() {
// 这其实就是 .all()
let mut all_healthy = true;
for server in &fleet {
    if !server.is_healthy() {
        all_healthy = false;
        break;
    }
}

// 应该写成这样
let all_healthy = fleet.iter().all(|s| s.is_healthy());
}

标准库提供这些工具是有原因的。学会这些术语,代码模式就会变得显而易见。


关键要点

  • Option 和 Result 是包含一个元素的集合。 它们的组合子 (.map(), .and_then(), .unwrap_or_else(), .filter(), .zip()) 可以取代大部分 if let / match 的模板代码。
  • 使用 bool::then_some() —— 在任何情况下,它都能取代 if cond { Some(x) } else { None }。
  • 数据流水线优先考虑迭代器链 —— 具有零可变状态的 filter/map/collect。它们在编译后与手写循环的机器码性能一致。
  • 多输出状态机优先考虑循环 —— 当你需要构建多个集合、在分支中执行 I/O 或管理状态转换时。
  • ? 运算符是两全其美的选择 —— 既有函数式的错误传播,又有指令式的可读性。
  • 在适配器超过 ~4 个时拆分链条 —— 请使用有命名的中间变量以提高可读性。过度函数化与函数化不足同样糟糕。
  • 学习标准库中的术语 —— .any(), .all(), .find(), .position(), .sum(), .min_by_key() —— 每一个都能将多行循环替换为一个能够揭示意图的调用。

另请参阅: 第 7 章 关于闭包机制和 Fn 特性层级。第 10 章 关于错误组合子模式。第 15 章 关于流式 API 设计。


练习:将指令式重构为函数式 ★★ (~30 分钟)

将以下函数从指令式风格重构为函数式风格。然后,找出函数式版本比原版 更糟 的一个地方,并解释原因。

#![allow(unused)]
fn main() {
fn summarize_fleet(fleet: &[Server]) -> FleetSummary {
    let mut healthy = Vec::new();
    let mut degraded = Vec::new();
    let mut failed = Vec::new();
    let mut total_power = 0.0;
    let mut max_temp = f64::NEG_INFINITY;

    for server in fleet {
        match server.health_status() {
            Health::Healthy => healthy.push(server.id.clone()),
            Health::Degraded(reason) => degraded.push((server.id.clone(), reason)),
            Health::Failed(err) => failed.push((server.id.clone(), err)),
        }
        total_power += server.power_draw();
        if server.max_temperature() > max_temp {
            max_temp = server.max_temperature();
        }
    }

    FleetSummary {
        healthy,
        degraded,
        failed,
        avg_power: total_power / fleet.len() as f64,
        max_temp,
    }
}
}
🔑 参考答案

total_power 和 max_temp 可以非常整洁地改写为函数式:

#![allow(unused)]
fn main() {
fn summarize_fleet(fleet: &[Server]) -> FleetSummary {
    let avg_power: f64 = fleet.iter().map(|s| s.power_draw()).sum::<f64>()
        / fleet.len() as f64;

    let max_temp = fleet.iter()
        .map(|s| s.max_temperature())
        .fold(f64::NEG_INFINITY, f64::max);

    // 但是,三路分区 (three-way partition) 最好还是用循环。
    // 函数式版本将需要三次独立的遍历,
    // 或者在一个笨拙的 fold 中使用三个可变累加器。
    let mut healthy = Vec::new();
    let mut degraded = Vec::new();
    let mut failed = Vec::new();

    for server in fleet {
        match server.health_status() {
            Health::Healthy => healthy.push(server.id.clone()),
            Health::Degraded(reason) => degraded.push((server.id.clone(), reason)),
            Health::Failed(err) => failed.push((server.id.clone(), err)),
        }
    }

    FleetSummary { healthy, degraded, failed, avg_power, max_temp }
}
}

为什么对于三路分区来说循环更好: 函数式版本要么需要三次 .filter().collect() (3 倍的遍历开销),要么需要在一个 .fold() 中处理包含三个 mut Vec 累加器的元祖 —— 这本质上只是换了一种更糟糕的语法的循环。指令式的单次遍历循环更清晰、更高效,也更容易扩展。


English Original

第 9 章:智能指针 (Smart Pointers) 与内部可变性 (Interior Mutability) 🟡

你将学到:

  • 使用 Box, Rc, Arc 进行堆分配与共享所有权
  • 使用弱引用 (Weak references) 打破 Rc/Arc 的引用循环
  • Cell, RefCell 和 Cow 等内部可变性模式
  • 用于自引用类型的 Pin,以及用于生命周期控制的 ManuallyDrop

Box, Rc, Arc —— 堆分配与共享

#![allow(unused)]
fn main() {
// --- Box<T>:单一所有者,堆分配 ---
// 适用场景:递归类型、大型数值、Trait 对象 (Trait objects)
let boxed: Box<i32> = Box::new(42);
println!("{}", *boxed); // 解引用为 i32

// 递归类型需要 Box (否则大小将是无限的):
enum List<T> {
    Cons(T, Box<List<T>>),
    Nil,
}

// Trait 对象 (动态分发):
let writer: Box<dyn std::io::Write> = Box::new(std::io::stdout());

// --- Rc<T>:多所有者,单线程 ---
// 适用场景:在单个线程内的共享所有权 (未实现 Send/Sync)
use std::rc::Rc;

let a = Rc::new(vec![1, 2, 3]);
let b = Rc::clone(&a); // 增加引用计数 (并非深拷贝)
let c = Rc::clone(&a);
println!("引用计数: {}", Rc::strong_count(&a)); // 3

// 三个指针都指向同一个 Vec。当最后一个 Rc 被 drop 时,
// 该 Vec 内存将被释放。

// --- Arc<T>:多所有者,线程安全 ---
// 适用场景:跨线程的共享所有权
use std::sync::Arc;

let shared = Arc::new(String::from("共享数据"));
let handles: Vec<_> = (0..5).map(|_| {
    let shared = Arc::clone(&shared);
    std::thread::spawn(move || println!("{shared}"))
}).collect();
for h in handles { h.join().unwrap(); }
}

弱引用 —— 打破引用循环

Rc 和 Arc 使用的是引用计数,无法自动释放循环引用 (A → B → A)。 Weak<T> 是一种不持有所有权的句柄,它 不会 增加强引用计数:

#![allow(unused)]
fn main() {
use std::rc::{Rc, Weak};
use std::cell::RefCell;

struct Node {
    value: i32,
    parent: RefCell<Weak<Node>>,   // 不会让父节点保持存活
    children: RefCell<Vec<Rc<Node>>>,
}

let parent = Rc::new(Node {
    value: 0, parent: RefCell::new(Weak::new()), children: RefCell::new(vec![]),
});
let child = Rc::new(Node {
    value: 1, parent: RefCell::new(Rc::downgrade(&parent)), children: RefCell::new(vec![]),
});
parent.children.borrow_mut().push(Rc::clone(&child));

// 从子节点访问父节点 —— 返回 Option<Rc<Node>>:
if let Some(p) = child.parent.borrow().upgrade() {
    println!("子节点的父节点值: {}", p.value); // 0
}
// 当 `parent` 被 drop 时,其强引用计数强行归零,内存释放。
// 此时 `child.parent.upgrade()` 将返回 `None`。
}

经验法则:对于所有权关系的边使用 Rc/Arc,对于回溯引用 (Back-references) 和缓存使用 Weak。在线程安全的代码中,请组合使用 Arc<T> 与 sync::Weak<T>。

Cell 与 RefCell —— 内部可变性

有时你需要在共享 (&) 引用之后修改数据。Rust 通过“内部可变性 (Interior Mutability)”及运行时借用检查提供了这一功能:

#![allow(unused)]
fn main() {
use std::cell::{Cell, RefCell};

// --- Cell<T>:基于复制 (Copy-based) 的内部可变性 ---
// 仅适用于 Copy 类型 (或支持 swap/replace 的类型)
struct Counter {
    count: Cell<u32>,
}

impl Counter {
    fn new() -> Self { Counter { count: Cell::new(0) } }

    fn increment(&self) { // 注意是 &self,而非 &mut self!
        self.count.set(self.count.get() + 1);
    }

    fn value(&self) -> u32 { self.count.get() }
}

// --- RefCell<T>:运行时借用检查 ---
// 如果你在运行时违反借用规则,程序将发生 Panic
struct Cache {
    data: RefCell<Vec<String>>,
}

impl Cache {
    fn new() -> Self { Cache { data: RefCell::new(Vec::new()) } }

    fn add(&self, item: String) { // &self —— 从外部看是不可变的
        self.data.borrow_mut().push(item); // 运行时检查的 &mut
    }

    fn get_all(&self) -> Vec<String> {
        self.data.borrow().clone() // 运行时检查的 &
    }

    fn bad_example(&self) {
        let _guard1 = self.data.borrow();
        // let _guard2 = self.data.borrow_mut();
        // ❌ 运行时 Panic —— 无法在持有 & 的同时获取 &mut
    }
}
}

Cell vs RefCell:Cell 永远不会发生 Panic (它通过拷贝或交换值来工作),但它仅适用于 Copy 类型,或需要通过 swap()/replace() 操作。RefCell 适用于任何类型,但在发生双重可变借用时会 Panic。两者都不是 Sync 的 —— 如需在多线程中使用,请参考 Mutex/RwLock。

Cow —— 写时复制 (Clone on Write)

Cow (Clone on Write) 持有一个借用或拥有的值。它 仅在需要修改时 才会执行克隆:

use std::borrow::Cow;

// 当不需要修改时,避免内存分配:
fn normalize(input: &str) -> Cow<'_, str> {
    if input.contains('\t') {
        // 仅在需要替换制表符时执行分配 (Allocate)
        Cow::Owned(input.replace('\t', "    "))
    } else {
        // 不分配执行分配 —— 直接返回引用
        Cow::Borrowed(input)
    }
}

fn main() {
    let clean = "no tabs here";
    let dirty = "tabs\there";

    let r1 = normalize(clean); // Cow::Borrowed —— 零分配
    let r2 = normalize(dirty); // Cow::Owned —— 分配了新的 String

    println!("{r1}");
    println!("{r2}");
}

// 对于“可能需要所有权”的函数参数也非常有用:
fn process(data: Cow<'_, [u8]>) {
    // 无需拷贝即可读取数据
    println!("长度: {}", data.len());
    // 如果需要修改,Cow 会自动执行克隆:
    let mut owned = data.into_owned(); // 仅当它是 Borrowed 时克隆
    owned.push(0xFF);
}

用于二进制数据的 Cow<'_, [u8]>

对于可能需要也可能不需要转换(如校验和插入、填充、转义)的字节导向型 API,Cow 特别有用。这可以避免在通用的快速路径 (Fast path) 上分配 Vec<u8>:

#![allow(unused)]
fn main() {
use std::borrow::Cow;

/// 将帧填充到最小长度,若不需要填充则借用。
fn pad_frame(frame: &[u8], min_len: usize) -> Cow<'_, [u8]> {
    if frame.len() >= min_len {
        Cow::Borrowed(frame)  // 长度已经足够 —— 零分配
    } else {
        let mut padded = frame.to_vec();
        padded.resize(min_len, 0x00);
        Cow::Owned(padded)    // 仅在需要填充时分配
    }
}

let short = pad_frame(&[0xDE, 0xAD], 8);    // Owned —— 已填充至 8 字节
let long  = pad_frame(&[0; 64], 8);          // Borrowed —— 已经 ≥ 8
}

提示:当你需要引用计数式的共享,且可能涉及转换后的缓冲区时,请配合使用 Cow<[u8]> 与 bytes::Bytes (见第 10 章)。


何时使用哪种指针

指针所有者数量线程安全可变性适用场景
Box<T>1✅ (若 T: Send)通过 &mut堆分配、Trait 对象、递归类型
Rc<T>N❌无 (需包裹在 Cell/RefCell 中)共享所有权、单线程、图/树结构
Arc<T>N✅无 (需包裹在 Mutex/RwLock 中)跨线程共享所有权
Cell<T>—❌.get() / .set()Copy 类型的内部可变性
RefCell<T>—❌.borrow() / .borrow_mut()任意类型的内部可变性,单线程
Cow<'_, T>0 或 1✅ (若 T: Send)写时复制避免在数据通常不改变时进行分配

Pin 与自引用类型

Pin<P> 能够防止一个值在内存中被移动。这对于 自引用类型 (Self-referential types) —— 即包含指向自身数据指针的结构体 —— 以及 Future 来说至关重要,因为 Future 可能会在跨越 .await 点时持有引用。

use std::pin::Pin;
use std::marker::PhantomPinned;

// 一个简单的自引用结构体示例:
struct SelfRef {
    data: String,
    ptr: *const String, // 指向本结构体中的 `data` 字段
    _pin: PhantomPinned, // 退出 Unpin —— 禁止移动该结构体
}

impl SelfRef {
    fn new(s: &str) -> Pin<Box<Self>> {
        let val = SelfRef {
            data: s.to_string(),
            ptr: std::ptr::null(),
            _pin: PhantomPinned,
        };
        let mut boxed = Box::pin(val);

        // 安全性 (SAFETY):在设置指针后,我们不再移动数据
        let self_ptr: *const String = &boxed.data;
        unsafe {
            let mut_ref = Pin::as_mut(&mut boxed);
            Pin::get_unchecked_mut(mut_ref).ptr = self_ptr;
        }
        boxed
    }

    fn data(&self) -> &str {
        &self.data
    }

    fn ptr_data(&self) -> &str {
        // 安全性 (SAFETY):ptr 在固定 (Pinned) 后被设置为指向 self.data
        unsafe { &*self.ptr }
    }
}

fn main() {
    let pinned = SelfRef::new("hello");
    assert_eq!(pinned.data(), pinned.ptr_data()); // 均为 "hello"
    // std::mem::swap 会使 ptr 失效 —— 但 Pin 防止了移动操作
}

核心概念:

概念含义
Unpin (自动 Trait)“移动该类型是安全的。” 大多数类型默认都是 Unpin。
!Unpin / PhantomPinned“我有内部指针 —— 请不要移动我。”
Pin<&mut T>一个保证 T 不会被移动的可变引用。
Pin<Box<T>>一个在堆上固定的、拥有所有权的值。

为什么这在异步中很重要: 每一个 async fn 都会被脱糖 (Desugars) 为一个 Future,它可能跨 .await 点持有引用 —— 这使它变成了自引用的。异步运行时使用 Pin<&mut Future> 来保证 Future 在被轮询 (Polled) 后不会被移动。

#![allow(unused)]
fn main() {
// 当你写下:
async fn fetch(url: &str) -> String {
    let response = http_get(url).await; // 引用被跨 await 持有
    response.text().await
}

// 编译器会生成一个实现 !Unpin 的状态机结构体,
// 运行时在调用 Future::poll() 之前会将其固定。
}

什么时候需要关心 Pin: (1) 手动实现 Future 时,(2) 编写异步运行时或组合子时,(3) 任何带有自引用指针的结构体。对于一般的应用层代码,async/await 会透明地处理固定 (Pinning)。参见配套的《异步 Rust 训练》获取更深入的覆盖。

替代库: 对于无需手动 Pin 的自引用结构体,可以考虑使用 ouroboros 或 self_cell —— 它们会生成具有正确固定和 drop 语义的安全包装。


Pin 投影 —— 结构性固定 (Structural Pinning)

当你持有 Pin<&mut MyStruct> 时,通常需要访问其单个字段。Pin 投影 (Pin projection) 是一种从 Pin<&mut Struct> 安全地转换到 Pin<&mut Field> (针对被固定的字段) 或 &mut Field (针对未固定的字段) 的模式。

示例:被固定类型中的字段访问

#![allow(unused)]
fn main() {
use std::pin::Pin;
use std::marker::PhantomPinned;

struct MyFuture {
    data: String,              // 普通字段 —— 移动是安全的
    state: InternalState,      // 自引用字段 —— 必须保持固定
    _pin: PhantomPinned,
}

enum InternalState {
    Waiting { ptr: *const String }, // 指向 `data` —— 自引用的
    Done,
}

// 给定 `Pin<&mut MyFuture>`,你该如何访问 `data` 和 `state`?
// 你不能直接通过 `pinned.data` 来访问 —— 
// 编译器不允许你在不使用 unsafe 的情况下获取固定值的字段引用。
}

手动 Pin 投影 (使用 unsafe)

#![allow(unused)]
fn main() {
impl MyFuture {
    // 投影到 `data` —— 该字段在结构上未固定 (移动是安全的)
    fn data(self: Pin<&mut Self>) -> &mut String {
        // 安全性 (SAFETY):`data` 未被结构性固定。仅移动 `data` 
        // 并不移动整个结构体,因此 Pin 的保证依然有效。
        unsafe { &mut self.get_unchecked_mut().data }
    }

    // 投影到 `state` —— 该字段在结构上是固定的 (Structurally pinned)
    fn state(self: Pin<&mut Self>) -> Pin<&mut InternalState> {
        // 安全性 (SAFETY):`state` 是结构性固定的 —— 我们通过
        // 返回 Pin<&mut InternalState> 来维持 Pin 的不变性。
        unsafe { Pin::new_unchecked(&mut self.get_unchecked_mut().state) }
    }
}
}

结构性固定 (Structural Pinning) 规则 —— 一个字段在以下情况是“结构性固定”的:

  1. 仅移动或交换该字段就可能使自引用失效。
  2. 结构体的 Drop 实现必须保证不移动该字段。
  3. 结构体必须是 !Unpin 的 (通过 PhantomPinned 或 !Unpin 字段强制执行)。

pin-project —— 安全的 Pin 投影 (零 Unsafe)

pin-project 库在编译时生成可证明正确的投影,消除了手动使用 unsafe 的需求:

#![allow(unused)]
fn main() {
use pin_project::pin_project;
use std::pin::Pin;
use std::future::Future;
use std::task::{Context, Poll};

#[pin_project]                   // <-- 生成投影方法
struct TimedFuture<F: Future> {
    #[pin]                       // <-- 结构性固定 (因为它是一个 Future)
    inner: F,
    started_at: std::time::Instant, // 未固定 —— 普通数据
}

impl<F: Future> Future for TimedFuture<F> {
    type Output = (F::Output, std::time::Duration);

    fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
        let this = self.project();  // 安全!由 pin_project 生成
        //   this.inner   : Pin<&mut F>              — 固定字段
        //   this.started_at : &mut std::time::Instant — 未固定字段

        match this.inner.poll(cx) {
            Poll::Ready(output) => {
                let elapsed = this.started_at.elapsed();
                Poll::Ready((output, elapsed))
            }
            Poll::Pending => Poll::Pending,
        }
    }
}
}

pin-project vs 手动投影

维度手动 (unsafe)pin-project
安全性你来自行证明不变性编译器验证
样板代码低 (但容易出错)零样板 —— 通过派生宏
Drop 交互必须保证不移动固定字段强制执行:#[pinned_drop]
编译成本无过程宏展开开销
适用场景原语、no_std 环境应用开发 / 库开发

#[pinned_drop] —— 被固定类型的 Drop

当一个类型包含 #[pin] 字段时,pin-project 要求使用 #[pinned_drop] 代替常规的 Drop 实现,以防止意外移动固定的字段:

#![allow(unused)]
fn main() {
use pin_project::{pin_project, pinned_drop};
use std::pin::Pin;

#[pin_project(PinnedDrop)]
struct Connection<F> {
    #[pin]
    future: F,
    buffer: Vec<u8>,  // 未固定 —— 可以在 drop 中移动
}

#[pinned_drop]
impl<F> PinnedDrop for Connection<F> {
    fn drop(self: Pin<&mut Self>) {
        let this = self.project();
        // `this.future` 是 Pin<&mut F> —— 无法移动,只能就地析构
        // `this.buffer` 是 &mut Vec<u8> —— 可以执行 drain、clear 等操作
        this.buffer.clear();
        println!("连接已关闭,缓冲区已清空");
    }
}
}

何时在实践中需要 Pin 投影

graph TD
    A["你是否需要手动实现 Future?"] -->|是| B["该 Future 是否跨 .await 点<br/>持有引用?"]
    A -->|否| C["async/await 会为你处理 Pin<br/>✅ 无需投影"]
    B -->|是| D["在你的 Future 结构体上<br/>使用 #[pin_project]"]
    B -->|否| E["你的 Future 是 Unpin 的<br/>✅ 无需投影"]
    D --> F["将内部 Future/Stream 标记为 #[pin]<br/>数据字段保持未固定状态"]
    
    style C fill:#91e5a3,color:#000
    style E fill:#91e5a3,color:#000
    style D fill:#ffa07a,color:#000
    style F fill:#ffa07a,color:#000

经验法则: 如果你是在包装另一个 Future 或 Stream,请使用 pin-project。如果你是在使用 async/await 编写应用程序代码,你永远不需要直接使用 Pin 投影。参见配套的《异步 Rust 训练》中有关使用 Pin 投影的异步组合子模式。


Drop 顺序与 ManuallyDrop

Rust 的 Drop 顺序是确定性的 (Deterministic),但有一些规则值得了解:

Drop 顺序规则

struct Label(&'static str);

impl Drop for Label {
    fn drop(&mut self) { println!("正在释放 {}", self.0); }
}

fn main() {
    let a = Label("第一个");   // 首先声明
    let b = Label("第二个");   // 其次声明
    let c = Label("第三个");   // 最后声明
}
// 输出:
//   正在释放 第三个    ← 局部变量按声明顺序的逆序释放
//   正在释放 第二个
//   正在释放 第一个

三项基本规则:

释放项Drop 顺序原理
局部变量声明顺序的逆序后定义的变量可能引用先定义的变量
结构体字段声明顺序 (从上到下)与构建顺序一致 (自 Rust 1.0 起稳定,由 RFC 1857 保证)
元组元素声明顺序 (从左到右)(a, b, c) → 先释放 a,再 b,最后 c
#![allow(unused)]
fn main() {
struct Server {
    listener: Label,  // 第 1 个释放
    handler: Label,   // 第 2 个释放
    logger: Label,    // 第 3 个释放
}
// 字段按从上到下的声明顺序释放。
// 当字段之间存在相互引用或持有资源依赖时,这一点非常重要。
}

实际影响: 如果你的结构体同时包含 JoinHandle 和 Sender,字段顺序决定了谁先被释放。如果线程正在从通道中读取数据,应先释放 Sender (关闭通道) 以使线程退出,然后再 join 该 handle。因此在结构体中应将 Sender 放在 JoinHandle 之上。

ManuallyDrop<T> —— 抑制自动 Drop

ManuallyDrop<T> 包装一个值并防止其析构函数 (Destructor) 自动运行。你将承担起手动释放它(或故意让其内存泄漏)的责任:

#![allow(unused)]
fn main() {
use std::mem::ManuallyDrop;

// 用例 1:防止 Unsafe 代码中的双重释放 (Double-free)
struct TwoPhaseBuffer {
    // 我们需要自行释放 Vec 以控制时机
    data: ManuallyDrop<Vec<u8>>,
    committed: bool,
}

impl TwoPhaseBuffer {
    fn new(capacity: usize) -> Self {
        TwoPhaseBuffer {
            data: ManuallyDrop::new(Vec::with_capacity(capacity)),
            committed: false,
        }
    }

    fn write(&mut self, bytes: &[u8]) {
        self.data.extend_from_slice(bytes);
    }

    fn commit(&mut self) {
        self.committed = true;
        println!("已提交 {} 字节", self.data.len());
    }
}

impl Drop for TwoPhaseBuffer {
    fn drop(&mut self) {
        if !self.committed {
            println!("正在回滚 —— 释放未提交的数据");
        }
        // 安全性 (SAFETY):由我们保证此处数据有效且仅被释放一次。
        unsafe { ManuallyDrop::drop(&mut self.data); }
    }
}
}
#![allow(unused)]
fn main() {
// 用例 2:故意的内存泄漏 (例如全局单例)
fn leaked_string() -> &'static str {
    // Box::leak() 是创建 &'static 引用的惯用方式:
    let s = String::from("永生不死");
    Box::leak(s.into_boxed_str())
    // ⚠️ 这是一个受控的内存泄漏。该 String 的堆内存永不释放。
    // 仅用于长寿的单例对象。
}

// ManuallyDrop 替代方案 (需要 unsafe):
// ⚠️ 优先使用上面的 Box::leak() —— 此处仅为演示 ManuallyDrop 的语义
// (在堆数据存活的同时抑制 Drop)。
fn leaked_string_manual() -> &'static str {
    use std::mem::ManuallyDrop;
    let md = ManuallyDrop::new(String::from("永生不死"));
    // 安全性 (SAFETY):ManuallyDrop 防止了内存释放;堆数据将永远
    // 存活,因此 &'static 引用是有效的。
    unsafe { &*(md.as_str() as *const str) }
}
}

ManuallyDrop vs mem::forget:

ManuallyDrop<T>mem::forget(value)
时机在构建时包装在稍后消耗
内部访问&*md / &mut *md值已消失,无法访问
稍后释放ManuallyDrop::drop(&mut md)不可行
适用场景细粒度的生命周期控制运行即不管 (Fire-and-forget) 的解构抑制

准则: 仅在需要 精确控制 析构函数运行波次的 Unsafe 抽象中使用 ManuallyDrop。在安全的应用程序代码中,你几乎永远不需要它 —— Rust 的自动 Drop 顺序足以正确处理绝大多数情况。


关键要点 —— 智能指针

  • Box 用于堆上的单一所有权;Rc/Arc 用于共享所有权 (单线程/多线程)。
  • Cell/RefCell 提供内部可变性;RefCell 在运行时检查违规并 Panic。
  • Cow 避免在通用路径上进行内存分配;Pin 防止自引用类型的内存移动。
  • Drop 顺序:字段按声明顺序释放 (RFC 1857);局部变量按声明顺序的逆序释放。

另请参阅: 第 6 章 —— 并发 了解 Arc + Mutex 模式。第 4 章 —— PhantomData 了解与智能指针配合使用的 PhantomData。

graph TD
    Box["Box&lt;T&gt;<br>堆上单一所有者"] --> Heap["堆分配 (Heap)"]
    Rc["Rc&lt;T&gt;<br>共享,单线程"] --> Heap
    Arc["Arc&lt;T&gt;<br>共享,多线程"] --> Heap

    Rc --> Weak1["Weak&lt;T&gt;<br>非所有权句柄"]
    Arc --> Weak2["Weak&lt;T&gt;<br>非所有权句柄"]

    Cell["Cell&lt;T&gt;<br>Copy 内部可变性"] --> Stack["栈 / 内部 (Stack)"]
    RefCell["RefCell&lt;T&gt;<br>运行时借用检查"] --> Stack
    Cow["Cow&lt;T&gt;<br>写时复制"] --> Stack

    style Box fill:#d4efdf,stroke:#27ae60,color:#000
    style Rc fill:#e8f4f8,stroke:#2980b9,color:#000
    style Arc fill:#e8f4f8,stroke:#2980b9,color:#000
    style Weak1 fill:#fef9e7,stroke:#f1c40f,color:#000
    style Weak2 fill:#fef9e7,stroke:#f1c40f,color:#000
    style Cell fill:#fdebd0,stroke:#e67e22,color:#000
    style RefCell fill:#fdebd0,stroke:#e67e22,color:#000
    style Cow fill:#fdebd0,stroke:#e67e22,color:#000
    style Heap fill:#f5f5f5,stroke:#999,color:#000
    style Stack fill:#f5f5f5,stroke:#999,color:#000

练习:引用计数图 (Reference-Counted Graph) ★★ (~30 分钟)

利用 Rc<RefCell<Node>> 构建一个有向图 (Directed graph),其中每个节点都有一个名称和子节点列表。创建一个循环 (A → B → C → A),并使用 Weak 来打破回边 (Back-edge)。通过 Rc::strong_count 验证是否存在内存泄漏。

🔑 参考答案
use std::cell::RefCell;
use std::rc::{Rc, Weak};

struct Node {
    name: String,
    children: Vec<Rc<RefCell<Node>>>,
    back_ref: Option<Weak<RefCell<Node>>>,
}

impl Node {
    fn new(name: &str) -> Rc<RefCell<Self>> {
        Rc::new(RefCell::new(Node {
            name: name.to_string(),
            children: Vec::new(),
            back_ref: None,
        }))
    }
}

impl Drop for Node {
    fn drop(&mut self) {
        println!("正在释放 {}", self.name);
    }
}

fn main() {
    let a = Node::new("A");
    let b = Node::new("B");
    let c = Node::new("C");

    // A → B → C,其中 C 通过 Weak 指向 A 构成的回边
    a.borrow_mut().children.push(Rc::clone(&b));
    b.borrow_mut().children.push(Rc::clone(&c));
    c.borrow_mut().back_ref = Some(Rc::downgrade(&a)); // 弱引用!

    println!("A 的强引用计数: {}", Rc::strong_count(&a)); // 1 (仅变量 `a` 绑定)
    println!("B 的强引用计数: {}", Rc::strong_count(&b)); // 2 (b + A 的子节点)
    println!("C 的强引用计数: {}", Rc::strong_count(&c)); // 2 (c + B 的子节点)

    // 升级弱引用证明其有效:
    let c_ref = c.borrow();
    if let Some(back) = &c_ref.back_ref {
        if let Some(a_ref) = back.upgrade() {
            println!("C 指回了: {}", a_ref.borrow().name);
        }
    }
    // 当 a, b, c 超出作用域时,所有 Node 都会被释放 (没有循环引用导致的泄漏!)
}

English Original

第 10 章:错误处理模式 (Error Handling Patterns) 🟢

你将学到:

  • 何时使用 thiserror (库开发) 与 anyhow (应用程序开发)
  • 处理带有 #[from] 和 .context() 包装器的错误转换链
  • ? 运算符在 main() 中的脱糖 (Desugars) 与工作原理
  • 什么时候应该 panic,什么时候返回错误,以及用于 FFI 边界的 catch_unwind

thiserror vs anyhow —— 库 vs 应用程序

Rust 的错误处理以 Result<T, E> 类型为中心。目前有两个库占据主导地位:

// --- thiserror:适用于“库 (LIBRARIES)” ---
// 通过派生宏生成 Display、Error 和 From 的实现 (impl)
use thiserror::Error;

#[derive(Error, Debug)]
pub enum DatabaseError {
    #[error("连接失败: {0}")]
    ConnectionFailed(String),

    #[error("查询错误: {source}")]
    QueryError {
        #[source]
        source: sqlx::Error,
    },

    #[error("未找到记录: table={table} id={id}")]
    NotFound { table: String, id: u64 },

    #[error(transparent)] // 将 Display 委派 (Delegate) 给内部错误
    Io(#[from] std::io::Error), // 自动生成 From<io::Error>
}

// --- anyhow:适用于“应用程序 (APPLICATIONS)” ---
// 动态错误类型 —— 非常适合你只想让错误向上传播的顶层代码
use anyhow::{Context, Result, bail, ensure};

fn read_config(path: &str) -> Result<Config> {
    let content = std::fs::read_to_string(path)
        .with_context(|| format!("无法从 {path} 读取配置"))?;

    let config: Config = serde_json::from_str(&content)
        .context("解析配置 JSON 失败")?;

    ensure!(config.port > 0, "端口必须为正数, 得到 {}", config.port);

    Ok(config)
}

fn main() -> Result<()> {
    let config = read_config("server.toml")?;

    if config.name.is_empty() {
        bail!("服务器名称不能为空"); // 立即返回 Err
    }

    Ok(())
}

何时使用哪一个:

特性thiserroranyhow
适用范围库、共享的 Crate应用程序、二进制执行文件
错误类型具体枚举 (Concrete enums) —— 调用者可匹配anyhow::Error —— 结构不透明
开发成本需要定义自己的错误枚举直接使用 Result<T> 即可
类型下行转换无需 —— 使用模式匹配需要 error.downcast_ref::<MyError>()

错误转换链 (#[from])

use thiserror::Error;

#[derive(Error, Debug)]
enum AppError {
    #[error("I/O 错误: {0}")]
    Io(#[from] std::io::Error),

    #[error("JSON 错误: {0}")]
    Json(#[from] serde_json::Error),

    #[error("HTTP 错误: {0}")]
    Http(#[from] reqwest::Error),
}

// 现在 `?` 可以自动执行转换:
fn fetch_and_parse(url: &str) -> Result<Config, AppError> {
    let body = reqwest::blocking::get(url)?.text()?;  // reqwest::Error → AppError::Http
    let config: Config = serde_json::from_str(&body)?; // serde_json::Error → AppError::Json
    Ok(config)
}

上下文与错误包装

在不丢失原始错误信息的情况下,为错误添加人类可读的上下文:

use anyhow::{Context, Result};

fn process_file(path: &str) -> Result<Data> {
    let content = std::fs::read_to_string(path)
        .with_context(|| format!("读取 {path} 失败"))?;

    let data = parse_content(&content)
        .with_context(|| format!("解析 {path} 失败"))?;

    validate(&data)
        .context("验证失败")?;

    Ok(data)
}

// 错误输出示例:
// Error: 验证失败
//
// Caused by:
//    0: 解析 config.json 失败
//    1: expected ',' at line 5 column 12

深入理解 ? 运算符

? 是语法糖 (Syntactic sugar),它等同于 match + From 转换 + 早期返回:

#![allow(unused)]
fn main() {
// 这一行:
let value = operation()?;

// 脱糖 (Desugars) 后等同于:
let value = match operation() {
    Ok(v) => v,
    Err(e) => return Err(From::from(e)),
    //                  ^^^^^^^^^^^^^^
    //                  通过 From Trait 执行自动转换
};
}

? 也能用于 Option (在返回 Option 的函数中):

#![allow(unused)]
fn main() {
fn find_user_email(users: &[User], name: &str) -> Option<String> {
    let user = users.iter().find(|u| u.name == name)?; // 若未找到则返回 None
    let email = user.email.as_ref()?; // 若 email 为 None 则返回 None
Some(email.to_uppercase())
}
}

Panic、catch_unwind 以及何时中止 (Abort)

#![allow(unused)]
fn main() {
// Panic:用于处理 BUG,而非预料中的错误
fn get_element(data: &[i32], index: usize) -> &i32 {
    // 如果这里发生了 Panic,说明是一个编程错误 (Bug)。
    // 不要试图“处理”它 —— 应该去修复调用者。
    &data[index]
}

// catch_unwind:用于处理边界逻辑 (如 FFI、线程池)
use std::panic;

let result = panic::catch_unwind(|| {
    // 安全地运行可能发生 Panic 的代码
    risky_operation()
});

match result {
    Ok(value) => println!("成功: {value:?}"),
    Err(_) => eprintln!("操作发生了 Panic —— 正在安全地继续运行"),
}

// 何时使用哪种方式:
// - Result<T, E>    → 预料中的失败 (如:文件未找到、网络超时)
// - panic!()        → 编程上的 Bug (如:索引越界、违反了不变性)
// - process::abort() → 不可恢复的状态 (如:安全性违规、数据损坏)
}

与 C++ 的比较:对于预料中的错误,Result<T, E> 替代了异常 (Exceptions)。panic!() 类似于 assert() 或 std::terminate() —— 它用于处理 Bug,而非控制流。Rust 的 ? 运算符使错误传播变得如异常般符合人体工程学 (Ergonomic),且没有不可预测的控制流。


关键要点 —— 错误处理

  • 库开发:使用 thiserror 定义结构化的错误枚举;应用程序开发:使用 anyhow 实现符合人体工程学的错误传播。
  • #[from] 自动生成 From 实现;.context() 添加人类可读的包装层。
  • ? 会被脱糖为 From::from() + 早期返回;它在返回 Result 的 main() 函数中同样有效。

另请参阅: 第 15 章 —— API 设计 了解“解析而非验证 (Parse, don’t validate)”模式。第 11 章 —— 序列化 了解 Serde 的错误处理。

flowchart LR
    A["std::io::Error"] -->|"#[from]"| B["AppError::Io"]
    C["serde_json::Error"] -->|"#[from]"| D["AppError::Json"]
    E["自定义验证"] -->|"手动"| F["AppError::Validation"]

    B --> G["? 运算符"]
    D --> G
    F --> G
    G --> H["Result&lt;T, AppError&gt;"]

    style A fill:#e8f4f8,stroke:#2980b9,color:#000
    style C fill:#e8f4f8,stroke:#2980b9,color:#000
    style E fill:#e8f4f8,stroke:#2980b9,color:#000
    style B fill:#fdebd0,stroke:#e67e22,color:#000
    style D fill:#fdebd0,stroke:#e67e22,color:#000
    style F fill:#fdebd0,stroke:#e67e22,color:#000
    style G fill:#fef9e7,stroke:#f1c40f,color:#000
    style H fill:#d4efdf,stroke:#27ae60,color:#000

练习:使用 thiserror 构建错误层级 ★★ (~30 分钟)

为一个文件处理应用程序设计错误层级结构。该程序可能在 I/O、解析 (JSON 与 CSV) 以及验证过程中失败。请使用 thiserror 并演示 ? 传播。

🔑 参考答案
use thiserror::Error;

#[derive(Error, Debug)]
pub enum AppError {
    #[error("I/O 错误: {0}")]
    Io(#[from] std::io::Error),

    #[error("JSON 解析错误: {0}")]
    Json(#[from] serde_json::Error),

    #[error("CSV 错误 (行 {line}): {message}")]
    Csv { line: usize, message: String },

    #[error("验证错误: {field} —— {reason}")]
    Validation { field: String, reason: String },
}

fn read_file(path: &str) -> Result<String, AppError> {
    Ok(std::fs::read_to_string(path)?) // 通过 #[from] 将 io::Error 转换为 AppError::Io
}

fn parse_json(content: &str) -> Result<serde_json::Value, AppError> {
    Ok(serde_json::from_str(content)?) // 将 serde_json::Error 转换为 AppError::Json
}

fn validate_name(value: &serde_json::Value) -> Result<String, AppError> {
    let name = value.get("name")
        .and_then(|v| v.as_str())
        .ok_or_else(|| AppError::Validation {
            field: "name".into(),
            reason: "必须是非空字符串".into(),
        })?;

    if name.is_empty() {
        return Err(AppError::Validation {
            field: "name".into(),
            reason: "不能为空".into(),
        });
    }

    Ok(name.to_string())
}

fn process_file(path: &str) -> Result<String, AppError> {
    let content = read_file(path)?;
    let json = parse_json(&content)?;
    let name = validate_name(&json)?;
    Ok(name)
}

fn main() {
    match process_file("config.json") {
        Ok(name) => println!("名称: {name}"),
        Err(e) => eprintln!("错误: {e}"),
    }
}

English Original

第 11 章:序列化、零拷贝与二进制数据 🟡

你将学到:

  • serde 基础知识:派生宏、属性与枚举表示形式。
  • 零拷贝 (Zero-copy) 反序列化:适用于高性能、读密集型的工作负载。
  • serde 格式生态系统:JSON、TOML、bincode、MessagePack。
  • 二进制数据处理:通过 repr(C)、zerocopy 和 bytes::Bytes 处理数据。

11.1 serde 基础知识

serde (SERialize/DEserialize) 是 Rust 中通用的序列化框架。它将 数据模型 (Data model) (你的结构体) 与 数据格式 (Format) (JSON、TOML、二进制等) 分离开来:

use serde::{Serialize, Deserialize};

#[derive(Debug, Serialize, Deserialize)]
struct ServerConfig {
    name: String,
    port: u16,
    #[serde(default)]                    // 如果缺失,则使用 Default::default()
    max_connections: usize,
    #[serde(skip_serializing_if = "Option::is_none")]
    tls_cert_path: Option<String>,
}

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // 从 JSON 反序列化:
    let json_input = r#"{
        "name": "hw-diag",
        "port": 8080
    }"#;
    let config: ServerConfig = serde_json::from_str(json_input)?;
    println!("{config:?}");
    // ServerConfig { name: "hw-diag", port: 8080, max_connections: 0, tls_cert_path: None }

    // 序列化为 JSON:
    let output = serde_json::to_string_pretty(&config)?;
    println!("{output}");

    // 同样的结构体,不同的格式 —— 无需修改代码:
    let toml_input = r#"
        name = "hw-diag"
        port = 8080
    "#;
    let config: ServerConfig = toml::from_str(toml_input)?;
    println!("{config:?}");

    Ok(())
}

核心洞察:你的结构体只需派生一次 Serialize 和 Deserialize。然后它就能与 任何 兼容 serde 的格式配合使用 —— 包括 JSON、TOML、YAML、bincode、MessagePack、CBOR、postcard 以及其他数十种格式。

常用的 serde 属性

serde 通过字段和容器属性提供了对序列化的细粒度控制:

use serde::{Serialize, Deserialize};

// --- 容器属性 (用于结构体/枚举上) ---
#[derive(Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]       // JSON 惯例:field_name → fieldName
#[serde(deny_unknown_fields)]            // 拒绝额外的键 —— 严格解析
struct DiagResult {
    test_name: String,                   // 序列化为 "testName"
    pass_count: u32,                     // 序列化为 "passCount"
    fail_count: u32,                     // 序列化为 "failCount"
}

// --- 字段属性 ---
#[derive(Serialize, Deserialize)]
struct Sensor {
    #[serde(rename = "sensor_id")]       // 覆盖序列化时的字段名称
    id: u64,

    #[serde(default)]                    // 如果输入中缺失,则使用 Default
    enabled: bool,

    #[serde(default = "default_threshold")]
    threshold: f64,

    #[serde(skip)]                       // 永远不进行序列化或反序列化
    cached_value: Option<f64>,

    #[serde(skip_serializing_if = "Vec::is_empty")]
    tags: Vec<String>,

    #[serde(flatten)]                    // 内联嵌套结构体的字段
    metadata: Metadata,

    #[serde(with = "hex_bytes")]         // 使用自定义的序列化/反序列化模块
    raw_data: Vec<u8>,
}

fn default_threshold() -> f64 { 1.0 }

#[derive(Serialize, Deserialize)]
struct Metadata {
    vendor: String,
    model: String,
}
// 使用 #[serde(flatten)] 后,得到的 JSON 如下所示:
// { "sensor_id": 1, "vendor": "Intel", "model": "X200", ... }
// 而不是:{ "sensor_id": 1, "metadata": { "vendor": "Intel", ... } }

常用属性速查表:

属性作用层级效果
rename_all = "camelCase"容器将所有字段重命名为小驼峰/蛇形/大写蛇形等
deny_unknown_fields容器对非预期的键报错 (严格模式)
default字段当字段缺失时使用 Default::default()
rename = "..."字段自定义序列化名称
skip字段完全排除在序列化/反序列化之外
skip_serializing_if = "fn"字段有条件地排除 (例如 Option::is_none)
flatten字段内联嵌套结构体的字段
with = "module"字段使用自定义的序列化/反序列化函数
alias = "..."字段在反序列化期间接受替代名称
deserialize_with = "fn"字段仅自定义反序列化函数
untagged枚举按顺序尝试每个变体 (输出中没有判别式)

枚举表示形式

对于 JSON 等格式,serde 提供了四种枚举表示形式:

use serde::{Serialize, Deserialize};

// 1. 外部标记 (Externally Tagged) —— 默认方式:
#[derive(Serialize, Deserialize)]
enum Command {
    Reboot,
    RunDiag { test_name: String, timeout_secs: u64 },
    SetFanSpeed(u8),
}
// "Reboot"                                          → Command::Reboot
// {"RunDiag": {"test_name": "gpu", "timeout_secs": 60}}  → Command::RunDiag { ... }

// 2. 内部标记 (Internally Tagged) —— #[serde(tag = "type")]:
#[derive(Serialize, Deserialize)]
#[serde(tag = "type")]
enum Event {
    Start { timestamp: u64 },
    Error { code: i32, message: String },
    End   { timestamp: u64, success: bool },
}
// {"type": "Start", "timestamp": 1706000000}
// {"type": "Error", "code": 42, "message": "timeout"}

// 3. 相邻标记 (Adjacently Tagged) —— #[serde(tag = "t", content = "c")]:
#[derive(Serialize, Deserialize)]
#[serde(tag = "t", content = "c")]
enum Payload {
    Text(String),
    Binary(Vec<u8>),
}
// {"t": "Text", "c": "hello"}
// {"t": "Binary", "c": [0, 1, 2]}

// 4. 无标记 (Untagged) —— #[serde(untagged)]:
#[derive(Serialize, Deserialize)]
#[serde(untagged)]
enum StringOrNumber {
    Str(String),
    Num(f64),
}
// "hello" → StringOrNumber::Str("hello")
// 42.0    → StringOrNumber::Num(42.0)
// ⚠️ 按顺序尝试 —— 第一个匹配的变体胜出

如何选择表示形式:对于大多数 JSON API,请使用内部标记 (tag = "type") —— 它是最可读的,且符合 Go、Python 和 TypeScript 的惯例。仅在形状 (Shape) 本身就足以区分的“联合”类型中使用无标记形式。

零拷贝 (Zero-Copy) 反序列化

serde 可以在不分配新字符串的情况下进行反序列化 —— 直接从输入缓冲区借用。这是高性能解析的关键:

use serde::Deserialize;

// --- 所有权类型 (涉及分配) ---
// 每个 String 字段都会将字节从输入拷贝到新的堆分配中。
#[derive(Deserialize)]
struct OwnedRecord {
    name: String,           // 分配一个新的 String
    value: String,          // 分配另一个 String
}

// --- 零拷贝 (借用方式) ---
// &'de str 字段直接从输入中借用 —— 零分配。
#[derive(Deserialize)]
struct BorrowedRecord<'a> {
    name: &'a str,          // 指向输入缓冲区
    value: &'a str,         // 指向输入缓冲区
}

fn main() {
    let input = r#"{"name": "cpu_temp", "value": "72.5"}"#;

    // 所有权类型:分配两个 String 对象
    let owned: OwnedRecord = serde_json::from_str(input).unwrap();

    // 零拷贝:`name` 和 `value` 直接指向 `input` —— 无分配
    let borrowed: BorrowedRecord = serde_json::from_str(input).unwrap();

    // 输出受到生命周期的约束:borrowed 的存活时间不能超过 input
    println!("{}: {}", borrowed.name, borrowed.value);
}

理解生命周期:

// Deserialize<'de> —— 结构体可以从生命周期为 'de 的数据中借用:
//   struct BorrowedRecord<'a> where 'a == 'de
//   仅在输入缓冲区存活时间足够长时才有效

// DeserializeOwned —— 结构体拥有其所有数据,不进行借用:
//   trait DeserializeOwned: for<'de> Deserialize<'de> {}
//   适用于任何输入生命周期 (结构体是独立的)

use serde::de::DeserializeOwned;

// 此函数要求所有权类型 —— 输入可以是临时的
fn parse_owned<T: DeserializeOwned>(input: &str) -> T {
    serde_json::from_str(input).unwrap()
}

// 此函数允许借用 —— 更高效,但会限制生命周期
fn parse_borrowed<'a, T: Deserialize<'a>>(input: &'a str) -> T {
    serde_json::from_str(input).unwrap()
}

何时使用零拷贝:

  • 解析大型文件,但你只需要其中的几个字段。
  • 高吞吐量管道 (网络数据包、日志行)。
  • 当输入缓冲区已经存活足够长时 (例如内存映射文件)。

何时不要使用零拷贝:

  • 输入是瞬时的 (例如会被重用的网络读取缓冲区)。
  • 你需要将结果存储到超过输入生命周期的时间。
  • 字段需要进行转换 (如转义字符的处理、规范化)。

实用建议:Cow<'a, str> 可以让你两全其美 —— 尽可能借用,必要时进行分配 (例如当 JSON 转义序列需要反转义时)。serde 原生支持 Cow。

格式生态系统

格式Crate人类可读大小速度使用场景
JSONserde_json✅大良好配置文件、REST API、日志
TOMLtoml✅中良好配置文件 (Cargo.toml 风格)
YAMLserde_yaml✅中良好配置文件 (复杂嵌套)
bincodebincode❌小极快IPC、缓存、Rust 之间通信
postcardpostcard❌极小极快嵌入式工程、no_std
MessagePackrmp-serde❌小良好跨语言二进制协议
CBORciborium❌小良好IoT、受限环境
#![allow(unused)]
fn main() {
// 同样的结构体,多种格式 —— 这正是 serde 的威力所在:

#[derive(serde::Serialize, serde::Deserialize, Debug)]
struct DiagConfig {
    name: String,
    tests: Vec<String>,
    timeout_secs: u64,
}

let config = DiagConfig {
    name: "accel_diag".into(),
    tests: vec!["memory".into(), "compute".into()],
    timeout_secs: 300,
};

// JSON:   {"name":"accel_diag","tests":["memory","compute"],"timeout_secs":300}
let json = serde_json::to_string(&config).unwrap();       // 67 字节

// bincode: 紧凑的二进制 —— 约 40 字节,没有字段名称
let bin = bincode::serialize(&config).unwrap();            // 显著更小
}

如何选择格式:

  • 人类可编辑的配置文件 → TOML 或 JSON
  • Rust 之间的 IPC/缓存 → bincode (快速、紧凑,但不跨语言)
  • 跨语言二进制通信 → MessagePack 或 CBOR
  • 嵌入式 / no_std → postcard

二进制数据与 repr(C)

在硬件诊断中,解析二进制协议数据非常常见。Rust 提供了用于安全、零拷贝二进制数据处理的工具:

#![allow(unused)]
fn main() {
// --- #[repr(C)]: 可预测的内存布局 ---
// 确保字段按照声明顺序排列,并遵循 C 语言的填充规则。
// 对于匹配硬件寄存器布局和协议头至关重要。

#[repr(C)]
#[derive(Debug, Clone, Copy)]
struct IpmiHeader {
    rs_addr: u8,
    net_fn_lun: u8,
    checksum: u8,
    rq_addr: u8,
    rq_seq_lun: u8,
    cmd: u8,
}

// --- 通过手动解析实现安全的二进制解析 ---
impl IpmiHeader {
    fn from_bytes(data: &[u8]) -> Option<Self> {
        if data.len() < size_of::<Self>() {
            return None;
        }
        Some(IpmiHeader {
            rs_addr:     data[0],
            net_fn_lun:  data[1],
            checksum:    data[2],
            rq_addr:     data[3],
            rq_seq_lun:  data[4],
            cmd:         data[5],
        })
    }

    fn net_fn(&self) -> u8 { self.net_fn_lun >> 2 }
    fn lun(&self)    -> u8 { self.net_fn_lun & 0x03 }
}

// --- 字节序 (Endianness) 感知解析 ---
fn read_u16_le(data: &[u8], offset: usize) -> u16 {
    u16::from_le_bytes([data[offset], data[offset + 1]])
}

fn read_u32_be(data: &[u8], offset: usize) -> u32 {
    u32::from_be_bytes([
        data[offset], data[offset + 1],
        data[offset + 2], data[offset + 3],
    ])
}

// --- #[repr(C, packed)]: 移除填充 (alignment = 1) ---
#[repr(C, packed)]
#[derive(Debug, Clone, Copy)]
struct PcieCapabilityHeader {
    cap_id: u8,        // 能力 ID
    next_cap: u8,      // 指向下一个能力的指针
    cap_reg: u16,      // 能力特定的寄存器
}
// ⚠️ Packed 结构体:获取 &field 会创建非对齐引用 —— 这是未定义行为 (UB)。
// 务必将字段拷贝出来:let id = header.cap_id;  // 正常 (Copy)
// 千万不要:let r = &header.cap_reg;           // 如果未对齐则是 UB
}

zerocopy 与 bytemuck —— 安全转换 (Transmutation)

与其使用不安全的 transmute,不如使用那些能在编译时验证布局安全性的 crate:

#![allow(unused)]
fn main() {
// --- zerocopy: 编译时检查的零拷贝转换 ---
// Cargo.toml: zerocopy = { version = "0.8", features = ["derive"] }

use zerocopy::{FromBytes, IntoBytes, KnownLayout, Immutable};

#[derive(FromBytes, IntoBytes, KnownLayout, Immutable, Debug)]
#[repr(C)]
struct SensorReading {
    sensor_id: u16,
    flags: u8,
    _reserved: u8,
    value: u32,     // 定点数:实际值 = value / 1000.0
}

fn parse_sensor(raw: &[u8]) -> Option<&SensorReading> {
    // 安全零拷贝:在编译时验证对齐和大小
    SensorReading::ref_from_bytes(raw).ok()
    // 返回指向 raw 内部的 &SensorReading —— 无拷贝,无分配
}

// --- bytemuck: 简单、经过实战检验 ---
// Cargo.toml: bytemuck = { version = "1", features = ["derive"] }

use bytemuck::{Pod, Zeroable};

#[derive(Pod, Zeroable, Clone, Copy, Debug)]
#[repr(C)]
struct GpuRegister {
    address: u32,
    value: u32,
}

fn cast_registers(data: &[u8]) -> &[GpuRegister] {
    // 安全转换:Pod 保证所有位模式 (bit pattern) 都是有效的
    bytemuck::cast_slice(data)
}
}

如何选择:

方法安全性开销使用场景
手动逐字段解析✅ 安全拷贝字段小型结构体、复杂布局
zerocopy✅ 安全零拷贝大型缓冲区、多次读取、编译时检查
bytemuck✅ 安全零拷贝简单的 Pod 类型、转换切片
unsafe { transmute() }❌ 不安全零拷贝最后的手段 —— 在应用代码中应避免使用

bytes::Bytes —— 引用计数缓冲区

bytes crate (被 tokio、hyper、tonic 使用) 提供了带有引用计数的零拷贝字节缓冲区 —— Bytes 之于 Vec<u8> 类似于 Arc<[u8]> 之于拥有所有权的切片:

use bytes::{Bytes, BytesMut, Buf, BufMut};

fn main() {
    // --- BytesMut: 用于构建数据的可变缓冲区 ---
    let mut buf = BytesMut::with_capacity(1024);
    buf.put_u8(0x01);                    // 写入一个字节
    buf.put_u16(0x1234);                 // 写入 u16 (大端序)
    buf.put_slice(b"hello");             // 写入原始字节
    buf.put(&b"world"[..]);              // 从切片中写入

    // 冻结为不可变的 Bytes (零成本):
    let data: Bytes = buf.freeze();

    // --- Bytes: 不可变、引用计数、可克隆 ---
    let data2 = data.clone();            // 开销极低:增加引用计数,而非深拷贝
    let slice = data.slice(3..8);        // 零拷贝子切片 (共享底层缓冲区)

    // 使用 Buf trait 从 Bytes 中读取:
    let mut reader = &data[..];
    let byte = reader.get_u8();          // 0x01
    let short = reader.get_u16();        // 0x1234

    // 无需拷贝即可拆分:
    let mut original = Bytes::from_static(b"HEADER\x00PAYLOAD");
    let header = original.split_to(6);   // header = "HEADER", original = "\x00PAYLOAD"

    println!("header: {:?}", &header[..]);
    println!("payload: {:?}", &original[1..]);
}

bytes vs Vec<u8>:

特性Vec<u8>Bytes
克隆成本O(n) 深拷贝O(1) 增加引用计数
子切片具有生命周期的借用拥有所有权,由引用计数追踪
线程安全非 Sync (需要 Arc)内置 Send + Sync
可变性直接使用 &mut需要先拆分为 BytesMut
生态系统标准库tokio, hyper, tonic, axum

何时使用 bytes:网络协议、报文解析,以及任何当你收到一个缓冲区并需要将其拆分为多个部分供不同组件或线程处理的场景。零拷贝拆分是它的杀手锏级特性。

关键要点 —— 序列化与二进制数据

  • serde 的派生宏可处理 90% 的情况;其余情况使用属性 (rename、skip、default) 处理。
  • 零拷贝反序列化 (结构体中使用 &'a str) 避免了读密集型工作负载中的分配。
  • repr(C) + zerocopy/bytemuck 适用于硬件寄存器布局;bytes::Bytes 适用于引用计数缓冲区。

另请参阅: 第 10 章 关于将 serde 错误与 thiserror 结合使用。第 12 章 关于 repr(C) 和 FFI 数据布局。

flowchart LR
    subgraph 输入
        JSON["JSON"]
        TOML["TOML"]
        Bin["bincode"]
        MsgP["MessagePack"]
    end

    subgraph serde["serde 数据模型"]
        Ser["Serialize"]
        De["Deserialize"]
    end

    subgraph 输出
        Struct["Rust 结构体"]
        Enum["Rust 枚举"]
    end

    JSON --> De
    TOML --> De
    Bin --> De
    MsgP --> De
    De --> Struct
    De --> Enum
    Struct --> Ser
    Enum --> Ser
    Ser --> JSON
    Ser --> Bin

    style JSON fill:#e8f4f8,stroke:#2980b9,color:#000
    style TOML fill:#e8f4f8,stroke:#2980b9,color:#000
    style Bin fill:#e8f4f8,stroke:#2980b9,color:#000
    style MsgP fill:#e8f4f8,stroke:#2980b9,color:#000
    style Ser fill:#fef9e7,stroke:#f1c40f,color:#000
    style De fill:#fef9e7,stroke:#f1c40f,color:#000
    style Struct fill:#d4efdf,stroke:#27ae60,color:#000
    style Enum fill:#d4efdf,stroke:#27ae60,color:#000

练习:自定义 serde 反序列化 ★★★ (~45 分钟)

设计一个 HumanDuration 包装器,它能够使用自定义的 serde 反序列化器,从类似 "30s"、"5m"、"2h" 的人类可读字符串中进行反序列化。它还应当能够序列化回相同的格式。

🔑 参考答案
use serde::{Deserialize, Deserializer, Serialize, Serializer};
use std::fmt;

#[derive(Debug, Clone, PartialEq)]
struct HumanDuration(std::time::Duration);

impl HumanDuration {
    fn from_str(s: &str) -> Result<Self, String> {
        let s = s.trim();
        if s.is_empty() { return Err("空持续时间字符串".into()); }

        let (num_str, suffix) = s.split_at(
            s.find(|c: char| !c.is_ascii_digit()).unwrap_or(s.len())
        );
        let value: u64 = num_str.parse()
            .map_err(|_| format!("无效数字: {num_str}"))?;

        let duration = match suffix {
            "s" | "sec"  => std::time::Duration::from_secs(value),
            "m" | "min"  => std::time::Duration::from_secs(value * 60),
            "h" | "hr"   => std::time::Duration::from_secs(value * 3600),
            "ms"         => std::time::Duration::from_millis(value),
            other        => return Err(format!("未知后缀: {other}")),
        };
        Ok(HumanDuration(duration))
    }
}

impl fmt::Display for HumanDuration {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        let secs = self.0.as_secs();
        if secs == 0 {
            write!(f, "{}ms", self.0.as_millis())
        } else if secs % 3600 == 0 {
            write!(f, "{}h", secs / 3600)
        } else if secs % 60 == 0 {
            write!(f, "{}m", secs / 60)
        } else {
            write!(f, "{}s", secs)
        }
    }
}

impl Serialize for HumanDuration {
    fn serialize<S: Serializer>(&self, serializer: S) -> Result<S::Ok, S::Error> {
        serializer.serialize_str(&self.to_string())
    }
}

impl<'de> Deserialize<'de> for HumanDuration {
    fn deserialize<D: Deserializer<'de>>(deserializer: D) -> Result<Self, D::Error> {
        let s = String::deserialize(deserializer)?;
        HumanDuration::from_str(&s).map_err(serde::de::Error::custom)
    }
}

#[derive(Debug, Deserialize, Serialize)]
struct Config {
    timeout: HumanDuration,
    retry_interval: HumanDuration,
}

fn main() {
    let json = r#"{ "timeout": "30s", "retry_interval": "5m" }"#;
    let config: Config = serde_json::from_str(json).unwrap();

    assert_eq!(config.timeout.0, std::time::Duration::from_secs(30));
    assert_eq!(config.retry_interval.0, std::time::Duration::from_secs(300));

    let serialized = serde_json::to_string(&config).unwrap();
    assert!(serialized.contains("30s"));
    println!("配置: {serialized}");
}

English Original

第 12 章:Unsafe Rust —— 受控的危险 🔴

你将学到:

  • 五种 Unsafe “超能力” 及其各自的适用场景。
  • 编写可靠的抽象 (Sound Abstractions):安全 API 与 Unsafe 内部实现。
  • FFI 模式:在 Rust 中调用 C 代码 (以及反向调用)。
  • 常见的未定义行为 (UB) 陷阱 以及 Arena/Slab 分配器模式。

12.1 五种 Unsafe “超能力”

unsafe 开启了五项编译器无法验证的操作:

#![allow(unused)]
fn main() {
// SAFETY: 下文内联解释了每项操作的安全性。
unsafe {
    // 1. 解引用裸指针 (Raw Pointer)
    let ptr: *const i32 = &42;
    let value = *ptr; // 指针可能是悬空的或为空

    // 2. 调用 unsafe 函数
    let layout = std::alloc::Layout::new::<u64>();
    let mem = std::alloc::alloc(layout);

    // 3. 访问可变静态变量 (Mutable Static Variable)
    static mut COUNTER: u32 = 0;
    COUNTER += 1; // 如果多个线程访问,会产生数据竞争 (Data race)

    // 4. 实现 unsafe trait
    // unsafe impl Send for MyType {}

    // 5. 访问 union 的字段
    // union IntOrFloat { i: i32, f: f32 }
    // let u = IntOrFloat { i: 42 };
    // let f = u.f; // 重新解释位模式 —— 可能是垃圾数据
}
}

核心原则:unsafe 并没有关闭借用检查器或类型系统。它仅仅开启了这五种特定的功能。所有其他的 Rust 规则依然适用。

编写可靠的抽象 (Sound Abstractions)

unsafe 的目的是围绕不安全的操作构建 安全抽象:

#![allow(unused)]
fn main() {
/// 一个容量固定的、栈分配的缓冲区。
/// 所有公共方法都是安全的 —— unsafe 已被封装在内部。
pub struct StackBuf<T, const N: usize> {
    data: [std::mem::MaybeUninit<T>; N],
    len: usize,
}

impl<T, const N: usize> StackBuf<T, N> {
    pub fn new() -> Self {
        StackBuf {
            // 每个元素都是独立的 MaybeUninit —— 无需使用 unsafe。
            // `const { ... }` 代码块(Rust 1.79+)允许我们重复
            // 一个非 Copy 的 const 表达式 N 次。
            data: [const { std::mem::MaybeUninit::uninit() }; N],
            len: 0,
        }
    }

    pub fn push(&mut self, value: T) -> Result<(), T> {
        if self.len >= N {
            return Err(value); // 缓冲区已满 —— 将原值返回给调用者
        }
        // SAFETY: len < N,因此 data[len] 位于边界内。
        // 我们将一个有效的 T 写入 MaybeUninit 槽位中。
        self.data[self.len] = std::mem::MaybeUninit::new(value);
        self.len += 1;
        Ok(())
    }

    pub fn get(&self, index: usize) -> Option<&T> {
        if index < self.len {
            // SAFETY: index < len,且 data[0..len] 均已初始化。
            Some(unsafe { self.data[index].assume_init_ref() })
        } else {
            None
        }
    }
}

impl<T, const N: usize> Drop for StackBuf<T, N> {
    fn drop(&mut self) {
        // SAFETY: data[0..len] 已初始化 —— 需要正确地释放 (drop) 它们。
        for i in 0..self.len {
            unsafe { self.data[i].assume_init_drop(); }
        }
    }
}
}

编写可靠 (Sound) Unsafe 代码的三原则:

  1. 记录不变式 (Invariants) —— 每个 // SAFETY: 注释都应解释该操作为何有效。
  2. 封装 (Encapsulate) —— 将 Unsafe 细节隐藏在安全 API 内部;确保用户无法触发 UB。
  3. 最小化 (Minimize) —— 尽量减小 unsafe 代码块的范围。

FFI 模式:从 Rust 调用 C

#![allow(unused)]
fn main() {
// 声明 C 函数签名:
extern "C" {
    fn strlen(s: *const std::ffi::c_char) -> usize;
    fn printf(format: *const std::ffi::c_char, ...) -> std::ffi::c_int;
}

// 安全包装器 (Safe wrapper):
fn safe_strlen(s: &str) -> usize {
    let c_string = std::ffi::CString::new(s).expect("字符串包含空字节");
    // SAFETY: c_string 是一个有效的、以 null 结尾的字符串,在调用期间保持存活。
    unsafe { strlen(c_string.as_ptr()) }
}

// 从 C 调用 Rust (导出函数):
#[no_mangle]
pub extern "C" fn rust_add(a: i32, b: i32) -> i32 {
    a + b
}
}

常用的 FFI 类型:

Rust 类型C 类型备注
i32 / u32int32_t / uint32_t固定宽度,安全
*const T / *mut Tconst T* / T*裸指针 (Raw pointers)
std::ffi::CStrconst char* (借用)以 null 结尾,借用方式
std::ffi::CStringchar* (所有权)以 null 结尾,所有权方式
std::ffi::c_voidvoid不透明指针目标 (Opaque)
Option<fn(...)>可为空的函数指针None 等同于 NULL

常见的未定义行为 (UB) 陷阱

陷阱示例为什么是 UB
空指针解引用*std::ptr::null::<i32>()解引用空指针在任何时候都是 UB
悬空指针在 drop() 后解析指针内存可能已被重用
数据竞争两个线程同时写入 static mut未同步的并发写入
错误的 assume_initMaybeUninit::<String>::uninit().assume_init()读取未初始化的内存。注意:[const { MaybeUninit::uninit() }; N] (Rust 1.79+) 是创建 MaybeUninit 数组的安全方式 —— 无需 unsafe 或 assume_init (见上文 StackBuf::new())
别名违规对同一数据创建两个 &mut违反了 Rust 的别名模型 (Aliasing Model)
无效的枚举值std::mem::transmute::<u8, bool>(2)bool 只能是 0 或 1

何时在生产环境中使用 unsafe:

  • FFI 边界 (调用 C/C++ 代码)。
  • 性能关键的内层循环 (为了避免边界检查)。
  • 构建底层原语 (Vec、HashMap —— 它们的内部实现都使用了 unsafe)。
  • 只要能避免,就绝不要在应用逻辑中使用它。

12.2 自定义分配器 —— Arena 与 Slab 模式

在 C 语言中,你会针对特定的分配模式编写自定义的 malloc() 替代品 —— 比如一次性释放所有内存的 Arena 分配器、用于固定大小对象的 Slab 分配器,或者用于高吞吐量系统的池分配器。Rust 通过 GlobalAlloc trait 和分配器库提供了同样的能力,并增加了 在编译时防止“释放后使用” (Use-after-free) 的优势,这可以通过生命周期约束的 Arena 来实现。

Arena 分配器 —— 批量分配,批量释放

Arena 分配器通过向前移动指针来分配内存。单个条目无法被单独释放 —— 整个 Arena 会被一次性释放。这非常适合处理请求作用域或帧作用域 (Frame-scoped) 的分配:

#![allow(unused)]
fn main() {
use bumpalo::Bump;

fn process_sensor_frame(raw_data: &[u8]) {
    // 为这一帧的分配创建一个 Arena
    let arena = Bump::new();

    // 在 Arena 中分配对象 —— 每个约耗时 2ns (仅仅是指针移动)
    let header = arena.alloc(parse_header(raw_data));
    let readings: &mut [f32] = arena.alloc_slice_fill_default(header.sensor_count);

    for (i, chunk) in raw_data[header.payload_offset..].chunks(4).enumerate() {
        if i < readings.len() {
            readings[i] = f32::from_le_bytes(chunk.try_into().unwrap());
        }
    }

    // 使用 readings...
    let avg = readings.iter().sum::<f32>() / readings.len() as f32;
    println!("帧平均值: {avg:.2}");

    // `arena` 在此处被释放 —— 所有分配在 O(1) 时间内一次性释放
    // 没有逐个对象的析构开销,也没有碎片化问题
}
fn parse_header(_: &[u8]) -> Header { Header { sensor_count: 4, payload_offset: 8 } }
struct Header { sensor_count: usize, payload_offset: usize }
}

Arena vs 标准分配器:

特性Vec::new() / Box::new()Bump Arena
分配速度~25ns (调用 malloc)~2ns (指针移动)
释放速度逐个对象的析构函数O(1) 批量释放
碎片化会 (针对长寿命进程)Arena 内部无碎片
生命周期安全堆内存 —— 在 Drop 时释放Arena 引用 —— 编译时作用域约束
使用场景通用目的请求/帧/批处理

typed-arena —— 类型安全的 Arena

当 Arena 中的所有对象都是同一类型时,typed-arena 提供了一个更简单的 API,返回绑定到 Arena 生命周期的引用:

#![allow(unused)]
fn main() {
use typed_arena::Arena;

struct AstNode<'a> {
    value: i32,
    children: Vec<&'a AstNode<'a>>,
}

fn build_tree() {
    let arena: Arena<AstNode<'_>> = Arena::new();

    // 分配节点 —— 返回存活时间与 Arena 一致的 &AstNode
    let root = arena.alloc(AstNode { value: 1, children: vec![] });
    let left = arena.alloc(AstNode { value: 2, children: vec![] });
    let right = arena.alloc(AstNode { value: 3, children: vec![] });

    // 构建树 —— 只要 `arena` 存活,所有引用就都是有效的
    // (对于真正的可变树,修改需要内部可变性)

    println!("根节点: {}, 左子节点: {}, 右子节点: {}", root.value, left.value, right.value);

    // `arena` 在此处释放 —— 所有节点一次性释放
}
}

Slab 分配器 —— 固定大小的对象池

Slab 分配器预先分配一个由固定大小槽位组成的池。对象是单独分配和返回的,但由于所有槽位大小相同,因此消除了碎片,并实现了 O(1) 的分配/释放:

#![allow(unused)]
fn main() {
use slab::Slab;

struct Connection {
    id: u64,
    buffer: [u8; 1024],
    active: bool,
}

fn connection_pool_example() {
    // 预先为连接分配一个 Slab
    let mut connections: Slab<Connection> = Slab::with_capacity(256);

    // 插入返回一个键 (usize 索引) —— O(1)
    let key1 = connections.insert(Connection {
        id: 1001,
        buffer: [0; 1024],
        active: true,
    });

    let key2 = connections.insert(Connection {
        id: 1002,
        buffer: [0; 1024],
        active: true,
    });

    // 通过键访问 —— O(1)
    if let Some(conn) = connections.get_mut(key1) {
        conn.buffer[0..5].copy_from_slice(b"hello");
    }

    // 移除返回该值 —— O(1),该槽位将被下次插入重用
    let removed = connections.remove(key2);
    assert_eq!(removed.id, 1002);

    // 下次插入会重用已释放的槽位 —— 无碎片
    let key3 = connections.insert(Connection {
        id: 1003,
        buffer: [0; 1024],
        active: true,
    });
    assert_eq!(key3, key2); // 同一个槽位被重用了!
}
}

为 no_std 实现极简 Arena

对于无法引入 bumpalo 的裸机环境,这里有一个基于 unsafe 构建的极简 Arena:

#![allow(unused)]
#![cfg_attr(not(test), no_std)]

fn main() {
use core::alloc::Layout;
use core::cell::{Cell, UnsafeCell};

/// 一个由固定大小字节数组支持的简单堆块分配器 (Bump Allocator)。
/// 非线程安全 —— 在多核环境中请配合锁或按核心独立使用。
///
/// **重要提示**:与 `bumpalo` 类似,此 Arena 在释放时不调用已分配条目的析构函数。
/// 实现 `Drop` 的类型(如文件句柄、套接字等)会产生资源泄漏。
/// 请仅分配不含重要 `Drop` 实现的类型,或在 Arena 释放前手动 drop 它们。
pub struct FixedArena<const N: usize> {
    // 此处必须使用 UnsafeCell:我们要通过 `&self` 修改 `buf`。
    // 如果没有 UnsafeCell,将 &self.buf 转换为 *mut u8 将是 UB
    // (违反了 Rust 的别名模型 —— 共享引用意味着不可变)。
    buf: UnsafeCell<[u8; N]>,
    offset: Cell<usize>, // 用于 &self 分配的内部可变性
}

impl<const N: usize> FixedArena<N> {
    pub const fn new() -> Self {
        FixedArena {
            buf: UnsafeCell::new([0; N]),
            offset: Cell::new(0),
        }
    }

    /// 在 Arena 中分配一个 `T`。空间不足时返回 `None`。
    pub fn alloc<T>(&self, value: T) -> Option<&mut T> {
        let layout = Layout::new::<T>();
        let current = self.offset.get();

        // 向上对齐 (Align up)
        let aligned = (current + layout.align() - 1) & !(layout.align() - 1);
        let new_offset = aligned + layout.size();

        if new_offset > N {
            return None; // Arena 已满
        }

        self.offset.set(new_offset);

        // SAFETY:
        // - `aligned` 位于 `buf` 边界内 (已在上方检查)
        // - 对齐正确 (已对齐至 T 的要求)
        // - 无别名冲突:每次分配返回一个唯一的、非重叠的区域
        // - UnsafeCell 授权了通过 &self 进行修改的权限
        // - Arena 的存活时间超过返回的引用 (调用者需确保)
        let ptr = unsafe {
            let base = (self.buf.get() as *mut u8).add(aligned);
            let typed = base as *mut T;
            typed.write(value);
            &mut *typed
        };

        Some(ptr)
    }

    /// 重置 Arena —— 会使之前所有的分配失效。
    ///
    /// # Safety
    /// 调用者必须确保不存在任何指向 Arena 分配数据的引用。
    pub unsafe fn reset(&self) {
        self.offset.set(0);
    }

    pub fn used(&self) -> usize {
        self.offset.get()
    }

    pub fn remaining(&self) -> usize {
        N - self.offset.get()
    }
}
}

选择分配器策略

注意:下方的图表使用了 Mermaid 语法。它可以在 GitHub 以及支持 Mermaid 的工具(如带有 mermaid 插件的 mdBook)中渲染。

graph TD
    A["你的分配模式是什么?"] --> B{是否为同一类型?}
    A --> I{"运行环境?"}
    B -->|是| C{是否需要单独释放?}
    B -->|否| D{是否需要单独释放?}
    C -->|是| E["<b>Slab<b><br/>slab crate<br/>O(1) 分配 + 释放<br/>基于索引访问"]
    C -->|否| F["<b>typed-arena<b><br/>批量分配,批量释放<br/>生命周期受限的引用"]
    D -->|是| G["<b>标准分配器<b><br/>Box, Vec 等<br/>通用的 malloc"]
    D -->|否| H["<b>Bump Arena<b><br/>bumpalo crate<br/>~2ns 分配,O(1) 批量释放"]
    
    I -->|no_std| J["FixedArena (自定义)<br/>或 embedded-alloc"]
    I -->|std| K["bumpalo / typed-arena / slab"]
    
    style E fill:#91e5a3,color:#000
    style F fill:#91e5a3,color:#000
    style G fill:#89CFF0,color:#000
    style H fill:#91e5a3,color:#000
    style J fill:#ffa07a,color:#000
    style K fill:#91e5a3,color:#000
C 语言模式Rust 等效方案关键优势
自定义 malloc() 池#[global_allocator] 实现类型安全、易于调试
obstack (GNU)bumpalo::Bump生命周期约束,无“释放后使用”
内核 Slab (kmem_cache)slab::Slab<T>类型安全、基于索引
栈分配的临时缓冲区FixedArena<N> (见上文)无需堆内存、const 可构造
alloca()[T; N] 或 SmallVec编译时确定大小,无 UB

交叉引用:关于裸机环境分配器的设置 (在使用 embedded-alloc 时配合 #[global_allocator]),请参阅《面向 C 程序员的 Rust 培训》第 15.1 节“全局分配器设置”,该节涵盖了嵌入式特定的引导过程。

关键要点 —— Unsafe Rust

  • 记录不变式 (SAFETY: 注释)、在安全 API 后进行封装、最小化 Unsafe 作用域。
  • [const { MaybeUninit::uninit() }; N] (Rust 1.79+) 取代了旧的 assume_init 反模式。
  • FFI 需要 extern "C"、#[repr(C)] 以及对空值和生命周期的仔细处理。
  • Arena 和 Slab 分配器以牺牲通用灵活性为代价,换取了极高的分配速度。

另请参阅: 第 4 章 关于 Unsafe 代码在型变和 Drop 检查方面的交互。第 9 章 关于 Pin 和自引用类型。


练习:围绕 Unsafe 编写安全包装器 ★★★ (~45 分钟)

编写一个 FixedVec<T, const N: usize> —— 一个固定容量、栈分配的向量。 要求如下:

  • push(&mut self, value: T) -> Result<(), T> 当缓冲区满时返回 Err(value)。
  • pop(&mut self) -> Option<T> 返回并移除最后一个元素。
  • as_slice(&self) -> &[T] 借用已初始化的元素。
  • 所有公共方法必须是安全的;所有 Unsafe 部分必须使用 SAFETY: 注释进行封装。
  • Drop 必须清理所有已初始化的元素。
🔑 参考答案
use std::mem::MaybeUninit;

pub struct FixedVec<T, const N: usize> {
    data: [MaybeUninit<T>; N],
    len: usize,
}

impl<T, const N: usize> FixedVec<T, N> {
    pub fn new() -> Self {
        FixedVec {
            data: [const { MaybeUninit::uninit() }; N],
            len: 0,
        }
    }

    pub fn push(&mut self, value: T) -> Result<(), T> {
        if self.len >= N { return Err(value); }
        // SAFETY: len < N,因此 data[len] 在边界内。
        self.data[self.len] = MaybeUninit::new(value);
        self.len += 1;
        Ok(())
    }

    pub fn pop(&mut self) -> Option<T> {
        if self.len == 0 { return None; }
        self.len -= 1;
        // SAFETY: data[len] 之前已初始化 (减量前 len > 0)。
        Some(unsafe { self.data[self.len].assume_init_read() })
    }

    pub fn as_slice(&self) -> &[T] {
        // SAFETY: data[0..len] 均已初始化,且 MaybeUninit<T>
        // 与 T 的内存布局相同。
        unsafe { std::slice::from_raw_parts(self.data.as_ptr() as *const T, self.len) }
    }

    pub fn len(&self) -> usize { self.len }
    pub fn is_empty(&self) -> bool { self.len == 0 }
}

impl<T, const N: usize> Drop for FixedVec<T, N> {
    fn drop(&mut self) {
        // SAFETY: data[0..len] 已初始化 —— 需要逐个释放。
        for i in 0..self.len {
            unsafe { self.data[i].assume_init_drop(); }
        }
    }
}

fn main() {
    let mut v = FixedVec::<String, 4>::new();
    v.push("hello".into()).unwrap();
    v.push("world".into()).unwrap();
    assert_eq!(v.as_slice(), &["hello", "world"]);
    assert_eq!(v.pop(), Some("world".into()));
    assert_eq!(v.len(), 1);
}

English Original

第 13 章:宏 —— 生成代码的代码 🟡

你将学到:

  • 声明式宏 (macro_rules!):带有模式匹配和重复机制。
  • 何时宏是正确工具:以及何时应该优先使用泛型/特性。
  • 过程宏:派生宏 (Derive)、属性宏 (Attribute) 和函数式宏。
  • 使用 syn 和 quote 编写 自定义派生宏。

13.1 声明式宏 (macro_rules!)

宏会在编译时对语法模式进行匹配,并将其展开为代码:

#![allow(unused)]
fn main() {
// 一个简单的创建 HashMap 的宏
macro_rules! hashmap {
    // 匹配:由逗号分隔的 key => value 键值对
    ( $( $key:expr => $value:expr ),* $(,)? ) => {
        {
            let mut map = std::collections::HashMap::new();
            $( map.insert($key, $value); )*
            map
        }
    };
}

let scores = hashmap! {
    "Alice" => 95,
    "Bob" => 87,
    "Carol" => 92,
};
// 展开为:
// let mut map = HashMap::new();
// map.insert("Alice", 95);
// map.insert("Bob", 87);
// map.insert("Carol", 92);
// map
}

宏片段类型 (Fragment types):

片段匹配内容示例
$x:expr任何表达式42, a + b, foo()
$x:ty类型i32, Vec<String>
$x:ident标识符my_var, Config
$x:pat模式Some(x), _
$x:stmt语句let x = 5;
$x:tt单个标记树 (Token tree)任何内容 (最灵活)
$x:literal字面量值42, "hello", true

重复机制:$( ... ),* 意味着“零个或多个,由逗号分隔”。

#![allow(unused)]
fn main() {
// 自动生成测试函数
macro_rules! test_cases {
    ( $( $name:ident: $input:expr => $expected:expr ),* $(,)? ) => {
        $(
            #[test]
            fn $name() {
                assert_eq!(process($input), $expected);
            }
        )*
    };
}

test_cases! {
    test_empty: "" => "",
    test_hello: "hello" => "HELLO",
    test_trim: "  spaces  " => "SPACES",
}
// 生成三个独立的 #[test] 函数
}

何时 (不) 使用宏

在以下情况下使用宏:

  • 减少特性/泛型无法处理的样板代码 (如变长参数、符合 DRY 原则的测试生成)。
  • 创建 DSL (领域特定语言,如 html!、sql!、vec!)。
  • 条件代码生成 (cfg!、compile_error!)。

在以下情况下不要使用宏:

  • 函数或泛型可以完成的任务 (宏更难调试,且自动补全无法提供帮助)。
  • 你需要在宏内部进行类型检查 (宏操作的是标记 tokens,而不是类型)。
  • 该模式只被使用一两次 (不值得付出抽象成本)。
#![allow(unused)]
fn main() {
// ❌ 不必要的宏 —— 函数完全可以胜任:
macro_rules! double {
    ($x:expr) => { $x * 2 };
}

// ✅ 直接使用函数即可:
fn double(x: i32) -> i32 { x * 2 }

// ✅ 宏的良好用例 —— 变长参数,无法通过函数实现:
macro_rules! println {
    ($($arg:tt)*) => { /* 格式化字符串 + 参数 */ };
}
}

过程宏 (Procedural Macros) 概述

过程宏是转换标记流 (token streams) 的 Rust 函数。它们需要一个设置了 proc-macro = true 的独立 crate:

#![allow(unused)]
fn main() {
// 三种类型的过程宏:

// 1. 派生宏 (Derive macros) —— #[derive(MyTrait)]
// 根据结构体定义生成特性实现
#[derive(Debug, Clone, Serialize, Deserialize)]
struct Config {
    name: String,
    port: u16,
}

// 2. 属性宏 (Attribute macros) —— #[my_attribute]
// 对被注解的项进行转换
#[route(GET, "/api/users")]
async fn list_users() -> Json<Vec<User>> { /* ... */ }

// 3. 函数式宏 (Function-like macros) —— my_macro!(...)
// 允许自定义语法
let query = sql!(SELECT * FROM users WHERE id = ?);
}

派生宏实践

这是最常用的过程宏类型。以下是 #[derive(Debug)] 在概念上的工作原理:

#![allow(unused)]
fn main() {
// 输入 (你的结构体):
#[derive(Debug)]
struct Point {
    x: f64,
    y: f64,
}

// 派生宏会生成如下代码:
impl std::fmt::Debug for Point {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        f.debug_struct("Point")
            .field("x", &self.x)
            .field("y", &self.y)
            .finish()
    }
}
}

常用的派生宏:

派生宏所在的 Crate生成的内容
Debugstdfmt::Debug 实现 (用于调试打印)
Clone, Copystd值拷贝/克隆
PartialEq, Eqstd等值比较
Hashstd为 HashMap 键提供哈希支持
Serialize, DeserializeserdeJSON/YAML 等编解码
Errorthiserrorstd::error::Error + Display 实现
ParserclapCLI 参数解析
Builderderive_builder构建器 (Builder) 模式

实践建议:请大胆地使用派生宏 —— 它们能消除容易出错的样板代码。编写自己的过程宏是一个进阶主题;在构建自定义宏之前,请先尝试使用现有的宏 (serde、thiserror、clap)。

宏的卫生性 (Macro Hygiene) 与 $crate

卫生性 意味着在宏内部创建的标识符不会与调用者作用域中的标识符发生冲突。Rust 的 macro_rules! 是 部分卫生的:

macro_rules! make_var {
    () => {
        let x = 42; // 此处的 'x' 位于宏的作用域内
    };
}

fn main() {
    let x = 10;
    make_var!();   // 创建了一个不同的 'x' (卫生的)
    println!("{x}"); // 打印 10,而不是 42 —— 宏内部的 x 不会泄漏出来
}

$crate:在库中编写宏时,请使用 $crate 来引用你自己的 crate —— 无论用户如何导入你的 crate,它都能正确解析:

#![allow(unused)]
fn main() {
// 在 my_diagnostics crate 中:

pub fn log_result(msg: &str) {
    println!("[diag] {msg}");
}

#[macro_export]
macro_rules! diag_log {
    ($($arg:tt)*) => {
        // ✅ $crate 始终会解析为 my_diagnostics,
        // 即使该用户在他们的 Cargo.toml 中重命名了该 crate
        $crate::log_result(&format!($($arg)*))
    };
}

// ❌ 如果没有 $crate:
// my_diagnostics::log_result(...)  ← 如果用户这样写,宏就会失效:
//   [dependencies]
//   diag = { package = "my_diagnostics", version = "1" }
}

规则:在 #[macro_export] 宏中务必使用 $crate::。绝不要直接使用你的 crate 名称。

递归宏与 tt munching

递归宏每次处理一个标记 (token) 输入 —— 这种技术被称为 tt munching (标记树咀嚼):

// 计算传递给宏的表达式数量
macro_rules! count {
    // 基础案例:没有剩余标记
    () => { 0usize };
    // 递归案例:消耗一个表达式,计算剩余部分
    ($head:expr $(, $tail:expr)* $(,)?) => {
        1usize + count!($($tail),*)
    };
}

fn main() {
    let n = count!("a", "b", "c", "d");
    assert_eq!(n, 4);

    // 在编译时也同样有效:
    const N: usize = count!(1, 2, 3);
    assert_eq!(N, 3);
}
#![allow(unused)]
fn main() {
// 从一组表达式构建异构元组 (heterogeneous tuple):
macro_rules! tuple_from {
    // 基础案例:单个元素
    ($single:expr $(,)?) => { ($single,) };
    // 递归案例:首个元素 + 剩余部分
    ($head:expr, $($tail:expr),+ $(,)?) => {
        ($head, tuple_from!($($tail),+))
    };
}

let t = tuple_from!(1, "hello", 3.14, true);
// 展开为:(1, ("hello", (3.14, (true,))))
}

片段限定符的微妙之处:

片段陷阱
$x:expr贪婪解析 —— 1 + 2 是一个单一表达式,而不是三个标记
$x:ty贪婪解析 —— Vec<String> 是一个单一类型;后面不能跟 + 或 <
$x:tt仅匹配 一个 标记树 —— 最灵活,但检查最少
$x:ident仅限普通标识符 —— 不能是像 std::io 之前的路径
$x:pat在 Rust 2021 中匹配 A | B 模式;对于单一模式请使用 $x:pat_param

何时使用 tt:当你需要将标记转发给另一个宏且不想被解析器约束时。$($args:tt)* 是“接受一切”的模式(被 println!、format!、vec! 所使用)。

使用 syn 和 quote 编写派生宏

派生宏存储在独立的 crate 中 (proc-macro = true),并使用 syn (解析 Rust) 和 quote (生成 Rust) 来转换标记流:

# my_derive/Cargo.toml
[lib]
proc-macro = true

[dependencies]
syn = { version = "2", features = ["full"] }
quote = "1"
proc-macro2 = "1"
#![allow(unused)]
fn main() {
// my_derive/src/lib.rs
use proc_macro::TokenStream;
use quote::quote;
use syn::{parse_macro_input, DeriveInput};

/// 派生宏:生成一个 `describe()` 方法,
/// 该方法返回结构体名称及其字段名称。
#[proc_macro_derive(Describe)]
pub fn derive_describe(input: TokenStream) -> TokenStream {
    let input = parse_macro_input!(input as DeriveInput);
    let name = &input.ident;
    let name_str = name.to_string();

    // 提取字段名称 (仅针对具有命名字段的结构体)
    let fields = match &input.data {
        syn::Data::Struct(data) => {
            data.fields.iter()
                .filter_map(|f| f.ident.as_ref())
                .map(|id| id.to_string())
                .collect::<Vec<_>>()
        }
        _ => vec![],
    };

    let field_list = fields.join(", ");

    let expanded = quote! {
        impl #name {
            pub fn describe() -> String {
                format!("{} {{ {} }}", #name_str, #field_list)
            }
        }
    };

    TokenStream::from(expanded)
}
}
// 在应用程序 crate 中:
use my_derive::Describe;

#[derive(Describe)]
struct SensorReading {
    sensor_id: u16,
    value: f64,
    timestamp: u64,
}

fn main() {
    println!("{}", SensorReading::describe());
    // "SensorReading { sensor_id, value, timestamp }"
}

工作流程:TokenStream (原始标记) → syn::parse (生成 AST) → 检查/转换 → quote! (生成标记) → TokenStream (回传给编译器)。

Crate角色关键类型
proc-macro编译器接口TokenStream
syn将 Rust 源码解析为 ASTDeriveInput, ItemFn, Type
quote从模板生成 Rust 标记quote!{}, #variable 插值
proc-macro2syn/quote 与 proc-macro 之间的桥梁TokenStream, Span

实践建议:在自己编写派生宏之前,先研究一下像 thiserror 或 derive_more 这样简单的宏源码。cargo expand 命令(通过 cargo-expand 工具)可以显示宏展开后的代码 —— 这对调试非常有价值。

关键要点 —— 宏

  • 使用 macro_rules! 处理简单的代码生成;使用过程宏 (syn + quote) 处理复杂的派生操作。
  • 只要可能,优先选择泛型/特性而非宏 —— 宏更难调试和维护。
  • $crate 确保了卫生性;tt munching 实现了递归模式匹配。

另请参阅: 第 2 章 了解特性/泛型优于宏的场景。第 14 章 了解如何测试宏生成的代码。

flowchart LR
    A["源码"] --> B["macro_rules!<br>模式匹配"]
    A --> C["#[derive(MyMacro)]<br>过程宏"]

    B --> D["标记展开"]
    C --> E["syn: 解析 AST"]
    E --> F["转换"]
    F --> G["quote!: 生成标记"]
    G --> D

    D --> H["已编译的代码"]

    style A fill:#e8f4f8,stroke:#2980b9,color:#000
    style B fill:#d4efdf,stroke:#27ae60,color:#000
    style C fill:#fdebd0,stroke:#e67e22,color:#000
    style D fill:#fef9e7,stroke:#f1c40f,color:#000
    style E fill:#fdebd0,stroke:#e67e22,color:#000
    style F fill:#fdebd0,stroke:#e67e22,color:#000
    style G fill:#fdebd0,stroke:#e67e22,color:#000
    style H fill:#d4efdf,stroke:#27ae60,color:#000

练习:声明式宏 —— map! ★ (~15 分钟)

编写一个 map! 宏,它可以从键值对创建 HashMap:

let m = map! {
    "host" => "localhost",
    "port" => "8080",
};
assert_eq!(m.get("host"), Some(&"localhost"));

要求:支持尾随逗号和空调用 map!{}。

🔑 参考答案
macro_rules! map {
    () => { std::collections::HashMap::new() };
    ( $( $key:expr => $val:expr ),+ $(,)? ) => {{
        let mut m = std::collections::HashMap::new();
        $( m.insert($key, $val); )+
        m
    }};
}

fn main() {
    let config = map! {
        "host" => "localhost",
        "port" => "8080",
        "timeout" => "30",
    };
    assert_eq!(config.len(), 3);
    assert_eq!(config["host"], "localhost");

    let empty: std::collections::HashMap<String, String> = map!();
    assert!(empty.is_empty());

    let scores = map! { 1 => 100, 2 => 200 };
    assert_eq!(scores[&1], 100);
}

English Original

第 14 章:测试与基准模式 🟢

你将学到:

  • Rust 的三级测试体系:单元测试、集成测试和文档测试。
  • 基于属性的测试 (Property-based testing):使用 proptest 发现边界情况。
  • 基准测试:使用 criterion 进行可靠的性能测量。
  • Mock 策略:无需重型框架的依赖注入方案。

14.1 单元测试、集成测试与文档测试

Rust 语言内置了三个层级的测试体系:

#![allow(unused)]
fn main() {
// --- 单元测试:与代码位于同一文件中 ---
pub fn factorial(n: u64) -> u64 {
    (1..=n).product()
}

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_factorial_zero() {
        // (1..=0).product() 返回 1 —— 这是空范围的乘法单位元
        assert_eq!(factorial(0), 1);
    }

    #[test]
    fn test_factorial_five() {
        assert_eq!(factorial(5), 120);
    }

    #[test]
    #[cfg(debug_assertions)] // 仅在调试模式下启用溢出检查
    #[should_panic(expected = "overflow")]
    fn test_factorial_overflow() {
        // ⚠️ 此测试仅在调试模式下通过 (开启了溢出检查)。
        // 在发布模式 (`cargo test --release`) 下,u64 算术会静默回绕,
        // 且不会发生 panic。为了发布模式的安全性,请使用 `checked_mul` 
        // 或设置 `overflow-checks = true` 的配置项。
        factorial(100); // 应该在溢出时 panic
    }

    #[test]
    fn test_with_result() -> Result<(), Box<dyn std::error::Error>> {
        // 测试可以返回 Result —— 内部可以使用 `?`!
        let value: u64 = "42".parse()?;
        assert_eq!(value, 42);
        Ok(())
    }
}
}
#![allow(unused)]
fn main() {
// --- 集成测试:位于 tests/ 目录中 ---
// tests/integration_test.rs
// 这些测试仅针对你的 crate 的 公共 API

use my_crate::factorial;

#[test]
fn test_factorial_from_outside() {
    assert_eq!(factorial(10), 3_628_800);
}
}
#![allow(unused)]
fn main() {
// --- 文档测试:位于文档注释中 ---
/// 计算 `n` 的阶乘。
///
/// # 示例
///
/// ```
/// use my_crate::factorial;
/// assert_eq!(factorial(5), 120);
/// ```
///
/// # Panics
///
/// 如果结果超出 `u64` 范围则会发生 Panic。
///
/// ```should_panic
/// my_crate::factorial(100);
/// ```
pub fn factorial(n: u64) -> u64 {
    (1..=n).product()
}
// 文档测试会被 `cargo test` 编译并运行 —— 它们能确保示例代码的真实有效。
}

测试固件 (Fixtures) 与设置

#![allow(unused)]
fn main() {
#[cfg(test)]
mod tests {
    use super::*;

    // 共享设置 —— 创建一个辅助函数
    fn setup_database() -> TestDb {
        let db = TestDb::new_in_memory();
        db.run_migrations();
        db.seed_test_data();
        db
    }

    #[test]
    fn test_user_creation() {
        let db = setup_database();
        let user = db.create_user("Alice", "[email protected]").unwrap();
        assert_eq!(user.name, "Alice");
    }

    #[test]
    fn test_user_deletion() {
        let db = setup_database();
        db.create_user("Bob", "[email protected]").unwrap();
        assert!(db.delete_user("Bob").is_ok());
        assert!(db.get_user("Bob").is_none());
    }

    // 使用 Drop 实现清理 (RAII):
    struct TempDir {
        path: std::path::PathBuf,
    }

    impl TempDir {
        fn new() -> Self {
            // Cargo.toml: rand = "0.8"
            let path = std::env::temp_dir().join(format!("test_{}", rand::random::<u32>()));
            std::fs::create_dir_all(&path).unwrap();
            TempDir { path }
        }
    }

    impl Drop for TempDir {
        fn drop(&mut self) {
            let _ = std::fs::remove_dir_all(&self.path);
        }
    }

    #[test]
    fn test_file_operations() {
        let dir = TempDir::new(); // 已创建
        std::fs::write(dir.path.join("test.txt"), "hello").unwrap();
        assert!(dir.path.join("test.txt").exists());
    } // dir 在此处被释放 (drop) → 临时目录被清理
}
}

基于属性的测试 (proptest)

与其测试特定的值,不如测试那些应该 始终成立 的“属性 (Properties)”:

#![allow(unused)]
fn main() {
// Cargo.toml: proptest = "1"
use proptest::prelude::*;

fn reverse(v: &[i32]) -> Vec<i32> {
    v.iter().rev().cloned().collect()
}

proptest! {
    #[test]
    fn test_reverse_twice_is_identity(v in prop::collection::vec(any::<i32>(), 0..100)) {
        // 属性:反转两次会得到原始值
        assert_eq!(reverse(&reverse(&v)), v);
    }

    #[test]
    fn test_reverse_preserves_length(v in prop::collection::vec(any::<i32>(), 0..100)) {
        assert_eq!(reverse(&v).len(), v.len());
    }

    #[test]
    fn test_sort_is_idempotent(mut v in prop::collection::vec(any::<i32>(), 0..100)) {
        v.sort();
        let sorted_once = v.clone();
        v.sort();
        assert_eq!(v, sorted_once); // 排序两次 = 排序一次
    }

    #[test]
    fn test_parse_roundtrip(x in any::<f64>().prop_filter("finite", |x| x.is_finite())) {
        // 属性:先格式化再解析会得到原始值
        let s = format!("{x}");
        let parsed: f64 = s.parse().unwrap();
        prop_assert!((x - parsed).abs() < f64::EPSILON);
    }
}
}

何时使用 proptest:当你需要测试一个具有庞大输入空间的函数,并希望确保它在那些你没想到的边界情况下也能工作时。proptest 会生成数百个随机输入,并在失败时将案例自动最小化 (Shrinking),从而找出最小复现用例。

使用 criterion 进行基准测试

#![allow(unused)]
fn main() {
// Cargo.toml:
// [dev-dependencies]
// criterion = { version = "0.5", features = ["html_reports"] }
//
// [[bench]]
// name = "my_benchmarks"
// harness = false

// benches/my_benchmarks.rs
use criterion::{criterion_group, criterion_main, Criterion, black_box};

fn fibonacci(n: u64) -> u64 {
    match n {
        0 | 1 => n,
        _ => fibonacci(n - 1) + fibonacci(n - 2),
    }
}

fn bench_fibonacci(c: &mut Criterion) {
    c.bench_function("fibonacci 20", |b| {
        b.iter(|| fibonacci(black_box(20)))
    });

    // 比较不同的实现:
    let mut group = c.benchmark_group("fibonacci_compare");
    for size in [10, 15, 20, 25] {
        group.bench_with_input(
            criterion::BenchmarkId::from_parameter(size),
            &size,
            |b, &size| b.iter(|| fibonacci(black_box(size))),
        );
    }
    group.finish();
}

criterion_group!(benches, bench_fibonacci);
criterion_main!(benches);

// 运行:cargo bench
// 在 target/criterion/ 目录下生成 HTML 报告
}

无框架的 Mock 策略

Rust 的特性 (trait) 系统提供了天然的依赖注入方式 —— 无需 Mock 框架:

#![allow(unused)]
fn main() {
// 通过特性定义行为
trait Clock {
    fn now(&self) -> std::time::Instant;
}

trait HttpClient {
    fn get(&self, url: &str) -> Result<String, String>;
}

// 生产环境下的实现
struct RealClock;
impl Clock for RealClock {
    fn now(&self) -> std::time::Instant { std::time::Instant::now() }
}

// 服务依赖于抽象
struct CacheService<C: Clock, H: HttpClient> {
    clock: C,
    client: H,
    ttl: std::time::Duration,
}

impl<C: Clock, H: HttpClient> CacheService<C, H> {
    fn fetch(&self, url: &str) -> Result<String, String> {
        // 利用 self.clock 和 self.client —— 这是可注入的
        self.client.get(url)
    }
}

// 使用 Mock 实现进行测试 —— 无需任何框架!
#[cfg(test)]
mod tests {
    use super::*;

    struct MockClock {
        fixed_time: std::time::Instant,
    }
    impl Clock for MockClock {
        fn now(&self) -> std::time::Instant { self.fixed_time }
    }

    struct MockHttpClient {
        response: String,
    }
    impl HttpClient for MockHttpClient {
        fn get(&self, _url: &str) -> Result<String, String> {
            Ok(self.response.clone())
        }
    }

    #[test]
    fn test_cache_service() {
        let service = CacheService {
            clock: MockClock { fixed_time: std::time::Instant::now() },
            client: MockHttpClient { response: "cached data".into() },
            ttl: std::time::Duration::from_secs(300),
        };

        assert_eq!(service.fetch("http://example.com").unwrap(), "cached data");
    }
}
}

测试哲学:在集成测试中优先使用真实的依赖,在单元测试中使用基于特性的 Mock。除非你的依赖图极其复杂,否则应避免使用 Mock 框架 —— Rust 的特性泛型自然地处理了绝大多数情况。

关键要点 —— 测试

  • 文档测试 (///) 同时兼具文档和回归测试的功能 —— 它们会被编译并运行。
  • proptest 会生成随机输入来寻找那些你永远不会手动去写的边界情况。
  • criterion 提供了统计学上严谨的基准测试,并带有 HTML 报告。
  • 通过特性泛型 + 测试双身 (Test doubles) 进行 Mock,而不是使用 Mock 框架。

另请参阅: 第 13 章 关于测试宏生成的代码。第 15 章 关于模块布局如何影响测试组织。


练习:使用 proptest 进行基于属性的测试 ★★ (~25 分钟)

编写一个 SortedVec<T: Ord> 包装器,它必须维护一个“排序不变式”。使用 proptest 来验证:

  1. 在任何插入序列之后,内部向量始终是已排序的。
  2. contains() 的结果与标准库的 Vec::contains() 一致。
  3. 长度等于插入次数。
🔑 参考答案
#[derive(Debug)]
struct SortedVec<T: Ord> {
    inner: Vec<T>,
}

impl<T: Ord> SortedVec<T> {
    fn new() -> Self { SortedVec { inner: Vec::new() } }

    fn insert(&mut self, value: T) {
        let pos = self.inner.binary_search(&value).unwrap_or_else(|p| p);
        self.inner.insert(pos, value);
    }

    fn contains(&self, value: &T) -> bool {
        self.inner.binary_search(value).is_ok()
    }

    fn len(&self) -> usize { self.inner.len() }
    fn as_slice(&self) -> &[T] { &self.inner }
}

#[cfg(test)]
mod tests {
    use super::*;
    use proptest::prelude::*;

    proptest! {
        #[test]
        fn always_sorted(values in proptest::collection::vec(-1000i32..1000, 0..100)) {
            let mut sv = SortedVec::new();
            for v in &values {
                sv.insert(*v);
            }
            for w in sv.as_slice().windows(2) {
                prop_assert!(w[0] <= w[1]);
            }
            prop_assert_eq!(sv.len(), values.len());
        }

        #[test]
        fn contains_matches_stdlib(values in proptest::collection::vec(0i32..50, 1..30)) {
            let mut sv = SortedVec::new();
            for v in &values {
                sv.insert(*v);
            }
            for v in &values {
                prop_assert!(sv.contains(v));
            }
            prop_assert!(!sv.contains(&9999));
        }
    }
}

English Original

第 15 章:Crate 架构与 API 设计 🟡

你将学到:

  • 模块布局惯例 与重导出策略。
  • 完善的 crate 公共 API 设计清单。
  • 易用的参数模式:impl Into、AsRef、Cow。
  • “以解析代替校验”:使用 TryFrom 和经过验证的类型。
  • 特性标志 (Feature flags)、条件编译与工作空间组织。

15.1 模块布局惯例

my_crate/
├── Cargo.toml
├── src/
│   ├── lib.rs          # Crate 根节点 —— 重导出与公共 API
│   ├── config.rs       # 功能模块
│   ├── parser/         # 带有子模块的复杂模块
│   │   ├── mod.rs      # 或在父级目录下的 parser.rs (Rust 2018+ 风格)
│   │   ├── lexer.rs
│   │   └── ast.rs
│   ├── error.rs        # 错误类型
│   └── utils.rs        # 内部辅助工具 (pub(crate))
├── tests/
│   └── integration.rs  # 集成测试
├── benches/
│   └── perf.rs         # 基准测试
└── examples/
    └── basic.rs        # 通过 cargo run --example basic 运行
#![allow(unused)]
fn main() {
// lib.rs —— 通过重导出来精挑细选你的公共 API:
mod config;
mod error;
mod parser;
mod utils;

// 重导出用户需要的项:
pub use config::Config;
pub use error::Error;
pub use parser::Parser;

// 公共类型位于 crate 根部 —— 用户只需编写:
// use my_crate::Config;
// 而不是:use my_crate::config::Config;
}

可见性修饰符:

修饰符对谁可见
pub所有人
pub(crate)仅限当前 crate
pub(super)父模块
pub(in path)指定的祖先模块
(不写)当前模块及其子模块

公共 API 设计清单

  1. 接收引用,返回所有权类型 —— fn process(input: &str) -> String。
  2. 在参数中使用 impl Trait —— 相比 fn read<R: Read>(r: R),使用 fn read(r: impl Read) 能让签名更整洁。
  3. 返回 Result 而非使用 panic! —— 让调用者决定如何处理错误。
  4. 实现标准特性 —— 如 Debug、Display、Clone、Default、From/Into。
  5. 使非法状态无法被表示 —— 使用类型状态 (Type states) 和新类型 (Newtypes)。
  6. 针对复杂的配置使用构建器模式 (Builder pattern) —— 如果有必填字段,可结合类型状态。
  7. 密封那些你不希望用户实现的特性 —— pub trait Sealed: private::Sealed {}。
  8. 为类型和函数标记 #[must_use] —— 防止调用者无意中忽略重要的 Result、Guard 或返回值。对于任何忽视其返回值几乎必然导致 bug 的类型,都应应用此标记:
    #![allow(unused)]
    fn main() {
    #[must_use = "立即丢弃 guard 会导致锁被立即释放"]
    pub struct LockGuard<'a, T> { /* ... */ }
    
    #[must_use]
    pub fn validate(input: &str) -> Result<ValidInput, ValidationError> { /* ... */ }
    }
#![allow(unused)]
fn main() {
// 密封特性 (Sealed trait) 模式 —— 用户可以使用但无法实现:
mod private {
    pub trait Sealed {}
}

pub trait DatabaseDriver: private::Sealed {
    fn connect(&self, url: &str) -> Connection;
}

// 只有当前 crate 中的类型才能实现 Sealed → 也就意味着只有我们能实现 DatabaseDriver
pub struct PostgresDriver;
impl private::Sealed for PostgresDriver {}
impl DatabaseDriver for PostgresDriver {
    fn connect(&self, url: &str) -> Connection { /* ... */ }
}
}

#[non_exhaustive] —— 为公共枚举和结构体打上此标记,这样添加新变体或字段就不属于破坏性变更。下游 crate 在 match 语句中必须使用通配符分支 (_ =>),且无法通过结构体字面量语法构造该类型:

#![allow(unused)]
fn main() {
#[non_exhaustive]
pub enum DiagError {
    Timeout,
    HardwareFault,
    // 在未来的版本中添加新变体 不会 破坏语义化版本 (semver)。
}
}

易用的参数模式 —— impl Into、AsRef、Cow

Rust 中影响力最大的 API 模式之一是在函数参数中接收 最通用的类型,这样调用者就不必在每个调用处重复编写 .to_string()、&*s 或 .as_ref()。这是 Rust 特有的“在接收时保持宽容”的设计哲学。

impl Into<T> —— 接收任何可转换的类型

#![allow(unused)]
fn main() {
// ❌ 不便之处:调用者必须手动转换
fn connect(host: String, port: u16) -> Connection {
    // ...
}
connect("localhost".to_string(), 5432);  // 烦人的 .to_string()
connect(hostname.clone(), 5432);          // 如果我们已经有了 String,这会导致不必要的克隆

// ✅ 易用:接收任何能转换为 String 的类型
fn connect(host: impl Into<String>, port: u16) -> Connection {
    let host = host.into();  // 在函数内部进行一次转换
    // ...
}
connect("localhost", 5432);     // &str —— 零摩擦
connect(hostname, 5432);        // String —— 直接转移所有权,无克隆
}

之所以可行,是因为 Rust 的 From/Into 特性对提供了“一揽子转换”功能。当你接收 impl Into<T> 时,你的意思是:“给我任何知道如何变成 T 的东西。”

AsRef<T> —— 作为引用进行借用

AsRef<T> 是 Into<T> 在借用方面的对应物。当你只需要 读取 数据而不需要获取其所有权时,请使用它:

#![allow(unused)]
fn main() {
use std::path::Path;

// ❌ 强制调用者转换为 &Path
fn file_exists(path: &Path) -> bool {
    path.exists()
}
file_exists(Path::new("/tmp/test.txt"));  // 比较笨拙

// ✅ 接收任何能作为 &Path 使用的类型
fn file_exists(path: impl AsRef<Path>) -> bool {
    path.as_ref().exists()
}
file_exists("/tmp/test.txt");                    // &str ✅
file_exists(String::from("/tmp/test.txt"));      // String ✅
file_exists(Path::new("/tmp/test.txt"));         // &Path ✅
file_exists(PathBuf::from("/tmp/test.txt"));     // PathBuf ✅

// 对于类字符串参数,使用同样的模式:
fn log_message(msg: impl AsRef<str>) {
    println!("[LOG] {}", msg.as_ref());
}
log_message("hello");                    // &str ✅
log_message(String::from("hello"));      // String ✅
}

Cow<T> —— 写时克隆 (Clone on Write)

Cow<'a, T> (Clone on Write) 会将内存分配推迟到需要修改时。它持有借用的 &T 或拥有所有权类型的 T::Owned。这非常适合那些大多数调用都不需要修改数据的场景:

#![allow(unused)]
fn main() {
use std::borrow::Cow;

/// 对诊断消息进行规范化 —— 仅在需要修改时才进行分配。
fn normalize_message(msg: &str) -> Cow<'_, str> {
    if msg.contains('\t') || msg.contains('\r') {
        // 必须进行分配 —— 我们需要修改内容
        Cow::Owned(msg.replace('\t', "    ").replace('\r', ""))
    } else {
        // 无分配 —— 直接借用原始字符串
        Cow::Borrowed(msg)
    }
}

// 大多数消息在没有分配的情况下通过:
let clean = normalize_message("All tests passed");          // 借用方式 —— 免费
let fixed = normalize_message("Error:\tfailed\r\n");        // 所有权方式 —— 发生了分配

// Cow<str> 实现了 Deref<Target=str>,因此它的用法与 &str 类似:
println!("{}", clean);
println!("{}", fixed.to_uppercase());
}

快速参考:该使用哪一个

你是否需要在函数内部获取数据的所有权?
├── 是 → impl Into<T>
│         "给我任何能变成 T 的东西"
└── 否  → 你是否只需要读取它?
     ├── 是 → impl AsRef<T> 或 &T
     │         "给我任何我能作为 &T 借用的东西"
     └── 也许 (有时可能需要修改?)
          └── Cow<'_, T>
              "尽可能借用,仅在必须时克隆"
模式所有权状态分配情况何时使用
&str借用从不简单的字符串参数
impl AsRef<str>借用从不接收 String, &str 等 —— 仅限读取
impl Into<String>所有权转换时可能发生接收 &str, String —— 将存储/拥有它
Cow<'_, str>二者择一仅在修改时处理过程通常不修改数据的场景
&[u8] / AsRef<[u8]>借用从不面向字节的 API

Borrow<T> vs AsRef<T>:二者都能提供 &T,但 Borrow<T> 额外保证了原始形式与借用形式之间的 Eq、Ord 和 Hash 是 一致的。这就是为什么 HashMap<String, V>::get() 接收的是 &Q where String: Borrow<Q> —— 而不是 AsRef。当借用形式被用作查找键 (Lookup key) 时请使用 Borrow;对于通用的“给我一个引用”参数,请使用 AsRef。

在 API 中组合使用转换

#![allow(unused)]
fn main() {
/// 一个设计良好的、使用了易用参数模式的诊断 API:
pub struct DiagRunner {
    name: String,
    config_path: PathBuf,
    results: HashMap<String, TestResult>,
}

impl DiagRunner {
    /// 针对名称接收任何类字符串类型,针对配置接收任何类路径类型。
    pub fn new(
        name: impl Into<String>,
        config_path: impl Into<PathBuf>,
    ) -> Self {
        DiagRunner {
            name: name.into(),
            config_path: config_path.into(),
        }
    }

    /// 为只读查找接收任何 AsRef<str>。
    pub fn get_result(&self, test_name: impl AsRef<str>) -> Option<&TestResult> {
        self.results.get(test_name.as_ref())
    }
}

// 所有这些调用在调用方都没有摩擦:
let runner = DiagRunner::new("GPU Diag", "/etc/diag_tool/config.json");
let runner = DiagRunner::new(format!("Diag-{}", node_id), config_path);
let runner = DiagRunner::new(name_string, path_buf);
}

15.2 案例研究:设计公共 Crate API —— 演进过程

一个将强依赖字符串的内部 API 演变为易用、类型安全的公共 API 的真实案例。考虑一个配置文件解析 crate:

重构前 (强依赖字符串,容易被误用):

#![allow(unused)]
fn main() {
// ❌ 所有参数都是字符串 —— 缺乏编译时校验
pub fn parse_config(path: &str, format: &str, strict: bool) -> Result<Config, String> {
    // 哪些格式是有效的?"json"?"JSON"?"Json"?
    // path 是文件路径还是 URL?
    // "strict" 到底是什么意思?
    todo!()
}
}

重构后 (类型安全,自带文档说明):

#![allow(unused)]
fn main() {
use std::path::Path;

/// 支持的配置格式。
#[derive(Debug, Clone, Copy)]
#[non_exhaustive]  // 添加格式不会破坏下游代码
pub enum Format {
    Json,
    Toml,
    Yaml,
}

/// 控制解析的严格程度。
#[derive(Debug, Clone, Copy, Default)]
pub enum Strictness {
    /// 拒绝未知字段 (库的默认行为)
    #[default]
    Strict,
    /// 忽略未知字段 (适用于向前兼容的配置)
    Lenient,
}

pub fn parse_config(
    path: &Path,          // 类型强制:必须是文件系统路径
    format: Format,       // 枚举:不可能传入无效格式
    strictness: Strictness,  // 命名的选项,而非裸布尔值
) -> Result<Config, ConfigError> {
    todo!()
}
}

改进之处:

维度重构前重构后
格式校验运行时的字符串比较编译时的枚举
路径类型原始的 &str (可以是任何内容)&Path (文件系统专用)
严格程度神秘的 bool自带文档说明的枚举
错误类型String (不透明)ConfigError (结构化的)
可扩展性容易造成破坏性变更#[non_exhaustive]

“以解析代替校验” —— TryFrom 与验证过的类型

“以解析代替校验 (Parse, don’t validate)”原则指出:不要检查数据后依然传递原始的、未经检查的形式 —— 相反,应将其解析为一种只有在数据有效时才能存在的类型。 Rust 的 TryFrom 特性是实现这一目标的标准工具。

问题所在:只有校验,没有约束

#![allow(unused)]
fn main() {
// ❌ 先校验后使用:检查后没有什么能阻止使用无效值
fn process_port(port: u16) {
    if port == 0 || port > 65535 {
        panic!("无效端口");           // 我们检查了,但是...
    }
    start_server(port);                    // 如果有人直接调用 start_server(0) 怎么办?
}

// ❌ 强依赖字符串:电子邮件只是一个 String —— 任何垃圾数据都能混进来
fn send_email(to: String, body: String) {
    // `to` 真的是有效的邮件地址吗?我们不知道。
    // 有人可能会传入 "not-an-email",而我们只有到了 SMTP 服务器端才能发现。
}
}

解决方案:通过 TryFrom 解析为经过验证的新类型

use std::convert::TryFrom;
use std::fmt;

/// 一个经过验证的 TCP 端口号 (1–65535)。
/// 如果你拥有一个 `Port` 实例,它就被保证是有效的。
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct Port(u16);

impl TryFrom<u16> for Port {
    type Error = PortError;

    fn try_from(value: u16) -> Result<Self, Self::Error> {
        if value == 0 {
            Err(PortError::Zero)
        } else {
            Ok(Port(value))
        }
    }
}

impl Port {
    pub fn get(&self) -> u16 { self.0 }
}

#[derive(Debug)]
pub enum PortError {
    Zero,
    InvalidFormat,
}

impl fmt::Display for PortError {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        match self {
            PortError::Zero => write!(f, "端口号不能为零"),
            PortError::InvalidFormat => write!(f, "无效的端口格式"),
        }
    }
}

impl std::error::Error for PortError {}

// 现在类型系统强制了有效性:
fn start_server(port: Port) {
    // 无需校验 —— Port 只能通过 TryFrom 构造,
    // 而 TryFrom 已经验证过它的有效性。
    println!("正在监听端口 {}", port.get());
}

// 使用方式:
fn main() -> Result<(), Box<dyn std::error::Error>> {
    let port = Port::try_from(8080)?;   // ✅ 在系统边界处进行一次验证
    start_server(port);                  // 下游的任何地方都无需重复验证

    let bad = Port::try_from(0);         // ❌ 返回 Err(PortError::Zero)
    Ok(())
}

字符串解析与 FromStr

对于通常从文本 (CLI 参数、配置文件) 中解析出的类型,请实现 FromStr:

#![allow(unused)]
fn main() {
use std::str::FromStr;

impl FromStr for Port {
    type Err = PortError;

    fn from_str(s: &str) -> Result<Self, Self::Err> {
        let n: u16 = s.parse().map_err(|_| PortError::InvalidFormat)?;
        Port::try_from(n)
    }
}

// 现在可以配合 .parse() 使用:
let port: Port = "8080".parse()?;   // 一步完成验证

// 也可以配合 clap 进行 CLI 解析:
// #[derive(Parser)]
// struct Args {
//     #[arg(short, long)]
//     port: Port,   // clap 会自动调用 FromStr
// }
}

总结:校验 vs 解析

方法是否检查数据?编译器是否强制执行有效性?是否需要重复校验?
运行时检查 (if/assert)✅❌每个函数边界都需要
验证后的新类型 + TryFrom✅✅从不需要 —— 类型就是证明

规则是:在边界处解析,在内部各处都使用验证后的类型。 原始字符串、整数和字节切片进入你的系统,通过 TryFrom/FromStr 解析为经过验证的类型,之后类型系统就会保证它们的有效性。

特性标志与条件编译

# Cargo.toml
[features]
default = ["json"]          # 默认开启
json = ["dep:serde_json"]   # 开启 JSON 支持
xml = ["dep:quick-xml"]     # 开启 XML 支持
full = ["json", "xml"]      # 元特性:开启所有功能

[dependencies]
serde = "1"
serde_json = { version = "1", optional = true }
quick-xml = { version = "0.31", optional = true }
#![allow(unused)]
fn main() {
// 基于特性的条件编译:
#[cfg(feature = "json")]
pub fn to_json<T: serde::Serialize>(value: &T) -> String {
    serde_json::to_string(value).unwrap()
}

#[cfg(feature = "xml")]
pub fn to_xml<T: serde::Serialize>(value: &T) -> String {
    quick_xml::se::to_string(value).unwrap()
}

// 如果一个必须的特性没有开启,则产生编译错误:
#[cfg(not(any(feature = "json", feature = "xml")))]
compile_error!("必须至少开启一个格式特性 (json, xml)");
}

最佳实践:

  • 保持 default 特性尽可能少 —— 让用户按需开启。
  • 使用 dep: 语法 (Rust 1.60+) 来定义可选依赖,避免创建隐式特性。
  • 在你的 README 和 crate 级文档中记录特性标志。

工作空间 (Workspace) 组织

对于大型项目,请使用 Cargo 工作空间来共享依赖项和编译产物:

# 根目录的 Cargo.toml
[workspace]
members = [
    "core",         // 共享的类型和特性
    "parser",       // 解析库
    "server",       // 二进制程序 —— 主应用
    "client",       // 客户端库
    "cli",          // CLI 二进制程序
]

# 共享依赖版本:
[workspace.dependencies]
serde = { version = "1", features = ["derive"] }
tokio = { version = "1", features = ["full"] }
tracing = "0.1"

# 在每个成员的 Cargo.toml 中:
# [dependencies]
# serde = { workspace = true }

优势:

  • 单一的 Cargo.lock —— 所有 crate 使用相同的依赖版本。
  • cargo test --workspace 运行所有测试。
  • 共享编译缓存 —— 编译一个 crate 会使所有 crate 受益。
  • 组件之间拥有清晰的依赖边界。

.cargo/config.toml:项目级配置

.cargo/config.toml 文件(位于工作空间根目录或 $HOME/.cargo/ 中)可以在不修改 Cargo.toml 的情况下自定义 Cargo 行为:

# .cargo/config.toml

# 当前工作空间的默认目标平台
[build]
target = "x86_64-unknown-linux-gnu"

# 自定义运行程序 —— 例如,通过 QEMU 运行交叉编译的二进制程序
[target.aarch64-unknown-linux-gnu]
runner = "qemu-aarch64-static"
linker = "aarch64-linux-gnu-gcc"

# Cargo 别名 —— 自定义快捷命令
[alias]
xt = "test --workspace --release"        # cargo xt = 以发布模式运行所有测试
ci = "clippy --workspace -- -D warnings" # cargo ci = 将警告视为错误进行 lint 检查
cov = "llvm-cov --workspace"             # cargo cov = 覆盖率检查 (需要 cargo-llvm-cov)

# 编译脚本的环境变量
[env]
IPMI_LIB_PATH = "/usr/lib/bmc"

编译时环境变量:env!() 与 option_env!()

Rust 可以在编译时将环境变量嵌入二进制程序中 —— 对于版本字符串、编译元数据和配置非常有用:

#![allow(unused)]
fn main() {
// env!() —— 如果变量缺失,则在编译时产生 panic
const VERSION: &str = env!("CARGO_PKG_VERSION"); // 来自 Cargo.toml 的 "0.1.0"
const PKG_NAME: &str = env!("CARGO_PKG_NAME");   // 来自 Cargo.toml 的 crate 名称

// option_env!() —— 返回 Option<&str>,如果缺失则不产生 panic
const BUILD_SHA: Option<&str> = option_env!("GIT_SHA");
const BUILD_TIME: Option<&str> = option_env!("BUILD_TIMESTAMP");

fn print_version() {
    println!("{PKG_NAME} v{VERSION}");
    if let Some(sha) = BUILD_SHA {
        println!("  提交 ID: {sha}");
    }
    if let Some(time) = BUILD_TIME {
        println!("  编译时间: {time}");
    }
}
}

Cargo 会自动设置许多有用的环境变量:

变量值使用场景
CARGO_PKG_VERSION"1.2.3"版本报告
CARGO_PKG_NAME"diag_tool"二进制标识
CARGO_MANIFEST_DIRCargo.toml 的绝对路径定位测试数据文件
OUT_DIR编译输出目录build.rs 的代码生成目标

你也可以从 build.rs 中设置自定义环境变量:

// build.rs
fn main() {
    println!("cargo::rustc-env=GIT_SHA={}", git_sha());
    println!("cargo::rustc-env=BUILD_TIMESTAMP={}", timestamp());
}

cfg_attr:条件属性

cfg_attr 仅当条件满足时才会应用属性。这比 #[cfg()] 更有针对性,因为后者会包含或排除整个项:

#![allow(unused)]
fn main() {
// 仅当开启了 "serde" 特性时才派生 Serialize:
#[cfg_attr(feature = "serde", derive(serde::Serialize, serde::Deserialize))]
#[derive(Debug, Clone)]
pub struct DiagResult {
    pub fc: u32,
    pub passed: bool,
    pub message: String,
}
// 没有 "serde" 特性时:完全不需要依赖 serde
// 开启 "serde" 特性后:DiagResult 变得可序列化

// 测试时的条件属性:
#[cfg_attr(test, derive(PartialEq))]  // 仅在测试构建中派生 PartialEq
pub struct LargeStruct { /* ... */ }

// platform 特定的函数属性:
#[cfg_attr(target_os = "linux", link_name = "ioctl")]
#[cfg_attr(target_os = "freebsd", link_name = "__ioctl")]
extern "C" fn platform_ioctl(fd: i32, request: u64) -> i32;
}
模式作用
#[cfg(feature = "x")]包含或排除整个项
#[cfg_attr(feature = "x", derive(Foo))]仅在特性 “x” 开启时添加 derive(Foo)
#[cfg_attr(test, allow(unused))]仅在测试构建中抑制警告
#[cfg_attr(doc, doc = "...")]仅在运行 cargo doc 时可见的文档

cargo deny 与 cargo audit:供应链安全

# 安装安全审计工具
cargo install cargo-deny
cargo install cargo-audit

# 检查依赖项中的已知漏洞 (CVE)
cargo audit

# 进行全面检查:许可证、封禁列表、公告、来源
cargo deny check

可以在工作空间根目录通过 deny.toml 来配置 cargo deny:

# deny.toml
[advisories]
vulnerability = "deny"      # 发现已知漏洞时失败
unmaintained = "warn"        # 对停止维护的 crate 发出警告

[licenses]
allow = ["MIT", "Apache-2.0", "BSD-2-Clause", "BSD-3-Clause"]
deny = ["GPL-3.0"]          # 拒绝“左版” (Copyleft) 许可证

[bans]
multiple-versions = "warn"  # 如果同一 crate 存在多个版本则警告
deny = [
    { name = "openssl" },   # 强制改用 rustls
]

[sources]
allow-git = []              # 生产环境中不允许使用 git 依赖

文档测试:文档内部的测试

Rust 的文档注释 (///) 可以包含代码块,这些代码块会被 作为测试编译并运行:

#![allow(unused)]
fn main() {
/// 从字符串中解析诊断错误码 (Fault Code)。
///
/// # 示例
///
/// ```
/// use my_crate::parse_fc;
///
/// let fc = parse_fc("FC:12345").unwrap();
/// assert_eq!(fc, 12345);
/// ```
///
/// 无效输入会返回错误:
///
/// ```
/// use my_crate::parse_fc;
///
/// assert!(parse_fc("not-a-fc").is_err());
/// ```
pub fn parse_fc(input: &str) -> Result<u32, ParseError> {
    input.strip_prefix("FC:")
        .ok_or(ParseError::MissingPrefix)?
        .parse()
        .map_err(ParseError::InvalidNumber)
}
}
cargo test --doc  # 仅运行文档测试
cargo test        # 运行单元测试 + 集成测试 + 文档测试

模块级文档 在文件顶部使用 //!:

#![allow(unused)]
fn main() {
//! # 诊断框架 (Diagnostic Framework)
//!
//! 本 crate 提供了核心的诊断执行引擎。
//! 它支持运行诊断测试、收集结果,并经由 IPMI 向 BMC 报告。
//!
//! ## 快速开始
//!
//! ```no_run
//! use diag_framework::Framework;
//!
//! let mut fw = Framework::new("config.json")?;
//! fw.run_all_tests()?;
//! ```
}

使用 Criterion 进行基准测试

完整覆盖:关于 criterion 的完整设置、API 示例以及与 cargo bench 的对比表,请参阅第 14 章(测试与基准模式)中的使用 criterion 进行基准测试部分。以下是针对架构设计的快速参考。

在对你的 crate 的公共 API 进行基准测试时,请将基准测试代码放置在 benches/ 目录下,并专注于 热点路径 (Hot path) —— 通常是解析器、序列化器或校验边界:

cargo bench                  # 运行所有基准测试
cargo bench -- parse_config  # 运行特定的基准测试
# 结果位于 target/criterion/,附带 HTML 报告

关键要点 —— 架构与 API 设计

  • 接收最通用的类型 (impl Into、impl AsRef、Cow);返回最具体的类型信号。
  • 以解析代替校验:使用 TryFrom 创建“构造即有效”的类型。
  • 在公共枚举上使用 #[non_exhaustive] 可以防止添加新变体时造成破坏性变更。
  • #[must_use] 能捕获那些被无意忽略的重要返回值。

另请参阅: 第 10 章 了解公共 API 中的错误类型设计。第 14 章 了解如何测试你的 crate 公共 API。


练习:Crate API 重构 ★★ (~30 分钟)

将以下“强依赖字符串”的 API 重构为使用 TryFrom、新类型和构建器模式的 API:

// 重构前:容易被误用
fn create_server(host: &str, port: &str, max_conn: &str) -> Server { ... }

设计一个带有如下验证过类型的 ServerConfig:Host、Port (1–65535) 和 MaxConnections (1–10000) ,要求在解析时就能拒绝无效值。

🔑 参考答案
#[derive(Debug, Clone)]
struct Host(String);

impl TryFrom<&str> for Host {
    type Error = String;
    fn try_from(s: &str) -> Result<Self, String> {
        if s.is_empty() { return Err("主机名不能为空".into()); }
        if s.contains(' ') { return Err("主机名不能包含空格".into()); }
        Ok(Host(s.to_string()))
    }
}

#[derive(Debug, Clone, Copy)]
struct Port(u16);

impl TryFrom<u16> for Port {
    type Error = String;
    fn try_from(p: u16) -> Result<Self, String> {
        if p == 0 { return Err("端口号必须 >= 1".into()); }
        Ok(Port(p))
    }
}

#[derive(Debug, Clone, Copy)]
struct MaxConnections(u32);

impl TryFrom<u32> for MaxConnections {
    type Error = String;
    fn try_from(n: u32) -> Result<Self, String> {
        if n == 0 || n > 10_000 {
            return Err(format!("max_connections 必须在 1–10000 之间,得到了 {n}"));
        }
        Ok(MaxConnections(n))
    }
}

#[derive(Debug)]
struct ServerConfig {
    host: Host,
    port: Port,
    max_connections: MaxConnections,
}

impl ServerConfig {
    fn new(host: Host, port: Port, max_connections: MaxConnections) -> Self {
        ServerConfig { host, port, max_connections }
    }
}

fn main() {
    let config = ServerConfig::new(
        Host::try_from("localhost").unwrap(),
        Port::try_from(8080).unwrap(),
        MaxConnections::try_from(100).unwrap(),
    );
    println!("{config:?}");

    // 无效值在解析时就会被捕获:
    assert!(Host::try_from("").is_err());
    assert!(Port::try_from(0).is_err());
    assert!(MaxConnections::try_from(99999).is_err());
}

English Original

第 16 章:Async/Await 核心要点 🔴

你将学到:

  • Rust 的 Future 特性与 Go 的 goroutine 以及 Python 的 asyncio 有何不同。
  • Tokio 快速上手:生成任务、join! 以及运行时配置。
  • 常见的异步陷阱及其修复方法。
  • 何时使用 spawn_blocking 转移阻塞性工作。

16.1 Future、运行时与 async fn

Rust 的异步模型与 Go 的 goroutine 或 Python 的 asyncio 有着 本质的区别。了解以下三个概念即可开始上手:

  1. Future 是一个惰性状态机 —— 调用 async fn 不会执行任何操作;它会返回一个必须进行轮询 (poll) 的 Future。
  2. 你需要一个运行时 来轮询 Future —— 例如 tokio、async-std 或 smol。标准库定义了 Future 但不提供运行时。
  3. async fn 是语法糖 —— 编译器会将其转换为一个实现了 Future 的状态机。
#![allow(unused)]
fn main() {
// Future 只是一个特性:
pub trait Future {
    type Output;
    fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output>;
}

// async fn 脱糖后变为:
// fn fetch_data(url: &str) -> impl Future<Output = Result<Vec<u8>, Error>>
async fn fetch_data(url: &str) -> Result<Vec<u8>, reqwest::Error> {
    let response = reqwest::get(url).await?;  // .await 会让出控制权直至就绪
    let bytes = response.bytes().await?;
    Ok(bytes.to_vec())
}
}

Tokio 快速上手

# Cargo.toml
[dependencies]
tokio = { version = "1", features = ["full"] }
use tokio::time::{sleep, Duration};
use tokio::task;

#[tokio::main]
async fn main() {
    // 生成并发任务 (类似轻量级线程):
    let handle_a = task::spawn(async {
        sleep(Duration::from_millis(100)).await;
        "任务 A 已完成"
    });

    let handle_b = task::spawn(async {
        sleep(Duration::from_millis(50)).await;
        "任务 B 已完成"
    });

    // 对二者执行 .await —— 它们并发运行,而非顺序运行:
    let (a, b) = tokio::join!(handle_a, handle_b);
    println!("{}, {}", a.unwrap(), b.unwrap());
}

异步常见陷阱

陷阱发生原因修复方法
异步中阻塞std::thread::sleep 或 CPU 密集任务阻塞了执行器使用 tokio::task::spawn_blocking 或 rayon
Send 约束错误跨 .await 持有的 Future 包含非 Send 类型 (如 Rc、MutexGuard)重构代码,在 .await 之前丢弃非 Send 的值
Future 未轮询调用 async fn 但未执行 .await 或 spawn —— 导致没有任何反应务必对返回的 Future 执行 .await 或 tokio::spawn
跨 .await 持有 MutexGuardstd::sync::MutexGuard 是 !Send;异步任务可能会在不同线程上恢复执行使用 tokio::sync::Mutex 或在 .await 之前手动 drop
意外的顺序执行let a = foo().await; let b = bar().await; 会由于依次等待而顺序执行使用 tokio::join! 或 tokio::spawn 来实现并发
#![allow(unused)]
fn main() {
// ❌ 阻塞异步执行器:
async fn bad() {
    std::thread::sleep(std::time::Duration::from_secs(5)); // 阻塞整个线程!
}

// ✅ 转移阻塞工作:
async fn good() {
    tokio::task::spawn_blocking(|| {
        std::thread::sleep(std::time::Duration::from_secs(5)); // 在阻塞线程池中运行
    }).await.unwrap();
}
}

深度异步覆盖:关于 Stream、select!、取消安全性、结构化并发以及 tower 中间件,请参阅我们的 Async Rust 进阶指南。本节仅涵盖阅读和编写基础异步代码所需的核心内容。

任务生成与结构化并发

Tokio 的 spawn 会创建一个新的异步任务 —— 类似于 thread::spawn 但更轻量:

use tokio::task;
use tokio::time::{sleep, Duration};

#[tokio::main]
async fn main() {
    // 生成三个并发任务
    let h1 = task::spawn(async {
        sleep(Duration::from_millis(200)).await;
        "已获取用户个人资料"
    });

    let h2 = task::spawn(async {
        sleep(Duration::from_millis(100)).await;
        "已获取订单历史"
    });

    let h3 = task::spawn(async {
        sleep(Duration::from_millis(150)).await;
        "已获取推荐信息"
    });

    // 并发(而非顺序!)等待所有三项任务
    let (r1, r2, r3) = tokio::join!(h1, h2, h3);
    println!("{}", r1.unwrap());
    println!("{}", r2.unwrap());
    println!("{}", r3.unwrap());
}

join! vs try_join! vs select!:

宏行为使用场景
join!等待所有 Future 完成所有任务都必须完成的情况
try_join!等待所有任务,但在遇到第一个 Err 时短路任务返回 Result 的情况
select!在第一个 Future 完成时返回用于超时处理、任务取消
use tokio::time::{timeout, Duration};

async fn fetch_with_timeout() -> Result<String, Box<dyn std::error::Error>> {
    let result = timeout(Duration::from_secs(5), async {
        // 模拟慢速网络调用
        tokio::time::sleep(Duration::from_millis(100)).await;
        Ok::<_, Box<dyn std::error::Error>>("数据".to_string())
    }).await??; // 第一个 ? 解包 Elapsed (超时),第二个 ? 解包内部 Result

    Ok(result)
}

Send 约束以及为何 Future 必须满足 Send

当你使用 tokio::spawn 生成一个 Future 时,它可能会在不同的 OS 线程上恢复执行。这意味着该 Future 必须满足 Send 约束。常见陷阱如下:

use std::rc::Rc;

async fn not_send() {
    let rc = Rc::new(42); // Rc 不满足 Send
    tokio::time::sleep(std::time::Duration::from_millis(10)).await;
    println!("{}", rc); // rc 跨 .await 被持有 —— 导致 Future 不满足 Send
}

// 修复 1:在 .await 之前丢弃 (Drop)
async fn fixed_drop() {
    let data = {
        let rc = Rc::new(42);
        *rc // 拷贝出其值
    }; // rc 在此处被丢弃
    tokio::time::sleep(std::time::Duration::from_millis(10)).await;
    println!("{}", data); // 只是一个满足 Send 的 i32
}

// 修复 2:使用 Arc 替代 Rc
async fn fixed_arc() {
    let arc = std::sync::Arc::new(42); // Arc 满足 Send
    tokio::time::sleep(std::time::Duration::from_millis(10)).await;
    println!("{}", arc); // ✅ Future 满足 Send
}

另请参阅: 第 5 章 了解同步通道。第 6 章 了解 OS 线程与异步任务的对比。

关键要点 —— 异步

  • async fn 返回一个 惰性 Future —— 除非你执行 .await 或 spawn 它,否则不会运行任何代码。
  • 在异步上下文中使用 tokio::task::spawn_blocking 处理重型 CPU 任务或阻塞性工作。
  • 不要跨 .await 持有 std::sync::MutexGuard —— 请改用 tokio::sync::Mutex。
  • 被生成 (spawn) 的 Future 必须满足 Send 约束 —— 在跨 .await 点之前应丢弃非 Send 类型。

练习:带超时的并发获取器 ★★ (~25 分钟)

编写一个异步函数 fetch_all,它生成三个 tokio::spawn 任务,每个任务都使用 tokio::time::sleep 模拟网络调用。使用 tokio::try_join! 将三个任务组合在一起,并将其包装在 tokio::time::timeout(Duration::from_secs(5), ...) 中。返回 Result<Vec<String>, ...>,或者在任何任务失败或截止时间到期时返回错误。

🔑 参考答案
use tokio::time::{sleep, timeout, Duration};

async fn fake_fetch(name: &'static str, delay_ms: u64) -> Result<String, String> {
    sleep(Duration::from_millis(delay_ms)).await;
    Ok(format!("{name}: OK"))
}

async fn fetch_all() -> Result<Vec<String>, Box<dyn std::error::Error>> {
    let deadline = Duration::from_secs(5);

    let (a, b, c) = timeout(deadline, async {
        let h1 = tokio::spawn(fake_fetch("svc-a", 100));
        let h2 = tokio::spawn(fake_fetch("svc-b", 200));
        let h3 = tokio::spawn(fake_fetch("svc-c", 150));
        tokio::try_join!(h1, h2, h3)
    })
    .await??;

    Ok(vec![a?, b?, c?])
}

#[tokio::main]
async fn main() {
    let results = fetch_all().await.unwrap();
    for r in &results {
        println!("{r}");
    }
}

练习题

练习 1:类型安全的状态机 ★★ (~30 分钟)

使用类型状态 (Type-state) 模式构建一个红绿灯状态机。该灯必须按照 红 → 绿 → 黄 → 红 的顺序进行切换,且不允许出现其他顺序。

🔑 参考答案
use std::marker::PhantomData;

struct Red;
struct Green;
struct Yellow;

struct TrafficLight<State> {
    _state: PhantomData<State>,
}

impl TrafficLight<Red> {
    fn new() -> Self {
        println!("🔴 红灯 — 停止");
        TrafficLight { _state: PhantomData }
    }

    fn go(self) -> TrafficLight<Green> {
        println!("🟢 绿灯 — 行驶");
        TrafficLight { _state: PhantomData }
    }
}

impl TrafficLight<Green> {
    fn caution(self) -> TrafficLight<Yellow> {
        println!("🟡 黄灯 — 注意");
        TrafficLight { _state: PhantomData }
    }
}

impl TrafficLight<Yellow> {
    fn stop(self) -> TrafficLight<Red> {
        println!("🔴 红灯 — 停止");
        TrafficLight { _state: PhantomData }
    }
}

fn main() {
    let light = TrafficLight::new(); // 红灯
    let light = light.go();          // 绿灯
    let light = light.caution();     // 黄灯
    let light = light.stop();        // 红灯

    // light.caution(); // ❌ 编译错误:红灯状态下没有 `caution` 方法
    // TrafficLight::new().stop(); // ❌ 编译错误:红灯状态下没有 `stop` 方法
}

关键要点:非法的状态切换会导致编译错误,而不是运行时 panic。


练习 2:使用 PhantomData 实现单位计量 ★★ (~30 分钟)

扩展第 4 章中的单位计量模式,以支持:

  • Meters (米)、Seconds (秒)、Kilograms (千克)
  • 相同单位的加法
  • 乘法:Meters * Meters = SquareMeters (平方米)
  • 除法:Meters / Seconds = MetersPerSecond (米/秒)
🔑 参考答案
use std::marker::PhantomData;
use std::ops::{Add, Mul, Div};

#[derive(Clone, Copy)]
struct Meters;
#[derive(Clone, Copy)]
struct Seconds;
#[derive(Clone, Copy)]
struct Kilograms;
#[derive(Clone, Copy)]
struct SquareMeters;
#[derive(Clone, Copy)]
struct MetersPerSecond;

#[derive(Debug, Clone, Copy)]
struct Qty<U> {
    value: f64,
    _unit: PhantomData<U>,
}

impl<U> Qty<U> {
    fn new(v: f64) -> Self { Qty { value: v, _unit: PhantomData } }
}

impl<U> Add for Qty<U> {
    type Output = Qty<U>;
    fn add(self, rhs: Self) -> Self::Output { Qty::new(self.value + rhs.value) }
}

impl Mul<Qty<Meters>> for Qty<Meters> {
    type Output = Qty<SquareMeters>;
    fn mul(self, rhs: Qty<Meters>) -> Qty<SquareMeters> {
        Qty::new(self.value * rhs.value)
    }
}

impl Div<Qty<Seconds>> for Qty<Meters> {
    type Output = Qty<MetersPerSecond>;
    fn div(self, rhs: Qty<Seconds>) -> Qty<MetersPerSecond> {
        Qty::new(self.value / rhs.value)
    }
}

fn main() {
    let width = Qty::<Meters>::new(5.0);
    let height = Qty::<Meters>::new(3.0);
    let area = width * height; // Qty<SquareMeters>
    println!("面积: {:.1} m²", area.value);

    let dist = Qty::<Meters>::new(100.0);
    let time = Qty::<Seconds>::new(9.58);
    let speed = dist / time;
    println!("速度: {:.2} m/s", speed.value);

    let sum = width + height; // 相同单位 ✅
    println!("总和: {:.1} m", sum.value);

    // let bad = width + time; // ❌ 编译错误:无法将“米”与“秒”相加
}

练习 3:基于通道的工作池 (Worker Pool) ★★★ (~45 分钟)

使用通道构建一个工作池,满足以下要求:

  • 一个调度器 (Dispatcher) 通过通道发送 Job 结构体。
  • N 个工作者 (Workers) 消耗这些任务并将结果发回。
  • 使用 std::sync::mpsc 实现。
🔑 参考答案
use std::sync::mpsc;
use std::thread;

struct Job {
    id: u64,
    data: String,
}

struct JobResult {
    job_id: u64,
    output: String,
    worker_id: usize,
}

fn worker_pool(jobs: Vec<Job>, num_workers: usize) -> Vec<JobResult> {
    let (job_tx, job_rx) = mpsc::channel::<Job>();
    let (result_tx, result_rx) = mpsc::channel::<JobResult>();

    // 将接收器封装在 Arc<Mutex> 中以便在工作者之间共享
    let job_rx = std::sync::Arc::new(std::sync::Mutex::new(job_rx));

    // 生成工作者
    let mut handles = Vec::new();
    for worker_id in 0..num_workers {
        let job_rx = job_rx.clone();
        let result_tx = result_tx.clone();
        handles.push(thread::spawn(move || {
            loop {
                // 加锁、接收、解锁 —— 保持极短的临界区
                let job = {
                    let rx = job_rx.lock().unwrap();
                    rx.recv() // 阻塞直到接收到任务或通道关闭
                };
                match job {
                    Ok(job) => {
                        let output = format!("由工作者 {worker_id} 处理了任务 '{}'", job.data);
                        result_tx.send(JobResult {
                            job_id: job.id,
                            output,
                            worker_id,
                        }).unwrap();
                    }
                    Err(_) => break, // 通道已关闭 —— 退出
                }
            }
        }));
    }
    drop(result_tx); // 丢弃我们手中的副本,这样当所有工作者结束时,结果通道才会关闭

    // 调度任务
    let num_jobs = jobs.len();
    for job in jobs {
        job_tx.send(job).unwrap();
    }
    drop(job_tx); // 关闭任务通道 —— 工作者在排空任务后将退出

    // 收集结果
    let mut results = Vec::new();
    for result in result_rx {
        results.push(result);
    }
    assert_eq!(results.len(), num_jobs);

    for h in handles { h.join().unwrap(); }
    results
}

fn main() {
    let jobs: Vec<Job> = (0..20).map(|i| Job {
        id: i,
        data: format!("task-{i}"),
    }).collect();

    let results = worker_pool(jobs, 4);
    for r in &results {
        println!("[工作者 {}] 任务 {}: {}", r.worker_id, r.job_id, r.output);
    }
}

练习 4:高阶组合子流水线 (Pipeline) ★★ (~25 分钟)

创建一个 Pipeline 结构体来链接一系列变换操作。它应该支持通过 .pipe(f) 添加变换,并通过 .execute(input) 运行完整的流水线。

🔑 参考答案
struct Pipeline<T> {
    transforms: Vec<Box<dyn Fn(T) -> T>>,
}

impl<T: 'static> Pipeline<T> {
    fn new() -> Self {
        Pipeline { transforms: Vec::new() }
    }

    fn pipe(mut self, f: impl Fn(T) -> T + 'static) -> Self {
        self.transforms.push(Box::new(f));
        self
    }

    fn execute(self, input: T) -> T {
        self.transforms.into_iter().fold(input, |val, f| f(val))
    }
}

fn main() {
    let result = Pipeline::new()
        .pipe(|s: String| s.trim().to_string())
        .pipe(|s| s.to_uppercase())
        .pipe(|s| format!(">>> {s} <<<"))
        .execute("  hello world  ".to_string());

    println!("{result}"); // >>> HELLO WORLD <<<

    // 数字流水线:
    let result = Pipeline::new()
        .pipe(|x: i32| x * 2)
        .pipe(|x| x + 10)
        .pipe(|x| x * x)
        .execute(5);

    println!("{result}"); // (5*2 + 10)^2 = 400
}

加分项:如果想要让流水线支持在各个阶段改变类型,则需要不同的设计 —— 每次 .pipe() 调用都返回一个具有不同输出类型的 Pipeline(这需要更进阶的泛型处理能力)。


练习 5:使用 thiserror 构建错误层级 ★★ (~30 分钟)

为一个文件处理应用设计一个错误类型层级,该应用可能会在 I/O、解析(JSON 和 CSV)以及校验阶段发生故障。使用 thiserror 并演示 ? 操作符的传播。

🔑 参考答案
use thiserror::Error;

#[derive(Error, Debug)]
pub enum AppError {
    #[error("I/O 错误: {0}")]
    Io(#[from] std::io::Error),

    #[error("JSON 解析错误: {0}")]
    Json(#[from] serde_json::Error),

    #[error("第 {line} 行发生 CSV 错误: {message}")]
    Csv { line: usize, message: String },

    #[error("校验错误: {field} — {reason}")]
    Validation { field: String, reason: String },
}

fn read_file(path: &str) -> Result<String, AppError> {
    Ok(std::fs::read_to_string(path)?) // io::Error → 通过 #[from] 转换为 AppError::Io
}

fn parse_json(content: &str) -> Result<serde_json::Value, AppError> {
    Ok(serde_json::from_str(content)?) // serde_json::Error → AppError::Json
}

fn validate_name(value: &serde_json::Value) -> Result<String, AppError> {
    let name = value.get("name")
        .and_then(|v| v.as_str())
        .ok_or_else(|| AppError::Validation {
            field: "name".into(),
            reason: "必须是非空字符串".into(),
        })?;

    if name.is_empty() {
        return Err(AppError::Validation {
            field: "name".into(),
            reason: "不能为空".into(),
        });
    }

    Ok(name.to_string())
}

fn process_file(path: &str) -> Result<String, AppError> {
    let content = read_file(path)?;
    let json = parse_json(&content)?;
    let name = validate_name(&json)?;
    Ok(name)
}

fn main() {
    match process_file("config.json") {
        Ok(name) => println!("名称: {name}"),
        Err(e) => eprintln!("错误: {e}"),
    }
}

练习 6:带有关联类型的泛型特性 ★★★ (~40 分钟)

设计一个 Repository<T> 特性,包含关联的 Error 和 Id 类型。为内存存储实现该特性,并演示编译时类型安全性。

🔑 参考答案
use std::collections::HashMap;

trait Repository {
    type Item;
    type Id;
    type Error;

    fn get(&self, id: &Self::Id) -> Result<Option<&Self::Item>, Self::Error>;
    fn insert(&mut self, item: Self::Item) -> Result<Self::Id, Self::Error>;
    fn delete(&mut self, id: &Self::Id) -> Result<bool, Self::Error>;
}

#[derive(Debug, Clone)]
struct User {
    name: String,
    email: String,
}

struct InMemoryUserRepo {
    data: HashMap<u64, User>,
    next_id: u64,
}

impl InMemoryUserRepo {
    fn new() -> Self {
        InMemoryUserRepo { data: HashMap::new(), next_id: 1 }
    }
}

// 错误类型为 Infallible —— 内存操作绝不失败
impl Repository for InMemoryUserRepo {
    type Item = User;
    type Id = u64;
    type Error = std::convert::Infallible;

    fn get(&self, id: &u64) -> Result<Option<&User>, Self::Error> {
        Ok(self.data.get(id))
    }

    fn insert(&mut self, item: User) -> Result<u64, Self::Error> {
        let id = self.next_id;
        self.next_id += 1;
        self.data.insert(id, item);
        Ok(id)
    }

    fn delete(&mut self, id: &u64) -> Result<bool, Self::Error> {
        Ok(self.data.remove(id).is_some())
    }
}

// 针对 任何 仓库的通用函数:
fn create_and_fetch<R: Repository>(repo: &mut R, item: R::Item) -> Result<(), R::Error>
where
    R::Item: std::fmt::Debug,
    R::Id: std::fmt::Debug,
{
    let id = repo.insert(item)?;
    println!("插入成功,ID 为: {id:?}");
    let retrieved = repo.get(&id)?;
    println!("检索到: {retrieved:?}");
    Ok(())
}

fn main() {
    let mut repo = InMemoryUserRepo::new();
    create_and_fetch(&mut repo, User {
        name: "Alice".into(),
        email: "[email protected]".into(),
    }).unwrap();
}

练习 7:环绕 Unsafe 的安全包装器 (对应第 12 章) ★★★ (~45 分钟)

编写一个 FixedVec<T, const N: usize> —— 一个固定容量、栈分配的向量。 要求:

  • push(&mut self, value: T) -> Result<(), T> 当满时返回 Err(value)。
  • pop(&mut self) -> Option<T> 返回并移除最后一个元素。
  • as_slice(&self) -> &[T] 借用已初始化的元素。
  • 所有公共方法必须是安全的;所有 unsafe 代码块必须附带 SAFETY: 注释。
  • Drop 必须清理已初始化的元素。

提示:使用 MaybeUninit<T> 和 [const { MaybeUninit::uninit() }; N]。

🔑 参考答案
use std::mem::MaybeUninit;

pub struct FixedVec<T, const N: usize> {
    data: [MaybeUninit<T>; N],
    len: usize,
}

impl<T, const N: usize> FixedVec<T, N> {
    pub fn new() -> Self {
        FixedVec {
            data: [const { MaybeUninit::uninit() }; N],
            len: 0,
        }
    }

    pub fn push(&mut self, value: T) -> Result<(), T> {
        if self.len >= N { return Err(value); }
        // SAFETY: len < N,因此 data[len] 在范围内。
        self.data[self.len] = MaybeUninit::new(value);
        self.len += 1;
        Ok(())
    }

    pub fn pop(&mut self) -> Option<T> {
        if self.len == 0 { return None; }
        self.len -= 1;
        // SAFETY: data[len] 已初始化(递减前 len > 0)。
        Some(unsafe { self.data[self.len].assume_init_read() })
    }

    pub fn as_slice(&self) -> &[T] {
        // SAFETY: data[0..len] 均已初始化,且 MaybeUninit<T> 
        // 与 T 的内存布局相同。
        unsafe { std::slice::from_raw_parts(self.data.as_ptr() as *const T, self.len) }
    }

    pub fn len(&self) -> usize { self.len }
    pub fn is_empty(&self) -> bool { self.len == 0 }
}

impl<T, const N: usize> Drop for FixedVec<T, N> {
    fn drop(&mut self) {
        // SAFETY: data[0..len] 已初始化 —— 对每一个进行 drop。
        for i in 0..self.len {
            unsafe { self.data[i].assume_init_drop(); }
        }
    }
}

fn main() {
    let mut v = FixedVec::<String, 4>::new();
    v.push("hello".into()).unwrap();
    v.push("world".into()).unwrap();
    assert_eq!(v.as_slice(), &["hello", "world"]);
    assert_eq!(v.pop(), Some("world".into()));
    assert_eq!(v.len(), 1);
    // Drop 时会清理剩余的 "hello"
}

练习 8:声明式宏 —— map! (对应第 13 章) ★ (~15 分钟)

编写一个 map! 宏,用于从键值对创建 HashMap,类似于 vec![]:

#![allow(unused)]
fn main() {
let m = map! {
    "host" => "localhost",
    "port" => "8080",
};
assert_eq!(m.get("host"), Some(&"localhost"));
assert_eq!(m.len(), 2);
}

要求:

  • 支持尾随逗号。
  • 支持空调用 map!{}。
  • 为了最大的灵活性,应适用于任何实现了 Into<K> 和 Into<V> 的类型。
🔑 参考答案
macro_rules! map {
    // 空案例
    () => {
        std::collections::HashMap::new()
    };
    // 一个或多个 key => value 键值对(尾随逗号可选)
    ( $( $key:expr => $val:expr ),+ $(,)? ) => {{
        let mut m = std::collections::HashMap::new();
        $( m.insert($key, $val); )+
        m
    }};
}

fn main() {
    // 基础用法:
    let config = map! {
        "host" => "localhost",
        "port" => "8080",
        "timeout" => "30",
    };
    assert_eq!(config.len(), 3);
    assert_eq!(config["host"], "localhost");

    // 空 map:
    let empty: std::collections::HashMap<String, String> = map!();
    assert!(empty.is_empty());

    // 不同类型:
    let scores = map! {
        1 => 100,
        2 => 200,
    };
    assert_eq!(scores[&1], 100);
}

练习 9:自定义 serde 反序列化 (对应第 11 章) ★★★ (~45 分钟)

设计一个 Duration 包装器,使用自定义 serde 反序列化器从 "30s"、"5m"、"2h" 等人类可读的字符串中进行反序列化。该结构体还应该能够序列化回相同的格式。

🔑 参考答案
use serde::{Deserialize, Deserializer, Serialize, Serializer};
use std::fmt;

#[derive(Debug, Clone, PartialEq)]
struct HumanDuration(std::time::Duration);

impl HumanDuration {
    fn from_str(s: &str) -> Result<Self, String> {
        let s = s.trim();
        if s.is_empty() { return Err("时长字符串不能为空".into()); }

        let (num_str, suffix) = s.split_at(
            s.find(|c: char| !c.is_ascii_digit()).unwrap_or(s.len())
        );
        let value: u64 = num_str.parse()
            .map_err(|_| format!("无效数字: {num_str}"))?;

        let duration = match suffix {
            "s" | "sec"  => std::time::Duration::from_secs(value),
            "m" | "min"  => std::time::Duration::from_secs(value * 60),
            "h" | "hr"   => std::time::Duration::from_secs(value * 3600),
            "ms"         => std::time::Duration::from_millis(value),
            other        => return Err(format!("未知后缀: {other}")),
        };
        Ok(HumanDuration(duration))
    }
}

impl fmt::Display for HumanDuration {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        let secs = self.0.as_secs();
        if secs == 0 {
            write!(f, "{}ms", self.0.as_millis())
        } else if secs % 3600 == 0 {
            write!(f, "{}h", secs / 3600)
        } else if secs % 60 == 0 {
            write!(f, "{}m", secs / 60)
        } else {
            write!(f, "{}s", secs)
        }
    }
}

impl Serialize for HumanDuration {
    fn serialize<S: Serializer>(&self, serializer: S) -> Result<S::Ok, S::Error> {
        serializer.serialize_str(&self.to_string())
    }
}

impl<'de> Deserialize<'de> for HumanDuration {
    fn deserialize<D: Deserializer<'de>>(deserializer: D) -> Result<Self, D::Error> {
        let s = String::deserialize(deserializer)?;
        HumanDuration::from_str(&s).map_err(serde::de::Error::custom)
    }
}

#[derive(Debug, Deserialize, Serialize)]
struct Config {
    timeout: HumanDuration,
    retry_interval: HumanDuration,
}

fn main() {
    let json = r#"{ "timeout": "30s", "retry_interval": "5m" }"#;
    let config: Config = serde_json::from_str(json).unwrap();

    assert_eq!(config.timeout.0, std::time::Duration::from_secs(30));
    assert_eq!(config.retry_interval.0, std::time::Duration::from_secs(300));

    // 往返转换正确:
    let serialized = serde_json::to_string(&config).unwrap();
    assert!(serialized.contains("30s"));
    assert!(serialized.contains("5m"));
    println!("配置: {serialized}");
}

练习 10 —— 带超时的并发获取器 (对应第 16 章) ★★ (~25 分钟)

编写一个异步函数 fetch_all,生成三个 tokio::spawn 任务,每个任务都使用 tokio::time::sleep 模拟网络调用。使用 tokio::try_join! 将三者组合在一起,并将其封装在 tokio::time::timeout(Duration::from_secs(5), ...) 中。返回 Result<Vec<String>, ...>,或者在任何任务失败或截止时间到期时返回错误。

学习目标:tokio::spawn、try_join!、timeout 以及跨任务边界的错误传播。

提示

每个生成的任务都返回 Result<String, _>。try_join! 会解包这三个结果。将整个 try_join! 封装在 timeout() 中 —— Elapsed 错误意味着达到了截止时间。

参考答案
use tokio::time::{sleep, timeout, Duration};

async fn fake_fetch(name: &'static str, delay_ms: u64) -> Result<String, String> {
    sleep(Duration::from_millis(delay_ms)).await;
    Ok(format!("{name}: OK"))
}

async fn fetch_all() -> Result<Vec<String>, Box<dyn std::error::Error>> {
    let deadline = Duration::from_secs(5);

    let (a, b, c) = timeout(deadline, async {
        let h1 = tokio::spawn(fake_fetch("svc-a", 100));
        let h2 = tokio::spawn(fake_fetch("svc-b", 200));
        let h3 = tokio::spawn(fake_fetch("svc-c", 150));
        tokio::try_join!(h1, h2, h3)
    })
    .await??; // 第一个 ? 为 timeout,第二个 ? 为 join

    Ok(vec![a?, b?, c?]) // 解包内部的 Result
}

#[tokio::main]
async fn main() {
    let results = fetch_all().await.unwrap();
    for r in &results {
        println!("{r}");
    }
}

练习 11 —— 异步通道流水线 (Async Channel Pipeline) ★★★ (~40 分钟)

使用 tokio::sync::mpsc 构建一个“生产者 → 转换器 → 消费者”流水线:

  1. 生产者 (Producer):将整数 1..=20 发送到通道 A(容量为 4)。
  2. 转换器 (Transformer):从通道 A 读取,对每个值求平方,然后发送到通道 B。
  3. 消费者 (Consumer):从通道 B 读取,收集到 Vec<u64> 中并返回。

这三个阶段都作为并发的 tokio::spawn 任务运行。使用有界通道 (Bounded channels) 来演示背压 (Back-pressure)。断言最终的向量等于 [1, 4, 9, ..., 400]。

学习目标:mpsc::channel、有界背压、带 move 闭包的 tokio::spawn、通过通道关闭实现的优雅停机。

参考答案
use tokio::sync::mpsc;

#[tokio::main]
async fn main() {
    let (tx_a, mut rx_a) = mpsc::channel::<u64>(4); // 有界 —— 提供背压
    let (tx_b, mut rx_b) = mpsc::channel::<u64>(4);

    // 生产者
    let producer = tokio::spawn(async move {
        for i in 1..=20u64 {
            tx_a.send(i).await.unwrap();
        }
        // tx_a 在此处被丢弃 → 通道 A 关闭
    });

    // 转换器
    let transformer = tokio::spawn(async move {
        while let Some(val) = rx_a.recv().await {
            tx_b.send(val * val).await.unwrap();
        }
        // tx_b 在此处被丢弃 → 通道 B 关闭
    });

    // 消费者
    let consumer = tokio::spawn(async move {
        let mut results = Vec::new();
        while let Some(val) = rx_b.recv().await {
            results.push(val);
        }
        results
    });

    producer.await.unwrap();
    transformer.await.unwrap();
    let results = consumer.await.unwrap();

    let expected: Vec<u64> = (1..=20).map(|x: u64| x * x).collect();
    assert_eq!(results, expected);
    println!("流水线已完成: {results:?}");
}

总结与速查表

快速参考卡片

模式决策指南

需要为原生类型提供类型安全?
└── 新类型 (Newtype) 模式 (第 3 章)

需要编译时状态约束?
└── 类型状态 (Type-state) 模式 (第 3 章)

需要一个不含运行时数据的“标签”?
└── PhantomData (第 4 章)

需要打破 Rc/Arc 的引用循环?
└── Weak<T> / sync::Weak<T> (第 9 章)

需要等待某个条件而不进行忙碌轮询 (busy-looping)?
└── Condvar + Mutex (第 6 章)

需要处理“N 种类型之一”?
├── 已知的封闭集 → 枚举 (Enum)
├── 开放集,热点路径 → 泛型 (Generics)
├── 开放集,冷点路径 → dyn Trait
└── 完全未知的类型 → Any + TypeId (第 2 章)

需要跨线程共享状态?
├── 简单的计数器/标志 → 原子操作 (Atomics)
├── 短临界区 → 互斥锁 (Mutex)
├── 读多写少 → 读写锁 (RwLock)
├── 延迟一次性初始化 → OnceLock / LazyLock (第 6 章)
└── 复杂状态 → Actor + 通道 (Channels)

需要并行化计算?
├── 集合处理 → rayon::par_iter
├── 后台任务 → thread::spawn
└── 借用本地数据 → thread::scope (线程作用域)

需要异步 I/O 或并发网络?
├── 基础 → tokio + async/await (第 16 章)
└── 高阶 (流、中间件) → 请参阅 Async Rust 进阶指南

需要进行错误处理?
├── 库 (Library) → thiserror (#[derive(Error)])
└── 应用程序 (Application) → anyhow (Result<T>)

需要防止某个值被移动 (Move)?
└── Pin<T> (第 9 章) —— Future 和自引用类型所必需

特性约束 (Trait Bounds) 速查表

约束含义
T: Clone可被克隆
T: Send可被移动到另一个线程
T: Sync&T 可在线程间共享
T: 'static不包含非静态引用
T: Sized编译时大小已知 (默认情况)
T: ?Sized大小可能未知 ([T]、dyn Trait)
T: Unpin在固定 (Pin) 后仍可安全移动
T: Default拥有默认值
T: Into<U>可转换为类型 U
T: AsRef<U>可被借用为 &U
T: Deref<Target = U>自动解引用为 &U
F: Fn(A) -> B可调用,以不可变方式借用状态
F: FnMut(A) -> B可调用,可能会修改状态
F: FnOnce(A) -> B仅限调用一次,可能会消耗状态

生命周期消除 (Elision) 规则

在以下三种情况下,编译器会自动插入生命周期(这样你就不必手动编写):

#![allow(unused)]
fn main() {
// 规则 1:每个引用参数都有自己的生命周期
// fn foo(x: &str, y: &str)  →  fn foo<'a, 'b>(x: &'a str, y: &'b str)

// 规则 2:如果正好只有一个输入生命周期,它将用于所有输出
// fn foo(x: &str) -> &str   →  fn foo<'a>(x: &'a str) -> &'a str

// 规则 3:如果有一个参数是 &self 或 &mut self,则使用其生命周期
// fn foo(&self, x: &str) -> &str  →  fn foo<'a>(&'a self, x: &str) -> &'a str
}

当你 必须 编写显式生命周期时:

  • 存在多个输入引用且存在引用输出(编译器无法推断应使用哪个输入)。
  • 结构体字段包含引用:struct Ref<'a> { data: &'a str }。
  • 当你需要不包含借用引用的数据时,使用 'static 约束。

常用的派生特性

#![allow(unused)]
fn main() {
#[derive(
    Debug,          // {:?} 格式化
    Clone,          // .clone() 方法
    Copy,           // 隐式复制 (仅限简单类型)
    PartialEq, Eq,  // == 比较
    PartialOrd, Ord, // < > 比较 + 排序
    Hash,           // HashMap/HashSet 的键
    Default,        // Type::default() 方法
)]
struct MyType { /* ... */ }
}

模块可见性速查参考

pub           → 到处可见
pub(crate)    → 仅在当前 crate 内可见
pub(super)    → 对父模块可见
pub(in path)  → 在特定路径内可见
(不写)        → 仅对当前模块及其子模块私有

进一步阅读

资源推荐理由
Rust 设计模式惯用法与反面模式(anti-patterns)目录
Rust API 指南完善的公共 API 设计官方清单
Rust 原子操作与锁Mara Bos 对并发原语的深入探讨
Rustonomicon (死灵书)针对 Unsafe Rust 与底层细节的官方指南
Rust 错误处理Andrew Gallant 撰写的详尽指南
Jon Gjengset —— Crust of Rust 系列视频对迭代器、生命周期、通道等内容的深度剖析
Effective Rust35 种改进 Rust 代码的具体方式

《Rust 设计模式与工程实践》 完

综合项目:类型安全的任务调度器

本项目将书中各章节的模式整合到一个生产级的系统中。你将构建一个 类型安全的并发任务调度器,它综合运用了泛型、特性、类型状态 (Typestate)、通道、错误处理以及测试。

预计耗时:4–6 小时 | 难度:★★★

你将练习到的内容:

  • 泛型与特性约束 (第 1–2 章)
  • 用于任务生命周期的类型状态模式 (第 3 章)
  • 用于零成本状态标记的 PhantomData (第 4 章)
  • 用于工作者通信的通道 (第 5 章)
  • 使用线程作用域 (Scoped threads) 的并发处理 (第 6 章)
  • 使用 thiserror 的错误处理 (第 10 章)
  • 使用基于属性的测试进行测试 (第 14 章)
  • 使用 TryFrom 和验证后的类型进行 API 设计 (第 15 章)

问题描述

构建一个满足以下要求的任务调度器:

  1. 任务 具有类型化的生命周期:挂起 (Pending) → 运行中 (Running) → 已完成 (Completed)(或 已失败 (Failed))。
  2. 工作者 (Workers) 从通道中拉取任务、执行任务并报告结果。
  3. 调度器 (Scheduler) 管理任务提交、工作者协调以及结果收集。
  4. 非法的状态转换在 编译时报错。
stateDiagram-v2
    [*] --> 挂起: scheduler.submit(task)
    挂起 --> 运行中: 工作者提取任务
    运行中 --> 已完成: 任务成功执行
    运行中 --> 已失败: 任务返回 Err
    已完成 --> [*]: scheduler.results()
    已失败 --> [*]: scheduler.results()

    挂起 --> 挂起: ❌ 无法直接执行
    已完成 --> 运行中: ❌ 无法重复运行

第一步:定义任务类型

首先定义类型状态标记和通用的 Task 结构体:

#![allow(unused)]
fn main() {
use std::marker::PhantomData;

// --- 状态标记 (零大小类型) ---
struct Pending;
struct Running;
struct Completed;
struct Failed;

// --- 任务 ID (用于类型安全的新类型) ---
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
struct TaskId(u64);

// --- Task 结构体,由生命周期状态参数化 ---
struct Task<State, R> {
    id: TaskId,
    name: String,
    _state: PhantomData<State>,
    _result: PhantomData<R>,
}
}

你的任务:实现状态转换,满足:

  • Task<Pending, R> 可以转换到 Task<Running, R> (通过 start() 方法)。
  • Task<Running, R> 可以转换到 Task<Completed, R> 或 Task<Failed, R>。
  • 其他任何转换都无法通过编译。
💡 提示

每个转换方法都应该消耗 (consume) self 并返回新状态:

#![allow(unused)]
fn main() {
impl<R> Task<Pending, R> {
    fn start(self) -> Task<Running, R> {
        Task {
            id: self.id,
            name: self.name,
            _state: PhantomData,
            _result: PhantomData,
        }
    }
}
}

第二步:定义执行函数

任务需要一个可执行的函数。请使用装箱的闭包 (Boxed closure):

#![allow(unused)]
fn main() {
struct WorkItem<R: Send + 'static> {
    id: TaskId,
    name: String,
    work: Box<dyn FnOnce() -> Result<R, String> + Send>,
}
}

你的任务:实现 WorkItem::new(),接收任务名称和闭包。添加一个 TaskId 生成器(简单的原子计数器或由互斥锁保护的计数器)。

第三步:错误处理

使用 thiserror 定义调度器的错误类型:

use thiserror::Error;

#[derive(Error, Debug)]
pub enum SchedulerError {
    #[error("调度器已关闭")]
    ShutDown,

    #[error("任务 {0:?} 失败: {1}")]
    TaskFailed(TaskId, String),

    #[error("通道发送错误")]
    ChannelError(#[from] std::sync::mpsc::SendError<()>),

    #[error("工作者发生 panic")]
    WorkerPanic,
}

第四步:调度器

使用通道 (第 5 章) 和线程作用域 (第 6 章) 构建调度器:

#![allow(unused)]
fn main() {
use std::sync::mpsc;

struct Scheduler<R: Send + 'static> {
    sender: Option<mpsc::Sender<WorkItem<R>>>,
    results: mpsc::Receiver<TaskResult<R>>,
    num_workers: usize,
}

struct TaskResult<R> {
    id: TaskId,
    name: String,
    outcome: Result<R, String>,
}
}

你的任务:实现:

  • Scheduler::new(num_workers: usize) -> Self —— 创建通道并生成工作者。
  • Scheduler::submit(&self, item: WorkItem<R>) -> Result<TaskId, SchedulerError>。
  • Scheduler::shutdown(self) -> Vec<TaskResult<R>> —— 丢弃发送器,等待工作者结束并收集结果。
💡 提示 —— 工作者循环
#![allow(unused)]
fn main() {
fn worker_loop<R: Send + 'static>(
    rx: std::sync::Arc<std::sync::Mutex<mpsc::Receiver<WorkItem<R>>>>,
    result_tx: mpsc::Sender<TaskResult<R>>,
    worker_id: usize,
) {
    loop {
        let item = {
            let rx = rx.lock().unwrap();
            rx.recv()
        };
        match item {
            Ok(work_item) => {
                let outcome = (work_item.work)();
                let _ = result_tx.send(TaskResult {
                    id: work_item.id,
                    name: work_item.name,
                    outcome,
                });
            }
            Err(_) => break, // 通道已关闭
        }
    }
}
}

第五步:集成测试

编写测试来验证:

  1. 成功路径:提交 10 个任务,关闭调度器,验证所有 10 个结果均为 Ok。
  2. 错误处理:提交会失败的任务,验证 TaskResult.outcome 为 Err。
  3. 空调度器:创建后立即关闭 —— 不应发生 panic。
  4. 属性测试 (加分项):使用 proptest 验证对于任何数量 N 的任务 (1..100),调度器始终准确返回 N 个结果。
#![allow(unused)]
fn main() {
#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn happy_path() {
        let scheduler = Scheduler::<String>::new(4);

        for i in 0..10 {
            let item = WorkItem::new(
                format!("task-{i}"),
                move || Ok(format!("result-{i}")),
            );
            scheduler.submit(item).unwrap();
        }

        let results = scheduler.shutdown();
        assert_eq!(results.len(), 10);
        for r in &results {
            assert!(r.outcome.is_ok());
        }
    }

    #[test]
    fn handles_failures() {
        let scheduler = Scheduler::<String>::new(2);

        scheduler.submit(WorkItem::new("good", || Ok("ok".into()))).unwrap();
        scheduler.submit(WorkItem::new("bad", || Err("boom".into()))).unwrap();

        let results = scheduler.shutdown();
        assert_eq!(results.len(), 2);

        let failures: Vec<_> = results.iter()
            .filter(|r| r.outcome.is_err())
            .collect();
        assert_eq!(failures.len(), 1);
    }
}
}

第六步:综合运用

以下 main() 函数演示了整个系统的运行:

fn main() {
    let scheduler = Scheduler::<String>::new(4);

    // 提交负载各异的任务
    for i in 0..20 {
        let item = WorkItem::new(
            format!("compute-{i}"),
            move || {
                // 模拟工作
                std::thread::sleep(std::time::Duration::from_millis(10));
                if i % 7 == 0 {
                    Err(format!("任务 {i} 遇到了模拟错误"))
                } else {
                    Ok(format!("任务 {i} 已完成,返回值为 {}", i * i))
                }
            },
        );
        // 注意:此处使用 .unwrap() 是为了简洁 —— 在生产环境中请处理 SendError。
        scheduler.submit(item).unwrap();
    }

    println!("所有任务已提交。正在关闭调度器...");
    let results = scheduler.shutdown();

    let (ok, err): (Vec<_>, Vec<_>) = results.iter()
        .partition(|r| r.outcome.is_ok());

    println!("\n✅ 成功: {}", ok.len());
    for r in &ok {
        println!("  {} → {}", r.name, r.outcome.as_ref().unwrap());
    }

    println!("\n❌ 失败: {}", err.len());
    for r in &err {
        println!("  {} → {}", r.name, r.outcome.as_ref().unwrap_err());
    }
}

评估标准

标准目标
类型安全非法的状态转换无法通过编译
并发性工作者并行运行,无数据竞争
错误处理所有故障均在 TaskResult 中捕获,无 panic
测试至少包含 3 个测试;proptest 为加分项
代码组织模块结构清晰,公共 API 使用验证后的类型
文档关键类型附带解释其不变性 (Invariants) 的文档注释

扩展思路

在基础调度器工作正常后,可以尝试以下增强功能:

  1. 优先队列:添加一个 Priority 新类型 (1–10),并优先处理高优先级任务。
  2. 重试策略:失败的任务在被标记为永久失败前,最多重试 N 次。
  3. 取消功能:添加 cancel(TaskId) 方法来移除挂起中的任务。
  4. 异步版本:移植到 tokio::spawn 并使用 tokio::sync::mpsc 通道 (第 16 章)。
  5. 指标监控 (Metrics):跟踪每个工作者的任务计数、平均执行时间和失败率。

Rust Patterns & Engineering How-Tos

Speaker Intro

  • Principal Firmware Architect in Microsoft SCHIE (Silicon and Cloud Hardware Infrastructure Engineering) team
  • Industry veteran with expertise in security, systems programming (firmware, operating systems, hypervisors), CPU and platform architecture, and C++ systems
  • Started programming in Rust in 2017 (@AWS EC2), and have been in love with the language ever since

A practical guide to intermediate-and-above Rust patterns that arise in real codebases. This is not a language tutorial — it assumes you can write basic Rust and want to level up. Each chapter isolates one concept, explains when and why to use it, and provides compilable examples with inline exercises.

Who This Is For

  • Developers who have finished The Rust Programming Language but struggle with “how do I actually design this?”
  • C++/C# engineers translating production systems into Rust
  • Anyone who has hit a wall with generics, trait bounds, or lifetime errors and wants a systematic toolkit

Prerequisites

Before starting, you should be comfortable with:

  • Ownership, borrowing, and lifetimes (basic level)
  • Enums, pattern matching, and Option/Result
  • Structs, methods, and basic traits (Display, Debug, Clone)
  • Cargo basics: cargo build, cargo test, cargo run

How to Use This Book

Difficulty Legend

Each chapter is tagged with a difficulty level:

SymbolLevelMeaning
🟢FundamentalsCore concepts every Rust developer needs
🟡IntermediatePatterns used in production codebases
🔴AdvancedDeep language mechanics — revisit as needed

Pacing Guide

ChaptersTopicSuggested TimeCheckpoint
Part I: Type-Level Patterns
1. Generics 🟢Monomorphization, const generics, const fn1–2 hoursCan explain when dyn Trait beats generics
2. Traits 🟡Associated types, GATs, blanket impls, vtables3–4 hoursCan design a trait with associated types
3. Newtype & Type-State 🟡Zero-cost safety, compile-time FSMs2–3 hoursCan build a type-state builder pattern
4. PhantomData 🔴Lifetime branding, variance, drop check2–3 hoursCan explain why PhantomData<fn(T)> differs from PhantomData<T>
Part II: Concurrency & Runtime
5. Channels 🟢mpsc, crossbeam, select!, actors1–2 hoursCan implement a channel-based worker pool
6. Concurrency 🟡Threads, rayon, Mutex, RwLock, atomics2–3 hoursCan pick the right sync primitive for a scenario
7. Closures 🟢Fn/FnMut/FnOnce, combinators1–2 hoursCan write a higher-order function that accepts closures
8. Functional vs. Imperative 🟡Combinators, iterator adapters, functional patterns2–3 hoursCan explain when functional style beats imperative
9. Smart Pointers 🟡Box, Rc, Arc, RefCell, Cow, Pin2–3 hoursCan explain when to use each smart pointer
Part III: Systems & Production
10. Error Handling 🟢thiserror, anyhow, ? operator1–2 hoursCan design an error type hierarchy
11. Serialization 🟡serde, zero-copy, binary data2–3 hoursCan write a custom serde deserializer
12. Unsafe 🔴Superpowers, FFI, UB pitfalls, allocators2–3 hoursCan wrap unsafe code in a sound safe API
13. Macros 🟡macro_rules!, proc macros, syn/quote2–3 hoursCan write a declarative macro with tt munching
14. Testing 🟢Unit/integration/doc tests, proptest, criterion1–2 hoursCan set up property-based tests
15. API Design 🟡Module layout, ergonomic APIs, feature flags2–3 hoursCan apply the “parse, don’t validate” pattern
16. Async 🔴Futures, Tokio, common pitfalls1–2 hoursCan identify async anti-patterns
Appendices
Reference CardQuick-look trait bounds, lifetimes, patternsAs needed—
Capstone ProjectType-safe task scheduler4–6 hoursSubmit a working implementation

Total estimated time: 30–45 hours for thorough study with exercises.

Working Through Exercises

Every chapter ends with a hands-on exercise. For maximum learning:

  1. Try it yourself first — spend at least 15 minutes before opening the solution
  2. Type the code — don’t copy-paste; typing builds muscle memory
  3. Modify the solution — add a feature, change a constraint, break something on purpose
  4. Check cross-references — most exercises combine patterns from multiple chapters

The capstone project (Appendix) ties together patterns from across the book into a single, production-quality system.

Table of Contents

Part I: Type-Level Patterns

1. Generics — The Full Picture 🟢 Monomorphization, code bloat trade-offs, generics vs enums vs trait objects, const generics, const fn.

2. Traits In Depth 🟡 Associated types, GATs, blanket impls, marker traits, vtables, HRTBs, extension traits, enum dispatch.

3. The Newtype and Type-State Patterns 🟡 Zero-cost type safety, compile-time state machines, builder patterns, config traits.

4. PhantomData — Types That Carry No Data 🔴 Lifetime branding, unit-of-measure pattern, drop check, variance.

Part II: Concurrency & Runtime

5. Channels and Message Passing 🟢 std::sync::mpsc, crossbeam, select!, backpressure, actor pattern.

6. Concurrency vs Parallelism vs Threads 🟡 OS threads, scoped threads, rayon, Mutex/RwLock/Atomics, Condvar, OnceLock, lock-free patterns.

7. Closures and Higher-Order Functions 🟢 Fn/FnMut/FnOnce, closures as parameters/return values, combinators, higher-order APIs.

8. Functional vs. Imperative: When Elegance Wins (and When It Doesn’t) 🟡 Combinators, iterator adapters, functional patterns.

9. Smart Pointers and Interior Mutability 🟡 Box, Rc, Arc, Weak, Cell/RefCell, Cow, Pin, ManuallyDrop.

Part III: Systems & Production

10. Error Handling Patterns 🟢 thiserror vs anyhow, #[from], .context(), ? operator, panics.

11. Serialization, Zero-Copy, and Binary Data 🟡 serde fundamentals, enum representations, zero-copy deserialization, repr(C), bytes::Bytes.

12. Unsafe Rust — Controlled Danger 🔴 Five superpowers, sound abstractions, FFI, UB pitfalls, arena/slab allocators.

13. Macros — Code That Writes Code 🟡 macro_rules!, when (not) to use macros, proc macros, derive macros, syn/quote.

14. Testing and Benchmarking Patterns 🟢 Unit/integration/doc tests, proptest, criterion, mocking strategies.

15. Crate Architecture and API Design 🟡 Module layout, API design checklist, ergonomic parameters, feature flags, workspaces.

16. Async/Await Essentials 🔴 Futures, Tokio quick-start, common pitfalls. (For deep async coverage, see our Async Rust Training.)

Appendices

Summary and Reference Card Pattern decision guide, trait bounds cheat sheet, lifetime elision rules, further reading.

Capstone Project: Type-Safe Task Scheduler Integrate generics, traits, typestate, channels, error handling, and testing into a complete system.


1. Generics — The Full Picture 🟢

What you’ll learn:

  • How monomorphization gives zero-cost generics — and when it causes code bloat
  • The decision framework: generics vs enums vs trait objects
  • Const generics for compile-time array sizes and const fn for compile-time evaluation
  • When to trade static dispatch for dynamic dispatch on cold paths

Monomorphization and Zero Cost

Generics in Rust are monomorphized — the compiler generates a specialized copy of each generic function for every concrete type it’s used with. This is the opposite of Java/C# where generics are erased at runtime.

fn max_of<T: PartialOrd>(a: T, b: T) -> T {
    if a >= b { a } else { b }
}

fn main() {
    max_of(3_i32, 5_i32);     // Compiler generates max_of_i32
    max_of(2.0_f64, 7.0_f64); // Compiler generates max_of_f64
    max_of("a", "z");         // Compiler generates max_of_str
}

What the compiler actually produces (conceptually):

#![allow(unused)]
fn main() {
// Three separate functions — no runtime dispatch, no vtable:
fn max_of_i32(a: i32, b: i32) -> i32 { if a >= b { a } else { b } }
fn max_of_f64(a: f64, b: f64) -> f64 { if a >= b { a } else { b } }
fn max_of_str<'a>(a: &'a str, b: &'a str) -> &'a str { if a >= b { a } else { b } }
}

Why does max_of_str need <'a> but max_of_i32 doesn’t? i32 and f64 are Copy types — the function returns an owned value. But &str is a reference, so the compiler must know the returned reference’s lifetime. The <'a> annotation says “the returned &str lives at least as long as both inputs.”

Advantages: Zero runtime cost — identical to hand-written specialized code. The optimizer can inline, vectorize, and specialize each copy independently.

Comparison with C++: Rust generics work like C++ templates but with one crucial difference — bounds checking happens at definition, not instantiation. In C++, a template compiles only when used with a specific type, leading to cryptic error messages deep in library code. In Rust, T: PartialOrd is checked when you define the function, so errors are caught early and messages are clear.

#![allow(unused)]
fn main() {
// Rust: error at definition site — "T doesn't implement Display"
fn broken<T>(val: T) {
    println!("{val}"); // ❌ Error: T doesn't implement Display
}
}
#![allow(unused)]
fn main() {
// Fix: add the bound
fn fixed<T: std::fmt::Display>(val: T) {
    println!("{val}"); // ✅
}
}

When Generics Hurt: Code Bloat

Monomorphization has a cost — binary size. Each unique instantiation duplicates the function body:

// This innocent function...
fn serialize<T: serde::Serialize>(value: &T) -> Vec<u8> {
    serde_json::to_vec(value).unwrap()
}

// ...used with 50 different types → 50 copies in the binary.

Mitigation strategies:

// 1. Extract the non-generic core ("outline" pattern)
fn serialize<T: serde::Serialize>(value: &T) -> Result<Vec<u8>, serde_json::Error> {
    // Generic part: only the serialization call
    let json_value = serde_json::to_value(value)?;
    // Non-generic part: extracted into a separate function
    serialize_value(json_value)
}

fn serialize_value(value: serde_json::Value) -> Result<Vec<u8>, serde_json::Error> {
    // This function exists only ONCE in the binary
    serde_json::to_vec(&value)
}

// 2. Use trait objects (dynamic dispatch) when inlining isn't critical
fn log_item(item: &dyn std::fmt::Display) {
    // One copy — uses vtable for dispatch
    println!("[LOG] {item}");
}

Rule of thumb: Use generics for hot paths where inlining matters. Use dyn Trait for cold paths (error handling, logging, configuration) where a vtable call is negligible.

Generics vs Enums vs Trait Objects — Decision Guide

Three ways to handle “different types, same interface” in Rust:

ApproachDispatchKnown atExtensible?Overhead
Generics (impl Trait / <T: Trait>)Static (monomorphized)Compile time✅ (open set)Zero — inlined
EnumMatch armCompile time❌ (closed set)Zero — no vtable
Trait object (dyn Trait)Dynamic (vtable)Runtime✅ (open set)Vtable pointer + indirect call
// --- GENERICS: Open set, zero cost, compile-time ---
fn process<H: Handler>(handler: H, request: Request) -> Response {
    handler.handle(request) // Monomorphized — one copy per H
}

// --- ENUM: Closed set, zero cost, exhaustive matching ---
enum Shape {
    Circle(f64),
    Rect(f64, f64),
    Triangle(f64, f64, f64),
}

impl Shape {
    fn area(&self) -> f64 {
        match self {
            Shape::Circle(r) => std::f64::consts::PI * r * r,
            Shape::Rect(w, h) => w * h,
            Shape::Triangle(a, b, c) => {
                let s = (a + b + c) / 2.0;
                (s * (s - a) * (s - b) * (s - c)).sqrt()
            }
        }
    }
}
// Adding a new variant forces updating ALL match arms — the compiler
// enforces exhaustiveness. Great for "I control all the variants."

// --- TRAIT OBJECT: Open set, runtime cost, extensible ---
fn log_all(items: &[Box<dyn std::fmt::Display>]) {
    for item in items {
        println!("{item}"); // vtable dispatch
    }
}

Decision flowchart:

flowchart TD
    A["Do you know ALL<br>possible types at<br>compile time?"]
    A -->|"Yes, small<br>closed set"| B["Enum"]
    A -->|"Yes, but set<br>is open"| C["Generics<br>(monomorphized)"]
    A -->|"No — types<br>determined at runtime"| D["dyn Trait"]

    C --> E{"Hot path?<br>(millions of calls)"}
    E -->|Yes| F["Generics<br>(inlineable)"]
    E -->|No| G["dyn Trait<br>is fine"]

    D --> H{"Need mixed types<br>in one collection?"}
    H -->|Yes| I["Vec&lt;Box&lt;dyn Trait&gt;&gt;"]
    H -->|No| C

    style A fill:#e8f4f8,stroke:#2980b9,color:#000
    style B fill:#d4efdf,stroke:#27ae60,color:#000
    style C fill:#d4efdf,stroke:#27ae60,color:#000
    style D fill:#fdebd0,stroke:#e67e22,color:#000
    style F fill:#d4efdf,stroke:#27ae60,color:#000
    style G fill:#fdebd0,stroke:#e67e22,color:#000
    style I fill:#fdebd0,stroke:#e67e22,color:#000
    style E fill:#fef9e7,stroke:#f1c40f,color:#000
    style H fill:#fef9e7,stroke:#f1c40f,color:#000

Const Generics

Since Rust 1.51, you can parameterize types and functions over constant values, not just types:

#![allow(unused)]
fn main() {
// Array wrapper parameterized over size
struct Matrix<const ROWS: usize, const COLS: usize> {
    data: [[f64; COLS]; ROWS],
}

impl<const ROWS: usize, const COLS: usize> Matrix<ROWS, COLS> {
    fn new() -> Self {
        Matrix { data: [[0.0; COLS]; ROWS] }
    }

    fn transpose(&self) -> Matrix<COLS, ROWS> {
        let mut result = Matrix::<COLS, ROWS>::new();
        for r in 0..ROWS {
            for c in 0..COLS {
                result.data[c][r] = self.data[r][c];
            }
        }
        result
    }
}

// The compiler enforces dimensional correctness:
fn multiply<const M: usize, const N: usize, const P: usize>(
    a: &Matrix<M, N>,
    b: &Matrix<N, P>, // N must match!
) -> Matrix<M, P> {
    let mut result = Matrix::<M, P>::new();
    for i in 0..M {
        for j in 0..P {
            for k in 0..N {
                result.data[i][j] += a.data[i][k] * b.data[k][j];
            }
        }
    }
    result
}

// Usage:
let a = Matrix::<2, 3>::new(); // 2×3
let b = Matrix::<3, 4>::new(); // 3×4
let c = multiply(&a, &b);      // 2×4 ✅

// let d = Matrix::<5, 5>::new();
// multiply(&a, &d); // ❌ Compile error: expected Matrix<3, _>, got Matrix<5, 5>
}

C++ comparison: This is similar to template<int N> in C++, but Rust const generics are type-checked eagerly and don’t suffer from SFINAE complexity.

Const Functions (const fn)

const fn marks a function as evaluable at compile time — Rust’s equivalent of C++ constexpr. The result can be used in const and static contexts:

#![allow(unused)]
fn main() {
// Basic const fn — evaluated at compile time when used in const context
const fn celsius_to_fahrenheit(c: f64) -> f64 {
    c * 9.0 / 5.0 + 32.0
}

const BOILING_F: f64 = celsius_to_fahrenheit(100.0); // Computed at compile time
const FREEZING_F: f64 = celsius_to_fahrenheit(0.0);  // 32.0

// Const constructors — create statics without lazy_static!
struct BitMask(u32);

impl BitMask {
    const fn new(bit: u32) -> Self {
        BitMask(1 << bit)
    }

    const fn or(self, other: BitMask) -> Self {
        BitMask(self.0 | other.0)
    }

    const fn contains(&self, bit: u32) -> bool {
        self.0 & (1 << bit) != 0
    }
}

// Static lookup table — no runtime cost, no lazy initialization
const GPIO_INPUT:  BitMask = BitMask::new(0);
const GPIO_OUTPUT: BitMask = BitMask::new(1);
const GPIO_IRQ:    BitMask = BitMask::new(2);
const GPIO_IO:     BitMask = GPIO_INPUT.or(GPIO_OUTPUT);

// Register maps as const arrays:
const SENSOR_THRESHOLDS: [u16; 4] = {
    let mut table = [0u16; 4];
    table[0] = 50;   // Warning
    table[1] = 70;   // High
    table[2] = 85;   // Critical
    table[3] = 100;  // Shutdown
    table
};
// The entire table exists in the binary — no heap, no runtime init.
}

What you CAN do in const fn (as of Rust 1.79+):

  • Arithmetic, bit operations, comparisons
  • if/else, match, loop, while (control flow)
  • Creating and modifying local variables (let mut)
  • Calling other const fns
  • References (&, &mut — within the const context)
  • panic!() (becomes a compile error if reached at compile time)
  • Basic floating-point arithmetic (+, -, *, /; complex ops like sqrt/sin are not const-eligible)

What you CANNOT do (yet):

  • Heap allocation (Box, Vec, String)
  • Trait method calls (only inherent methods)
  • I/O or side effects
#![allow(unused)]
fn main() {
// const fn with panic — becomes a compile-time error:
const fn checked_div(a: u32, b: u32) -> u32 {
    if b == 0 {
        panic!("division by zero"); // Compile error if b is 0 at const time
    }
    a / b
}

const RESULT: u32 = checked_div(100, 4);  // ✅ 25
// const BAD: u32 = checked_div(100, 0);  // ❌ Compile error: "division by zero"
}

C++ comparison: const fn is Rust’s constexpr. The key difference: Rust’s version is opt-in and the compiler rigorously verifies that only const-compatible operations are used. In C++, constexpr functions can silently fall back to runtime evaluation — in Rust, a const context requires compile-time evaluation or it’s a hard error.

Practical advice: Make constructors and simple utility functions const fn whenever possible — it costs nothing and enables callers to use them in const contexts. For hardware diagnostic code, const fn is ideal for register definitions, bitmask construction, and threshold tables.

Key Takeaways — Generics

  • Monomorphization gives zero-cost abstractions but can cause code bloat — use dyn Trait for cold paths
  • Const generics ([T; N]) replace C++ template tricks with compile-time–checked array sizes
  • const fn eliminates lazy_static! for compile-time–computable values

See also: Ch 2 — Traits In Depth for trait bounds, associated types, and trait objects. Ch 4 — PhantomData for zero-sized generic markers.


Exercise: Generic Cache with Eviction ★★ (~30 min)

Build a generic Cache<K, V> struct that stores key-value pairs with a configurable maximum capacity. When full, the oldest entry is evicted (FIFO). Requirements:

  • fn new(capacity: usize) -> Self
  • fn insert(&mut self, key: K, value: V) — evicts the oldest if at capacity
  • fn get(&self, key: &K) -> Option<&V>
  • fn len(&self) -> usize
  • Constrain K: Eq + Hash + Clone
🔑 Solution
use std::collections::{HashMap, VecDeque};
use std::hash::Hash;

struct Cache<K, V> {
    map: HashMap<K, V>,
    order: VecDeque<K>,
    capacity: usize,
}

impl<K: Eq + Hash + Clone, V> Cache<K, V> {
    fn new(capacity: usize) -> Self {
        Cache {
            map: HashMap::with_capacity(capacity),
            order: VecDeque::with_capacity(capacity),
            capacity,
        }
    }

    fn insert(&mut self, key: K, value: V) {
        if self.capacity == 0 {
            // no capacity!
            return;
        }
        if self.map.contains_key(&key) {
            self.map.insert(key, value);
            return;
        }
        if self.map.len() >= self.capacity {
            if let Some(oldest) = self.order.pop_front() {
                self.map.remove(&oldest);
            }
        }
        self.order.push_back(key.clone());
        self.map.insert(key, value);
    }

    fn get(&self, key: &K) -> Option<&V> {
        self.map.get(key)
    }

    fn len(&self) -> usize {
        self.map.len()
    }
}

fn main() {
    // Test of a basic cache
    let mut cache = Cache::new(3);
    cache.insert("a", 1);
    cache.insert("b", 2);
    cache.insert("c", 3);
    assert_eq!(cache.len(), 3);

    cache.insert("d", 4); // Evicts "a"
    assert_eq!(cache.get(&"a"), None);
    assert_eq!(cache.get(&"d"), Some(&4));

    // Left to the reader: what type should `capacity` attribute be,
    // to ensure that such a useless cache cannot be defined?
    let mut empty_cache = Cache::new(0);
    empty_cache.insert("0", 0);
    assert_eq!(empty_cache.get(&"0"), None);
    assert_eq!(empty_cache.len(), 0);

    println!("Cache works! len = {}", cache.len());
}

2. Traits In Depth 🟡

What you’ll learn:

  • Associated types vs generic parameters — and when to use each
  • GATs, blanket impls, marker traits, and trait object safety rules
  • How vtables and fat pointers work under the hood
  • Extension traits, enum dispatch, and typed command patterns

Associated Types vs Generic Parameters

Both let a trait work with different types, but they serve different purposes:

#![allow(unused)]
fn main() {
// --- ASSOCIATED TYPE: One implementation per type ---
trait Iterator {
    type Item; // Each iterator produces exactly ONE kind of item

    fn next(&mut self) -> Option<Self::Item>;
}

// A custom iterator that always yields i32 — there's no choice
struct Counter { max: i32, current: i32 }

impl Iterator for Counter {
    type Item = i32; // Exactly one Item type per implementation
    fn next(&mut self) -> Option<i32> {
        if self.current < self.max {
            self.current += 1;
            Some(self.current)
        } else {
            None
        }
    }
}

// --- GENERIC PARAMETER: Multiple implementations per type ---
trait Convert<T> {
    fn convert(&self) -> T;
}

// A single type can implement Convert for MANY target types:
impl Convert<f64> for i32 {
    fn convert(&self) -> f64 { *self as f64 }
}
impl Convert<String> for i32 {
    fn convert(&self) -> String { self.to_string() }
}
}

When to use which:

UseWhen
Associated typeThere’s exactly ONE natural output/result per implementing type. Iterator::Item, Deref::Target, Add::Output
Generic parameterA type can meaningfully implement the trait for MANY different types. From<T>, AsRef<T>, PartialEq<Rhs>

Intuition: If it makes sense to ask “what is the Item of this iterator?”, use associated type. If it makes sense to ask “can this convert to f64? to String? to bool?”, use a generic parameter.

#![allow(unused)]
fn main() {
// Real-world example: std::ops::Add
trait Add<Rhs = Self> {
    type Output; // Associated type — addition has ONE result type
    fn add(self, rhs: Rhs) -> Self::Output;
}

// Rhs is a generic parameter — you can add different types to Meters:
struct Meters(f64);
struct Centimeters(f64);

impl Add<Meters> for Meters {
    type Output = Meters;
    fn add(self, rhs: Meters) -> Meters { Meters(self.0 + rhs.0) }
}
impl Add<Centimeters> for Meters {
    type Output = Meters;
    fn add(self, rhs: Centimeters) -> Meters { Meters(self.0 + rhs.0 / 100.0) }
}
}

Generic Associated Types (GATs)

Since Rust 1.65, associated types can have generic parameters of their own. This enables lending iterators — iterators that return references tied to the iterator rather than to the underlying collection:

#![allow(unused)]
fn main() {
// Without GATs — impossible to express a lending iterator:
// trait LendingIterator {
//     type Item<'a>;  // ← This was rejected before 1.65
// }

// With GATs (Rust 1.65+):
trait LendingIterator {
    type Item<'a> where Self: 'a;

    fn next(&mut self) -> Option<Self::Item<'_>>;
}

// Example: an iterator that yields overlapping windows
struct WindowIter<'data> {
    data: &'data [u8],
    pos: usize,
    window_size: usize,
}

impl<'data> LendingIterator for WindowIter<'data> {
    type Item<'a> = &'a [u8] where Self: 'a;

    fn next(&mut self) -> Option<&[u8]> {
        if self.pos + self.window_size <= self.data.len() {
            let window = &self.data[self.pos..self.pos + self.window_size];
            self.pos += 1;
            Some(window)
        } else {
            None
        }
    }
}
}

When you need GATs: Lending iterators, streaming parsers, or any trait where the associated type’s lifetime depends on the &self borrow. For most code, plain associated types are sufficient.

Supertraits and Trait Hierarchies

Traits can require other traits as prerequisites, forming hierarchies:

graph BT
    Display["Display"]
    Debug["Debug"]
    Error["Error"]
    Clone["Clone"]
    Copy["Copy"]
    PartialEq["PartialEq"]
    Eq["Eq"]
    PartialOrd["PartialOrd"]
    Ord["Ord"]

    Error --> Display
    Error --> Debug
    Copy --> Clone
    Eq --> PartialEq
    Ord --> Eq
    Ord --> PartialOrd
    PartialOrd --> PartialEq

    style Display fill:#e8f4f8,stroke:#2980b9,color:#000
    style Debug fill:#e8f4f8,stroke:#2980b9,color:#000
    style Error fill:#fdebd0,stroke:#e67e22,color:#000
    style Clone fill:#d4efdf,stroke:#27ae60,color:#000
    style Copy fill:#d4efdf,stroke:#27ae60,color:#000
    style PartialEq fill:#fef9e7,stroke:#f1c40f,color:#000
    style Eq fill:#fef9e7,stroke:#f1c40f,color:#000
    style PartialOrd fill:#fef9e7,stroke:#f1c40f,color:#000
    style Ord fill:#fef9e7,stroke:#f1c40f,color:#000

Arrows point from subtrait to supertrait: implementing Error requires Display + Debug.

A trait can require that implementors also implement other traits:

#![allow(unused)]
fn main() {
use std::fmt;

// Display is a supertrait of Error
trait Error: fmt::Display + fmt::Debug {
    fn source(&self) -> Option<&(dyn Error + 'static)> { None }
}
// Any type implementing Error MUST also implement Display and Debug

// Build your own hierarchies:
trait Identifiable {
    fn id(&self) -> u64;
}

trait Timestamped {
    fn created_at(&self) -> chrono::DateTime<chrono::Utc>;
}

// Entity requires both:
trait Entity: Identifiable + Timestamped {
    fn is_active(&self) -> bool;
}

// Implementing Entity forces you to implement all three:
struct User { id: u64, name: String, created: chrono::DateTime<chrono::Utc> }

impl Identifiable for User {
    fn id(&self) -> u64 { self.id }
}
impl Timestamped for User {
    fn created_at(&self) -> chrono::DateTime<chrono::Utc> { self.created }
}
impl Entity for User {
    fn is_active(&self) -> bool { true }
}
}

Blanket Implementations

Implement a trait for ALL types that satisfy some bound:

#![allow(unused)]
fn main() {
// std does this: any type that implements Display automatically gets ToString
impl<T: fmt::Display> ToString for T {
    fn to_string(&self) -> String {
        format!("{self}")
    }
}
// Now i32, &str, your custom types — anything with Display — gets to_string() for free.

// Your own blanket impl:
trait Loggable {
    fn log(&self);
}

// Every Debug type is automatically Loggable:
impl<T: std::fmt::Debug> Loggable for T {
    fn log(&self) {
        eprintln!("[LOG] {self:?}");
    }
}

// Now ANY Debug type has .log():
// 42.log();              // [LOG] 42
// "hello".log();         // [LOG] "hello"
// vec![1, 2, 3].log();   // [LOG] [1, 2, 3]
}

Caution: Blanket impls are powerful but irreversible — you can’t add a more specific impl for a type that’s already covered by a blanket impl (orphan rules + coherence). Design them carefully.

Marker Traits

Traits with no methods — they mark a type as having some property:

#![allow(unused)]
fn main() {
// Standard library marker traits:
// Send    — safe to transfer between threads
// Sync    — safe to share (&T) between threads
// Unpin   — safe to move after pinning
// Sized   — has a known size at compile time
// Copy    — can be duplicated with memcpy

// Your own marker trait:
/// Marker: this sensor has been factory-calibrated
trait Calibrated {}

struct RawSensor { reading: f64 }
struct CalibratedSensor { reading: f64 }

impl Calibrated for CalibratedSensor {}

// Only calibrated sensors can be used in production:
fn record_measurement<S: Calibrated>(sensor: &S) {
    // ...
}
// record_measurement(&RawSensor { reading: 0.0 }); // ❌ Compile error
// record_measurement(&CalibratedSensor { reading: 0.0 }); // ✅
}

This connects directly to the type-state pattern in Chapter 3.

Trait Object Safety Rules

Not every trait can be used as dyn Trait. A trait is object-safe only if:

  1. No Self: Sized bound on the trait itself
  2. No generic type parameters on methods
  3. No use of Self in return position (except via indirection like Box<Self>)
  4. No associated functions (methods must have &self, &mut self, or self)
#![allow(unused)]
fn main() {
// ✅ Object-safe — can be used as dyn Drawable
trait Drawable {
    fn draw(&self);
    fn bounding_box(&self) -> (f64, f64, f64, f64);
}

let shapes: Vec<Box<dyn Drawable>> = vec![/* ... */]; // ✅ Works

// ❌ NOT object-safe — uses Self in return position
trait Cloneable {
    fn clone_self(&self) -> Self;
    //                       ^^^^ Can't know the concrete size at runtime
}
// let items: Vec<Box<dyn Cloneable>> = ...; // ❌ Compile error

// ❌ NOT object-safe — generic method
trait Converter {
    fn convert<T>(&self) -> T;
    //        ^^^ The vtable can't contain infinite monomorphizations
}

// ❌ NOT object-safe — associated function (no self)
trait Factory {
    fn create() -> Self;
    // No &self — how would you call this through a trait object?
}
}

Workarounds:

#![allow(unused)]
fn main() {
// Add `where Self: Sized` to exclude a method from the vtable:
trait MyTrait {
    fn regular_method(&self); // Included in vtable

    fn generic_method<T>(&self) -> T
    where
        Self: Sized; // Excluded from vtable — can't be called via dyn MyTrait
}

// Now dyn MyTrait is valid, but generic_method can only be called
// when the concrete type is known.
}

Rule of thumb: If you plan to use dyn Trait, keep methods simple — no generics, no Self in return types, no Sized bounds. When in doubt, try let _: Box<dyn YourTrait>; and let the compiler tell you.

Trait Objects Under the Hood — vtables and Fat Pointers

A &dyn Trait (or Box<dyn Trait>) is a fat pointer — two machine words:

┌──────────────────────────────────────────────────┐
│  &dyn Drawable (on 64-bit: 16 bytes total)       │
├──────────────┬───────────────────────────────────┤
│  data_ptr    │  vtable_ptr                       │
│  (8 bytes)   │  (8 bytes)                        │
│  ↓           │  ↓                                │
│  ┌─────────┐ │  ┌──────────────────────────────┐ │
│  │ Circle  │ │  │ vtable for <Circle as        │ │
│  │ {       │ │  │           Drawable>          │ │
│  │  r: 5.0 │ │  │                              │ │
│  │ }       │ │  │  drop_in_place: 0x7f...a0    │ │
│  └─────────┘ │  │  size:           8           │ │
│              │  │  align:          8           │ │
│              │  │  draw:          0x7f...b4    │ │
│              │  │  bounding_box:  0x7f...c8    │ │
│              │  └──────────────────────────────┘ │
└──────────────┴───────────────────────────────────┘

How a vtable call works (e.g., shape.draw()):

  1. Load vtable_ptr from the fat pointer (second word)
  2. Index into the vtable to find the draw function pointer
  3. Call it, passing data_ptr as the self argument

This is similar to C++ virtual dispatch in cost (one pointer indirection per call), but Rust stores the vtable pointer in the fat pointer rather than inside the object — so a plain Circle on the stack carries no vtable pointer at all.

trait Drawable {
    fn draw(&self);
    fn area(&self) -> f64;
}

struct Circle { radius: f64 }

impl Drawable for Circle {
    fn draw(&self) { println!("Drawing circle r={}", self.radius); }
    fn area(&self) -> f64 { std::f64::consts::PI * self.radius * self.radius }
}

struct Square { side: f64 }

impl Drawable for Square {
    fn draw(&self) { println!("Drawing square s={}", self.side); }
    fn area(&self) -> f64 { self.side * self.side }
}

fn main() {
    let shapes: Vec<Box<dyn Drawable>> = vec![
        Box::new(Circle { radius: 5.0 }),
        Box::new(Square { side: 3.0 }),
    ];

    // Each element is a fat pointer: (data_ptr, vtable_ptr)
    // The vtable for Circle and Square are DIFFERENT
    for shape in &shapes {
        shape.draw();  // vtable dispatch → Circle::draw or Square::draw
        println!("  area = {:.2}", shape.area());
    }

    // Size comparison:
    println!("size_of::<&Circle>()        = {}", size_of::<&Circle>());
    // → 8 bytes (one pointer — the compiler knows the type)
    println!("size_of::<&dyn Drawable>()  = {}", size_of::<&dyn Drawable>());
    // → 16 bytes (data_ptr + vtable_ptr)
}

Performance cost model:

AspectStatic dispatch (impl Trait / generics)Dynamic dispatch (dyn Trait)
Call overheadZero — inlined by LLVMOne pointer indirection per call
Inlining✅ Compiler can inline❌ Opaque function pointer
Binary sizeLarger (one copy per type)Smaller (one shared function)
Pointer sizeThin (1 word)Fat (2 words)
Heterogeneous collections❌✅ Vec<Box<dyn Trait>>

When vtable cost matters: In tight loops calling a trait method millions of times, the indirection and inability to inline can be significant (2-10× slower). For cold paths, configuration, or plugin architectures, the flexibility of dyn Trait is worth the small cost.

Higher-Ranked Trait Bounds (HRTBs)

Sometimes you need a function that works with references of any lifetime, not a specific one. This is where for<'a> syntax appears:

// Problem: this function needs a closure that can process
// references with ANY lifetime, not just one specific lifetime.

// ❌ This is too restrictive — 'a is fixed by the caller:
// fn apply<'a, F: Fn(&'a str) -> &'a str>(f: F, data: &'a str) -> &'a str

// ✅ HRTB: F must work for ALL possible lifetimes:
fn apply<F>(f: F, data: &str) -> &str
where
    F: for<'a> Fn(&'a str) -> &'a str,
{
    f(data)
}

fn main() {
    let result = apply(|s| s.trim(), "  hello  ");
    println!("{result}"); // "hello"
}

When you encounter HRTBs:

  • Fn(&T) -> &U traits — the compiler infers for<'a> automatically in most cases
  • Custom trait implementations that must work across different borrows
  • Deserialization with serde: for<'de> Deserialize<'de>
// serde's DeserializeOwned is defined as:
// trait DeserializeOwned: for<'de> Deserialize<'de> {}
// Meaning: "can be deserialized from data with ANY lifetime"
// (i.e., the result doesn't borrow from the input)

use serde::de::DeserializeOwned;

fn parse_json<T: DeserializeOwned>(input: &str) -> T {
    serde_json::from_str(input).unwrap()
}

Practical advice: You’ll rarely write for<'a> yourself. It mostly appears in trait bounds on closure parameters, where the compiler handles it implicitly. But recognizing it in error messages (“expected a for<'a> Fn(&'a ...) bound”) helps you understand what the compiler is asking for.

impl Trait — Argument Position vs Return Position

impl Trait appears in two positions with different semantics:

#![allow(unused)]
fn main() {
// --- Argument-Position impl Trait (APIT) ---
// "Caller chooses the type" — syntactic sugar for a generic parameter
fn print_all(items: impl Iterator<Item = i32>) {
    for item in items { println!("{item}"); }
}
// Equivalent to:
fn print_all_verbose<I: Iterator<Item = i32>>(items: I) {
    for item in items { println!("{item}"); }
}
// Caller decides: print_all(vec![1,2,3].into_iter())
//                 print_all(0..10)

// --- Return-Position impl Trait (RPIT) ---
// "Callee chooses the type" — the function picks one concrete type
fn evens(limit: i32) -> impl Iterator<Item = i32> {
    (0..limit).filter(|x| x % 2 == 0)
    // The concrete type is Filter<Range<i32>, Closure>
    // but the caller only sees "some Iterator<Item = i32>"
}
}

Key difference:

APIT (fn foo(x: impl T))RPIT (fn foo() -> impl T)
Who picks the type?CallerCallee (function body)
Monomorphized?Yes — one copy per typeYes — one concrete type
Turbofish?No (foo::<X>() not allowed)N/A
Equivalent tofn foo<X: T>(x: X)Existential type

RPIT in Trait Definitions (RPITIT)

Since Rust 1.75, you can use -> impl Trait directly in trait definitions:

#![allow(unused)]
fn main() {
trait Container {
    fn items(&self) -> impl Iterator<Item = &str>;
    //                 ^^^^ Each implementor returns its own concrete type
}

struct CsvRow {
    fields: Vec<String>,
}

impl Container for CsvRow {
    fn items(&self) -> impl Iterator<Item = &str> {
        self.fields.iter().map(String::as_str)
    }
}

struct FixedFields;

impl Container for FixedFields {
    fn items(&self) -> impl Iterator<Item = &str> {
        ["host", "port", "timeout"].into_iter()
    }
}
}

Before Rust 1.75, you had to use Box<dyn Iterator> or an associated type to achieve this in traits. RPITIT removes the allocation.

impl Trait vs dyn Trait — Decision Guide

Do you know the concrete type at compile time?
├── YES → Use impl Trait or generics (zero cost, inlinable)
└── NO  → Do you need a heterogeneous collection?
     ├── YES → Use dyn Trait (Box<dyn T>, &dyn T)
     └── NO  → Do you need the SAME trait object across an API boundary?
          ├── YES → Use dyn Trait
          └── NO  → Use generics / impl Trait
Featureimpl Traitdyn Trait
DispatchStatic (monomorphized)Dynamic (vtable)
PerformanceBest — inlinableOne indirection per call
Heterogeneous collections❌✅
Binary size per typeOne copy eachShared code
Trait must be object-safe?NoYes
Works in trait definitions✅ (Rust 1.75+)Always

Type Erasure with Any and TypeId

Sometimes you need to store values of unknown types and downcast them later — a pattern familiar from void* in C or object in C#. Rust provides this through std::any::Any:

use std::any::Any;

// Store heterogeneous values:
fn log_value(value: &dyn Any) {
    if let Some(s) = value.downcast_ref::<String>() {
        println!("String: {s}");
    } else if let Some(n) = value.downcast_ref::<i32>() {
        println!("i32: {n}");
    } else {
        // TypeId lets you inspect the type at runtime:
        println!("Unknown type: {:?}", value.type_id());
    }
}

// Useful for plugin systems, event buses, or ECS-style architectures:
struct AnyMap(std::collections::HashMap<std::any::TypeId, Box<dyn Any + Send>>);

impl AnyMap {
    fn new() -> Self { AnyMap(std::collections::HashMap::new()) }

    fn insert<T: Any + Send + 'static>(&mut self, value: T) {
        self.0.insert(std::any::TypeId::of::<T>(), Box::new(value));
    }

    fn get<T: Any + Send + 'static>(&self) -> Option<&T> {
        self.0.get(&std::any::TypeId::of::<T>())?
            .downcast_ref()
    }
}

fn main() {
    let mut map = AnyMap::new();
    map.insert(42_i32);
    map.insert(String::from("hello"));

    assert_eq!(map.get::<i32>(), Some(&42));
    assert_eq!(map.get::<String>().map(|s| s.as_str()), Some("hello"));
    assert_eq!(map.get::<f64>(), None); // Never inserted
}

When to use Any: Plugin/extension systems, type-indexed maps (typemap), error downcasting (anyhow::Error::downcast_ref). Prefer generics or trait objects when the set of types is known at compile time — Any is a last resort that trades compile-time safety for flexibility.


Extension Traits — Adding Methods to Types You Don’t Own

Rust’s orphan rule prevents you from implementing a foreign trait on a foreign type. Extension traits are the standard workaround: define a new trait in your crate whose methods have a blanket implementation for any type that meets a bound. The caller imports the trait and the new methods appear on existing types.

This pattern is pervasive in the Rust ecosystem: itertools::Itertools, futures::StreamExt, tokio::io::AsyncReadExt, tower::ServiceExt.

The Problem

#![allow(unused)]
fn main() {
// We want to add a .mean() method to all iterators that yield f64.
// But Iterator is defined in std and f64 is a primitive — orphan rule prevents:
//
// impl<I: Iterator<Item = f64>> I {   // ❌ Cannot add inherent methods to a foreign type
//     fn mean(self) -> f64 { ... }
// }
}

The Solution: An Extension Trait

#![allow(unused)]
fn main() {
/// Extension methods for iterators over numeric values.
pub trait IteratorExt: Iterator {
    /// Computes the arithmetic mean. Returns `None` for empty iterators.
    fn mean(self) -> Option<f64>
    where
        Self: Sized,
        Self::Item: Into<f64>;
}

// Blanket implementation — automatically applies to ALL iterators
impl<I: Iterator> IteratorExt for I {
    fn mean(self) -> Option<f64>
    where
        Self: Sized,
        Self::Item: Into<f64>,
    {
        let mut sum: f64 = 0.0;
        let mut count: u64 = 0;
        for item in self {
            sum += item.into();
            count += 1;
        }
        if count == 0 { None } else { Some(sum / count as f64) }
    }
}

// Usage — just import the trait:
use crate::IteratorExt;  // One import and the method appears on all iterators

fn analyze_temperatures(readings: &[f64]) -> Option<f64> {
    readings.iter().copied().mean()  // .mean() is now available!
}

fn analyze_sensor_data(data: &[i32]) -> Option<f64> {
    data.iter().copied().mean()  // Works on i32 too (i32: Into<f64>)
}
}

Real-World Example: Diagnostic Result Extensions

#![allow(unused)]
fn main() {
use std::collections::HashMap;

struct DiagResult {
    component: String,
    passed: bool,
    message: String,
}

/// Extension trait for Vec<DiagResult> — adds domain-specific analysis methods.
pub trait DiagResultsExt {
    fn passed_count(&self) -> usize;
    fn failed_count(&self) -> usize;
    fn overall_pass(&self) -> bool;
    fn failures_by_component(&self) -> HashMap<String, Vec<&DiagResult>>;
}

impl DiagResultsExt for Vec<DiagResult> {
    fn passed_count(&self) -> usize {
        self.iter().filter(|r| r.passed).count()
    }

    fn failed_count(&self) -> usize {
        self.iter().filter(|r| !r.passed).count()
    }

    fn overall_pass(&self) -> bool {
        self.iter().all(|r| r.passed)
    }

    fn failures_by_component(&self) -> HashMap<String, Vec<&DiagResult>> {
        let mut map = HashMap::new();
        for r in self.iter().filter(|r| !r.passed) {
            map.entry(r.component.clone()).or_default().push(r);
        }
        map
    }
}

// Now any Vec<DiagResult> has these methods:
fn report(results: Vec<DiagResult>) {
    if !results.overall_pass() {
        let failures = results.failures_by_component();
        for (component, fails) in &failures {
            eprintln!("{component}: {} failures", fails.len());
        }
    }
}
}

Naming Convention

The Rust ecosystem uses a consistent Ext suffix:

CrateExtension TraitExtends
itertoolsItertoolsIterator
futuresStreamExt, FutureExtStream, Future
tokioAsyncReadExt, AsyncWriteExtAsyncRead, AsyncWrite
towerServiceExtService
bytesBufMut (partial)&mut [u8]
Your crateDiagResultsExtVec<DiagResult>

When to Use

SituationUse Extension Trait?
Adding convenience methods to a foreign type✅
Grouping domain-specific logic on generic collections✅
The method needs access to private fields❌ (use a wrapper/newtype)
The method logically belongs on a new type you control❌ (just add it to your type)
You want the method available without any import❌ (inherent methods only)

Enum Dispatch — Static Polymorphism Without dyn

When you have a closed set of types implementing a trait, you can replace dyn Trait with an enum whose variants hold the concrete types. This eliminates the vtable indirection and heap allocation while preserving the same caller-facing interface.

The Problem with dyn Trait

#![allow(unused)]
fn main() {
trait Sensor {
    fn read(&self) -> f64;
    fn name(&self) -> &str;
}

struct Gps { lat: f64, lon: f64 }
struct Thermometer { temp_c: f64 }
struct Accelerometer { g_force: f64 }

impl Sensor for Gps {
    fn read(&self) -> f64 { self.lat }
    fn name(&self) -> &str { "GPS" }
}
impl Sensor for Thermometer {
    fn read(&self) -> f64 { self.temp_c }
    fn name(&self) -> &str { "Thermometer" }
}
impl Sensor for Accelerometer {
    fn read(&self) -> f64 { self.g_force }
    fn name(&self) -> &str { "Accelerometer" }
}

// Heterogeneous collection with dyn — works, but has costs:
fn read_all_dyn(sensors: &[Box<dyn Sensor>]) -> Vec<f64> {
    sensors.iter().map(|s| s.read()).collect()
    // Each .read() goes through a vtable indirection
    // Each Box allocates on the heap
}
}

The Enum Dispatch Solution

// Replace the trait object with an enum:
enum AnySensor {
    Gps(Gps),
    Thermometer(Thermometer),
    Accelerometer(Accelerometer),
}

impl AnySensor {
    fn read(&self) -> f64 {
        match self {
            AnySensor::Gps(s) => s.read(),
            AnySensor::Thermometer(s) => s.read(),
            AnySensor::Accelerometer(s) => s.read(),
        }
    }

    fn name(&self) -> &str {
        match self {
            AnySensor::Gps(s) => s.name(),
            AnySensor::Thermometer(s) => s.name(),
            AnySensor::Accelerometer(s) => s.name(),
        }
    }
}

// Now: no heap allocation, no vtable, stored inline
fn read_all(sensors: &[AnySensor]) -> Vec<f64> {
    sensors.iter().map(|s| s.read()).collect()
    // Each .read() is a match branch — compiler can inline everything
}

fn main() {
    let sensors = vec![
        AnySensor::Gps(Gps { lat: 47.6, lon: -122.3 }),
        AnySensor::Thermometer(Thermometer { temp_c: 72.5 }),
        AnySensor::Accelerometer(Accelerometer { g_force: 1.02 }),
    ];

    for sensor in &sensors {
        println!("{}: {:.2}", sensor.name(), sensor.read());
    }
}

Implement the Trait on the Enum

For interoperability, you can implement the original trait on the enum itself:

#![allow(unused)]
fn main() {
impl Sensor for AnySensor {
    fn read(&self) -> f64 {
        match self {
            AnySensor::Gps(s) => s.read(),
            AnySensor::Thermometer(s) => s.read(),
            AnySensor::Accelerometer(s) => s.read(),
        }
    }

    fn name(&self) -> &str {
        match self {
            AnySensor::Gps(s) => s.name(),
            AnySensor::Thermometer(s) => s.name(),
            AnySensor::Accelerometer(s) => s.name(),
        }
    }
}

// Now AnySensor works anywhere a Sensor is expected via generics:
fn report<S: Sensor>(s: &S) {
    println!("{}: {:.2}", s.name(), s.read());
}
}

Reducing Boilerplate with a Macro

The match-arm delegation is repetitive. A macro eliminates it:

#![allow(unused)]
fn main() {
macro_rules! dispatch_sensor {
    ($self:expr, $method:ident $(, $arg:expr)*) => {
        match $self {
            AnySensor::Gps(s) => s.$method($($arg),*),
            AnySensor::Thermometer(s) => s.$method($($arg),*),
            AnySensor::Accelerometer(s) => s.$method($($arg),*),
        }
    };
}

impl Sensor for AnySensor {
    fn read(&self) -> f64     { dispatch_sensor!(self, read) }
    fn name(&self) -> &str    { dispatch_sensor!(self, name) }
}
}

For larger projects, the enum_dispatch crate automates this entirely:

#![allow(unused)]
fn main() {
use enum_dispatch::enum_dispatch;

#[enum_dispatch]
trait Sensor {
    fn read(&self) -> f64;
    fn name(&self) -> &str;
}

#[enum_dispatch(Sensor)]
enum AnySensor {
    Gps,
    Thermometer,
    Accelerometer,
}
// All delegation code is generated automatically.
}

dyn Trait vs Enum Dispatch — Decision Guide

Is the set of types closed (known at compile time)?
├── YES → Prefer enum dispatch (faster, no heap allocation)
│         ├── Few variants (< ~20)?     → Manual enum
│         └── Many variants or growing? → enum_dispatch crate
└── NO  → Must use dyn Trait (plugins, user-provided types)
Propertydyn TraitEnum Dispatch
Dispatch costVtable indirection (~2ns)Branch prediction (~0.3ns)
Heap allocationUsually (Box)None (inline)
Cache-friendlyNo (pointer chasing)Yes (contiguous)
Open to new types✅ (anyone can impl)❌ (closed set)
Code sizeSharedOne copy per variant
Trait must be object-safeYesNo
Adding a variantNo code changesUpdate enum + match arms

When to Use Enum Dispatch

ScenarioRecommendation
Diagnostic test types (CPU, GPU, NIC, Memory, …)✅ Enum dispatch — closed set, known at compile time
Bus protocols (SPI, I2C, UART, …)✅ Enum dispatch or Config trait
Plugin system (user loads .so at runtime)❌ Use dyn Trait
2-3 variants✅ Manual enum dispatch
10+ variants with many methods✅ enum_dispatch crate
Performance-critical inner loop✅ Enum dispatch (eliminates vtable)

Capability Mixins — Associated Types as Zero-Cost Composition

Ruby developers compose behaviour with mixins — include SomeModule injects methods into a class. Rust traits with associated types + default methods + blanket impls produce the same result, except:

  • Everything resolves at compile time — no method-missing surprises
  • Each associated type is a knob that changes what the default methods produce
  • The compiler monomorphises each combination — zero vtable overhead

The Problem: Cross-Cutting Bus Dependencies

Hardware diagnostic routines share common operations — read an IPMI sensor, toggle a GPIO rail, sample a temperature over SPI — but different diagnostics need different combinations. Inheritance hierarchies don’t exist in Rust. Passing every bus handle as a function argument creates unwieldy signatures. We need a way to mix in bus capabilities à la carte.

Step 1 — Define “Ingredient” Traits

Each ingredient provides one hardware capability via an associated type:

#![allow(unused)]
fn main() {
use std::io;

// ── Bus abstractions (traits the hardware team provides) ──────────
pub trait SpiBus {
    fn spi_transfer(&self, tx: &[u8], rx: &mut [u8]) -> io::Result<()>;
}

pub trait I2cBus {
    fn i2c_read(&self, addr: u8, reg: u8, buf: &mut [u8]) -> io::Result<()>;
    fn i2c_write(&self, addr: u8, reg: u8, data: &[u8]) -> io::Result<()>;
}

pub trait GpioPin {
    fn set_high(&self) -> io::Result<()>;
    fn set_low(&self) -> io::Result<()>;
    fn read_level(&self) -> io::Result<bool>;
}

pub trait IpmiBmc {
    fn raw_command(&self, net_fn: u8, cmd: u8, data: &[u8]) -> io::Result<Vec<u8>>;
    fn read_sensor(&self, sensor_id: u8) -> io::Result<f64>;
}

// ── Ingredient traits — one per bus, carries an associated type ───
pub trait HasSpi {
    type Spi: SpiBus;
    fn spi(&self) -> &Self::Spi;
}

pub trait HasI2c {
    type I2c: I2cBus;
    fn i2c(&self) -> &Self::I2c;
}

pub trait HasGpio {
    type Gpio: GpioPin;
    fn gpio(&self) -> &Self::Gpio;
}

pub trait HasIpmi {
    type Ipmi: IpmiBmc;
    fn ipmi(&self) -> &Self::Ipmi;
}
}

Each ingredient is tiny, generic, and testable in isolation.

Step 2 — Define “Mixin” Traits

A mixin trait declares its required ingredients as supertraits, then provides all its methods via defaults — implementors get them for free:

#![allow(unused)]
fn main() {
/// Mixin: fan diagnostics — needs I2C (tachometer) + GPIO (PWM enable)
pub trait FanDiagMixin: HasI2c + HasGpio {
    /// Read fan RPM from the tachometer IC over I2C.
    fn read_fan_rpm(&self, fan_id: u8) -> io::Result<u32> {
        let mut buf = [0u8; 2];
        self.i2c().i2c_read(0x48 + fan_id, 0x00, &mut buf)?;
        Ok(u16::from_be_bytes(buf) as u32 * 60) // tach counts → RPM
    }

    /// Enable or disable the fan PWM output via GPIO.
    fn set_fan_pwm(&self, enable: bool) -> io::Result<()> {
        if enable { self.gpio().set_high() }
        else      { self.gpio().set_low() }
    }

    /// Full fan health check — read RPM + verify within threshold.
    fn check_fan_health(&self, fan_id: u8, min_rpm: u32) -> io::Result<bool> {
        let rpm = self.read_fan_rpm(fan_id)?;
        Ok(rpm >= min_rpm)
    }
}

/// Mixin: temperature monitoring — needs SPI (thermocouple ADC) + IPMI (BMC sensors)
pub trait TempMonitorMixin: HasSpi + HasIpmi {
    /// Read a thermocouple via the SPI ADC (e.g. MAX31855).
    fn read_thermocouple(&self) -> io::Result<f64> {
        let mut rx = [0u8; 4];
        self.spi().spi_transfer(&[0x00; 4], &mut rx)?;
        let raw = i32::from_be_bytes(rx) >> 18; // 14-bit signed
        Ok(raw as f64 * 0.25)
    }

    /// Read a BMC-managed temperature sensor via IPMI.
    fn read_bmc_temp(&self, sensor_id: u8) -> io::Result<f64> {
        self.ipmi().read_sensor(sensor_id)
    }

    /// Cross-validate: thermocouple vs BMC must agree within delta.
    fn validate_temps(&self, sensor_id: u8, max_delta: f64) -> io::Result<bool> {
        let tc = self.read_thermocouple()?;
        let bmc = self.read_bmc_temp(sensor_id)?;
        Ok((tc - bmc).abs() <= max_delta)
    }
}

/// Mixin: power sequencing — needs GPIO (rail enable) + IPMI (event logging)
pub trait PowerSeqMixin: HasGpio + HasIpmi {
    /// Assert the power-good GPIO and verify via IPMI sensor.
    fn enable_power_rail(&self, sensor_id: u8) -> io::Result<bool> {
        self.gpio().set_high()?;
        std::thread::sleep(std::time::Duration::from_millis(50));
        let voltage = self.ipmi().read_sensor(sensor_id)?;
        Ok(voltage > 0.8) // above 80% nominal = good
    }

    /// De-assert power and log shutdown via IPMI OEM command.
    fn disable_power_rail(&self) -> io::Result<()> {
        self.gpio().set_low()?;
        // Log OEM "power rail disabled" event to BMC
        self.ipmi().raw_command(0x2E, 0x01, &[0x00, 0x01])?;
        Ok(())
    }
}
}

Step 3 — Blanket Impls Make It Truly “Mixin”

The magic line — provide the ingredients, get the methods:

#![allow(unused)]
fn main() {
impl<T: HasI2c + HasGpio>  FanDiagMixin    for T {}
impl<T: HasSpi  + HasIpmi>  TempMonitorMixin for T {}
impl<T: HasGpio + HasIpmi>  PowerSeqMixin   for T {}
}

Any struct that implements the right ingredient traits automatically gains every mixin method — no boilerplate, no forwarding, no inheritance.

Step 4 — Wire Up Production

#![allow(unused)]
fn main() {
// ── Concrete bus implementations (Linux platform) ────────────────
struct LinuxSpi  { dev: String }
struct LinuxI2c  { dev: String }
struct SysfsGpio { pin: u32 }
struct IpmiTool  { timeout_secs: u32 }

impl SpiBus for LinuxSpi {
    fn spi_transfer(&self, _tx: &[u8], _rx: &mut [u8]) -> io::Result<()> {
        // spidev ioctl — omitted for brevity
        Ok(())
    }
}
impl I2cBus for LinuxI2c {
    fn i2c_read(&self, _addr: u8, _reg: u8, _buf: &mut [u8]) -> io::Result<()> {
        // i2c-dev ioctl — omitted for brevity
        Ok(())
    }
    fn i2c_write(&self, _addr: u8, _reg: u8, _data: &[u8]) -> io::Result<()> { Ok(()) }
}
impl GpioPin for SysfsGpio {
    fn set_high(&self) -> io::Result<()>  { /* /sys/class/gpio */ Ok(()) }
    fn set_low(&self) -> io::Result<()>   { Ok(()) }
    fn read_level(&self) -> io::Result<bool> { Ok(true) }
}
impl IpmiBmc for IpmiTool {
    fn raw_command(&self, _nf: u8, _cmd: u8, _data: &[u8]) -> io::Result<Vec<u8>> {
        // shells out to ipmitool — omitted for brevity
        Ok(vec![])
    }
    fn read_sensor(&self, _id: u8) -> io::Result<f64> { Ok(25.0) }
}

// ── Production platform — all four buses ─────────────────────────
struct DiagPlatform {
    spi:  LinuxSpi,
    i2c:  LinuxI2c,
    gpio: SysfsGpio,
    ipmi: IpmiTool,
}

impl HasSpi  for DiagPlatform { type Spi  = LinuxSpi;  fn spi(&self)  -> &LinuxSpi  { &self.spi  } }
impl HasI2c  for DiagPlatform { type I2c  = LinuxI2c;  fn i2c(&self)  -> &LinuxI2c  { &self.i2c  } }
impl HasGpio for DiagPlatform { type Gpio = SysfsGpio; fn gpio(&self) -> &SysfsGpio { &self.gpio } }
impl HasIpmi for DiagPlatform { type Ipmi = IpmiTool;  fn ipmi(&self) -> &IpmiTool  { &self.ipmi } }

// DiagPlatform now has ALL mixin methods:
fn production_diagnostics(platform: &DiagPlatform) -> io::Result<()> {
    let rpm = platform.read_fan_rpm(0)?;       // from FanDiagMixin
    let tc  = platform.read_thermocouple()?;   // from TempMonitorMixin
    let ok  = platform.enable_power_rail(42)?;  // from PowerSeqMixin
    println!("Fan: {rpm} RPM, Temp: {tc}°C, Power: {ok}");
    Ok(())
}
}

Step 5 — Test With Mocks (No Hardware Required)

#![allow(unused)]
fn main() {
#[cfg(test)]
mod tests {
    use super::*;
    use std::cell::Cell;

    struct MockSpi  { temp: Cell<f64> }
    struct MockI2c  { rpm: Cell<u32> }
    struct MockGpio { level: Cell<bool> }
    struct MockIpmi { sensor_val: Cell<f64> }

    impl SpiBus for MockSpi {
        fn spi_transfer(&self, _tx: &[u8], rx: &mut [u8]) -> io::Result<()> {
            // Encode mock temp as MAX31855 format
            let raw = ((self.temp.get() / 0.25) as i32) << 18;
            rx.copy_from_slice(&raw.to_be_bytes());
            Ok(())
        }
    }
    impl I2cBus for MockI2c {
        fn i2c_read(&self, _addr: u8, _reg: u8, buf: &mut [u8]) -> io::Result<()> {
            let tach = (self.rpm.get() / 60) as u16;
            buf.copy_from_slice(&tach.to_be_bytes());
            Ok(())
        }
        fn i2c_write(&self, _: u8, _: u8, _: &[u8]) -> io::Result<()> { Ok(()) }
    }
    impl GpioPin for MockGpio {
        fn set_high(&self)  -> io::Result<()>   { self.level.set(true);  Ok(()) }
        fn set_low(&self)   -> io::Result<()>   { self.level.set(false); Ok(()) }
        fn read_level(&self) -> io::Result<bool> { Ok(self.level.get()) }
    }
    impl IpmiBmc for MockIpmi {
        fn raw_command(&self, _: u8, _: u8, _: &[u8]) -> io::Result<Vec<u8>> { Ok(vec![]) }
        fn read_sensor(&self, _: u8) -> io::Result<f64> { Ok(self.sensor_val.get()) }
    }

    // ── Partial platform: only fan-related buses ─────────────────
    struct FanTestRig {
        i2c:  MockI2c,
        gpio: MockGpio,
    }
    impl HasI2c  for FanTestRig { type I2c  = MockI2c;  fn i2c(&self)  -> &MockI2c  { &self.i2c  } }
    impl HasGpio for FanTestRig { type Gpio = MockGpio; fn gpio(&self) -> &MockGpio { &self.gpio } }
    // FanTestRig gets FanDiagMixin but NOT TempMonitorMixin or PowerSeqMixin

    #[test]
    fn fan_health_check_passes_above_threshold() {
        let rig = FanTestRig {
            i2c:  MockI2c  { rpm: Cell::new(6000) },
            gpio: MockGpio { level: Cell::new(false) },
        };
        assert!(rig.check_fan_health(0, 4000).unwrap());
    }

    #[test]
    fn fan_health_check_fails_below_threshold() {
        let rig = FanTestRig {
            i2c:  MockI2c  { rpm: Cell::new(2000) },
            gpio: MockGpio { level: Cell::new(false) },
        };
        assert!(!rig.check_fan_health(0, 4000).unwrap());
    }
}
}

Notice that FanTestRig only implements HasI2c + HasGpio — it gets FanDiagMixin automatically, but the compiler refuses rig.read_thermocouple() because HasSpi is not satisfied. This is mixin scoping enforced at compile time.

Conditional Methods — Beyond What Ruby Can Do

Add where bounds to individual default methods. The method only exists when the associated type satisfies the extra bound:

#![allow(unused)]
fn main() {
/// Marker trait for DMA-capable SPI controllers
pub trait DmaCapable: SpiBus {
    fn dma_transfer(&self, tx: &[u8], rx: &mut [u8]) -> io::Result<()>;
}

/// Marker trait for interrupt-capable GPIO pins
pub trait InterruptCapable: GpioPin {
    fn wait_for_edge(&self, timeout_ms: u32) -> io::Result<bool>;
}

pub trait AdvancedDiagMixin: HasSpi + HasGpio {
    // Always available
    fn basic_probe(&self) -> io::Result<bool> {
        let mut rx = [0u8; 1];
        self.spi().spi_transfer(&[0xFF], &mut rx)?;
        Ok(rx[0] != 0x00)
    }

    // Only exists when the SPI controller supports DMA
    fn bulk_sensor_read(&self, buf: &mut [u8]) -> io::Result<()>
    where
        Self::Spi: DmaCapable,
    {
        self.spi().dma_transfer(&vec![0x00; buf.len()], buf)
    }

    // Only exists when the GPIO pin supports interrupts
    fn wait_for_fault_signal(&self, timeout_ms: u32) -> io::Result<bool>
    where
        Self::Gpio: InterruptCapable,
    {
        self.gpio().wait_for_edge(timeout_ms)
    }
}

impl<T: HasSpi + HasGpio> AdvancedDiagMixin for T {}
}

If your platform’s SPI doesn’t support DMA, calling bulk_sensor_read() is a compile error, not a runtime crash. Ruby’s respond_to? check is the closest equivalent — but it happens at deploy time, not compile time.

Composability: Stacking Mixins

Multiple mixins can share the same ingredient — no diamond problem:

┌─────────────┐    ┌───────────┐    ┌──────────────┐
│ FanDiagMixin│    │TempMonitor│    │ PowerSeqMixin│
│  (I2C+GPIO) │    │ (SPI+IPMI)│    │  (GPIO+IPMI) │
└──────┬──────┘    └─────┬─────┘    └──────┬───────┘
       │                 │                 │
       │   ┌─────────────┴─────────────┐   │
       └──►│      DiagPlatform         │◄──┘
           │ HasSpi+HasI2c+HasGpio     │
           │        +HasIpmi           │
           └───────────────────────────┘

DiagPlatform implements HasGpio once, and both FanDiagMixin and PowerSeqMixin use the same self.gpio(). In Ruby, this would be two modules both calling self.gpio_pin — but if they expected different pin numbers, you’d discover the conflict at runtime. In Rust, you can disambiguate at the type level.

Comparison: Ruby Mixins vs Rust Capability Mixins

DimensionRuby MixinsRust Capability Mixins
DispatchRuntime (method table lookup)Compile-time (monomorphised)
Safe compositionMRO linearisation hides conflictsCompiler rejects ambiguity
Conditional methodsrespond_to? at runtimewhere bounds at compile time
OverheadMethod dispatch + GCZero-cost (inlined)
TestabilityStub/mock via metaprogrammingGeneric over mock types
Adding new busesinclude at runtimeAdd ingredient trait, recompile
Runtime flexibilityextend, prepend, open classesNone (fully static)

When to Use Capability Mixins

ScenarioUse Mixins?
Multiple diagnostics share bus-reading logic✅
Test harness needs different bus subsets✅ (partial ingredient structs)
Methods only valid for certain bus capabilities (DMA, IRQ)✅ (conditional where bounds)
You need runtime module loading (plugins)❌ (use dyn Trait or enum dispatch)
Single struct with one bus — no sharing needed❌ (keep it simple)
Cross-crate ingredients with coherence issues⚠️ (use newtype wrappers)

Key Takeaways — Capability Mixins

  1. Ingredient trait = associated type + accessor method (e.g., HasSpi)
  2. Mixin trait = supertrait bounds on ingredients + default method bodies
  3. Blanket impl = impl<T: HasX + HasY> Mixin for T {} — auto-injects methods
  4. Conditional methods = where Self::Spi: DmaCapable on individual defaults
  5. Partial platforms = test structs that only impl the needed ingredients
  6. No runtime cost — the compiler generates specialised code for each platform type

Typed Commands — GADT-Style Return Type Safety

In Haskell, Generalised Algebraic Data Types (GADTs) let each constructor of a data type refine the type parameter — so Expr Int and Expr Bool are enforced by the type checker. Rust has no direct GADT syntax, but traits with associated types achieve the same guarantee: the command type determines the response type, and mixing them up is a compile error.

This pattern is particularly powerful for hardware diagnostics, where IPMI commands, register reads, and sensor queries each return different physical quantities that should never be confused.

The Problem: The Untyped Vec<u8> Swamp

Most C/C++ IPMI stacks — and naïve Rust ports — use raw bytes everywhere:

#![allow(unused)]
fn main() {
use std::io;

struct BmcConnectionUntyped { timeout_secs: u32 }

impl BmcConnectionUntyped {
    fn raw_command(&self, net_fn: u8, cmd: u8, data: &[u8]) -> io::Result<Vec<u8>> {
        // ... shells out to ipmitool ...
        Ok(vec![0x00, 0x19, 0x00]) // stub
    }
}

fn diagnose_thermal_untyped(bmc: &BmcConnectionUntyped) -> io::Result<()> {
    // Read CPU temperature — sensor ID 0x20
    let raw = bmc.raw_command(0x04, 0x2D, &[0x20])?;
    let cpu_temp = raw[0] as f64;  // 🤞 hope byte 0 is the reading

    // Read fan speed — sensor ID 0x30
    let raw = bmc.raw_command(0x04, 0x2D, &[0x30])?;
    let fan_rpm = raw[0] as u32;  // 🐛 BUG: fan speed is 2 bytes LE

    // Read inlet voltage — sensor ID 0x40
    let raw = bmc.raw_command(0x04, 0x2D, &[0x40])?;
    let voltage = raw[0] as f64;  // 🐛 BUG: need to divide by 1000

    // 🐛 Comparing °C to RPM — compiles, but nonsensical
    if cpu_temp > fan_rpm as f64 {
        println!("uh oh");
    }

    // 🐛 Passing Volts as temperature — compiles fine
    log_temp_untyped(voltage);
    log_volts_untyped(cpu_temp);

    Ok(())
}

fn log_temp_untyped(t: f64)  { println!("Temp: {t}°C"); }
fn log_volts_untyped(v: f64) { println!("Voltage: {v}V"); }
}

Every reading is f64 — the compiler has no idea that one is a temperature, another is RPM, another is voltage. Four distinct bugs compile without warning:

#BugConsequenceDiscovered
1Fan RPM parsed as 1 byte instead of 2Reads 25 RPM instead of 6400Production, 3 AM fan-failure flood
2Voltage not divided by 100012000V instead of 12.0VThreshold check flags every PSU
3Comparing °C to RPMMeaningless booleanPossibly never
4Voltage passed to log_temp_untyped()Silent data corruption in logs6 months later, reading history

The Solution: Typed Commands via Associated Types

Step 1 — Domain newtypes

#![allow(unused)]
fn main() {
#[derive(Debug, Clone, Copy, PartialEq, PartialOrd)]
struct Celsius(f64);

#[derive(Debug, Clone, Copy, PartialEq, PartialOrd)]
struct Rpm(u32);

#[derive(Debug, Clone, Copy, PartialEq, PartialOrd)]
struct Volts(f64);

#[derive(Debug, Clone, Copy, PartialEq, PartialOrd)]
struct Watts(f64);
}

Step 2 — The command trait (the GADT equivalent)

The associated type Response is the key — it binds each command to its return type:

#![allow(unused)]
fn main() {
trait IpmiCmd {
    /// The GADT "index" — determines what execute() returns.
    type Response;

    fn net_fn(&self) -> u8;
    fn cmd_byte(&self) -> u8;
    fn payload(&self) -> Vec<u8>;

    /// Parsing is encapsulated HERE — each command knows its own byte layout.
    fn parse_response(&self, raw: &[u8]) -> io::Result<Self::Response>;
}
}

Step 3 — One struct per command, parsing written once

#![allow(unused)]
fn main() {
struct ReadTemp { sensor_id: u8 }
impl IpmiCmd for ReadTemp {
    type Response = Celsius;  // ← "this command returns a temperature"
    fn net_fn(&self) -> u8 { 0x04 }
    fn cmd_byte(&self) -> u8 { 0x2D }
    fn payload(&self) -> Vec<u8> { vec![self.sensor_id] }
    fn parse_response(&self, raw: &[u8]) -> io::Result<Celsius> {
        // Signed byte per IPMI SDR — written once, tested once
        Ok(Celsius(raw[0] as i8 as f64))
    }
}

struct ReadFanSpeed { fan_id: u8 }
impl IpmiCmd for ReadFanSpeed {
    type Response = Rpm;     // ← "this command returns RPM"
    fn net_fn(&self) -> u8 { 0x04 }
    fn cmd_byte(&self) -> u8 { 0x2D }
    fn payload(&self) -> Vec<u8> { vec![self.fan_id] }
    fn parse_response(&self, raw: &[u8]) -> io::Result<Rpm> {
        // 2-byte LE — the correct layout, encoded once
        Ok(Rpm(u16::from_le_bytes([raw[0], raw[1]]) as u32))
    }
}

struct ReadVoltage { rail: u8 }
impl IpmiCmd for ReadVoltage {
    type Response = Volts;   // ← "this command returns voltage"
    fn net_fn(&self) -> u8 { 0x04 }
    fn cmd_byte(&self) -> u8 { 0x2D }
    fn payload(&self) -> Vec<u8> { vec![self.rail] }
    fn parse_response(&self, raw: &[u8]) -> io::Result<Volts> {
        // Millivolts → Volts, always correct
        Ok(Volts(u16::from_le_bytes([raw[0], raw[1]]) as f64 / 1000.0))
    }
}

struct ReadFru { fru_id: u8 }
impl IpmiCmd for ReadFru {
    type Response = String;
    fn net_fn(&self) -> u8 { 0x0A }
    fn cmd_byte(&self) -> u8 { 0x11 }
    fn payload(&self) -> Vec<u8> { vec![self.fru_id, 0x00, 0x00, 0xFF] }
    fn parse_response(&self, raw: &[u8]) -> io::Result<String> {
        Ok(String::from_utf8_lossy(raw).to_string())
    }
}
}

Step 4 — The executor (zero dyn, monomorphised)

#![allow(unused)]
fn main() {
struct BmcConnection { timeout_secs: u32 }

impl BmcConnection {
    /// Generic over any command — compiler generates one version per command type.
    fn execute<C: IpmiCmd>(&self, cmd: &C) -> io::Result<C::Response> {
        let raw = self.raw_send(cmd.net_fn(), cmd.cmd_byte(), &cmd.payload())?;
        cmd.parse_response(&raw)
    }

    fn raw_send(&self, _nf: u8, _cmd: u8, _data: &[u8]) -> io::Result<Vec<u8>> {
        Ok(vec![0x19, 0x00]) // stub — real impl calls ipmitool
    }
}
}

Step 5 — Caller code: all four bugs become compile errors

#![allow(unused)]
fn main() {
fn diagnose_thermal(bmc: &BmcConnection) -> io::Result<()> {
    let cpu_temp: Celsius = bmc.execute(&ReadTemp { sensor_id: 0x20 })?;
    let fan_rpm:  Rpm     = bmc.execute(&ReadFanSpeed { fan_id: 0x30 })?;
    let voltage:  Volts   = bmc.execute(&ReadVoltage { rail: 0x40 })?;

    // Bug #1 — IMPOSSIBLE: parsing lives in ReadFanSpeed::parse_response
    // Bug #2 — IMPOSSIBLE: scaling lives in ReadVoltage::parse_response

    // Bug #3 — COMPILE ERROR:
    // if cpu_temp > fan_rpm { }
    //    ^^^^^^^^   ^^^^^^^
    //    Celsius    Rpm      → "mismatched types" ❌

    // Bug #4 — COMPILE ERROR:
    // log_temperature(voltage);
    //                 ^^^^^^^  Volts, expected Celsius ❌

    // Only correct comparisons compile:
    if cpu_temp > Celsius(85.0) {
        println!("CPU overheating: {:?}", cpu_temp);
    }
    if fan_rpm < Rpm(4000) {
        println!("Fan too slow: {:?}", fan_rpm);
    }

    Ok(())
}

fn log_temperature(t: Celsius) { println!("Temp: {:?}", t); }
fn log_voltage(v: Volts)       { println!("Voltage: {:?}", v); }
}

Macro DSL for Diagnostic Scripts

For large diagnostic routines that run many commands in sequence, a macro gives concise declarative syntax while preserving full type safety:

#![allow(unused)]
fn main() {
/// Execute a series of typed IPMI commands, returning a tuple of results.
/// Each element of the tuple has the command's own Response type.
macro_rules! diag_script {
    ($bmc:expr; $($cmd:expr),+ $(,)?) => {{
        ( $( $bmc.execute(&$cmd)?, )+ )
    }};
}

fn full_pre_flight(bmc: &BmcConnection) -> io::Result<()> {
    // Expands to: (Celsius, Rpm, Volts, String) — every type tracked
    let (temp, rpm, volts, board_pn) = diag_script!(bmc;
        ReadTemp     { sensor_id: 0x20 },
        ReadFanSpeed { fan_id:    0x30 },
        ReadVoltage  { rail:      0x40 },
        ReadFru      { fru_id:    0x00 },
    );

    println!("Board: {:?}", board_pn);
    println!("CPU: {:?}, Fan: {:?}, 12V: {:?}", temp, rpm, volts);

    // Type-safe threshold checks:
    assert!(temp  < Celsius(95.0), "CPU too hot");
    assert!(rpm   > Rpm(3000),     "Fan too slow");
    assert!(volts > Volts(11.4),   "12V rail sagging");

    Ok(())
}
}

The macro is just syntactic sugar — the tuple type (Celsius, Rpm, Volts, String) is fully inferred by the compiler. Swap two commands and the destructuring breaks at compile time, not at runtime.

Enum Dispatch for Heterogeneous Command Lists

When you need a Vec of mixed commands (e.g., a configurable script loaded from JSON), use enum dispatch to stay dyn-free:

#![allow(unused)]
fn main() {
enum AnyReading {
    Temp(Celsius),
    Rpm(Rpm),
    Volt(Volts),
    Text(String),
}

enum AnyCmd {
    Temp(ReadTemp),
    Fan(ReadFanSpeed),
    Voltage(ReadVoltage),
    Fru(ReadFru),
}

impl AnyCmd {
    fn execute(&self, bmc: &BmcConnection) -> io::Result<AnyReading> {
        match self {
            AnyCmd::Temp(c)    => Ok(AnyReading::Temp(bmc.execute(c)?)),
            AnyCmd::Fan(c)     => Ok(AnyReading::Rpm(bmc.execute(c)?)),
            AnyCmd::Voltage(c) => Ok(AnyReading::Volt(bmc.execute(c)?)),
            AnyCmd::Fru(c)     => Ok(AnyReading::Text(bmc.execute(c)?)),
        }
    }
}

/// Dynamic diagnostic script — commands loaded at runtime
fn run_script(bmc: &BmcConnection, script: &[AnyCmd]) -> io::Result<Vec<AnyReading>> {
    script.iter().map(|cmd| cmd.execute(bmc)).collect()
}
}

You lose per-element type tracking (everything is AnyReading), but you gain runtime flexibility — and the parsing is still encapsulated in each IpmiCmd impl.

Testing Typed Commands

#![allow(unused)]
fn main() {
#[cfg(test)]
mod tests {
    use super::*;

    struct StubBmc {
        responses: std::collections::HashMap<u8, Vec<u8>>,
    }

    impl StubBmc {
        fn execute<C: IpmiCmd>(&self, cmd: &C) -> io::Result<C::Response> {
            let key = cmd.payload()[0]; // sensor ID as key
            let raw = self.responses.get(&key)
                .ok_or_else(|| io::Error::new(io::ErrorKind::NotFound, "no stub"))?;
            cmd.parse_response(raw)
        }
    }

    #[test]
    fn read_temp_parses_signed_byte() {
        let bmc = StubBmc {
            responses: [( 0x20, vec![0xE7] )].into() // -25 as i8 = 0xE7
        };
        let temp = bmc.execute(&ReadTemp { sensor_id: 0x20 }).unwrap();
        assert_eq!(temp, Celsius(-25.0));
    }

    #[test]
    fn read_fan_parses_two_byte_le() {
        let bmc = StubBmc {
            responses: [( 0x30, vec![0x00, 0x19] )].into() // 0x1900 = 6400
        };
        let rpm = bmc.execute(&ReadFanSpeed { fan_id: 0x30 }).unwrap();
        assert_eq!(rpm, Rpm(6400));
    }

    #[test]
    fn read_voltage_scales_millivolts() {
        let bmc = StubBmc {
            responses: [( 0x40, vec![0xE8, 0x2E] )].into() // 0x2EE8 = 12008 mV
        };
        let v = bmc.execute(&ReadVoltage { rail: 0x40 }).unwrap();
        assert!((v.0 - 12.008).abs() < 0.001);
    }
}
}

Each command’s parsing is tested independently. If ReadFanSpeed changes from 2-byte LE to 4-byte BE in a new IPMI spec revision, you update one parse_response and the test catches regressions.

How This Maps to Haskell GADTs

Haskell GADT                         Rust Equivalent
────────────────                     ───────────────────────
data Cmd a where                     trait IpmiCmd {
  ReadTemp :: SensorId -> Cmd Temp       type Response;
  ReadFan  :: FanId    -> Cmd Rpm        ...
                                     }

eval :: Cmd a -> IO a                fn execute<C: IpmiCmd>(&self, cmd: &C)
                                         -> io::Result<C::Response>

Type refinement in case branches     Monomorphisation: compiler generates
                                     execute::<ReadTemp>() → returns Celsius
                                     execute::<ReadFanSpeed>() → returns Rpm

Both guarantee: the command determines the return type. Rust achieves it through generic monomorphisation instead of type-level case analysis — same safety, zero runtime cost.

Before vs After Summary

DimensionUntyped (Vec<u8>)Typed Commands
Lines per sensor~3 (duplicated at every call site)~15 (written and tested once)
Parsing errors possibleAt every call siteIn one parse_response impl
Unit confusion bugsUnlimitedZero (compile error)
Adding a new sensorTouch N files, copy-paste parsingAdd 1 struct + 1 impl
Runtime cost—Identical (monomorphised)
IDE autocompletef64 everywhereCelsius, Rpm, Volts — self-documenting
Code review burdenMust verify every raw byte parseVerify one parse_response per sensor
Macro DSLN/Adiag_script!(bmc; ReadTemp{..}, ReadFan{..}) → (Celsius, Rpm)
Dynamic scriptsManual dispatchAnyCmd enum — still dyn-free

When to Use Typed Commands

ScenarioRecommendation
IPMI sensor reads with distinct physical units✅ Typed commands
Register map with different-width fields✅ Typed commands
Network protocol messages (request → response)✅ Typed commands
Single command type with one return format❌ Overkill — just return the type directly
Prototyping / exploring an unknown device❌ Raw bytes first, type later
Plugin system where commands aren’t known at compile time⚠️ Use AnyCmd enum dispatch

Key Takeaways — Traits

  • Associated types = one impl per type; generic parameters = many impls per type
  • GATs unlock lending iterators and async-in-traits patterns
  • Use enum dispatch for closed sets (fast); dyn Trait for open sets (flexible)
  • Any + TypeId is the escape hatch when compile-time types are unknown

See also: Ch 1 — Generics for monomorphization and when generics cause code bloat. Ch 3 — Newtype & Type-State for using traits with the config trait pattern.


Exercise: Repository with Associated Types ★★★ (~40 min)

Design a Repository trait with associated Error, Id, and Item types. Implement it for an in-memory store and demonstrate compile-time type safety.

🔑 Solution
use std::collections::HashMap;

trait Repository {
    type Item;
    type Id;
    type Error;

    fn get(&self, id: &Self::Id) -> Result<Option<&Self::Item>, Self::Error>;
    fn insert(&mut self, item: Self::Item) -> Result<Self::Id, Self::Error>;
    fn delete(&mut self, id: &Self::Id) -> Result<bool, Self::Error>;
}

#[derive(Debug, Clone)]
struct User {
    name: String,
    email: String,
}

struct InMemoryUserRepo {
    data: HashMap<u64, User>,
    next_id: u64,
}

impl InMemoryUserRepo {
    fn new() -> Self {
        InMemoryUserRepo { data: HashMap::new(), next_id: 1 }
    }
}

impl Repository for InMemoryUserRepo {
    type Item = User;
    type Id = u64;
    type Error = std::convert::Infallible;

    fn get(&self, id: &u64) -> Result<Option<&User>, Self::Error> {
        Ok(self.data.get(id))
    }

    fn insert(&mut self, item: User) -> Result<u64, Self::Error> {
        let id = self.next_id;
        self.next_id += 1;
        self.data.insert(id, item);
        Ok(id)
    }

    fn delete(&mut self, id: &u64) -> Result<bool, Self::Error> {
        Ok(self.data.remove(id).is_some())
    }
}

fn create_and_fetch<R: Repository>(repo: &mut R, item: R::Item) -> Result<(), R::Error>
where
    R::Item: std::fmt::Debug,
    R::Id: std::fmt::Debug,
{
    let id = repo.insert(item)?;
    println!("Inserted with id: {id:?}");
    let retrieved = repo.get(&id)?;
    println!("Retrieved: {retrieved:?}");
    Ok(())
}

fn main() {
    let mut repo = InMemoryUserRepo::new();
    create_and_fetch(&mut repo, User {
        name: "Alice".into(),
        email: "[email protected]".into(),
    }).unwrap();
}

3. The Newtype and Type-State Patterns 🟡

What you’ll learn:

  • The newtype pattern for zero-cost compile-time type safety
  • Type-state pattern: making illegal state transitions unrepresentable
  • Builder pattern with type states for compile-time–enforced construction
  • Config trait pattern for taming generic parameter explosion

Newtype: Zero-Cost Type Safety

The newtype pattern wraps an existing type in a single-field tuple struct to create a distinct type with zero runtime overhead:

#![allow(unused)]
fn main() {
// Without newtypes — easy to mix up:
fn create_user(name: String, email: String, age: u32, employee_id: u32) { }
// create_user(name, email, age, id);  — but what if we swap age and id?
// create_user(name, email, id, age);  — COMPILES FINE, BUG

// With newtypes — the compiler catches mistakes:
struct UserName(String);
struct Email(String);
struct Age(u32);
struct EmployeeId(u32);

fn create_user(name: UserName, email: Email, age: Age, id: EmployeeId) { }
// create_user(name, email, EmployeeId(42), Age(30));
// ❌ Compile error: expected Age, got EmployeeId
}

impl Deref for Newtypes — Power and Pitfalls

Implementing Deref on a newtype lets it auto-coerce to the inner type’s reference, giving you all of the inner type’s methods “for free”:

#![allow(unused)]
fn main() {
use std::ops::Deref;

struct Email(String);

impl Email {
    fn new(raw: &str) -> Result<Self, &'static str> {
        if raw.contains('@') {
            Ok(Email(raw.to_string()))
        } else {
            Err("invalid email: missing @")
        }
    }
}

impl Deref for Email {
    type Target = str;
    fn deref(&self) -> &str { &self.0 }
}

// Now Email auto-derefs to &str:
let email = Email::new("[email protected]").unwrap();
println!("Length: {}", email.len()); // Uses str::len via Deref
}

This is convenient — but it effectively punches a hole through your newtype’s abstraction boundary because every method on the target type becomes callable on your wrapper.

When Deref IS appropriate

ScenarioExampleWhy it’s fine
Smart-pointer wrappersBox<T>, Arc<T>, MutexGuard<T>The wrapper’s whole purpose is to behave like T
Transparent “thin” wrappersString → str, PathBuf → Path, Vec<T> → [T]The wrapper IS-A superset of the target
Your newtype genuinely IS the inner typestruct Hostname(String) where you always want full string opsRestricting the API would add no value

When Deref is an anti-pattern

ScenarioProblem
Domain types with invariantsEmail derefs to &str, so callers can call .split_at(), .trim(), etc. — none of which preserve the “must contain @” invariant. If someone stores the trimmed &str and reconstructs, the invariant is lost.
Types where you want a restricted APIstruct Password(String) with Deref<Target = str> leaks .as_bytes(), .chars(), Debug output — exactly what you’re trying to hide.
Fake inheritanceUsing Deref to make ManagerWidget auto-deref to Widget simulates OOP inheritance. This is explicitly discouraged — see the Rust API Guidelines (C-DEREF).

Rule of thumb: If your newtype exists to add type safety or restrict the API, don’t implement Deref. If it exists to add capabilities while keeping the inner type’s full surface (like a smart pointer), Deref is the right choice.

DerefMut — doubles the risk

If you also implement DerefMut, callers can mutate the inner value directly, bypassing any validation in your constructors:

#![allow(unused)]
fn main() {
use std::ops::{Deref, DerefMut};

struct PortNumber(u16);

impl Deref for PortNumber {
    type Target = u16;
    fn deref(&self) -> &u16 { &self.0 }
}

impl DerefMut for PortNumber {
    fn deref_mut(&mut self) -> &mut u16 { &mut self.0 }
}

let mut port = PortNumber(443);
*port = 0; // Bypasses any validation — now an invalid port
}

Only implement DerefMut when the inner type has no invariants to protect.

Prefer explicit delegation instead

When you want only some of the inner type’s methods, delegate explicitly:

#![allow(unused)]
fn main() {
struct Email(String);

impl Email {
    fn new(raw: &str) -> Result<Self, &'static str> {
        if raw.contains('@') { Ok(Email(raw.to_string())) }
        else { Err("missing @") }
    }

    // Expose only what makes sense:
    pub fn as_str(&self) -> &str { &self.0 }
    pub fn len(&self) -> usize { self.0.len() }
    pub fn domain(&self) -> &str {
        self.0.split('@').nth(1).unwrap_or("")
    }
    // .split_at(), .trim(), .replace() — NOT exposed
}
}

Clippy and the ecosystem

  • clippy::wrong_self_convention can fire when Deref coercion makes method resolution surprising (e.g., is_empty() resolving to the inner type’s version instead of one you intended to shadow).
  • The Rust API Guidelines (C-DEREF) state: “only smart pointers should implement Deref.” Treat this as a strong default; deviate only with clear justification.
  • If you need trait compatibility (e.g., passing Email to functions expecting &str), consider implementing AsRef<str> and Borrow<str> instead — they’re explicit conversions without auto-coercion surprises.

Decision matrix

Do you want ALL methods of the inner type to be callable?
  ├─ YES → Does your type enforce invariants or restrict the API?
  │    ├─ NO  → impl Deref ✅  (smart-pointer / transparent wrapper)
  │    └─ YES → Don't impl Deref ❌ (invariant leaks)
  └─ NO  → Don't impl Deref ❌  (use AsRef / explicit delegation)

Type-State: Compile-Time Protocol Enforcement

The type-state pattern uses the type system to enforce that operations happen in the correct order. Invalid states become unrepresentable.

stateDiagram-v2
    [*] --> Disconnected: new()
    Disconnected --> Connected: connect()
    Connected --> Authenticated: authenticate()
    Authenticated --> Authenticated: request()
    Authenticated --> [*]: drop

    Disconnected --> Disconnected: ❌ request() won't compile
    Connected --> Connected: ❌ request() won't compile

Each transition consumes self and returns a new type — the compiler enforces valid ordering.

// Problem: A network connection that must be:
// 1. Created
// 2. Connected
// 3. Authenticated
// 4. Then used for requests
// Calling request() before authenticate() should be a COMPILE error.

// --- Type-state markers (zero-sized types) ---
struct Disconnected;
struct Connected;
struct Authenticated;

// --- Connection parameterized by state ---
struct Connection<State> {
    address: String,
    _state: std::marker::PhantomData<State>,
}

// Only Disconnected connections can connect:
impl Connection<Disconnected> {
    fn new(address: &str) -> Self {
        Connection {
            address: address.to_string(),
            _state: std::marker::PhantomData,
        }
    }

    fn connect(self) -> Connection<Connected> {
        println!("Connecting to {}...", self.address);
        Connection {
            address: self.address,
            _state: std::marker::PhantomData,
        }
    }
}

// Only Connected connections can authenticate:
impl Connection<Connected> {
    fn authenticate(self, _token: &str) -> Connection<Authenticated> {
        println!("Authenticating...");
        Connection {
            address: self.address,
            _state: std::marker::PhantomData,
        }
    }
}

// Only Authenticated connections can make requests:
impl Connection<Authenticated> {
    fn request(&self, path: &str) -> String {
        format!("GET {} from {}", path, self.address)
    }
}

fn main() {
    let conn = Connection::new("api.example.com");
    // conn.request("/data"); // ❌ Compile error: no method `request` on Connection<Disconnected>

    let conn = conn.connect();
    // conn.request("/data"); // ❌ Compile error: no method `request` on Connection<Connected>

    let conn = conn.authenticate("secret-token");
    let response = conn.request("/data"); // ✅ Only works after authentication
    println!("{response}");
}

Key insight: Each state transition consumes self and returns a new type. You can’t use the old state after transitioning — the compiler enforces it. Zero runtime cost — PhantomData is zero-sized, states are erased at compile time.

Comparison with C++/C#: In C++ or C#, you’d enforce this with runtime checks (if (!authenticated) throw ...). The Rust type-state pattern moves these checks to compile time — invalid states are literally unrepresentable in the type system.

Builder Pattern with Type States

A practical application — a builder that enforces required fields:

use std::marker::PhantomData;

// Marker types for required fields
struct NeedsName;
struct NeedsPort;
struct Ready;

struct ServerConfig<State> {
    name: Option<String>,
    port: Option<u16>,
    max_connections: usize, // Optional, has default
    _state: PhantomData<State>,
}

impl ServerConfig<NeedsName> {
    fn new() -> Self {
        ServerConfig {
            name: None,
            port: None,
            max_connections: 100,
            _state: PhantomData,
        }
    }

    fn name(self, name: &str) -> ServerConfig<NeedsPort> {
        ServerConfig {
            name: Some(name.to_string()),
            port: self.port,
            max_connections: self.max_connections,
            _state: PhantomData,
        }
    }
}

impl ServerConfig<NeedsPort> {
    fn port(self, port: u16) -> ServerConfig<Ready> {
        ServerConfig {
            name: self.name,
            port: Some(port),
            max_connections: self.max_connections,
            _state: PhantomData,
        }
    }
}

impl ServerConfig<Ready> {
    fn max_connections(mut self, n: usize) -> Self {
        self.max_connections = n;
        self
    }

    fn build(self) -> Server {
        Server {
            name: self.name.unwrap(),
            port: self.port.unwrap(),
            max_connections: self.max_connections,
        }
    }
}

struct Server {
    name: String,
    port: u16,
    max_connections: usize,
}

fn main() {
    // Must provide name, then port, then can build:
    let server = ServerConfig::new()
        .name("my-server")
        .port(8080)
        .max_connections(500)
        .build();

    // ServerConfig::new().port(8080); // ❌ Compile error: no method `port` on NeedsName
    // ServerConfig::new().name("x").build(); // ❌ Compile error: no method `build` on NeedsPort
}

Case Study: Type-Safe Connection Pool

Real-world systems need connection pools where connections move through well-defined states. Here’s how the typestate pattern enforces correctness in a production pool:

stateDiagram-v2
    [*] --> Idle: pool.acquire()
    Idle --> Active: conn.begin_transaction()
    Active --> Active: conn.execute(query)
    Active --> Idle: conn.commit() / conn.rollback()
    Idle --> [*]: pool.release(conn)

    Active --> [*]: ❌ cannot release mid-transaction
use std::marker::PhantomData;

// States
struct Idle;
struct InTransaction;

struct PooledConnection<State> {
    id: u32,
    _state: PhantomData<State>,
}

struct Pool {
    next_id: u32,
}

impl Pool {
    fn new() -> Self { Pool { next_id: 0 } }

    fn acquire(&mut self) -> PooledConnection<Idle> {
        self.next_id += 1;
        println!("[pool] Acquired connection #{}", self.next_id);
        PooledConnection { id: self.next_id, _state: PhantomData }
    }

    // Only idle connections can be released — prevents mid-transaction leaks
    fn release(&self, conn: PooledConnection<Idle>) {
        println!("[pool] Released connection #{}", conn.id);
    }
}

impl PooledConnection<Idle> {
    fn begin_transaction(self) -> PooledConnection<InTransaction> {
        println!("[conn #{}] BEGIN", self.id);
        PooledConnection { id: self.id, _state: PhantomData }
    }
}

impl PooledConnection<InTransaction> {
    fn execute(&self, query: &str) {
        println!("[conn #{}] EXEC: {}", self.id, query);
    }

    fn commit(self) -> PooledConnection<Idle> {
        println!("[conn #{}] COMMIT", self.id);
        PooledConnection { id: self.id, _state: PhantomData }
    }

    fn rollback(self) -> PooledConnection<Idle> {
        println!("[conn #{}] ROLLBACK", self.id);
        PooledConnection { id: self.id, _state: PhantomData }
    }
}

fn main() {
    let mut pool = Pool::new();

    let conn = pool.acquire();
    let conn = conn.begin_transaction();
    conn.execute("INSERT INTO users VALUES ('Alice')");
    conn.execute("INSERT INTO orders VALUES (1, 42)");
    let conn = conn.commit(); // Back to Idle
    pool.release(conn);       // ✅ Only works on Idle connections

    // pool.release(conn_active); // ❌ Compile error: can't release InTransaction
}

Why this matters in production: A connection leaked mid-transaction holds database locks indefinitely. The typestate pattern makes this impossible — you literally cannot return a connection to the pool until the transaction is committed or rolled back.


Config Trait Pattern — Taming Generic Parameter Explosion

The Problem

As a struct takes on more responsibilities, each backed by a trait-constrained generic, the type signature grows unwieldy:

#![allow(unused)]
fn main() {
trait SpiBus   { fn spi_transfer(&self, tx: &[u8], rx: &mut [u8]) -> Result<(), BusError>; }
trait ComPort  { fn com_send(&self, data: &[u8]) -> Result<usize, BusError>; }
trait I3cBus   { fn i3c_read(&self, addr: u8, buf: &mut [u8]) -> Result<(), BusError>; }
trait SmBus    { fn smbus_read_byte(&self, addr: u8, cmd: u8) -> Result<u8, BusError>; }
trait GpioBus  { fn gpio_set(&self, pin: u32, high: bool); }

// ❌ Every new bus trait adds another generic parameter
struct DiagController<S: SpiBus, C: ComPort, I: I3cBus, M: SmBus, G: GpioBus> {
    spi: S,
    com: C,
    i3c: I,
    smbus: M,
    gpio: G,
}
// impl blocks, function signatures, and callers all repeat the full list.
// Adding a 6th bus means editing every mention of DiagController<S, C, I, M, G>.
}

This is often called “generic parameter explosion.” It compounds across impl blocks, function parameters, and downstream consumers — each of which must repeat the full parameter list.

The Solution: A Config Trait

Bundle all associated types into a single trait. The struct then has one generic parameter regardless of how many component types it contains:

#![allow(unused)]
fn main() {
#[derive(Debug)]
enum BusError {
    Timeout,
    NakReceived,
    HardwareFault(String),
}

// --- Bus traits (unchanged) ---
trait SpiBus {
    fn spi_transfer(&self, tx: &[u8], rx: &mut [u8]) -> Result<(), BusError>;
    fn spi_write(&self, data: &[u8]) -> Result<(), BusError>;
}

trait ComPort {
    fn com_send(&self, data: &[u8]) -> Result<usize, BusError>;
    fn com_recv(&self, buf: &mut [u8], timeout_ms: u32) -> Result<usize, BusError>;
}

trait I3cBus {
    fn i3c_read(&self, addr: u8, buf: &mut [u8]) -> Result<(), BusError>;
    fn i3c_write(&self, addr: u8, data: &[u8]) -> Result<(), BusError>;
}

// --- The Config trait: one associated type per component ---
trait BoardConfig {
    type Spi: SpiBus;
    type Com: ComPort;
    type I3c: I3cBus;
}

// --- DiagController has exactly ONE generic parameter ---
struct DiagController<Cfg: BoardConfig> {
    spi: Cfg::Spi,
    com: Cfg::Com,
    i3c: Cfg::I3c,
}
}

DiagController<Cfg> will never gain another generic parameter. Adding a 4th bus means adding one associated type to BoardConfig and one field to DiagController — no downstream signature changes.

Implementing the Controller

#![allow(unused)]
fn main() {
impl<Cfg: BoardConfig> DiagController<Cfg> {
    fn new(spi: Cfg::Spi, com: Cfg::Com, i3c: Cfg::I3c) -> Self {
        DiagController { spi, com, i3c }
    }

    fn read_flash_id(&self) -> Result<u32, BusError> {
        let cmd = [0x9F]; // JEDEC Read ID
        let mut id = [0u8; 4];
        self.spi.spi_transfer(&cmd, &mut id)?;
        Ok(u32::from_be_bytes(id))
    }

    fn send_bmc_command(&self, cmd: &[u8]) -> Result<Vec<u8>, BusError> {
        self.com.com_send(cmd)?;
        let mut resp = vec![0u8; 256];
        let n = self.com.com_recv(&mut resp, 1000)?;
        resp.truncate(n);
        Ok(resp)
    }

    fn read_sensor_temp(&self, sensor_addr: u8) -> Result<i16, BusError> {
        let mut buf = [0u8; 2];
        self.i3c.i3c_read(sensor_addr, &mut buf)?;
        Ok(i16::from_be_bytes(buf))
    }

    fn run_full_diag(&self) -> Result<DiagReport, BusError> {
        let flash_id = self.read_flash_id()?;
        let bmc_resp = self.send_bmc_command(b"VERSION\n")?;
        let cpu_temp = self.read_sensor_temp(0x48)?;
        let gpu_temp = self.read_sensor_temp(0x49)?;

        Ok(DiagReport {
            flash_id,
            bmc_version: String::from_utf8_lossy(&bmc_resp).to_string(),
            cpu_temp_c: cpu_temp,
            gpu_temp_c: gpu_temp,
        })
    }
}

#[derive(Debug)]
struct DiagReport {
    flash_id: u32,
    bmc_version: String,
    cpu_temp_c: i16,
    gpu_temp_c: i16,
}
}

Production Wiring

One impl BoardConfig selects the concrete hardware drivers:

struct PlatformSpi  { dev: String, speed_hz: u32 }
struct UartCom      { dev: String, baud: u32 }
struct LinuxI3c     { dev: String }

impl SpiBus for PlatformSpi {
    fn spi_transfer(&self, tx: &[u8], rx: &mut [u8]) -> Result<(), BusError> {
        // ioctl(SPI_IOC_MESSAGE) in production
        rx[0..4].copy_from_slice(&[0xEF, 0x40, 0x18, 0x00]);
        Ok(())
    }
    fn spi_write(&self, _data: &[u8]) -> Result<(), BusError> { Ok(()) }
}

impl ComPort for UartCom {
    fn com_send(&self, _data: &[u8]) -> Result<usize, BusError> { Ok(0) }
    fn com_recv(&self, buf: &mut [u8], _timeout: u32) -> Result<usize, BusError> {
        let resp = b"BMC v2.4.1\n";
        buf[..resp.len()].copy_from_slice(resp);
        Ok(resp.len())
    }
}

impl I3cBus for LinuxI3c {
    fn i3c_read(&self, _addr: u8, buf: &mut [u8]) -> Result<(), BusError> {
        buf[0] = 0x00; buf[1] = 0x2D; // 45°C
        Ok(())
    }
    fn i3c_write(&self, _addr: u8, _data: &[u8]) -> Result<(), BusError> { Ok(()) }
}

// ✅ One struct, one impl — all concrete types resolved here
struct ProductionBoard;
impl BoardConfig for ProductionBoard {
    type Spi = PlatformSpi;
    type Com = UartCom;
    type I3c = LinuxI3c;
}

fn main() {
    let ctrl = DiagController::<ProductionBoard>::new(
        PlatformSpi { dev: "/dev/spidev0.0".into(), speed_hz: 10_000_000 },
        UartCom     { dev: "/dev/ttyS0".into(),     baud: 115200 },
        LinuxI3c    { dev: "/dev/i3c-0".into() },
    );
    let report = ctrl.run_full_diag().unwrap();
    println!("{report:#?}");
}

Test Wiring with Mocks

Swap the entire hardware layer by defining a different BoardConfig:

#![allow(unused)]
fn main() {
struct MockSpi  { flash_id: [u8; 4] }
struct MockCom  { response: Vec<u8> }
struct MockI3c  { temps: std::collections::HashMap<u8, i16> }

impl SpiBus for MockSpi {
    fn spi_transfer(&self, _tx: &[u8], rx: &mut [u8]) -> Result<(), BusError> {
        rx[..4].copy_from_slice(&self.flash_id);
        Ok(())
    }
    fn spi_write(&self, _data: &[u8]) -> Result<(), BusError> { Ok(()) }
}

impl ComPort for MockCom {
    fn com_send(&self, _data: &[u8]) -> Result<usize, BusError> { Ok(0) }
    fn com_recv(&self, buf: &mut [u8], _timeout: u32) -> Result<usize, BusError> {
        let n = self.response.len().min(buf.len());
        buf[..n].copy_from_slice(&self.response[..n]);
        Ok(n)
    }
}

impl I3cBus for MockI3c {
    fn i3c_read(&self, addr: u8, buf: &mut [u8]) -> Result<(), BusError> {
        let temp = self.temps.get(&addr).copied().unwrap_or(0);
        buf[..2].copy_from_slice(&temp.to_be_bytes());
        Ok(())
    }
    fn i3c_write(&self, _addr: u8, _data: &[u8]) -> Result<(), BusError> { Ok(()) }
}

struct TestBoard;
impl BoardConfig for TestBoard {
    type Spi = MockSpi;
    type Com = MockCom;
    type I3c = MockI3c;
}

#[cfg(test)]
mod tests {
    use super::*;

    fn make_test_controller() -> DiagController<TestBoard> {
        let mut temps = std::collections::HashMap::new();
        temps.insert(0x48, 45i16);
        temps.insert(0x49, 72i16);

        DiagController::<TestBoard>::new(
            MockSpi  { flash_id: [0xEF, 0x40, 0x18, 0x00] },
            MockCom  { response: b"BMC v2.4.1\n".to_vec() },
            MockI3c  { temps },
        )
    }

    #[test]
    fn test_flash_id() {
        let ctrl = make_test_controller();
        assert_eq!(ctrl.read_flash_id().unwrap(), 0xEF401800);
    }

    #[test]
    fn test_sensor_temps() {
        let ctrl = make_test_controller();
        assert_eq!(ctrl.read_sensor_temp(0x48).unwrap(), 45);
        assert_eq!(ctrl.read_sensor_temp(0x49).unwrap(), 72);
    }

    #[test]
    fn test_full_diag() {
        let ctrl = make_test_controller();
        let report = ctrl.run_full_diag().unwrap();
        assert_eq!(report.flash_id, 0xEF401800);
        assert_eq!(report.cpu_temp_c, 45);
        assert_eq!(report.gpu_temp_c, 72);
        assert!(report.bmc_version.contains("2.4.1"));
    }
}
}

Adding a New Bus Later

When you need a 4th bus, only two things change — BoardConfig and DiagController. No downstream signature changes. The generic parameter count stays at one:

#![allow(unused)]
fn main() {
trait SmBus {
    fn smbus_read_byte(&self, addr: u8, cmd: u8) -> Result<u8, BusError>;
}

// 1. Add one associated type:
trait BoardConfig {
    type Spi: SpiBus;
    type Com: ComPort;
    type I3c: I3cBus;
    type Smb: SmBus;     // ← new
}

// 2. Add one field:
struct DiagController<Cfg: BoardConfig> {
    spi: Cfg::Spi,
    com: Cfg::Com,
    i3c: Cfg::I3c,
    smb: Cfg::Smb,       // ← new
}

// 3. Provide the concrete type in each config impl:
impl BoardConfig for ProductionBoard {
    type Spi = PlatformSpi;
    type Com = UartCom;
    type I3c = LinuxI3c;
    type Smb = LinuxSmbus; // ← new
}
}

When to Use This Pattern

SituationUse Config Trait?Alternative
3+ trait-constrained generics on a struct✅ Yes—
Need to swap entire hardware/platform layer✅ Yes—
Only 1-2 generics❌ OverkillDirect generics
Need runtime polymorphism❌dyn Trait objects
Open-ended plugin system❌Type-map / Any
Component traits form a natural group (board, platform)✅ Yes—

Key Properties

  • One generic parameter forever — DiagController<Cfg> never gains more <A, B, C, ...>
  • Fully static dispatch — no vtables, no dyn, no heap allocation for trait objects
  • Clean test swapping — define TestBoard with mock impls, zero conditional compilation
  • Compile-time safety — forget an associated type → compile error, not runtime crash
  • Battle-tested — this is the pattern used by Substrate/Polkadot’s frame system to manage 20+ associated types through a single Config trait

Key Takeaways — Newtype & Type-State

  • Newtypes give compile-time type safety at zero runtime cost
  • Type-state makes illegal state transitions a compile error, not a runtime bug
  • Config traits tame generic parameter explosion in large systems

See also: Ch 4 — PhantomData for the zero-sized markers that power type-state. Ch 2 — Traits In Depth for associated types used in the config trait pattern.


Case Study: Dual-Axis Typestate — Vendor × Protocol State

The patterns above handle one axis at a time: typestate enforces protocol order, and trait abstraction handles multiple vendors. Real systems often need both simultaneously: a wrapper Handle<Vendor, State> where available methods depend on which vendor is plugged in and which state the handle is in.

This section shows the dual-axis conditional impl pattern — where impl blocks are gated on both a vendor trait bound and a state marker trait.

The Two-Dimensional Problem

Consider a debug probe interface (JTAG/SWD). Multiple vendors make probes, and every probe must be unlocked before registers become accessible. Some vendors additionally support direct memory reads — but only after an extended unlock that configures the memory access port:

graph LR
    subgraph "All vendors"
        L["🔒 Locked"] -- "unlock()" --> U["🔓 Unlocked"]
    end
    subgraph "Memory-capable vendors only"
        U -- "extended_unlock()" --> E["🔓🧠 ExtendedUnlocked"]
    end

    U -. "read_reg() / write_reg()" .-> U
    E -. "read_reg() / write_reg()" .-> E
    E -. "read_memory() / write_memory()" .-> E

    style L fill:#fee,stroke:#c33
    style U fill:#efe,stroke:#3a3
    style E fill:#eef,stroke:#33c

The capability matrix — which methods exist for which (vendor, state) combination — is two-dimensional:

block-beta
    columns 4
    space header1["Locked"] header2["Unlocked"] header3["ExtendedUnlocked"]
    basic["Basic Vendor"]:1 b1["unlock()"] b2["read_reg()\nwrite_reg()"] b3["— unreachable —"]
    memory["Memory Vendor"]:1 m1["unlock()"] m2["read_reg()\nwrite_reg()\nextended_unlock()"] m3["read_reg()\nwrite_reg()\nread_memory()\nwrite_memory()"]

    style b1 fill:#ffd,stroke:#aa0
    style b2 fill:#efe,stroke:#3a3
    style b3 fill:#eee,stroke:#999,stroke-dasharray: 5 5
    style m1 fill:#ffd,stroke:#aa0
    style m2 fill:#efe,stroke:#3a3
    style m3 fill:#eef,stroke:#33c

The challenge: express this matrix entirely at compile time, with static dispatch, so that calling extended_unlock() on a basic probe or read_memory() on an unlocked-but-not-extended handle is a compile error.

The Solution: Jtag<V, S> with Marker Traits

Step 1 — State tokens and capability markers:

use std::marker::PhantomData;

// Zero-sized state tokens — no runtime cost
struct Locked;
struct Unlocked;
struct ExtendedUnlocked;

// Marker traits express which capabilities each state has
trait HasRegAccess {}
impl HasRegAccess for Unlocked {}
impl HasRegAccess for ExtendedUnlocked {}

trait HasMemAccess {}
impl HasMemAccess for ExtendedUnlocked {}

Why marker traits, not just concrete states? Writing impl<V, S: HasRegAccess> Jtag<V, S> means read_reg() works in any state with register access — today that’s Unlocked and ExtendedUnlocked, but if you add DebugHalted tomorrow, you just add one line: impl HasRegAccess for DebugHalted {}. Every register function works with it automatically — zero code changes.

Step 2 — Vendor traits (raw operations):

// Every probe vendor implements these
trait JtagVendor {
    fn raw_unlock(&mut self);
    fn raw_read_reg(&self, addr: u32) -> u32;
    fn raw_write_reg(&mut self, addr: u32, val: u32);
}

// Vendors with memory access also implement this super-trait
trait JtagMemoryVendor: JtagVendor {
    fn raw_extended_unlock(&mut self);
    fn raw_read_memory(&self, addr: u64, buf: &mut [u8]);
    fn raw_write_memory(&mut self, addr: u64, data: &[u8]);
}

Step 3 — The wrapper with conditional impl blocks:

struct Jtag<V, S = Locked> {
    vendor: V,
    _state: PhantomData<S>,
}

// Construction — always starts Locked
impl<V: JtagVendor> Jtag<V, Locked> {
    fn new(vendor: V) -> Self {
        Jtag { vendor, _state: PhantomData }
    }

    fn unlock(mut self) -> Jtag<V, Unlocked> {
        self.vendor.raw_unlock();
        Jtag { vendor: self.vendor, _state: PhantomData }
    }
}

// Register I/O — any vendor, any state with HasRegAccess
impl<V: JtagVendor, S: HasRegAccess> Jtag<V, S> {
    fn read_reg(&self, addr: u32) -> u32 {
        self.vendor.raw_read_reg(addr)
    }
    fn write_reg(&mut self, addr: u32, val: u32) {
        self.vendor.raw_write_reg(addr, val);
    }
}

// Extended unlock — only memory-capable vendors, only from Unlocked
impl<V: JtagMemoryVendor> Jtag<V, Unlocked> {
    fn extended_unlock(mut self) -> Jtag<V, ExtendedUnlocked> {
        self.vendor.raw_extended_unlock();
        Jtag { vendor: self.vendor, _state: PhantomData }
    }
}

// Memory I/O — only memory-capable vendors, only ExtendedUnlocked
impl<V: JtagMemoryVendor, S: HasMemAccess> Jtag<V, S> {
    fn read_memory(&self, addr: u64, buf: &mut [u8]) {
        self.vendor.raw_read_memory(addr, buf);
    }
    fn write_memory(&mut self, addr: u64, data: &[u8]) {
        self.vendor.raw_write_memory(addr, data);
    }
}

Each impl block encodes one cell (or row) of the capability matrix. The compiler enforces the matrix — no runtime checks anywhere.

Vendor Implementations

Adding a vendor means implementing raw methods on one struct — no per-state struct duplication, no delegation boilerplate:

// Vendor A: basic probe — register access only
struct BasicProbe { port: u16 }

impl JtagVendor for BasicProbe {
    fn raw_unlock(&mut self)                    { /* TAP reset sequence */ }
    fn raw_read_reg(&self, addr: u32) -> u32    { /* DR scan */  0 }
    fn raw_write_reg(&mut self, addr: u32, val: u32) { /* DR scan */ }
}
// BasicProbe does NOT impl JtagMemoryVendor.
// extended_unlock() will not compile on Jtag<BasicProbe, _>.

// Vendor B: full-featured probe — registers + memory
struct DapProbe { serial: String }

impl JtagVendor for DapProbe {
    fn raw_unlock(&mut self)                    { /* SWD switch, read DPIDR */ }
    fn raw_read_reg(&self, addr: u32) -> u32    { /* AP register read */ 0 }
    fn raw_write_reg(&mut self, addr: u32, val: u32) { /* AP register write */ }
}

impl JtagMemoryVendor for DapProbe {
    fn raw_extended_unlock(&mut self)           { /* select MEM-AP, power up */ }
    fn raw_read_memory(&self, addr: u64, buf: &mut [u8])  { /* MEM-AP read */ }
    fn raw_write_memory(&mut self, addr: u64, data: &[u8]) { /* MEM-AP write */ }
}

What the Compiler Prevents

AttemptErrorWhy
Jtag<_, Locked>::read_reg()no method read_regLocked doesn’t impl HasRegAccess
Jtag<BasicProbe, _>::extended_unlock()no method extended_unlockBasicProbe doesn’t impl JtagMemoryVendor
Jtag<_, Unlocked>::read_memory()no method read_memoryUnlocked doesn’t impl HasMemAccess
Calling unlock() twicevalue used after moveunlock() consumes self

All four errors are caught at compile time. No panics, no Option, no runtime state enum.

Writing Generic Functions

Functions bind only the axes they care about:

/// Works with ANY vendor, ANY state that grants register access.
fn read_idcode<V: JtagVendor, S: HasRegAccess>(jtag: &Jtag<V, S>) -> u32 {
    jtag.read_reg(0x00)
}

/// Only compiles for memory-capable vendors in ExtendedUnlocked state.
fn dump_firmware<V: JtagMemoryVendor, S: HasMemAccess>(jtag: &Jtag<V, S>) {
    let mut buf = [0u8; 256];
    jtag.read_memory(0x0800_0000, &mut buf);
}

read_idcode doesn’t care whether you’re in Unlocked or ExtendedUnlocked — it only requires HasRegAccess. This is where marker traits pay off over hardcoding specific states in signatures.

Same Pattern, Different Domain: Storage Backends

The dual-axis technique isn’t hardware-specific. Here’s the same structure for a storage layer where some backends support transactions:

// States
struct Closed;
struct Open;
struct InTransaction;

trait HasReadWrite {}
impl HasReadWrite for Open {}
impl HasReadWrite for InTransaction {}

// Vendor traits
trait StorageBackend {
    fn raw_open(&mut self);
    fn raw_read(&self, key: &[u8]) -> Option<Vec<u8>>;
    fn raw_write(&mut self, key: &[u8], value: &[u8]);
}

trait TransactionalBackend: StorageBackend {
    fn raw_begin(&mut self);
    fn raw_commit(&mut self);
    fn raw_rollback(&mut self);
}

// Wrapper
struct Store<B, S = Closed> { backend: B, _s: PhantomData<S> }

impl<B: StorageBackend> Store<B, Closed> {
    fn open(mut self) -> Store<B, Open> { self.backend.raw_open(); /* ... */ todo!() }
}
impl<B: StorageBackend, S: HasReadWrite> Store<B, S> {
    fn read(&self, key: &[u8]) -> Option<Vec<u8>>  { self.backend.raw_read(key) }
    fn write(&mut self, key: &[u8], val: &[u8])    { self.backend.raw_write(key, val) }
}
impl<B: TransactionalBackend> Store<B, Open> {
    fn begin(mut self) -> Store<B, InTransaction>   { /* ... */ todo!() }
}
impl<B: TransactionalBackend> Store<B, InTransaction> {
    fn commit(mut self) -> Store<B, Open>           { /* ... */ todo!() }
    fn rollback(mut self) -> Store<B, Open>         { /* ... */ todo!() }
}

A flat-file backend implements StorageBackend only — begin() won’t compile. A database backend adds TransactionalBackend — the full Open → InTransaction → Open cycle becomes available.

When to Reach for This Pattern

SignalWhy dual-axis fits
Two independent axes: “who provides it” and “what state is it in”The impl block matrix directly encodes both
Some providers have strictly more capabilities than othersSuper-trait (MemoryVendor: Vendor) + conditional impl
Misusing state or capability is a safety/correctness bugCompile-time prevention > runtime checks
You want static dispatch (no vtables)PhantomData + generics = zero-cost
SignalConsider something simpler
Only one axis varies (state OR vendor, not both)Single-axis typestate or plain trait objects
Three or more independent axesConfig Trait Pattern (above) bundles axes into associated types
Runtime polymorphism is acceptableenum state + dyn dispatch is simpler

When two axes become three or more: If you find yourself writing Handle<V, S, D, T> — vendor, state, debug level, transport — the generic parameter list is telling you something. Consider collapsing the vendor axis into an associated-type config trait (the Config Trait Pattern from earlier in this chapter), keeping only the state axis as a generic parameter: Handle<Cfg, S>. The config trait bundles type Vendor, type Transport, etc. into one parameter, and the state axis retains its compile-time transition guarantees. This is a natural evolution, not a rewrite — you lift vendor-related types into Cfg and leave the typestate machinery untouched.

Key Takeaway: The dual-axis pattern is the intersection of typestate and trait-based abstraction. Each impl block maps to one cell of the (vendor × state) matrix. The compiler enforces the entire matrix — no runtime state checks, no impossible-state panics, no cost.


Exercise: Type-Safe State Machine ★★ (~30 min)

Build a traffic light state machine using the type-state pattern. The light must transition Red → Green → Yellow → Red and no other order should be possible.

🔑 Solution
use std::marker::PhantomData;

struct Red;
struct Green;
struct Yellow;

struct TrafficLight<State> {
    _state: PhantomData<State>,
}

impl TrafficLight<Red> {
    fn new() -> Self {
        println!("🔴 Red — STOP");
        TrafficLight { _state: PhantomData }
    }

    fn go(self) -> TrafficLight<Green> {
        println!("🟢 Green — GO");
        TrafficLight { _state: PhantomData }
    }
}

impl TrafficLight<Green> {
    fn caution(self) -> TrafficLight<Yellow> {
        println!("🟡 Yellow — CAUTION");
        TrafficLight { _state: PhantomData }
    }
}

impl TrafficLight<Yellow> {
    fn stop(self) -> TrafficLight<Red> {
        println!("🔴 Red — STOP");
        TrafficLight { _state: PhantomData }
    }
}

fn main() {
    let light = TrafficLight::new(); // Red
    let light = light.go();          // Green
    let light = light.caution();     // Yellow
    let _light = light.stop();       // Red

    // light.caution(); // ❌ Compile error: no method `caution` on Red
    // TrafficLight::new().stop(); // ❌ Compile error: no method `stop` on Red
}

Key takeaway: Invalid transitions are compile errors, not runtime panics.


4. PhantomData — Types That Carry No Data 🔴

What you’ll learn:

  • Why PhantomData<T> exists and the three problems it solves
  • Lifetime branding for compile-time scope enforcement
  • The unit-of-measure pattern for dimension-safe arithmetic
  • Variance (covariant, contravariant, invariant) and how PhantomData controls it

What PhantomData Solves

PhantomData<T> is a zero-sized type that tells the compiler “this struct is logically associated with T, even though it doesn’t contain a T.” It affects variance, drop checking, and auto-trait inference — without using any memory.

#![allow(unused)]
fn main() {
use std::marker::PhantomData;

// Without PhantomData:
struct Slice<'a, T> {
    ptr: *const T,
    len: usize,
    // Problem: compiler doesn't know this struct borrows from 'a
    // or that it's associated with T for drop-check purposes
}

// With PhantomData:
struct Slice<'a, T> {
    ptr: *const T,
    len: usize,
    _marker: PhantomData<&'a T>,
    // Now the compiler knows:
    // 1. This struct borrows data with lifetime 'a
    // 2. It's covariant over 'a (lifetimes can shrink)
    // 3. Drop check considers T
}
}

The three jobs of PhantomData:

JobExampleWhat It Does
Lifetime bindingPhantomData<&'a T>Struct is treated as borrowing 'a
Ownership simulationPhantomData<T>Drop check assumes struct owns a T
Variance controlPhantomData<fn(T)>Makes struct contravariant over T

Lifetime Branding

Use PhantomData to prevent mixing values from different “sessions” or “contexts”:

use std::marker::PhantomData;

/// A handle that's valid only within a specific arena's lifetime
struct ArenaHandle<'arena> {
    index: usize,
    _brand: PhantomData<&'arena ()>,
}

struct Arena {
    data: Vec<String>,
}

impl Arena {
    fn new() -> Self {
        Arena { data: Vec::new() }
    }

    /// Allocate a string and return a branded handle
    fn alloc<'a>(&'a mut self, value: String) -> ArenaHandle<'a> {
        let index = self.data.len();
        self.data.push(value);
        ArenaHandle { index, _brand: PhantomData }
    }

    /// Look up by handle — only accepts handles from THIS arena
    fn get<'a>(&'a self, handle: ArenaHandle<'a>) -> &'a str {
        &self.data[handle.index]
    }
}

fn main() {
    let mut arena1 = Arena::new();
    let handle1 = arena1.alloc("hello".to_string());

    // Can't use handle1 with a different arena — lifetimes won't match
    // let mut arena2 = Arena::new();
    // arena2.get(handle1); // ❌ Lifetime mismatch

    println!("{}", arena1.get(handle1)); // ✅
}

Unit-of-Measure Pattern

Prevent mixing incompatible units at compile time, with zero runtime cost:

use std::marker::PhantomData;
use std::ops::{Add, Mul};

// Unit marker types (zero-sized)
struct Meters;
struct Seconds;
struct MetersPerSecond;

#[derive(Debug, Clone, Copy)]
struct Quantity<Unit> {
    value: f64,
    _unit: PhantomData<Unit>,
}

impl<U> Quantity<U> {
    fn new(value: f64) -> Self {
        Quantity { value, _unit: PhantomData }
    }
}

// Can only add same units:
impl<U> Add for Quantity<U> {
    type Output = Quantity<U>;
    fn add(self, rhs: Self) -> Self::Output {
        Quantity::new(self.value + rhs.value)
    }
}

// Meters / Seconds = MetersPerSecond (custom trait)
impl std::ops::Div<Quantity<Seconds>> for Quantity<Meters> {
    type Output = Quantity<MetersPerSecond>;
    fn div(self, rhs: Quantity<Seconds>) -> Quantity<MetersPerSecond> {
        Quantity::new(self.value / rhs.value)
    }
}

fn main() {
    let dist = Quantity::<Meters>::new(100.0);
    let time = Quantity::<Seconds>::new(9.58);
    let speed = dist / time; // Quantity<MetersPerSecond>
    println!("Speed: {:.2} m/s", speed.value); // 10.44 m/s

    // let nonsense = dist + time; // ❌ Compile error: can't add Meters + Seconds
}

This is pure type-system magic — PhantomData<Meters> is zero-sized, so Quantity<Meters> has the same layout as f64. No wrapper overhead at runtime, but full unit safety at compile time.

PhantomData and Drop Check

When the compiler checks whether a struct’s destructor might access expired data, it uses PhantomData to decide:

#![allow(unused)]
fn main() {
use std::marker::PhantomData;

// PhantomData<T> — compiler assumes we MIGHT drop a T
// This means T must outlive our struct
struct OwningSemantic<T> {
    ptr: *const T,
    _marker: PhantomData<T>,  // "I logically own a T"
}

// PhantomData<*const T> — compiler assumes we DON'T own T
// More permissive — T doesn't need to outlive us
struct NonOwningSemantic<T> {
    ptr: *const T,
    _marker: PhantomData<*const T>,  // "I just point to T"
}
}

Practical rule: When wrapping raw pointers, choose PhantomData carefully:

  • Writing a container that owns its data? → PhantomData<T>
  • Writing a view/reference type? → PhantomData<&'a T> or PhantomData<*const T>

Variance — Why PhantomData’s Type Parameter Matters

Variance determines whether a generic type can be substituted with a sub- or super-type (in Rust, “subtype” means “has a longer lifetime”). Getting variance wrong causes either rejected-good-code or unsound-accepted-code.

graph LR
    subgraph Covariant
        direction TB
        A1["&'long T"] -->|"can become"| A2["&'short T"]
    end

    subgraph Contravariant
        direction TB
        B1["fn(&'short T)"] -->|"can become"| B2["fn(&'long T)"]
    end

    subgraph Invariant
        direction TB
        C1["&'a mut T"] ---|"NO substitution"| C2["&'b mut T"]
    end

    style A1 fill:#d4efdf,stroke:#27ae60,color:#000
    style A2 fill:#d4efdf,stroke:#27ae60,color:#000
    style B1 fill:#e8daef,stroke:#8e44ad,color:#000
    style B2 fill:#e8daef,stroke:#8e44ad,color:#000
    style C1 fill:#fadbd8,stroke:#e74c3c,color:#000
    style C2 fill:#fadbd8,stroke:#e74c3c,color:#000

The Three Variances

VarianceMeaning“Can I substitute…”Rust example
CovariantSubtype flows through'long where 'short expected ✅&'a T, Vec<T>, Box<T>
ContravariantSubtype flows against'short where 'long expected ✅fn(T) (in parameter position)
InvariantNo substitution allowedNeither direction ✅&mut T, Cell<T>, UnsafeCell<T>

Why &'a T is Covariant Over 'a

fn print_str(s: &str) {
    println!("{s}");
}

fn main() {
    let owned = String::from("hello");
    // owned lives for the entire function ('long)
    // print_str expects &'_ str ('short — just for the call)
    print_str(&owned); // ✅ Covariance: 'long → 'short is safe
    // A longer-lived reference can always be used where a shorter one is needed.
}

Why &mut T is Invariant Over T

#![allow(unused)]
fn main() {
// If &mut T were covariant over T, this would compile:
fn evil(s: &mut &'static str) {
    // We could write a shorter-lived &str into a &'static str slot!
    let local = String::from("temporary");
    // *s = &local; // ← Would create a dangling &'static str
}

// Invariance prevents this: &'static str ≠ &'a str when mutating.
// The compiler rejects the substitution entirely.
}

How PhantomData Controls Variance

PhantomData<X> gives your struct the same variance as X:

#![allow(unused)]
fn main() {
use std::marker::PhantomData;

// Covariant over 'a — a Ref<'long> can be used as Ref<'short>
struct Ref<'a, T> {
    ptr: *const T,
    _marker: PhantomData<&'a T>,  // Covariant over 'a, covariant over T
}

// Invariant over T — prevents unsound lifetime shortening of T
struct MutRef<'a, T> {
    ptr: *mut T,
    _marker: PhantomData<&'a mut T>,  // Covariant over 'a, INVARIANT over T
}

// Contravariant over T — useful for callback containers
struct CallbackSlot<T> {
    _marker: PhantomData<fn(T)>,  // Contravariant over T
}
}

PhantomData variance cheat sheet:

PhantomData typeVariance over TVariance over 'aUse when
PhantomData<T>Covariant—You logically own a T
PhantomData<&'a T>CovariantCovariantYou borrow a T with lifetime 'a
PhantomData<&'a mut T>InvariantCovariantYou mutably borrow T
PhantomData<*const T>Covariant—Non-owning pointer to T
PhantomData<*mut T>Invariant—Non-owning mutable pointer
PhantomData<fn(T)>Contravariant—T appears in argument position
PhantomData<fn() -> T>Covariant—T appears in return position
PhantomData<fn(T) -> T>Invariant—T in both positions cancels out

Worked Example: Why This Matters in Practice

use std::marker::PhantomData;

// A token that brands values with a session lifetime.
// MUST be covariant over 'a — otherwise callers can't shorten
// the lifetime when passing to functions that need a shorter borrow.
struct SessionToken<'a> {
    id: u64,
    _brand: PhantomData<&'a ()>,  // ✅ Covariant — callers can shorten 'a
    // _brand: PhantomData<fn(&'a ())>,  // ❌ Contravariant — breaks ergonomics
    // _brand: PhantomData<&'a mut ()>;  // Still covariant over 'a (invariant over T, but T is fixed as ())
}

fn use_token(token: &SessionToken<'_>) {
    println!("Using token {}", token.id);
}

fn main() {
    let token = SessionToken { id: 42, _brand: PhantomData };
    use_token(&token); // ✅ Works because SessionToken is covariant over 'a
}

Decision rule: Start with PhantomData<&'a T> (covariant). Switch to PhantomData<&'a mut T> (invariant) only if your abstraction hands out mutable access to T. Use PhantomData<fn(T)> (contravariant) almost never — it’s only correct for callback-storage scenarios.

Key Takeaways — PhantomData

  • PhantomData<T> carries type/lifetime information without runtime cost
  • Use it for lifetime branding, variance control, and unit-of-measure patterns
  • Drop check: PhantomData<T> tells the compiler your type logically owns a T

See also: Ch 3 — Newtype & Type-State for type-state patterns that use PhantomData. Ch 11 — Unsafe Rust for how PhantomData interacts with raw pointers.


Exercise: Unit-of-Measure with PhantomData ★★ (~30 min)

Extend the unit-of-measure pattern to support:

  • Meters, Seconds, Kilograms
  • Addition of same units
  • Multiplication: Meters * Meters = SquareMeters
  • Division: Meters / Seconds = MetersPerSecond
🔑 Solution
use std::marker::PhantomData;
use std::ops::{Add, Mul, Div};

#[derive(Clone, Copy)]
struct Meters;
#[derive(Clone, Copy)]
struct Seconds;
#[derive(Clone, Copy)]
struct Kilograms;
#[derive(Clone, Copy)]
struct SquareMeters;
#[derive(Clone, Copy)]
struct MetersPerSecond;

#[derive(Debug, Clone, Copy)]
struct Qty<U> {
    value: f64,
    _unit: PhantomData<U>,
}

impl<U> Qty<U> {
    fn new(v: f64) -> Self { Qty { value: v, _unit: PhantomData } }
}

impl<U> Add for Qty<U> {
    type Output = Qty<U>;
    fn add(self, rhs: Self) -> Self::Output { Qty::new(self.value + rhs.value) }
}

impl Mul<Qty<Meters>> for Qty<Meters> {
    type Output = Qty<SquareMeters>;
    fn mul(self, rhs: Qty<Meters>) -> Qty<SquareMeters> {
        Qty::new(self.value * rhs.value)
    }
}

impl Div<Qty<Seconds>> for Qty<Meters> {
    type Output = Qty<MetersPerSecond>;
    fn div(self, rhs: Qty<Seconds>) -> Qty<MetersPerSecond> {
        Qty::new(self.value / rhs.value)
    }
}

fn main() {
    let width = Qty::<Meters>::new(5.0);
    let height = Qty::<Meters>::new(3.0);
    let area = width * height; // Qty<SquareMeters>
    println!("Area: {:.1} m²", area.value);

    let dist = Qty::<Meters>::new(100.0);
    let time = Qty::<Seconds>::new(9.58);
    let speed = dist / time;
    println!("Speed: {:.2} m/s", speed.value);

    let sum = width + height; // Same unit ✅
    println!("Sum: {:.1} m", sum.value);

    // let bad = width + time; // ❌ Compile error: can't add Meters + Seconds
}

5. Channels and Message Passing 🟢

What you’ll learn:

  • std::sync::mpsc basics and when to upgrade to crossbeam-channel
  • Channel selection with select! for multi-source message handling
  • Bounded vs unbounded channels and backpressure strategies
  • The actor pattern for encapsulating concurrent state

std::sync::mpsc — The Standard Channel

Rust’s standard library provides a multi-producer, single-consumer channel:

use std::sync::mpsc;
use std::thread;
use std::time::Duration;

fn main() {
    // Create a channel: tx (transmitter) and rx (receiver)
    let (tx, rx) = mpsc::channel();

    // Spawn a producer thread
    let tx1 = tx.clone(); // Clone for multiple producers
    thread::spawn(move || {
        for i in 0..5 {
            tx1.send(format!("producer-1: msg {i}")).unwrap();
            thread::sleep(Duration::from_millis(100));
        }
    });

    // Second producer
    thread::spawn(move || {
        for i in 0..5 {
            tx.send(format!("producer-2: msg {i}")).unwrap();
            thread::sleep(Duration::from_millis(150));
        }
    });

    // Consumer: receive all messages
    for msg in rx {
        // rx iterator ends when ALL senders are dropped
        println!("Received: {msg}");
    }
    println!("All producers done.");
}

Note: .unwrap() on .send() is used for brevity. It panics if the receiver has been dropped. Production code should handle SendError gracefully.

Key properties:

  • Unbounded by default (can fill memory if consumer is slow)
  • mpsc::sync_channel(N) creates a bounded channel with backpressure
  • rx.recv() blocks the current thread until a message arrives
  • rx.try_recv() returns immediately with Err(TryRecvError::Empty) if nothing is ready
  • The channel closes when all Senders are dropped
#![allow(unused)]
fn main() {
// Bounded channel with backpressure:
let (tx, rx) = mpsc::sync_channel(10); // Buffer of 10 messages

thread::spawn(move || {
    for i in 0..1000 {
        tx.send(i).unwrap(); // BLOCKS if buffer is full — natural backpressure
    }
});
}

Note: .unwrap() is used for brevity. In production, handle SendError (receiver dropped) instead of panicking.

crossbeam-channel — The Production Workhorse

crossbeam-channel is the de facto standard for production channel usage. It’s faster than std::sync::mpsc and supports multi-consumer (mpmc):

// Cargo.toml:
//   [dependencies]
//   crossbeam-channel = "0.5"
use crossbeam_channel::{bounded, unbounded, select, Sender, Receiver};
use std::thread;
use std::time::Duration;

fn main() {
    // Bounded MPMC channel
    let (tx, rx) = bounded::<String>(100);

    // Multiple producers
    for id in 0..4 {
        let tx = tx.clone();
        thread::spawn(move || {
            for i in 0..10 {
                tx.send(format!("worker-{id}: item-{i}")).unwrap();
            }
        });
    }
    drop(tx); // Drop the original sender so the channel can close

    // Multiple consumers (not possible with std::sync::mpsc!)
    let rx2 = rx.clone();
    let consumer1 = thread::spawn(move || {
        while let Ok(msg) = rx.recv() {
            println!("[consumer-1] {msg}");
        }
    });
    let consumer2 = thread::spawn(move || {
        while let Ok(msg) = rx2.recv() {
            println!("[consumer-2] {msg}");
        }
    });

    consumer1.join().unwrap();
    consumer2.join().unwrap();
}

Channel Selection (select!)

Listen on multiple channels simultaneously — like select in Go:

use crossbeam_channel::{bounded, tick, after, select};
use std::time::Duration;

fn main() {
    let (work_tx, work_rx) = bounded::<String>(10);
    let ticker = tick(Duration::from_secs(1));        // Periodic tick
    let deadline = after(Duration::from_secs(10));     // One-shot timeout

    // Producer
    let tx = work_tx.clone();
    std::thread::spawn(move || {
        for i in 0..100 {
            tx.send(format!("job-{i}")).unwrap();
            std::thread::sleep(Duration::from_millis(500));
        }
    });
    drop(work_tx);

    loop {
        select! {
            recv(work_rx) -> msg => {
                match msg {
                    Ok(job) => println!("Processing: {job}"),
                    Err(_) => {
                        println!("Work channel closed");
                        break;
                    }
                }
            },
            recv(ticker) -> _ => {
                println!("Tick — heartbeat");
            },
            recv(deadline) -> _ => {
                println!("Deadline reached — shutting down");
                break;
            },
        }
    }
}

Go comparison: This is exactly like Go’s select statement over channels. crossbeam’s select! macro randomizes order to prevent starvation, just like Go.

Bounded vs Unbounded and Backpressure

TypeBehavior When FullMemoryUse Case
UnboundedNever blocks (grows heap)Unbounded ⚠️Rare — only when producer is slower than consumer
Boundedsend() blocks until spaceFixedProduction default — prevents OOM
Rendezvous (bounded(0))send() blocks until receiver is readyNoneSynchronization / handoff
#![allow(unused)]
fn main() {
// Rendezvous channel — zero capacity, direct handoff
let (tx, rx) = crossbeam_channel::bounded(0);
// tx.send(x) blocks until rx.recv() is called, and vice versa.
// This synchronizes the two threads precisely.
}

Rule: Always use bounded channels in production unless you can prove the producer will never outpace the consumer.

Actor Pattern with Channels

The actor pattern uses channels to serialize access to mutable state — no mutexes needed:

use std::sync::mpsc;
use std::thread;

// Messages the actor can receive
enum CounterMsg {
    Increment,
    Decrement,
    Get(mpsc::Sender<i64>), // Reply channel
}

struct CounterActor {
    count: i64,
    rx: mpsc::Receiver<CounterMsg>,
}

impl CounterActor {
    fn new(rx: mpsc::Receiver<CounterMsg>) -> Self {
        CounterActor { count: 0, rx }
    }

    fn run(mut self) {
        while let Ok(msg) = self.rx.recv() {
            match msg {
                CounterMsg::Increment => self.count += 1,
                CounterMsg::Decrement => self.count -= 1,
                CounterMsg::Get(reply) => {
                    let _ = reply.send(self.count);
                }
            }
        }
    }
}

// Actor handle — cheap to clone, Send + Sync
#[derive(Clone)]
struct Counter {
    tx: mpsc::Sender<CounterMsg>,
}

impl Counter {
    fn spawn() -> Self {
        let (tx, rx) = mpsc::channel();
        thread::spawn(move || CounterActor::new(rx).run());
        Counter { tx }
    }

    fn increment(&self) { let _ = self.tx.send(CounterMsg::Increment); }
    fn decrement(&self) { let _ = self.tx.send(CounterMsg::Decrement); }

    fn get(&self) -> i64 {
        let (reply_tx, reply_rx) = mpsc::channel();
        self.tx.send(CounterMsg::Get(reply_tx)).unwrap();
        reply_rx.recv().unwrap()
    }
}

fn main() {
    let counter = Counter::spawn();

    // Multiple threads can safely use the counter — no mutex!
    let handles: Vec<_> = (0..10).map(|_| {
        let counter = counter.clone();
        thread::spawn(move || {
            for _ in 0..1000 {
                counter.increment();
            }
        })
    }).collect();

    for h in handles { h.join().unwrap(); }
    println!("Final count: {}", counter.get()); // 10000
}

When to use actors vs mutexes: Actors are great when the state has complex invariants, operations take a long time, or you want to serialize access without thinking about lock ordering. Mutexes are simpler for short critical sections.

Key Takeaways — Channels

  • crossbeam-channel is the production workhorse — faster and more feature-rich than std::sync::mpsc
  • select! replaces complex multi-source polling with declarative channel selection
  • Bounded channels provide natural backpressure; unbounded channels risk OOM

See also: Ch 6 — Concurrency for threads, Mutex, and shared state. Ch 15 — Async for async channels (tokio::sync::mpsc).


Exercise: Channel-Based Worker Pool ★★★ (~45 min)

Build a worker pool using channels where:

  • A dispatcher sends Job structs through a channel
  • N workers consume jobs and send results back
  • Use std::sync::mpsc with Arc<Mutex<Receiver>> for a shared work queue
🔑 Solution
use std::sync::mpsc;
use std::thread;

struct Job {
    id: u64,
    data: String,
}

struct JobResult {
    job_id: u64,
    output: String,
    worker_id: usize,
}

fn worker_pool(jobs: Vec<Job>, num_workers: usize) -> Vec<JobResult> {
    let (job_tx, job_rx) = mpsc::channel::<Job>();
    let (result_tx, result_rx) = mpsc::channel::<JobResult>();

    let job_rx = std::sync::Arc::new(std::sync::Mutex::new(job_rx));

    let mut handles = Vec::new();
    for worker_id in 0..num_workers {
        let job_rx = job_rx.clone();
        let result_tx = result_tx.clone();
        handles.push(thread::spawn(move || {
            loop {
                let job = {
                    let rx = job_rx.lock().unwrap();
                    rx.recv()
                };
                match job {
                    Ok(job) => {
                        let output = format!("processed '{}' by worker {worker_id}", job.data);
                        result_tx.send(JobResult {
                            job_id: job.id, output, worker_id,
                        }).unwrap();
                    }
                    Err(_) => break,
                }
            }
        }));
    }
    drop(result_tx);

    let num_jobs = jobs.len();
    for job in jobs {
        job_tx.send(job).unwrap();
    }
    drop(job_tx);

    let results: Vec<_> = result_rx.into_iter().collect();
    assert_eq!(results.len(), num_jobs);

    for h in handles { h.join().unwrap(); }
    results
}

fn main() {
    let jobs: Vec<Job> = (0..20).map(|i| Job {
        id: i, data: format!("task-{i}"),
    }).collect();

    let results = worker_pool(jobs, 4);
    for r in &results {
        println!("[worker {}] job {}: {}", r.worker_id, r.job_id, r.output);
    }
}

6. Concurrency vs Parallelism vs Threads 🟡

What you’ll learn:

  • The precise distinction between concurrency and parallelism
  • OS threads, scoped threads, and rayon for data parallelism
  • Shared state primitives: Arc, Mutex, RwLock, Atomics, Condvar
  • Lazy initialization with OnceLock/LazyLock and lock-free patterns

Terminology: Concurrency ≠ Parallelism

These terms are often confused. Here is the precise distinction:

ConcurrencyParallelism
DefinitionManaging multiple tasks that can make progressExecuting multiple tasks simultaneously
Hardware requirementOne core is enoughRequires multiple cores
AnalogyOne cook, multiple dishes (switching between them)Multiple cooks, each working on a dish
Rust toolsasync/await, channels, select!rayon, thread::spawn, par_iter()
Concurrency (single core):           Parallelism (multi-core):
                                      
Task A: ██░░██░░██                   Task A: ██████████
Task B: ░░██░░██░░                   Task B: ██████████
─────────────────→ time              ─────────────────→ time
(interleaved on one core)           (simultaneous on two cores)

std::thread — OS Threads

Rust threads map 1:1 to OS threads. Each gets its own stack (typically 2-8 MB):

use std::thread;
use std::time::Duration;

fn main() {
    // Spawn a thread — takes a closure
    let handle = thread::spawn(|| {
        for i in 0..5 {
            println!("spawned thread: {i}");
            thread::sleep(Duration::from_millis(100));
        }
        42 // Return value
    });

    // Do work on the main thread simultaneously
    for i in 0..3 {
        println!("main thread: {i}");
        thread::sleep(Duration::from_millis(150));
    }

    // Wait for the thread to finish and get its return value
    let result = handle.join().unwrap(); // unwrap panics if thread panicked
    println!("Thread returned: {result}");
}

Thread::spawn type requirements:

#![allow(unused)]
fn main() {
// The closure must be:
// 1. Send — can be transferred to another thread
// 2. 'static — can't borrow from the calling scope
// 3. FnOnce — takes ownership of captured variables

let data = vec![1, 2, 3];

// ❌ Borrows data — not 'static
// thread::spawn(|| println!("{data:?}"));

// ✅ Move ownership into the thread
thread::spawn(move || println!("{data:?}"));
// data is no longer accessible here
}

Scoped Threads (std::thread::scope)

Since Rust 1.63, scoped threads solve the 'static requirement — threads can borrow from the parent scope:

use std::thread;

fn main() {
    let mut data = vec![1, 2, 3, 4, 5];

    thread::scope(|s| {
        // Thread 1: borrow shared reference
        s.spawn(|| {
            let sum: i32 = data.iter().sum();
            println!("Sum: {sum}");
        });

        // Thread 2: also borrow shared reference (multiple readers OK)
        s.spawn(|| {
            let max = data.iter().max().unwrap();
            println!("Max: {max}");
        });

        // ❌ Can't mutably borrow while shared borrows exist:
        // s.spawn(|| data.push(6));
    });
    // ALL scoped threads joined here — guaranteed before scope returns

    // Now safe to mutate — all threads have finished
    data.push(6);
    println!("Updated: {data:?}");
}

This is huge: Before scoped threads, you had to Arc::clone() everything to share with threads. Now you can borrow directly, and the compiler proves all threads finish before the data goes out of scope.

rayon — Data Parallelism

rayon provides parallel iterators that distribute work across a thread pool automatically:

// Cargo.toml: rayon = "1"
use rayon::prelude::*;

fn main() {
    let data: Vec<u64> = (0..1_000_000).collect();

    // Sequential:
    let sum_seq: u64 = data.iter().map(|x| x * x).sum();

    // Parallel — just change .iter() to .par_iter():
    let sum_par: u64 = data.par_iter().map(|x| x * x).sum();

    assert_eq!(sum_seq, sum_par);

    // Parallel sort:
    let mut numbers = vec![5, 2, 8, 1, 9, 3];
    numbers.par_sort();

    // Parallel processing with map/filter/collect:
    let results: Vec<_> = data
        .par_iter()
        .filter(|&&x| x % 2 == 0)
        .map(|&x| expensive_computation(x))
        .collect();
}

fn expensive_computation(x: u64) -> u64 {
    // Simulate CPU-heavy work
    (0..1000).fold(x, |acc, _| acc.wrapping_mul(7).wrapping_add(13))
}

When to use rayon vs threads:

UseWhen
rayon::par_iter()Processing collections in parallel (map, filter, reduce)
thread::spawnLong-running background tasks, I/O workers
thread::scopeShort-lived parallel tasks that borrow local data
async + tokioI/O-bound concurrency (networking, file I/O)

Shared State: Arc, Mutex, RwLock, Atomics

When threads need shared mutable state, Rust provides safe abstractions:

Note: .unwrap() on .lock(), .read(), and .write() is used for brevity throughout these examples. These calls fail only if another thread panicked while holding the lock (“poisoning”). Production code should decide whether to recover from poisoned locks or propagate the error.

#![allow(unused)]
fn main() {
use std::sync::{Arc, Mutex, RwLock};
use std::sync::atomic::{AtomicU64, Ordering};
use std::thread;

// --- Arc<Mutex<T>>: Shared + Exclusive access ---
fn mutex_example() {
    let counter = Arc::new(Mutex::new(0u64));
    let mut handles = vec![];

    for _ in 0..10 {
        let counter = Arc::clone(&counter);
        handles.push(thread::spawn(move || {
            for _ in 0..1000 {
                let mut guard = counter.lock().unwrap();
                *guard += 1;
            } // Guard dropped → lock released
        }));
    }

    for h in handles { h.join().unwrap(); }
    println!("Counter: {}", counter.lock().unwrap()); // 10000
}

// --- Arc<RwLock<T>>: Multiple readers OR one writer ---
fn rwlock_example() {
    let config = Arc::new(RwLock::new(String::from("initial")));

    // Many readers — don't block each other
    let readers: Vec<_> = (0..5).map(|id| {
        let config = Arc::clone(&config);
        thread::spawn(move || {
            let guard = config.read().unwrap();
            println!("Reader {id}: {guard}");
        })
    }).collect();

    // Writer — blocks and waits for all readers to finish
    {
        let mut guard = config.write().unwrap();
        *guard = "updated".to_string();
    }

    for r in readers { r.join().unwrap(); }
}

// --- Atomics: Lock-free for simple values ---
fn atomic_example() {
    let counter = Arc::new(AtomicU64::new(0));
    let mut handles = vec![];

    for _ in 0..10 {
        let counter = Arc::clone(&counter);
        handles.push(thread::spawn(move || {
            for _ in 0..1000 {
                counter.fetch_add(1, Ordering::Relaxed);
                // No lock, no mutex — hardware atomic instruction
            }
        }));
    }

    for h in handles { h.join().unwrap(); }
    println!("Atomic counter: {}", counter.load(Ordering::Relaxed)); // 10000
}
}

Quick Comparison

PrimitiveUse CaseCostContention
Mutex<T>Short critical sectionsLock + unlockThreads wait in line
RwLock<T>Read-heavy, rare writesReader-writer lockReaders concurrent, writer exclusive
AtomicU64 etc.Counters, flagsHardware CASLock-free — no waiting
ChannelsMessage passingQueue opsProducer/consumer decouple

Condition Variables (Condvar)

A Condvar lets a thread wait until another thread signals that a condition is true, without busy-looping. It is always paired with a Mutex:

#![allow(unused)]
fn main() {
use std::sync::{Arc, Mutex, Condvar};
use std::thread;

let pair = Arc::new((Mutex::new(false), Condvar::new()));
let pair2 = Arc::clone(&pair);

// Spawned thread: wait until ready == true
let handle = thread::spawn(move || {
    let (lock, cvar) = &*pair2;
    let mut ready = lock.lock().unwrap();
    while !*ready {
        ready = cvar.wait(ready).unwrap(); // atomically unlocks + sleeps
    }
    println!("Worker: condition met, proceeding");
});

// Main thread: set ready = true, then signal
{
    let (lock, cvar) = &*pair;
    let mut ready = lock.lock().unwrap();
    *ready = true;
    cvar.notify_one(); // wake one waiting thread (use notify_all for many)
}
handle.join().unwrap();
}

Pattern: Always re-check the condition in a while loop after wait() returns — spurious wakeups are allowed by the OS.

Lazy Initialization: OnceLock and LazyLock

Before Rust 1.80, initializing a global static that requires runtime computation (e.g., parsing a config, compiling a regex) needed the lazy_static! macro or the once_cell crate. The standard library now provides two types that cover these use cases natively:

#![allow(unused)]
fn main() {
use std::sync::{OnceLock, LazyLock};
use std::collections::HashMap;

// OnceLock — initialize on first use via `get_or_init`.
// Useful when the init value depends on runtime arguments.
static CONFIG: OnceLock<HashMap<String, String>> = OnceLock::new();

fn get_config() -> &'static HashMap<String, String> {
    CONFIG.get_or_init(|| {
        // Expensive: read & parse config file — happens exactly once.
        let mut m = HashMap::new();
        m.insert("log_level".into(), "info".into());
        m
    })
}

// LazyLock — initialize on first access, closure provided at definition site.
// Equivalent to lazy_static! but without a macro.
static REGEX: LazyLock<regex::Regex> = LazyLock::new(|| {
    regex::Regex::new(r"^[a-zA-Z0-9_]+$").unwrap()
});

fn is_valid_identifier(s: &str) -> bool {
    REGEX.is_match(s) // First call compiles the regex; subsequent calls reuse it.
}
}
TypeStabilizedInit TimingUse When
OnceLock<T>Rust 1.70Call-site (get_or_init)Init depends on runtime args
LazyLock<T>Rust 1.80Definition-site (closure)Init is self-contained
lazy_static!—Definition-site (macro)Pre-1.80 codebases (migrate away)
const fn + staticAlwaysCompile-timeValue is computable at compile time

Migration tip: Replace lazy_static! { static ref X: T = expr; } with static X: LazyLock<T> = LazyLock::new(|| expr); — same semantics, no macro, no external dependency.

Lock-Free Patterns

For high-performance code, avoid locks entirely:

#![allow(unused)]
fn main() {
use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering};
use std::sync::Arc;

// Pattern 1: Spin lock (educational — prefer std::sync::Mutex)
// ⚠️ WARNING: This is a teaching example only. Real spinlocks need:
//   - A RAII guard (so a panic while holding doesn't deadlock forever)
//   - Fairness guarantees (this starves under contention)
//   - Backoff strategies (exponential backoff, yield to OS)
// Use std::sync::Mutex or parking_lot::Mutex in production.
struct SpinLock {
    locked: AtomicBool,
}

impl SpinLock {
    fn new() -> Self { SpinLock { locked: AtomicBool::new(false) } }

    fn lock(&self) {
        while self.locked
            .compare_exchange_weak(false, true, Ordering::Acquire, Ordering::Relaxed)
            .is_err()
        {
            std::hint::spin_loop(); // CPU hint: we're spinning
        }
    }

    fn unlock(&self) {
        self.locked.store(false, Ordering::Release);
    }
}

// Pattern 2: Lock-free SPSC (single producer, single consumer)
// Use crossbeam::queue::ArrayQueue or similar in production
// roll-your-own only for learning.

// Pattern 3: Sequence counter for wait-free reads
// ⚠️ Best for single-machine-word types (u64, f64); wider T may tear on read.
struct SeqLock<T: Copy> {
    seq: AtomicUsize,
    data: std::cell::UnsafeCell<T>,
}

unsafe impl<T: Copy + Send> Sync for SeqLock<T> {}

impl<T: Copy> SeqLock<T> {
    fn new(val: T) -> Self {
        SeqLock {
            seq: AtomicUsize::new(0),
            data: std::cell::UnsafeCell::new(val),
        }
    }

    fn read(&self) -> T {
        loop {
            let s1 = self.seq.load(Ordering::Acquire);
            if s1 & 1 != 0 { continue; } // Writer in progress, retry

            // SAFETY: We use ptr::read_volatile to prevent the compiler from
            // reordering or caching the read. The SeqLock protocol (checking
            // s1 == s2 after reading) ensures we retry if a writer was active.
            // This mirrors the C SeqLock pattern where the data read must use
            // volatile/relaxed semantics to avoid tearing under concurrency.
            let value = unsafe { core::ptr::read_volatile(self.data.get() as *const T) };

            // Acquire fence: ensures the data read above is ordered before
            // we re-check the sequence counter.
            std::sync::atomic::fence(Ordering::Acquire);
            let s2 = self.seq.load(Ordering::Relaxed);

            if s1 == s2 { return value; } // No writer intervened
            // else retry
        }
    }

    /// # Safety contract
    /// Only ONE thread may call `write()` at a time. If multiple writers
    /// are needed, wrap the `write()` call in an external `Mutex`.
    fn write(&self, val: T) {
        // Increment to odd (signals write in progress).
        // AcqRel: the Acquire side prevents the subsequent data write
        // from being reordered before this increment (readers must see
        // odd before they could observe a partial write). The Release
        // side is technically unnecessary for a single writer but
        // harmless and consistent.
        self.seq.fetch_add(1, Ordering::AcqRel);
        // SAFETY: Single-writer invariant upheld by caller (see doc above).
        // UnsafeCell allows interior mutation; seq counter protects readers.
        unsafe { *self.data.get() = val; }
        // Increment to even (signals write complete).
        // Release: ensure the data write is visible before readers see the even seq.
        self.seq.fetch_add(1, Ordering::Release);
    }
}
}

⚠️ Rust memory model caveat: The non-atomic write through UnsafeCell in write() concurrent with the non-atomic ptr::read_volatile in read() is technically a data race under the Rust abstract machine — even though the SeqLock protocol ensures readers always retry on stale data. This mirrors the C kernel SeqLock pattern and is sound in practice on all modern hardware for types T that fit in a single machine word (e.g., u64). For wider types, consider using AtomicU64 for the data field or wrapping access in a Mutex. See the Rust unsafe code guidelines for the evolving story on UnsafeCell concurrency.

Practical advice: Lock-free code is hard to get right. Use Mutex or RwLock unless profiling shows lock contention is your bottleneck. When you do need lock-free, reach for proven crates (crossbeam, arc-swap, dashmap) rather than rolling your own.

Key Takeaways — Concurrency

  • Scoped threads (thread::scope) let you borrow stack data without Arc
  • rayon::par_iter() parallelizes iterators with one method call
  • Use OnceLock/LazyLock instead of lazy_static!; use Mutex before reaching for atomics
  • Lock-free code is hard — prefer proven crates over hand-rolled implementations

See also: Ch 5 — Channels for message-passing concurrency. Ch 8 — Smart Pointers for Arc/Rc details.

flowchart TD
    A["Need shared<br>mutable state?"] -->|Yes| B{"How much<br>contention?"}
    A -->|No| C["Use channels<br>(Ch 5)"]

    B -->|"Read-heavy"| D["RwLock"]
    B -->|"Short critical<br>section"| E["Mutex"]
    B -->|"Simple counter<br>or flag"| F["Atomics"]
    B -->|"Complex state"| G["Actor + channels"]

    H["Need parallelism?"] -->|"Collection<br>processing"| I["rayon::par_iter"]
    H -->|"Background task"| J["thread::spawn"]
    H -->|"Borrow local data"| K["thread::scope"]

    style A fill:#e8f4f8,stroke:#2980b9,color:#000
    style B fill:#fef9e7,stroke:#f1c40f,color:#000
    style C fill:#d4efdf,stroke:#27ae60,color:#000
    style D fill:#fdebd0,stroke:#e67e22,color:#000
    style E fill:#fdebd0,stroke:#e67e22,color:#000
    style F fill:#fdebd0,stroke:#e67e22,color:#000
    style G fill:#fdebd0,stroke:#e67e22,color:#000
    style H fill:#e8f4f8,stroke:#2980b9,color:#000
    style I fill:#d4efdf,stroke:#27ae60,color:#000
    style J fill:#d4efdf,stroke:#27ae60,color:#000
    style K fill:#d4efdf,stroke:#27ae60,color:#000

Exercise: Parallel Map with Scoped Threads ★★ (~25 min)

Write a function parallel_map<T, R>(data: &[T], f: fn(&T) -> R, num_threads: usize) -> Vec<R> that splits data into num_threads chunks and processes each in a scoped thread. Do not use rayon — use std::thread::scope.

🔑 Solution
fn parallel_map<T: Sync, R: Send>(data: &[T], f: fn(&T) -> R, num_threads: usize) -> Vec<R> {
    let chunk_size = (data.len() + num_threads - 1) / num_threads;
    let mut results = Vec::with_capacity(data.len());

    std::thread::scope(|s| {
        let mut handles = Vec::new();
        for chunk in data.chunks(chunk_size) {
            handles.push(s.spawn(move || {
                chunk.iter().map(f).collect::<Vec<_>>()
            }));
        }
        for h in handles {
            results.extend(h.join().unwrap());
        }
    });

    results
}

fn main() {
    let data: Vec<u64> = (1..=20).collect();
    let squares = parallel_map(&data, |x| x * x, 4);
    assert_eq!(squares, (1..=20).map(|x: u64| x * x).collect::<Vec<_>>());
    println!("Parallel squares: {squares:?}");
}

7. Closures and Higher-Order Functions 🟢

What you’ll learn:

  • The three closure traits (Fn, FnMut, FnOnce) and how capture works
  • Passing closures as parameters and returning them from functions
  • Combinator chains and iterator adapters for functional-style programming
  • Designing your own higher-order APIs with the right trait bounds

Fn, FnMut, FnOnce — The Closure Traits

Every closure in Rust implements one or more of three traits, based on how it captures variables:

#![allow(unused)]
fn main() {
// FnOnce — consumes captured values (can only be called once)
let name = String::from("Alice");
let greet = move || {
    println!("Hello, {name}!"); // Takes ownership of `name`
    drop(name); // name is consumed
};
greet(); // ✅ First call
// greet(); // ❌ Can't call again — `name` was consumed

// FnMut — mutably borrows captured values (can be called many times)
let mut count = 0;
let mut increment = || {
    count += 1; // Mutably borrows `count`
};
increment(); // count == 1
increment(); // count == 2

// Fn — immutably borrows captured values (can be called many times, concurrently)
let prefix = "Result";
let display = |x: i32| {
    println!("{prefix}: {x}"); // Immutably borrows `prefix`
};
display(1);
display(2);
}

The hierarchy: Fn : FnMut : FnOnce — each is a subtrait of the next:

FnOnce  ← everything can be called at least once
 ↑
FnMut   ← can be called repeatedly (may mutate state)
 ↑
Fn      ← can be called repeatedly and concurrently (no mutation)

If a closure implements Fn, it also implements FnMut and FnOnce.

Closures as Parameters and Return Values

// --- Parameters ---

// Static dispatch (monomorphized — fastest)
fn apply_twice<F: Fn(i32) -> i32>(f: F, x: i32) -> i32 {
    f(f(x))
}

// Also written with impl Trait:
fn apply_twice_v2(f: impl Fn(i32) -> i32, x: i32) -> i32 {
    f(f(x))
}

// Dynamic dispatch (trait object — flexible, slight overhead)
fn apply_dyn(f: &dyn Fn(i32) -> i32, x: i32) -> i32 {
    f(x)
}

// --- Return Values ---

// Can't return closures by value without boxing (they have anonymous types):
fn make_adder(n: i32) -> Box<dyn Fn(i32) -> i32> {
    Box::new(move |x| x + n)
}

// With impl Trait (simpler, monomorphized, but can't be dynamic):
fn make_adder_v2(n: i32) -> impl Fn(i32) -> i32 {
    move |x| x + n
}

fn main() {
    let double = |x: i32| x * 2;
    println!("{}", apply_twice(double, 3)); // 12

    let add5 = make_adder(5);
    println!("{}", add5(10)); // 15
}

Combinator Chains and Iterator Adapters

Higher-order functions shine with iterators — this is idiomatic Rust:

#![allow(unused)]
fn main() {
// C-style loop (imperative):
let data = vec![1, 2, 3, 4, 5, 6, 7, 8, 9, 10];
let mut result = Vec::new();
for x in &data {
    if x % 2 == 0 {
        result.push(x * x);
    }
}

// Idiomatic Rust (functional combinator chain):
let result: Vec<i32> = data.iter()
    .filter(|&&x| x % 2 == 0)
    .map(|&x| x * x)
    .collect();

// Same performance — iterators are lazy and optimized by LLVM
assert_eq!(result, vec![4, 16, 36, 64, 100]);
}

Common combinators cheat sheet:

CombinatorWhat It DoesExample
.map(f)Transform each element`.map(
.filter(p)Keep elements where predicate is true`.filter(
.filter_map(f)Map + filter in one step (returns Option)`.filter_map(
.flat_map(f)Map then flatten nested iterators`.flat_map(
.fold(init, f)Reduce to single value (like Aggregate in C#)`.fold(0,
.any(p) / .all(p)Short-circuit boolean check`.any(
.enumerate()Add index`.enumerate().map(
.zip(other)Pair with another iterator.zip(labels.iter())
.take(n) / .skip(n)First/skip N elements.take(10)
.chain(other)Concatenate two iterators.chain(extra.iter())
.peekable()Look ahead without consuming.peek()
.collect()Gather into a collection.collect::<Vec<_>>()

Implementing Your Own Higher-Order APIs

Design APIs that accept closures for customization:

#![allow(unused)]
fn main() {
/// Retry an operation with a configurable strategy
fn retry<T, E, F, S>(
    mut operation: F,
    mut should_retry: S,
    max_attempts: usize,
) -> Result<T, E>
where
    F: FnMut() -> Result<T, E>,
    S: FnMut(&E, usize) -> bool, // (error, attempt) → try again?
{
    for attempt in 1..=max_attempts {
        match operation() {
            Ok(val) => return Ok(val),
            Err(e) if attempt < max_attempts && should_retry(&e, attempt) => {
                continue;
            }
            Err(e) => return Err(e),
        }
    }
    unreachable!()
}

// Usage — caller controls retry logic:
}
#![allow(unused)]
fn main() {
fn connect_to_database() -> Result<(), String> { Ok(()) }
fn http_get(_url: &str) -> Result<String, String> { Ok(String::new()) }
trait TransientError { fn is_transient(&self) -> bool; }
impl TransientError for String { fn is_transient(&self) -> bool { true } }
let url = "http://example.com";
let result = retry(
    || connect_to_database(),
    |err, attempt| {
        eprintln!("Attempt {attempt} failed: {err}");
        true // Always retry
    },
    3,
);

// Usage — retry only specific errors:
let result = retry(
    || http_get(url),
    |err, _| err.is_transient(), // Only retry transient errors
    5,
);
}

The with Pattern — Bracketed Resource Access

Sometimes you need to guarantee that a resource is in a specific state for the duration of an operation, and restored afterward — regardless of how the caller’s code exits (early return, ?, panic). Instead of exposing the resource directly and hoping callers remember to set up and tear down, lend it through a closure:

set up → call closure with resource → tear down

The caller never touches setup or teardown. They can’t forget, can’t get it wrong, and can’t hold the resource beyond the closure’s scope.

Example: GPIO Pin Direction

A GPIO controller manages pins that support bidirectional I/O. Some callers need the pin configured as input, others as output. Rather than exposing raw pin access and trusting callers to set direction correctly, the controller provides with_pin_input and with_pin_output:

/// GPIO pin direction — not public, callers never set this directly.
#[derive(Debug, Clone, Copy, PartialEq)]
enum Direction { In, Out }

/// A GPIO pin handle lent to the closure. Cannot be stored or cloned —
/// it exists only for the duration of the callback.
pub struct GpioPin<'a> {
    pin_number: u8,
    _controller: &'a GpioController,
}

impl GpioPin<'_> {
    pub fn read(&self) -> bool {
        // Read pin level from hardware register
        println!("  reading pin {}", self.pin_number);
        true // stub
    }

    pub fn write(&self, high: bool) {
        // Drive pin level via hardware register
        println!("  writing pin {} = {high}", self.pin_number);
    }
}

pub struct GpioController {
    current_direction: std::cell::Cell<Option<Direction>>,
}

impl GpioController {
    pub fn new() -> Self {
        GpioController {
            current_direction: std::cell::Cell::new(None),
        }
    }

    /// Configure pin as input, run the closure, restore state.
    /// The caller receives a `GpioPin` that lives only for the callback.
    pub fn with_pin_input<R>(
        &self,
        pin: u8,
        mut f: impl FnMut(&GpioPin<'_>) -> R,
    ) -> R {
        let prev = self.current_direction.get();
        self.set_direction(pin, Direction::In);
        let handle = GpioPin { pin_number: pin, _controller: self };
        let result = f(&handle);
        // Restore previous direction (or leave as-is — policy choice)
        if let Some(dir) = prev {
            self.set_direction(pin, dir);
        }
        result
    }

    /// Configure pin as output, run the closure, restore state.
    pub fn with_pin_output<R>(
        &self,
        pin: u8,
        mut f: impl FnMut(&GpioPin<'_>) -> R,
    ) -> R {
        let prev = self.current_direction.get();
        self.set_direction(pin, Direction::Out);
        let handle = GpioPin { pin_number: pin, _controller: self };
        let result = f(&handle);
        if let Some(dir) = prev {
            self.set_direction(pin, dir);
        }
        result
    }

    fn set_direction(&self, pin: u8, dir: Direction) {
        println!("  [hw] pin {pin} → {dir:?}");
        self.current_direction.set(Some(dir));
    }
}

fn main() {
    let gpio = GpioController::new();

    // Caller 1: needs input — doesn't know or care how direction is managed
    let level = gpio.with_pin_input(4, |pin| {
        pin.read()
    });
    println!("Pin 4 level: {level}");

    // Caller 2: needs output — same API shape, different guarantee
    gpio.with_pin_output(4, |pin| {
        pin.write(true);
        // do more work...
        pin.write(false);
    });

    // Can't use the pin handle outside the closure:
    // let escaped_pin = gpio.with_pin_input(4, |pin| pin);
    // ❌ ERROR: borrowed value does not live long enough
}

What the with pattern guarantees:

  • Direction is always set before the caller’s code runs
  • Direction is always restored after, even if the closure returns early
  • The GpioPin handle cannot escape the closure — the borrow checker enforces this via the lifetime tied to the controller reference
  • Callers never import Direction, never call set_direction — the API is impossible to misuse

Where This Pattern Appears

The with pattern shows up throughout Rust’s standard library and ecosystem:

APISetupCallbackTeardown
std::thread::scopeCreate scope|s| { s.spawn(...) }Join all threads
Mutex::lockAcquire lockUse MutexGuard (RAII, not closure, but same idea)Release on drop
tempfile::tempdirCreate temp directoryUse pathDelete on drop
std::io::BufWriter::newBuffer writesWrite operationsFlush on drop
GPIO with_pin_* (above)Set directionUse pin handleRestore direction

The closure-based variant is strongest when:

  • Setup and teardown are paired and forgetting either is a bug
  • The resource shouldn’t outlive the operation — the borrow checker enforces this naturally
  • Multiple configurations exist (with_pin_input vs with_pin_output) — each with_* method encapsulates a different setup without exposing the configuration to the caller

with vs RAII (Drop): Both guarantee cleanup. Use RAII / Drop when the caller needs to hold the resource across multiple statements and function calls. Use with when the operation is bracketed — one setup, one block of work, one teardown — and you don’t want the caller to be able to break the bracket.

FnMut vs Fn in API design: Use FnMut as the default bound — it’s the most flexible (callers can pass Fn or FnMut closures). Only require Fn if you need to call the closure concurrently (e.g., from multiple threads). Only require FnOnce if you call it exactly once.

Key Takeaways — Closures

  • Fn borrows, FnMut borrows mutably, FnOnce consumes — accept the weakest bound your API needs
  • impl Fn in parameters, Box<dyn Fn> for storage, impl Fn in return (or Box<dyn Fn> if dynamic)
  • Combinator chains (map, filter, and_then) compose cleanly and inline to tight loops
  • The with pattern (bracketed access via closure) guarantees setup/teardown and prevents resource escape — use it when the caller shouldn’t manage configuration lifecycle

See also: Ch 2 — Traits In Depth for how Fn/FnMut/FnOnce relate to trait objects. Ch 8 — Functional vs. Imperative for when to choose combinators over loops. Ch 15 — API Design for ergonomic parameter patterns.

graph TD
    FnOnce["FnOnce<br>(can call once)"]
    FnMut["FnMut<br>(can call many times,<br>may mutate captures)"]
    Fn["Fn<br>(can call many times,<br>immutable captures)"]

    Fn -->|"implements"| FnMut
    FnMut -->|"implements"| FnOnce

    style Fn fill:#d4efdf,stroke:#27ae60,color:#000
    style FnMut fill:#fef9e7,stroke:#f1c40f,color:#000
    style FnOnce fill:#fadbd8,stroke:#e74c3c,color:#000

Every Fn is also FnMut, and every FnMut is also FnOnce. Accept FnMut by default — it’s the most flexible bound for callers.


Exercise: Higher-Order Combinator Pipeline ★★ (~25 min)

Create a Pipeline struct that chains transformations. It should support .pipe(f) to add a transformation and .execute(input) to run the full chain.

🔑 Solution
struct Pipeline<T> {
    transforms: Vec<Box<dyn Fn(T) -> T>>,
}

impl<T: 'static> Pipeline<T> {
    fn new() -> Self {
        Pipeline { transforms: Vec::new() }
    }

    fn pipe(mut self, f: impl Fn(T) -> T + 'static) -> Self {
        self.transforms.push(Box::new(f));
        self
    }

    fn execute(self, input: T) -> T {
        self.transforms.into_iter().fold(input, |val, f| f(val))
    }
}

fn main() {
    let result = Pipeline::new()
        .pipe(|s: String| s.trim().to_string())
        .pipe(|s| s.to_uppercase())
        .pipe(|s| format!(">>> {s} <<<"))
        .execute("  hello world  ".to_string());

    println!("{result}"); // >>> HELLO WORLD <<<

    let result = Pipeline::new()
        .pipe(|x: i32| x * 2)
        .pipe(|x| x + 10)
        .pipe(|x| x * x)
        .execute(5);

    println!("{result}"); // (5*2 + 10)^2 = 400
}

8. Functional vs. Imperative: When Elegance Wins (and When It Doesn’t)

Difficulty: 🟡 Intermediate | Time: 2–3 hours | Prerequisites: Ch 7 — Closures

Rust gives you genuine parity between functional and imperative styles. Unlike Haskell (functional by fiat) or C (imperative by default), Rust lets you choose — and the right choice depends on what you’re expressing. This chapter builds the judgment to pick well.

The core principle: Functional style shines when you’re transforming data through a pipeline. Imperative style shines when you’re managing state transitions with side effects. Most real code has both, and the skill is knowing where the boundary falls.


8.1 The Combinator You Didn’t Know You Wanted

Many Rust developers write this:

#![allow(unused)]
fn main() {
let value = if let Some(x) = maybe_config() {
    x
} else {
    default_config()
};
process(value);
}

When they could write this:

#![allow(unused)]
fn main() {
process(maybe_config().unwrap_or_else(default_config));
}

Or this common pattern:

#![allow(unused)]
fn main() {
let display_name = if let Some(name) = user.nickname() {
    name.to_uppercase()
} else {
    "ANONYMOUS".to_string()
};
}

Which is:

#![allow(unused)]
fn main() {
let display_name = user.nickname()
    .map(|n| n.to_uppercase())
    .unwrap_or_else(|| "ANONYMOUS".to_string());
}

The functional version isn’t just shorter — it tells you what is happening (transform, then default) without making you trace control flow. The if let version makes you read the branches to figure out that both paths end up in the same place.

The Option combinator family

Here’s the mental model: Option<T> is a one-element-or-empty collection. Every combinator on Option has an analogy to a collection operation.

You write…Instead of…What it communicates
opt.unwrap_or(default)if let Some(x) = opt { x } else { default }“Use this value or fall back”
opt.unwrap_or_else(|| expensive())if let Some(x) = opt { x } else { expensive() }Same, but default is lazy
opt.map(f)match opt { Some(x) => Some(f(x)), None => None }“Transform the inside, propagate absence”
opt.and_then(f)match opt { Some(x) => f(x), None => None }“Chain fallible operations” (flatmap)
opt.filter(|x| pred(x))match opt { Some(x) if pred(&x) => Some(x), _ => None }“Keep only if it passes”
opt.zip(other)if let (Some(a), Some(b)) = (opt, other) { Some((a,b)) } else { None }“Both or neither”
opt.or(fallback)if opt.is_some() { opt } else { fallback }“First available”
opt.or_else(|| try_another())if opt.is_some() { opt } else { try_another() }“Try alternatives in order”
opt.map_or(default, f)if let Some(x) = opt { f(x) } else { default }“Transform or default” — one-liner
opt.map_or_else(default_fn, f)if let Some(x) = opt { f(x) } else { default_fn() }Same, both sides are closures
opt?match opt { Some(x) => x, None => return None }“Propagate absence upward”

The Result combinator family

The same pattern applies to Result<T, E>:

You write…Instead of…What it communicates
res.map(f)match res { Ok(x) => Ok(f(x)), Err(e) => Err(e) }Transform the success path
res.map_err(f)match res { Ok(x) => Ok(x), Err(e) => Err(f(e)) }Transform the error
res.and_then(f)match res { Ok(x) => f(x), Err(e) => Err(e) }Chain fallible operations
res.unwrap_or_else(|e| default(e))match res { Ok(x) => x, Err(e) => default(e) }Recover from error
res.ok()match res { Ok(x) => Some(x), Err(_) => None }“I don’t care about the error”
res?match res { Ok(x) => x, Err(e) => return Err(e.into()) }Propagate errors upward

When if let IS better

The combinators lose when:

  • You need multiple statements in the Some branch. A map closure with 5 lines is worse than an if let with 5 lines.
  • The control flow is the point. if let Some(connection) = pool.try_get() { /* use it */ } else { /* log, retry, alert */ } — the two branches are genuinely different code paths, not a transform-or-default.
  • Side effects dominate. If both branches do I/O with different error handling, the combinator version obscures the important differences.

Rule of thumb: If the else branch produces the same type as the Some branch and the bodies are short expressions, use a combinator. If the branches do fundamentally different things, use if let or match.


8.2 Bool Combinators: .then() and .then_some()

Another pattern that’s more common than it should be:

#![allow(unused)]
fn main() {
let label = if is_admin {
    Some("ADMIN")
} else {
    None
};
}

Rust 1.62+ gives you:

#![allow(unused)]
fn main() {
let label = is_admin.then_some("ADMIN");
}

Or with a computed value:

#![allow(unused)]
fn main() {
let permissions = is_admin.then(|| compute_admin_permissions());
}

This is especially powerful in chains:

#![allow(unused)]
fn main() {
// Imperative
let mut tags = Vec::new();
if user.is_admin { tags.push("admin"); }
if user.is_verified { tags.push("verified"); }
if user.score > 100 { tags.push("power-user"); }

// Functional
let tags: Vec<&str> = [
    user.is_admin.then_some("admin"),
    user.is_verified.then_some("verified"),
    (user.score > 100).then_some("power-user"),
]
.into_iter()
.flatten()
.collect();
}

The functional version makes the pattern explicit: “build a list from conditional elements.” The imperative version makes you read each if to confirm they all do the same thing (push a tag).


8.3 Iterator Chains vs. Loops: The Decision Framework

Ch 7 showed the mechanics. This section builds the judgment.

When iterators win

Data pipelines — transforming a collection through a series of steps:

#![allow(unused)]
fn main() {
// Imperative: 8 lines, 2 mutable variables
let mut results = Vec::new();
for item in inventory {
    if item.category == Category::Server {
        if let Some(temp) = item.last_temperature() {
            if temp > 80.0 {
                results.push((item.id, temp));
            }
        }
    }
}

// Functional: 6 lines, 0 mutable variables, one pipeline
let results: Vec<_> = inventory.iter()
    .filter(|item| item.category == Category::Server)
    .filter_map(|item| item.last_temperature().map(|t| (item.id, t)))
    .filter(|(_, temp)| *temp > 80.0)
    .collect();
}

The functional version wins because:

  • Each filter is independently readable
  • No mut — the data flows in one direction
  • You can add/remove/reorder pipeline stages without restructuring
  • LLVM inlines iterator adapters to the same machine code as the loop

Aggregation — computing a single value from a collection:

#![allow(unused)]
fn main() {
// Imperative
let mut total_power = 0.0;
let mut count = 0;
for server in fleet {
    total_power += server.power_draw();
    count += 1;
}
let avg = total_power / count as f64;

// Functional
let (total_power, count) = fleet.iter()
    .map(|s| s.power_draw())
    .fold((0.0, 0usize), |(sum, n), p| (sum + p, n + 1));
let avg = total_power / count as f64;
}

Or even simpler if you just need the sum:

#![allow(unused)]
fn main() {
let total: f64 = fleet.iter().map(|s| s.power_draw()).sum();
}

When loops win

Early exit with complex state:

#![allow(unused)]
fn main() {
// This is clear and direct
let mut best_candidate = None;
for server in fleet {
    let score = evaluate(server);
    if score > threshold {
        if server.is_available() {
            best_candidate = Some(server);
            break; // Found one — stop immediately
        }
    }
}

// The functional version is strained
let best_candidate = fleet.iter()
    .filter(|s| evaluate(s) > threshold)
    .find(|s| s.is_available());
}

Wait — that functional version is actually pretty clean. Let’s try a case where it genuinely loses:

Building multiple outputs simultaneously:

#![allow(unused)]
fn main() {
// Imperative: clear, each branch does something different
let mut warnings = Vec::new();
let mut errors = Vec::new();
let mut stats = Stats::default();

for event in log_stream {
    match event.severity {
        Severity::Warn => {
            warnings.push(event.clone());
            stats.warn_count += 1;
        }
        Severity::Error => {
            errors.push(event.clone());
            stats.error_count += 1;
            if event.is_critical() {
                alert_oncall(&event);
            }
        }
        _ => stats.other_count += 1,
    }
}

// Functional version: forced, awkward, nobody wants to read this
let (warnings, errors, stats) = log_stream.iter().fold(
    (Vec::new(), Vec::new(), Stats::default()),
    |(mut w, mut e, mut s), event| {
        match event.severity {
            Severity::Warn => { w.push(event.clone()); s.warn_count += 1; }
            Severity::Error => {
                e.push(event.clone()); s.error_count += 1;
                if event.is_critical() { alert_oncall(event); }
            }
            _ => s.other_count += 1,
        }
        (w, e, s)
    },
);
}

The fold version is longer, harder to read, and has mutation anyway (the mut deconstructed accumulators). The loop wins because:

  • Multiple outputs being built in parallel
  • Side effects (alerting) mixed into the logic
  • Branch bodies are statements, not expressions

State machines with I/O:

#![allow(unused)]
fn main() {
// A parser that reads tokens — the loop IS the algorithm
let mut state = ParseState::Start;
loop {
    let token = lexer.next_token()?;
    state = match state {
        ParseState::Start => match token {
            Token::Keyword(k) => ParseState::GotKeyword(k),
            Token::Eof => break,
            _ => return Err(ParseError::UnexpectedToken(token)),
        },
        ParseState::GotKeyword(k) => match token {
            Token::Ident(name) => ParseState::GotName(k, name),
            _ => return Err(ParseError::ExpectedIdentifier),
        },
        // ...more states
    };
}
}

No functional equivalent is cleaner. The loop with match state is the natural expression of a state machine.

The decision flowchart

flowchart TB
    START{What are you doing?}

    START -->|"Transforming a collection\ninto another collection"| PIPE[Use iterator chain]
    START -->|"Computing a single value\nfrom a collection"| AGG{How complex?}
    START -->|"Multiple outputs from\none pass"| LOOP[Use a for loop]
    START -->|"State machine with\nI/O or side effects"| LOOP
    START -->|"One Option/Result\ntransform + default"| COMB[Use combinators]

    AGG -->|"Sum, count, min, max"| BUILTIN["Use .sum(), .count(),\n.min(), .max()"]
    AGG -->|"Custom accumulation"| FOLD{Accumulator has mutation\nor side effects?}
    FOLD -->|"No"| FOLDF["Use .fold()"]
    FOLD -->|"Yes"| LOOP

    style PIPE fill:#d4efdf,stroke:#27ae60,color:#000
    style COMB fill:#d4efdf,stroke:#27ae60,color:#000
    style BUILTIN fill:#d4efdf,stroke:#27ae60,color:#000
    style FOLDF fill:#d4efdf,stroke:#27ae60,color:#000
    style LOOP fill:#fef9e7,stroke:#f1c40f,color:#000

Rust blocks are expressions. This lets you confine mutation to a construction phase and bind the result immutably:

#![allow(unused)]
fn main() {
use rand::random;

let samples = {
    let mut buf = Vec::with_capacity(10);
    while buf.len() < 10 {
        let reading: f64 = random();
        buf.push(reading);
        if random::<u8>() % 3 == 0 { break; } // randomly stop early
    }
    buf
};
// samples is immutable — contains between 1 and 10 elements
}

The inner buf is mutable only inside the block. Once the block yields, the outer binding samples is immutable and the compiler will reject any later samples.push(...).

Why not an iterator chain? You might try:

#![allow(unused)]
fn main() {
let samples: Vec<f64> = std::iter::from_fn(|| Some(random()))
    .take(10)
    .take_while(|_| random::<u8>() % 3 != 0)
    .collect();
}

But take_while excludes the element that fails the predicate, producing anywhere from zero to nine elements instead of the guaranteed-at-least-one the imperative version provides. You can work around it with scan or chain, but the imperative version is clearer.

When scoped mutability genuinely wins:

ScenarioWhy iterators struggle
Sort-then-freeze (sort_unstable() + dedup())Both return () — no chainable output (itertools offers .sorted().dedup() if available)
Stateful termination (stop on a condition unrelated to the data)take_while drops the boundary element
Multi-step struct population (field-by-field from different sources)No natural single pipeline

Honest calibration: For most collection-building tasks, iterator chains or itertools are preferred. Reach for scoped mutability when the construction logic has branching, early exit, or in-place mutation that doesn’t map to a single pipeline. The pattern’s real value is teaching that mutation scope can be smaller than variable lifetime — a Rust fundamental that surprises developers coming from C++, C#, and Python.


8.4 The ? Operator: Where Functional Meets Imperative

The ? operator is Rust’s most elegant synthesis of both styles. It’s essentially .and_then() combined with early return:

#![allow(unused)]
fn main() {
// This chain of and_then...
fn load_config() -> Result<Config, Error> {
    read_file("config.toml")
        .and_then(|contents| parse_toml(&contents))
        .and_then(|table| validate_config(table))
        .and_then(|valid| Config::from_validated(valid))
}

// ...is exactly equivalent to this
fn load_config() -> Result<Config, Error> {
    let contents = read_file("config.toml")?;
    let table = parse_toml(&contents)?;
    let valid = validate_config(table)?;
    Config::from_validated(valid)
}
}

Both are functional in spirit (they propagate errors automatically) but the ? version gives you named intermediate variables, which matter when:

  • You need to use contents again later
  • You want to add .context("while parsing config")? per step
  • You’re debugging and want to inspect intermediate values

The anti-pattern: long .and_then() chains when ? is available. If every closure in the chain is |x| next_step(x), you’ve reinvented ? without the readability.

When .and_then() IS better than ?:

#![allow(unused)]
fn main() {
// Transforming inside an Option, without early return
let port: Option<u16> = config.get("port")
    .and_then(|v| v.parse::<u16>().ok())
    .filter(|&p| p > 0 && p < 65535);
}

You can’t use ? here because there’s no enclosing function to return from — you’re building an Option, not propagating it.


8.5 Collection Building: collect() vs. Push Loops

collect() is more powerful than most developers realize:

Collecting into a Result

#![allow(unused)]
fn main() {
// Imperative: parse a list, fail on first error
let mut numbers = Vec::new();
for s in input_strings {
    let n: i64 = s.parse().map_err(|_| Error::BadInput(s.clone()))?;
    numbers.push(n);
}

// Functional: collect into Result<Vec<_>, _>
let numbers: Vec<i64> = input_strings.iter()
    .map(|s| s.parse::<i64>().map_err(|_| Error::BadInput(s.clone())))
    .collect::<Result<_, _>>()?;
}

The collect::<Result<Vec<_>, _>>() trick works because Result implements FromIterator. It short-circuits on the first Err, just like the loop with ?.

Collecting into a HashMap

#![allow(unused)]
fn main() {
// Imperative
let mut index = HashMap::new();
for server in fleet {
    index.insert(server.id.clone(), server);
}

// Functional
let index: HashMap<_, _> = fleet.into_iter()
    .map(|s| (s.id.clone(), s))
    .collect();
}

Collecting into a String

#![allow(unused)]
fn main() {
// Imperative
let mut csv = String::new();
for (i, field) in fields.iter().enumerate() {
    if i > 0 { csv.push(','); }
    csv.push_str(field);
}

// Functional
let csv = fields.join(",");

// Or for more complex formatting:
let csv: String = fields.iter()
    .map(|f| format!("\"{f}\""))
    .collect::<Vec<_>>()
    .join(",");
}

When the loop version wins

collect() allocates a new collection. If you’re modifying in place, the loop is both clearer and more efficient:

#![allow(unused)]
fn main() {
// In-place update — no functional equivalent that's better
for server in &mut fleet {
    if server.needs_refresh() {
        server.refresh_telemetry()?;
    }
}
}

The functional version would require .iter_mut().for_each(|s| { ... }), which is just a loop with extra syntax.


8.6 Pattern Matching as Function Dispatch

Rust’s match is a functional construct that most developers use imperatively. Here’s the functional lens:

Match as a lookup table

#![allow(unused)]
fn main() {
// Imperative thinking: "check each case"
fn status_message(code: StatusCode) -> &'static str {
    if code == StatusCode::OK { "Success" }
    else if code == StatusCode::NOT_FOUND { "Not found" }
    else if code == StatusCode::INTERNAL { "Server error" }
    else { "Unknown" }
}

// Functional thinking: "map from domain to range"
fn status_message(code: StatusCode) -> &'static str {
    match code {
        StatusCode::OK => "Success",
        StatusCode::NOT_FOUND => "Not found",
        StatusCode::INTERNAL => "Server error",
        _ => "Unknown",
    }
}
}

The match version isn’t just style — the compiler verifies exhaustiveness. Add a new variant, and every match that doesn’t handle it becomes a compile error. The if/else chain silently falls through to the default.

Match + destructuring as a pipeline

#![allow(unused)]
fn main() {
// Parsing a command — each arm extracts and transforms
fn execute(cmd: Command) -> Result<Response, Error> {
    match cmd {
        Command::Get { key } => db.get(&key).map(Response::Value),
        Command::Set { key, value } => db.set(key, value).map(|_| Response::Ok),
        Command::Delete { key } => db.delete(&key).map(|_| Response::Ok),
        Command::Batch(cmds) => cmds.into_iter()
            .map(execute)
            .collect::<Result<Vec<_>, _>>()
            .map(Response::Batch),
    }
}
}

Each arm is an expression that returns the same type. This is pattern matching as function dispatch — the match arms are essentially a function table indexed by the enum variant.


8.7 Chaining Methods on Custom Types

The functional style extends beyond standard library types. Builder patterns and fluent APIs are functional programming in disguise:

#![allow(unused)]
fn main() {
// This is a combinator chain over your own type
let query = QueryBuilder::new("servers")
    .filter("status", Eq, "active")
    .filter("rack", In, &["A1", "A2", "B1"])
    .order_by("temperature", Desc)
    .limit(50)
    .build();
}

The key insight: if your type has methods that take self and return Self (or a transformed type), you’ve built a combinator. The same functional/imperative judgment applies:

#![allow(unused)]
fn main() {
// Good: chainable because each step is a simple transform
let config = Config::default()
    .with_timeout(Duration::from_secs(30))
    .with_retries(3)
    .with_tls(true);

// Bad: chainable but the chain is doing too many unrelated things
let result = processor
    .load_data(path)?       // I/O
    .validate()             // Pure
    .transform(rule_set)    // Pure
    .save_to_disk(output)?  // I/O
    .notify_downstream()?;  // Side effect

// Better: separate the pure pipeline from the I/O bookends
let data = load_data(path)?;
let processed = data.validate().transform(rule_set);
save_to_disk(output, &processed)?;
notify_downstream()?;
}

The chain fails when it mixes pure transforms with I/O. The reader can’t tell which calls might fail, which have side effects, and where the actual data transformations happen.


8.8 Performance: They’re the Same

A common misconception: “functional style is slower because of all the closures and allocations.”

In Rust, iterator chains compile to the same machine code as hand-written loops. LLVM inlines the closure calls, eliminates the iterator adapter structs, and often produces identical assembly. This is called zero-cost abstraction and it’s not aspirational — it’s measured.

#![allow(unused)]
fn main() {
// These produce identical assembly on release builds:

// Functional
let sum: i64 = (0..1000).filter(|n| n % 2 == 0).map(|n| n * n).sum();

// Imperative
let mut sum: i64 = 0;
for n in 0..1000 {
    if n % 2 == 0 {
        sum += n * n;
    }
}
}

The one exception: .collect() allocates. If you’re chaining .map().collect().iter().map().collect() with intermediate collections, you’re paying for allocations the loop version avoids. The fix: eliminate intermediate collects by chaining adapters directly, or use a loop if you need the intermediate collections for other reasons.


8.9 The Taste Test: A Catalog of Transformations

Here’s a reference table for the most common “I wrote 6 lines but there’s a one-liner” patterns:

Imperative patternFunctional equivalentWhen to prefer functional
if let Some(x) = opt { f(x) } else { default }opt.map_or(default, f)Short expressions on both sides
if let Some(x) = opt { Some(g(x)) } else { None }opt.map(g)Always — this is what map is for
if condition { Some(x) } else { None }condition.then_some(x)Always
if condition { Some(compute()) } else { None }condition.then(compute)Always
match opt { Some(x) if pred(x) => Some(x), _ => None }opt.filter(pred)Always
for x in iter { if pred(x) { result.push(f(x)); } }iter.filter(pred).map(f).collect()When the pipeline is readable in one screen
if a.is_some() && b.is_some() { Some((a?, b?)) }a.zip(b)Always — .zip() is exactly this
match (a, b) { (Some(x), Some(y)) => x + y, _ => 0 }a.zip(b).map(|(x,y)| x + y).unwrap_or(0)Judgment call — depends on complexity
iter.map(f).collect::<Vec<_>>()[0]iter.map(f).next().unwrap()Don’t allocate a Vec for one element
let mut v = vec; v.sort(); v{ let mut v = vec; v.sort(); v }Rust doesn’t have a .sorted() in std (use itertools)

8.10 The Anti-Patterns

Over-functionalizing: the 5-deep chain nobody can read

#![allow(unused)]
fn main() {
// This is not elegant. This is a puzzle.
let result = data.iter()
    .filter_map(|x| x.metadata.as_ref())
    .flat_map(|m| m.tags.iter())
    .filter(|t| t.starts_with("env:"))
    .map(|t| t.strip_prefix("env:").unwrap())
    .filter(|env| allowed_envs.contains(env))
    .map(|env| env.to_uppercase())
    .collect::<HashSet<_>>()
    .into_iter()
    .sorted()
    .collect::<Vec<_>>();
}

When a chain exceeds ~4 adapters, break it up with named intermediate variables or extract a helper:

#![allow(unused)]
fn main() {
let env_tags = data.iter()
    .filter_map(|x| x.metadata.as_ref())
    .flat_map(|m| m.tags.iter());

let allowed: Vec<_> = env_tags
    .filter_map(|t| t.strip_prefix("env:"))
    .filter(|env| allowed_envs.contains(env))
    .map(|env| env.to_uppercase())
    .sorted()
    .collect();
}

Under-functionalizing: the C-style loop that Rust has a word for

#![allow(unused)]
fn main() {
// This is just .any()
let mut found = false;
for item in &list {
    if item.is_expired() {
        found = true;
        break;
    }
}

// Write this instead
let found = list.iter().any(|item| item.is_expired());
}
#![allow(unused)]
fn main() {
// This is just .find()
let mut target = None;
for server in &fleet {
    if server.id == target_id {
        target = Some(server);
        break;
    }
}

// Write this instead
let target = fleet.iter().find(|s| s.id == target_id);
}
#![allow(unused)]
fn main() {
// This is just .all()
let mut all_healthy = true;
for server in &fleet {
    if !server.is_healthy() {
        all_healthy = false;
        break;
    }
}

// Write this instead
let all_healthy = fleet.iter().all(|s| s.is_healthy());
}

The standard library has these for a reason. Learn the vocabulary and the patterns become obvious.


Key Takeaways

  • Option and Result are one-element collections. Their combinators (.map(), .and_then(), .unwrap_or_else(), .filter(), .zip()) replace most if let / match boilerplate.
  • Use bool::then_some() — it replaces if cond { Some(x) } else { None } in every case.
  • Iterator chains win for data pipelines — filter/map/collect with zero mutable state. They compile to the same machine code as loops.
  • Loops win for multi-output state machines — when you’re building multiple collections, doing I/O in branches, or managing a state transition.
  • The ? operator is the best of both worlds — functional error propagation with imperative readability.
  • Break chains at ~4 adapters — use named intermediates for readability. Over-functionalizing is as bad as under-functionalizing.
  • Learn the standard-library vocabulary — .any(), .all(), .find(), .position(), .sum(), .min_by_key() — each one replaces a multi-line loop with a single intent-revealing call.

See also: Ch 7 for closure mechanics and the Fn trait hierarchy. Ch 10 for error combinator patterns. Ch 15 for fluent API design.


Exercise: Refactoring Imperative to Functional ★★ (~30 min)

Refactor the following function from imperative to functional style. Then identify one place where the functional version is worse and explain why.

#![allow(unused)]
fn main() {
fn summarize_fleet(fleet: &[Server]) -> FleetSummary {
    let mut healthy = Vec::new();
    let mut degraded = Vec::new();
    let mut failed = Vec::new();
    let mut total_power = 0.0;
    let mut max_temp = f64::NEG_INFINITY;

    for server in fleet {
        match server.health_status() {
            Health::Healthy => healthy.push(server.id.clone()),
            Health::Degraded(reason) => degraded.push((server.id.clone(), reason)),
            Health::Failed(err) => failed.push((server.id.clone(), err)),
        }
        total_power += server.power_draw();
        if server.max_temperature() > max_temp {
            max_temp = server.max_temperature();
        }
    }

    FleetSummary {
        healthy,
        degraded,
        failed,
        avg_power: total_power / fleet.len() as f64,
        max_temp,
    }
}
}
🔑 Solution

The total_power and max_temp are clean functional rewrites:

#![allow(unused)]
fn main() {
fn summarize_fleet(fleet: &[Server]) -> FleetSummary {
    let avg_power: f64 = fleet.iter().map(|s| s.power_draw()).sum::<f64>()
        / fleet.len() as f64;

    let max_temp = fleet.iter()
        .map(|s| s.max_temperature())
        .fold(f64::NEG_INFINITY, f64::max);

    // But the three-way partition is BETTER as a loop.
    // Functional version would require three separate passes
    // or an awkward fold with three mutable accumulators.
    let mut healthy = Vec::new();
    let mut degraded = Vec::new();
    let mut failed = Vec::new();

    for server in fleet {
        match server.health_status() {
            Health::Healthy => healthy.push(server.id.clone()),
            Health::Degraded(reason) => degraded.push((server.id.clone(), reason)),
            Health::Failed(err) => failed.push((server.id.clone(), err)),
        }
    }

    FleetSummary { healthy, degraded, failed, avg_power, max_temp }
}
}

Why the loop is better for the three-way partition: A functional version would either require three .filter().collect() passes (3x iteration), or a .fold() with three mut Vec accumulators inside a tuple — which is just the loop rewritten with worse syntax. The imperative single-pass loop is clearer, more efficient, and easier to extend.


9. Smart Pointers and Interior Mutability 🟡

What you’ll learn:

  • Box, Rc, Arc for heap allocation and shared ownership
  • Weak references for breaking Rc/Arc reference cycles
  • Cell, RefCell, and Cow for interior mutability patterns
  • Pin for self-referential types and ManuallyDrop for lifecycle control

Box, Rc, Arc — Heap Allocation and Sharing

#![allow(unused)]
fn main() {
// --- Box<T>: Single owner, heap allocation ---
// Use when: recursive types, large values, trait objects
let boxed: Box<i32> = Box::new(42);
println!("{}", *boxed); // Deref to i32

// Recursive type requires Box (otherwise infinite size):
enum List<T> {
    Cons(T, Box<List<T>>),
    Nil,
}

// Trait object (dynamic dispatch):
let writer: Box<dyn std::io::Write> = Box::new(std::io::stdout());

// --- Rc<T>: Multiple owners, single-threaded ---
// Use when: shared ownership within one thread (no Send/Sync)
use std::rc::Rc;

let a = Rc::new(vec![1, 2, 3]);
let b = Rc::clone(&a); // Increments reference count (NOT deep clone)
let c = Rc::clone(&a);
println!("Ref count: {}", Rc::strong_count(&a)); // 3

// All three point to the same Vec. When the last Rc is dropped,
// the Vec is deallocated.

// --- Arc<T>: Multiple owners, thread-safe ---
// Use when: shared ownership across threads
use std::sync::Arc;

let shared = Arc::new(String::from("shared data"));
let handles: Vec<_> = (0..5).map(|_| {
    let shared = Arc::clone(&shared);
    std::thread::spawn(move || println!("{shared}"))
}).collect();
for h in handles { h.join().unwrap(); }
}

Weak References — Breaking Reference Cycles

Rc and Arc use reference counting, which cannot free cycles (A → B → A). Weak<T> is a non-owning handle that does not increment the strong count:

#![allow(unused)]
fn main() {
use std::rc::{Rc, Weak};
use std::cell::RefCell;

struct Node {
    value: i32,
    parent: RefCell<Weak<Node>>,   // does NOT keep parent alive
    children: RefCell<Vec<Rc<Node>>>,
}

let parent = Rc::new(Node {
    value: 0, parent: RefCell::new(Weak::new()), children: RefCell::new(vec![]),
});
let child = Rc::new(Node {
    value: 1, parent: RefCell::new(Rc::downgrade(&parent)), children: RefCell::new(vec![]),
});
parent.children.borrow_mut().push(Rc::clone(&child));

// Access parent from child — returns Option<Rc<Node>>:
if let Some(p) = child.parent.borrow().upgrade() {
    println!("Child's parent value: {}", p.value); // 0
}
// When `parent` is dropped, strong_count → 0, memory is freed.
// `child.parent.upgrade()` would then return `None`.
}

Rule of thumb: Use Rc/Arc for ownership edges, Weak for back-references and caches. For thread-safe code, use Arc<T> with sync::Weak<T>.

Cell and RefCell — Interior Mutability

Sometimes you need to mutate data behind a shared (&) reference. Rust provides interior mutability with runtime borrow checking:

#![allow(unused)]
fn main() {
use std::cell::{Cell, RefCell};

// --- Cell<T>: Copy-based interior mutability ---
// Only for Copy types (or types you swap in/out)
struct Counter {
    count: Cell<u32>,
}

impl Counter {
    fn new() -> Self { Counter { count: Cell::new(0) } }

    fn increment(&self) { // &self, not &mut self!
        self.count.set(self.count.get() + 1);
    }

    fn value(&self) -> u32 { self.count.get() }
}

// --- RefCell<T>: Runtime borrow checking ---
// Panics if you violate borrow rules at runtime
struct Cache {
    data: RefCell<Vec<String>>,
}

impl Cache {
    fn new() -> Self { Cache { data: RefCell::new(Vec::new()) } }

    fn add(&self, item: String) { // &self — looks immutable from outside
        self.data.borrow_mut().push(item); // Runtime-checked &mut
    }

    fn get_all(&self) -> Vec<String> {
        self.data.borrow().clone() // Runtime-checked &
    }

    fn bad_example(&self) {
        let _guard1 = self.data.borrow();
        // let _guard2 = self.data.borrow_mut();
        // ❌ PANICS at runtime — can't have &mut while & exists
    }
}
}

Cell vs RefCell: Cell never panics (it copies/swaps values) but only works with Copy types or via swap()/replace(). RefCell works with any type but panics on double-mutable-borrow. Neither is Sync — for multithreaded use, see Mutex/RwLock.

Cow — Clone on Write

Cow (Clone on Write) holds either a borrowed or owned value. It clones only when mutation is needed:

use std::borrow::Cow;

// Avoids allocating when no modification is needed:
fn normalize(input: &str) -> Cow<'_, str> {
    if input.contains('\t') {
        // Only allocate if tabs need replacing
        Cow::Owned(input.replace('\t', "    "))
    } else {
        // No allocation — just return a reference
        Cow::Borrowed(input)
    }
}

fn main() {
    let clean = "no tabs here";
    let dirty = "tabs\there";

    let r1 = normalize(clean); // Cow::Borrowed — zero allocation
    let r2 = normalize(dirty); // Cow::Owned — allocated new String

    println!("{r1}");
    println!("{r2}");
}

// Also useful for function parameters that MIGHT need ownership:
fn process(data: Cow<'_, [u8]>) {
    // Can read data without copying
    println!("Length: {}", data.len());
    // If we need to mutate, Cow auto-clones:
    let mut owned = data.into_owned(); // Clone only if Borrowed
    owned.push(0xFF);
}

Cow<'_, [u8]> for Binary Data

Cow is especially useful for byte-oriented APIs where data may or may not need transformation (checksum insertion, padding, escaping). This avoids allocating a Vec<u8> on the common fast path:

#![allow(unused)]
fn main() {
use std::borrow::Cow;

/// Pads a frame to a minimum length, borrowing when no padding is needed.
fn pad_frame(frame: &[u8], min_len: usize) -> Cow<'_, [u8]> {
    if frame.len() >= min_len {
        Cow::Borrowed(frame)  // Already long enough — zero allocation
    } else {
        let mut padded = frame.to_vec();
        padded.resize(min_len, 0x00);
        Cow::Owned(padded)    // Allocate only when padding is required
    }
}

let short = pad_frame(&[0xDE, 0xAD], 8);    // Owned — padded to 8 bytes
let long  = pad_frame(&[0; 64], 8);          // Borrowed — already ≥ 8
}

Tip: Combine Cow<[u8]> with bytes::Bytes (Ch10) when you need reference-counted sharing of potentially-transformed buffers.

When to Use Which Pointer

PointerOwner CountThread-SafeMutabilityUse When
Box<T>1✅ (if T: Send)Via &mutHeap allocation, trait objects, recursive types
Rc<T>N❌None (wrap in Cell/RefCell)Shared ownership, single thread, graphs/trees
Arc<T>N✅None (wrap in Mutex/RwLock)Shared ownership across threads
Cell<T>—❌.get() / .set()Interior mutability for Copy types
RefCell<T>—❌.borrow() / .borrow_mut()Interior mutability for any type, single thread
Cow<'_, T>0 or 1✅ (if T: Send)Clone on writeAvoid allocation when data is often unchanged

Pin and Self-Referential Types

Pin<P> prevents a value from being moved in memory. This is essential for self-referential types — structs that contain a pointer to their own data — and for Futures, which may hold references across .await points.

use std::pin::Pin;
use std::marker::PhantomPinned;

// A self-referential struct (simplified):
struct SelfRef {
    data: String,
    ptr: *const String, // Points to `data` above
    _pin: PhantomPinned, // Opts out of Unpin — can't be moved
}

impl SelfRef {
    fn new(s: &str) -> Pin<Box<Self>> {
        let val = SelfRef {
            data: s.to_string(),
            ptr: std::ptr::null(),
            _pin: PhantomPinned,
        };
        let mut boxed = Box::pin(val);

        // SAFETY: we don't move the data after setting the pointer
        let self_ptr: *const String = &boxed.data;
        unsafe {
            let mut_ref = Pin::as_mut(&mut boxed);
            Pin::get_unchecked_mut(mut_ref).ptr = self_ptr;
        }
        boxed
    }

    fn data(&self) -> &str {
        &self.data
    }

    fn ptr_data(&self) -> &str {
        // SAFETY: ptr was set to point to self.data while pinned
        unsafe { &*self.ptr }
    }
}

fn main() {
    let pinned = SelfRef::new("hello");
    assert_eq!(pinned.data(), pinned.ptr_data()); // Both "hello"
    // std::mem::swap would invalidate ptr — but Pin prevents it
}

Key concepts:

ConceptMeaning
Unpin (auto-trait)“Moving this type is safe.” Most types are Unpin by default.
!Unpin / PhantomPinned“I have internal pointers — don’t move me.”
Pin<&mut T>A mutable reference that guarantees T won’t move
Pin<Box<T>>An owned, heap-pinned value

Why this matters for async: Every async fn desugars to a Future that may hold references across .await points — making it self-referential. The async runtime uses Pin<&mut Future> to guarantee the future isn’t moved once polled.

#![allow(unused)]
fn main() {
// When you write:
async fn fetch(url: &str) -> String {
    let response = http_get(url).await; // reference held across await
    response.text().await
}

// The compiler generates a state machine struct that is !Unpin,
// and the runtime pins it before calling Future::poll().
}

When to care about Pin: (1) Implementing Future manually, (2) writing async runtimes or combinators, (3) any struct with self-referential pointers. For normal application code, async/await handles pinning transparently. See the companion Async Rust Training for deeper coverage.

Crate alternatives: For self-referential structs without manual Pin, consider ouroboros or self_cell — they generate safe wrappers with correct pinning and drop semantics.

Pin Projections — Structural Pinning

When you have a Pin<&mut MyStruct>, you often need to access individual fields. Pin projection is the pattern for safely going from Pin<&mut Struct> to Pin<&mut Field> (for pinned fields) or &mut Field (for unpinned fields).

The Problem: Field Access on Pinned Types

#![allow(unused)]
fn main() {
use std::pin::Pin;
use std::marker::PhantomPinned;

struct MyFuture {
    data: String,              // Regular field — safe to move
    state: InternalState,      // Self-referential — must stay pinned
    _pin: PhantomPinned,
}

enum InternalState {
    Waiting { ptr: *const String }, // Points to `data` — self-referential
    Done,
}

// Given `Pin<&mut MyFuture>`, how do you access `data` and `state`?
// You CAN'T just do `pinned.data` — the compiler won't let you
// get a &mut to a field of a pinned value without unsafe.
}

Manual Pin Projection (unsafe)

#![allow(unused)]
fn main() {
impl MyFuture {
    // Project to `data` — this field is structurally unpinned (safe to move)
    fn data(self: Pin<&mut Self>) -> &mut String {
        // SAFETY: `data` is not structurally pinned. Moving `data` alone
        // doesn't move the whole struct, so Pin's guarantee is preserved.
        unsafe { &mut self.get_unchecked_mut().data }
    }

    // Project to `state` — this field IS structurally pinned
    fn state(self: Pin<&mut Self>) -> Pin<&mut InternalState> {
        // SAFETY: `state` is structurally pinned — we maintain the
        // pin invariant by returning Pin<&mut InternalState>.
        unsafe { Pin::new_unchecked(&mut self.get_unchecked_mut().state) }
    }
}
}

Structural pinning rules — a field is “structurally pinned” if:

  1. Moving/swapping that field alone could invalidate a self-reference
  2. The struct’s Drop impl must not move the field
  3. The struct must be !Unpin (enforced by PhantomPinned or a !Unpin field)

pin-project — Safe Pin Projections (Zero Unsafe)

The pin-project crate generates provably correct projections at compile time, eliminating the need for manual unsafe:

#![allow(unused)]
fn main() {
use pin_project::pin_project;
use std::pin::Pin;
use std::future::Future;
use std::task::{Context, Poll};

#[pin_project]                   // <-- Generates projection methods
struct TimedFuture<F: Future> {
    #[pin]                       // <-- Structurally pinned (it's a Future)
    inner: F,
    started_at: std::time::Instant, // NOT pinned — plain data
}

impl<F: Future> Future for TimedFuture<F> {
    type Output = (F::Output, std::time::Duration);

    fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
        let this = self.project();  // Safe! Generated by pin_project
        //   this.inner   : Pin<&mut F>              — pinned field
        //   this.started_at : &mut std::time::Instant — unpinned field

        match this.inner.poll(cx) {
            Poll::Ready(output) => {
                let elapsed = this.started_at.elapsed();
                Poll::Ready((output, elapsed))
            }
            Poll::Pending => Poll::Pending,
        }
    }
}
}

pin-project vs Manual Projection

AspectManual (unsafe)pin-project
SafetyYou prove invariantsCompiler-verified
BoilerplateLow (but error-prone)Zero — derive macro
Drop interactionMust not move pinned fieldsEnforced: #[pinned_drop]
Compile-time costNoneProc-macro expansion
Use casePrimitives, no_stdApplication / library code

#[pinned_drop] — Drop for Pinned Types

When a type has #[pin] fields, pin-project requires #[pinned_drop] instead of a regular Drop impl to prevent accidentally moving pinned fields:

#![allow(unused)]
fn main() {
use pin_project::{pin_project, pinned_drop};
use std::pin::Pin;

#[pin_project(PinnedDrop)]
struct Connection<F> {
    #[pin]
    future: F,
    buffer: Vec<u8>,  // Not pinned — can be moved in drop
}

#[pinned_drop]
impl<F> PinnedDrop for Connection<F> {
    fn drop(self: Pin<&mut Self>) {
        let this = self.project();
        // `this.future` is Pin<&mut F> — can't be moved, only dropped in place
        // `this.buffer` is &mut Vec<u8> — can be drained, cleared, etc.
        this.buffer.clear();
        println!("Connection dropped, buffer cleared");
    }
}
}

When Pin Projections Matter in Practice

Note: The diagram below uses Mermaid syntax. It renders on GitHub and in tools that support Mermaid (mdBook with mermaid plugin, VS Code with Mermaid extension). In plain Markdown viewers, you’ll see the raw source.

graph TD
    A["Do you implement Future manually?"] -->|Yes| B["Does the future hold references<br/>across .await points?"]
    A -->|No| C["async/await handles Pin for you<br/>✅ No projections needed"]
    B -->|Yes| D["Use #[pin_project] on your<br/>future struct"]
    B -->|No| E["Your future is Unpin<br/>✅ No projections needed"]
    D --> F["Mark futures/streams as #[pin]<br/>Leave data fields unpinned"]
    
    style C fill:#91e5a3,color:#000
    style E fill:#91e5a3,color:#000
    style D fill:#ffa07a,color:#000
    style F fill:#ffa07a,color:#000

Rule of thumb: If you’re wrapping another Future or Stream, use pin-project. If you’re writing application code with async/await, you’ll never need pin projections directly. See the companion Async Rust Training for async combinator patterns that use pin projections.

Drop Ordering and ManuallyDrop

Rust’s drop order is deterministic but has rules worth knowing:

Drop Order Rules

struct Label(&'static str);

impl Drop for Label {
    fn drop(&mut self) { println!("Dropping {}", self.0); }
}

fn main() {
    let a = Label("first");   // Declared first
    let b = Label("second");  // Declared second
    let c = Label("third");   // Declared third
}
// Output:
//   Dropping third    ← locals drop in REVERSE declaration order
//   Dropping second
//   Dropping first

The three rules:

WhatDrop OrderRationale
Local variablesReverse declaration orderLater variables might reference earlier ones
Struct fieldsDeclaration order (top to bottom)Matches construction order (stable since Rust 1.0, guaranteed by RFC 1857)
Tuple elementsDeclaration order (left to right)(a, b, c) → drop a, then b, then c
#![allow(unused)]
fn main() {
struct Server {
    listener: Label,  // Dropped 1st
    handler: Label,   // Dropped 2nd
    logger: Label,    // Dropped 3rd
}
// Fields drop top-to-bottom (declaration order).
// This matters when fields reference each other or hold resources.
}

Practical impact: If your struct has a JoinHandle and a Sender, field order determines which drops first. If the thread reads from the channel, drop the Sender first (close the channel) so the thread exits, then join the handle. Put Sender above JoinHandle in the struct.

ManuallyDrop<T> — Suppressing Automatic Drop

ManuallyDrop<T> wraps a value and prevents its destructor from running automatically. You take responsibility for dropping it (or intentionally leaking it):

#![allow(unused)]
fn main() {
use std::mem::ManuallyDrop;

// Use case 1: Prevent double-free in unsafe code
struct TwoPhaseBuffer {
    // We need to drop the Vec ourselves to control timing
    data: ManuallyDrop<Vec<u8>>,
    committed: bool,
}

impl TwoPhaseBuffer {
    fn new(capacity: usize) -> Self {
        TwoPhaseBuffer {
            data: ManuallyDrop::new(Vec::with_capacity(capacity)),
            committed: false,
        }
    }

    fn write(&mut self, bytes: &[u8]) {
        self.data.extend_from_slice(bytes);
    }

    fn commit(&mut self) {
        self.committed = true;
        println!("Committed {} bytes", self.data.len());
    }
}

impl Drop for TwoPhaseBuffer {
    fn drop(&mut self) {
        if !self.committed {
            println!("Rolling back — dropping uncommitted data");
        }
        // SAFETY: data is always valid here; we only drop it once.
        unsafe { ManuallyDrop::drop(&mut self.data); }
    }
}
}
#![allow(unused)]
fn main() {
// Use case 2: Intentional leak (e.g., global singletons)
fn leaked_string() -> &'static str {
    // Box::leak() is the idiomatic way to create a &'static reference:
    let s = String::from("lives forever");
    Box::leak(s.into_boxed_str())
    // ⚠️ This is a controlled memory leak. The String's heap allocation
    // is never freed. Only use for long-lived singletons.
}

// ManuallyDrop alternative (requires unsafe):
// ⚠️ Prefer Box::leak() above — this is shown only to illustrate
// ManuallyDrop semantics (suppressing Drop while the heap data survives).
fn leaked_string_manual() -> &'static str {
    use std::mem::ManuallyDrop;
    let md = ManuallyDrop::new(String::from("lives forever"));
    // SAFETY: ManuallyDrop prevents deallocation; the heap data lives
    // forever, so a 'static reference is valid.
    unsafe { &*(md.as_str() as *const str) }
}
}
#![allow(unused)]
fn main() {
// Use case 3: Union fields (only one variant is valid at a time)
use std::mem::ManuallyDrop;

union IntOrString {
    i: u64,
    s: ManuallyDrop<String>,
    // String has a Drop impl, so it MUST be wrapped in ManuallyDrop
    // inside a union — the compiler can't know which field is active.
}

// No automatic Drop — the code that constructs IntOrString must also
// handle cleanup. If the String variant is active, call:
//   unsafe { ManuallyDrop::drop(&mut value.s); }
// without a Drop impl, the union is simply leaked (no UB, just a leak).
}

ManuallyDrop vs mem::forget:

ManuallyDrop<T>mem::forget(value)
WhenWrap at constructionConsume later
Access inner&*md / &mut *mdValue is gone
Drop laterManuallyDrop::drop(&mut md)Not possible
Use caseFine-grained lifecycle controlFire-and-forget leak

Rule: Use ManuallyDrop in unsafe abstractions where you need to control exactly when a destructor runs. In safe application code, you almost never need it — Rust’s automatic drop ordering handles things correctly.

Key Takeaways — Smart Pointers

  • Box for single ownership on heap; Rc/Arc for shared ownership (single-/multi-threaded)
  • Cell/RefCell provide interior mutability; RefCell panics on violations at runtime
  • Cow avoids allocation on the common path; Pin prevents moves for self-referential types
  • Drop order: fields drop in declaration order (RFC 1857); locals drop in reverse declaration order

See also: Ch 6 — Concurrency for Arc + Mutex patterns. Ch 4 — PhantomData for PhantomData used with smart pointers.

graph TD
    Box["Box&lt;T&gt;<br>Single owner, heap"] --> Heap["Heap allocation"]
    Rc["Rc&lt;T&gt;<br>Shared, single-thread"] --> Heap
    Arc["Arc&lt;T&gt;<br>Shared, multi-thread"] --> Heap

    Rc --> Weak1["Weak&lt;T&gt;<br>Non-owning"]
    Arc --> Weak2["Weak&lt;T&gt;<br>Non-owning"]

    Cell["Cell&lt;T&gt;<br>Copy interior mut"] --> Stack["Stack / interior"]
    RefCell["RefCell&lt;T&gt;<br>Runtime borrow check"] --> Stack
    Cow["Cow&lt;T&gt;<br>Clone on write"] --> Stack

    style Box fill:#d4efdf,stroke:#27ae60,color:#000
    style Rc fill:#e8f4f8,stroke:#2980b9,color:#000
    style Arc fill:#e8f4f8,stroke:#2980b9,color:#000
    style Weak1 fill:#fef9e7,stroke:#f1c40f,color:#000
    style Weak2 fill:#fef9e7,stroke:#f1c40f,color:#000
    style Cell fill:#fdebd0,stroke:#e67e22,color:#000
    style RefCell fill:#fdebd0,stroke:#e67e22,color:#000
    style Cow fill:#fdebd0,stroke:#e67e22,color:#000
    style Heap fill:#f5f5f5,stroke:#999,color:#000
    style Stack fill:#f5f5f5,stroke:#999,color:#000

Exercise: Reference-Counted Graph ★★ (~30 min)

Build a directed graph using Rc<RefCell<Node>> where each node has a name and a list of children. Create a cycle (A → B → C → A) using Weak to break the back-edge. Verify no memory leak with Rc::strong_count.

🔑 Solution
use std::cell::RefCell;
use std::rc::{Rc, Weak};

struct Node {
    name: String,
    children: Vec<Rc<RefCell<Node>>>,
    back_ref: Option<Weak<RefCell<Node>>>,
}

impl Node {
    fn new(name: &str) -> Rc<RefCell<Self>> {
        Rc::new(RefCell::new(Node {
            name: name.to_string(),
            children: Vec::new(),
            back_ref: None,
        }))
    }
}

impl Drop for Node {
    fn drop(&mut self) {
        println!("Dropping {}", self.name);
    }
}

fn main() {
    let a = Node::new("A");
    let b = Node::new("B");
    let c = Node::new("C");

    // A → B → C, with C back-referencing A via Weak
    a.borrow_mut().children.push(Rc::clone(&b));
    b.borrow_mut().children.push(Rc::clone(&c));
    c.borrow_mut().back_ref = Some(Rc::downgrade(&a)); // Weak ref!

    println!("A strong count: {}", Rc::strong_count(&a)); // 1 (only `a` binding)
    println!("B strong count: {}", Rc::strong_count(&b)); // 2 (b + A's child)
    println!("C strong count: {}", Rc::strong_count(&c)); // 2 (c + B's child)

    // Upgrade the weak ref to prove it works:
    let c_ref = c.borrow();
    if let Some(back) = &c_ref.back_ref {
        if let Some(a_ref) = back.upgrade() {
            println!("C points back to: {}", a_ref.borrow().name);
        }
    }
    // When a, b, c go out of scope, all Nodes drop (no cycle leak!)
}

10. Error Handling Patterns 🟢

What you’ll learn:

  • When to use thiserror (libraries) vs anyhow (applications)
  • Error conversion chains with #[from] and .context() wrappers
  • How the ? operator desugars and works in main()
  • When to panic vs return errors, and catch_unwind for FFI boundaries

thiserror vs anyhow — Library vs Application

Rust error handling centers on the Result<T, E> type. Two crates dominate:

// --- thiserror: For LIBRARIES ---
// Generates Display, Error, and From impls via derive macros
use thiserror::Error;

#[derive(Error, Debug)]
pub enum DatabaseError {
    #[error("connection failed: {0}")]
    ConnectionFailed(String),

    #[error("query error: {source}")]
    QueryError {
        #[source]
        source: sqlx::Error,
    },

    #[error("record not found: table={table} id={id}")]
    NotFound { table: String, id: u64 },

    #[error(transparent)] // Delegate Display to the inner error
    Io(#[from] std::io::Error), // Auto-generates From<io::Error>
}

// --- anyhow: For APPLICATIONS ---
// Dynamic error type — great for top-level code where you just want errors to propagate
use anyhow::{Context, Result, bail, ensure};

fn read_config(path: &str) -> Result<Config> {
    let content = std::fs::read_to_string(path)
        .with_context(|| format!("failed to read config from {path}"))?;

    let config: Config = serde_json::from_str(&content)
        .context("failed to parse config JSON")?;

    ensure!(config.port > 0, "port must be positive, got {}", config.port);

    Ok(config)
}

fn main() -> Result<()> {
    let config = read_config("server.toml")?;

    if config.name.is_empty() {
        bail!("server name cannot be empty"); // Return Err immediately
    }

    Ok(())
}

When to use which:

thiserroranyhow
Use inLibraries, shared cratesApplications, binaries
Error typesConcrete enums — callers can matchanyhow::Error — opaque
EffortDefine your error enumJust use Result<T>
DowncastingNot needed — pattern matcherror.downcast_ref::<MyError>()

Error Conversion Chains (#[from])

use thiserror::Error;

#[derive(Error, Debug)]
enum AppError {
    #[error("I/O error: {0}")]
    Io(#[from] std::io::Error),

    #[error("JSON error: {0}")]
    Json(#[from] serde_json::Error),

    #[error("HTTP error: {0}")]
    Http(#[from] reqwest::Error),
}

// Now ? automatically converts:
fn fetch_and_parse(url: &str) -> Result<Config, AppError> {
    let body = reqwest::blocking::get(url)?.text()?;  // reqwest::Error → AppError::Http
    let config: Config = serde_json::from_str(&body)?; // serde_json::Error → AppError::Json
    Ok(config)
}

Context and Error Wrapping

Add human-readable context to errors without losing the original:

use anyhow::{Context, Result};

fn process_file(path: &str) -> Result<Data> {
    let content = std::fs::read_to_string(path)
        .with_context(|| format!("failed to read {path}"))?;

    let data = parse_content(&content)
        .with_context(|| format!("failed to parse {path}"))?;

    validate(&data)
        .context("validation failed")?;

    Ok(data)
}

// Error output:
// Error: validation failed
//
// Caused by:
//    0: failed to parse config.json
//    1: expected ',' at line 5 column 12

The ? Operator in Depth

? is syntactic sugar for a match + From conversion + early return:

#![allow(unused)]
fn main() {
// This:
let value = operation()?;

// Desugars to:
let value = match operation() {
    Ok(v) => v,
    Err(e) => return Err(From::from(e)),
    //                  ^^^^^^^^^^^^^^
    //                  Automatic conversion via From trait
};
}

? also works with Option (in functions returning Option):

#![allow(unused)]
fn main() {
fn find_user_email(users: &[User], name: &str) -> Option<String> {
    let user = users.iter().find(|u| u.name == name)?; // Returns None if not found
    let email = user.email.as_ref()?; // Returns None if email is None
    Some(email.to_uppercase())
}
}

Panics, catch_unwind, and When to Abort

#![allow(unused)]
fn main() {
// Panics: for BUGS, not expected errors
fn get_element(data: &[i32], index: usize) -> &i32 {
    // If this panics, it's a programming error (bug).
    // Don't "handle" it — fix the caller.
    &data[index]
}

// catch_unwind: for boundaries (FFI, thread pools)
use std::panic;

let result = panic::catch_unwind(|| {
    // Run potentially panicking code safely
    risky_operation()
});

match result {
    Ok(value) => println!("Success: {value:?}"),
    Err(_) => eprintln!("Operation panicked — continuing safely"),
}

// When to use which:
// - Result<T, E> → expected failures (file not found, network timeout)
// - panic!()     → programming bugs (index out of bounds, invariant violated)
// - process::abort() → unrecoverable state (security violation, corrupt data)
}

C++ comparison: Result<T, E> replaces exceptions for expected errors. panic!() is like assert() or std::terminate() — it’s for bugs, not control flow. Rust’s ? operator makes error propagation as ergonomic as exceptions without the unpredictable control flow.

Key Takeaways — Error Handling

  • Libraries: thiserror for structured error enums; applications: anyhow for ergonomic propagation
  • #[from] auto-generates From impls; .context() adds human-readable wrappers
  • ? desugars to From::from() + early return; works in main() returning Result

See also: Ch 14 — API Design for “parse, don’t validate” patterns. Ch 10 — Serialization for serde error handling.

flowchart LR
    A["std::io::Error"] -->|"#[from]"| B["AppError::Io"]
    C["serde_json::Error"] -->|"#[from]"| D["AppError::Json"]
    E["Custom validation"] -->|"manual"| F["AppError::Validation"]

    B --> G["? operator"]
    D --> G
    F --> G
    G --> H["Result&lt;T, AppError&gt;"]

    style A fill:#e8f4f8,stroke:#2980b9,color:#000
    style C fill:#e8f4f8,stroke:#2980b9,color:#000
    style E fill:#e8f4f8,stroke:#2980b9,color:#000
    style B fill:#fdebd0,stroke:#e67e22,color:#000
    style D fill:#fdebd0,stroke:#e67e22,color:#000
    style F fill:#fdebd0,stroke:#e67e22,color:#000
    style G fill:#fef9e7,stroke:#f1c40f,color:#000
    style H fill:#d4efdf,stroke:#27ae60,color:#000

Exercise: Error Hierarchy with thiserror ★★ (~30 min)

Design an error type hierarchy for a file-processing application that can fail during I/O, parsing (JSON and CSV), and validation. Use thiserror and demonstrate ? propagation.

🔑 Solution
use thiserror::Error;

#[derive(Error, Debug)]
pub enum AppError {
    #[error("I/O error: {0}")]
    Io(#[from] std::io::Error),

    #[error("JSON parse error: {0}")]
    Json(#[from] serde_json::Error),

    #[error("CSV error at line {line}: {message}")]
    Csv { line: usize, message: String },

    #[error("validation error: {field} — {reason}")]
    Validation { field: String, reason: String },
}

fn read_file(path: &str) -> Result<String, AppError> {
    Ok(std::fs::read_to_string(path)?) // io::Error → AppError::Io via #[from]
}

fn parse_json(content: &str) -> Result<serde_json::Value, AppError> {
    Ok(serde_json::from_str(content)?) // serde_json::Error → AppError::Json
}

fn validate_name(value: &serde_json::Value) -> Result<String, AppError> {
    let name = value.get("name")
        .and_then(|v| v.as_str())
        .ok_or_else(|| AppError::Validation {
            field: "name".into(),
            reason: "must be a non-null string".into(),
        })?;

    if name.is_empty() {
        return Err(AppError::Validation {
            field: "name".into(),
            reason: "must not be empty".into(),
        });
    }

    Ok(name.to_string())
}

fn process_file(path: &str) -> Result<String, AppError> {
    let content = read_file(path)?;
    let json = parse_json(&content)?;
    let name = validate_name(&json)?;
    Ok(name)
}

fn main() {
    match process_file("config.json") {
        Ok(name) => println!("Name: {name}"),
        Err(e) => eprintln!("Error: {e}"),
    }
}

11. Serialization, Zero-Copy, and Binary Data 🟡

What you’ll learn:

  • serde fundamentals: derive macros, attributes, and enum representations
  • Zero-copy deserialization for high-performance read-heavy workloads
  • The serde format ecosystem (JSON, TOML, bincode, MessagePack)
  • Binary data handling with repr(C), zerocopy, and bytes::Bytes

serde Fundamentals

serde (SERialize/DEserialize) is the universal serialization framework for Rust. It separates data model (your structs) from format (JSON, TOML, binary):

use serde::{Serialize, Deserialize};

#[derive(Debug, Serialize, Deserialize)]
struct ServerConfig {
    name: String,
    port: u16,
    #[serde(default)]                    // Use Default::default() if missing
    max_connections: usize,
    #[serde(skip_serializing_if = "Option::is_none")]
    tls_cert_path: Option<String>,
}

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Deserialize from JSON:
    let json_input = r#"{
        "name": "hw-diag",
        "port": 8080
    }"#;
    let config: ServerConfig = serde_json::from_str(json_input)?;
    println!("{config:?}");
    // ServerConfig { name: "hw-diag", port: 8080, max_connections: 0, tls_cert_path: None }

    // Serialize to JSON:
    let output = serde_json::to_string_pretty(&config)?;
    println!("{output}");

    // Same struct, different format — no code changes:
    let toml_input = r#"
        name = "hw-diag"
        port = 8080
    "#;
    let config: ServerConfig = toml::from_str(toml_input)?;
    println!("{config:?}");

    Ok(())
}

Key insight: Your struct derives Serialize and Deserialize once. Then it works with every serde-compatible format — JSON, TOML, YAML, bincode, MessagePack, CBOR, postcard, and dozens more.

Common serde Attributes

serde provides fine-grained control over serialization through field and container attributes:

use serde::{Serialize, Deserialize};

// --- Container attributes (on the struct/enum) ---
#[derive(Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]       // JSON convention: field_name → fieldName
#[serde(deny_unknown_fields)]            // Reject extra keys — strict parsing
struct DiagResult {
    test_name: String,                   // Serialized as "testName"
    pass_count: u32,                     // Serialized as "passCount"
    fail_count: u32,                     // Serialized as "failCount"
}

// --- Field attributes ---
#[derive(Serialize, Deserialize)]
struct Sensor {
    #[serde(rename = "sensor_id")]       // Override field name for serialization
    id: u64,

    #[serde(default)]                    // Use Default if missing from input
    enabled: bool,

    #[serde(default = "default_threshold")]
    threshold: f64,

    #[serde(skip)]                       // Never serialize or deserialize
    cached_value: Option<f64>,

    #[serde(skip_serializing_if = "Vec::is_empty")]
    tags: Vec<String>,

    #[serde(flatten)]                    // Inline nested struct fields
    metadata: Metadata,

    #[serde(with = "hex_bytes")]         // Custom ser/de module
    raw_data: Vec<u8>,
}

fn default_threshold() -> f64 { 1.0 }

#[derive(Serialize, Deserialize)]
struct Metadata {
    vendor: String,
    model: String,
}
// With #[serde(flatten)], the JSON looks like:
// { "sensor_id": 1, "vendor": "Intel", "model": "X200", ... }
// NOT: { "sensor_id": 1, "metadata": { "vendor": "Intel", ... } }

Most-used attributes cheat sheet:

AttributeLevelEffect
rename_all = "camelCase"ContainerRename all fields to camelCase/snake_case/SCREAMING_SNAKE_CASE
deny_unknown_fieldsContainerError on unexpected keys (strict mode)
defaultFieldUse Default::default() when field missing
rename = "..."FieldCustom serialized name
skipFieldExclude from ser/de entirely
skip_serializing_if = "fn"FieldConditionally exclude (e.g., Option::is_none)
flattenFieldInline a nested struct’s fields
with = "module"FieldUse custom serialize/deserialize functions
alias = "..."FieldAccept alternative names during deserialization
deserialize_with = "fn"FieldCustom deserialize function only
untaggedEnumTry each variant in order (no discriminant in output)

Enum Representations

serde provides four representations for enums in formats like JSON:

use serde::{Serialize, Deserialize};

// 1. Externally tagged (DEFAULT):
#[derive(Serialize, Deserialize)]
enum Command {
    Reboot,
    RunDiag { test_name: String, timeout_secs: u64 },
    SetFanSpeed(u8),
}
// "Reboot"                                          → Command::Reboot
// {"RunDiag": {"test_name": "gpu", "timeout_secs": 60}}  → Command::RunDiag { ... }

// 2. Internally tagged — #[serde(tag = "type")]:
#[derive(Serialize, Deserialize)]
#[serde(tag = "type")]
enum Event {
    Start { timestamp: u64 },
    Error { code: i32, message: String },
    End   { timestamp: u64, success: bool },
}
// {"type": "Start", "timestamp": 1706000000}
// {"type": "Error", "code": 42, "message": "timeout"}

// 3. Adjacently tagged — #[serde(tag = "t", content = "c")]:
#[derive(Serialize, Deserialize)]
#[serde(tag = "t", content = "c")]
enum Payload {
    Text(String),
    Binary(Vec<u8>),
}
// {"t": "Text", "c": "hello"}
// {"t": "Binary", "c": [0, 1, 2]}

// 4. Untagged — #[serde(untagged)]:
#[derive(Serialize, Deserialize)]
#[serde(untagged)]
enum StringOrNumber {
    Str(String),
    Num(f64),
}
// "hello" → StringOrNumber::Str("hello")
// 42.0    → StringOrNumber::Num(42.0)
// ⚠️ Tried IN ORDER — first matching variant wins

Which representation to choose: Use internally tagged (tag = "type") for most JSON APIs — it’s the most readable and matches conventions in Go, Python, and TypeScript. Use untagged only for “union” types where the shape alone disambiguates.

Zero-Copy Deserialization

serde can deserialize without allocating new strings — borrowing directly from the input buffer. This is the key to high-performance parsing:

use serde::Deserialize;

// --- Owned (allocating) ---
// Each String field copies bytes from the input into new heap allocations.
#[derive(Deserialize)]
struct OwnedRecord {
    name: String,           // Allocates a new String
    value: String,          // Allocates another String
}

// --- Zero-copy (borrowing) ---
// &'de str fields borrow directly from the input — ZERO allocation.
#[derive(Deserialize)]
struct BorrowedRecord<'a> {
    name: &'a str,          // Points into the input buffer
    value: &'a str,         // Points into the input buffer
}

fn main() {
    let input = r#"{"name": "cpu_temp", "value": "72.5"}"#;

    // Owned: allocates two String objects
    let owned: OwnedRecord = serde_json::from_str(input).unwrap();

    // Zero-copy: `name` and `value` point into `input` — no allocation
    let borrowed: BorrowedRecord = serde_json::from_str(input).unwrap();

    // The output is lifetime-bound: borrowed can't outlive input
    println!("{}: {}", borrowed.name, borrowed.value);
}

Understanding the lifetime:

// Deserialize<'de> — the struct can borrow from data with lifetime 'de:
//   struct BorrowedRecord<'a> where 'a == 'de
//   Only works when the input buffer lives long enough

// DeserializeOwned — the struct owns all its data, no borrowing:
//   trait DeserializeOwned: for<'de> Deserialize<'de> {}
//   Works with any input lifetime (the struct is independent)

use serde::de::DeserializeOwned;

// This function requires owned types — input can be temporary
fn parse_owned<T: DeserializeOwned>(input: &str) -> T {
    serde_json::from_str(input).unwrap()
}

// This function allows borrowing — more efficient but restricts lifetimes
fn parse_borrowed<'a, T: Deserialize<'a>>(input: &'a str) -> T {
    serde_json::from_str(input).unwrap()
}

When to use zero-copy:

  • Parsing large files where you only need a few fields
  • High-throughput pipelines (network packets, log lines)
  • When the input buffer already lives long enough (e.g., memory-mapped file)

When NOT to use zero-copy:

  • Input is ephemeral (network read buffer that’s reused)
  • You need to store the result beyond the input’s lifetime
  • Fields need transformation (escapes, normalization)

Practical tip: Cow<'a, str> gives you the best of both — borrow when possible, allocate when necessary (e.g., when JSON escape sequences need unescaping). serde supports Cow natively.

The Format Ecosystem

FormatCrateHuman-ReadableSizeSpeedUse Case
JSONserde_json✅LargeGoodConfig files, REST APIs, logging
TOMLtoml✅MediumGoodConfig files (Cargo.toml style)
YAMLserde_yaml✅MediumGoodConfig files (complex nesting)
bincodebincode❌SmallFastIPC, caches, Rust-to-Rust
postcardpostcard❌TinyVery fastEmbedded systems, no_std
MessagePackrmp-serde❌SmallFastCross-language binary protocol
CBORciborium❌SmallFastIoT, constrained environments
#![allow(unused)]
fn main() {
// Same struct, many formats — serde's power:

#[derive(serde::Serialize, serde::Deserialize, Debug)]
struct DiagConfig {
    name: String,
    tests: Vec<String>,
    timeout_secs: u64,
}

let config = DiagConfig {
    name: "accel_diag".into(),
    tests: vec!["memory".into(), "compute".into()],
    timeout_secs: 300,
};

// JSON:   {"name":"accel_diag","tests":["memory","compute"],"timeout_secs":300}
let json = serde_json::to_string(&config).unwrap();       // 67 bytes

// bincode: compact binary — ~40 bytes, no field names
let bin = bincode::serialize(&config).unwrap();            // Much smaller

// postcard: even smaller, varint encoding — great for embedded
// let post = postcard::to_allocvec(&config).unwrap();
}

Choose your format:

  • Config files humans edit → TOML or JSON
  • Rust-to-Rust IPC/caching → bincode (fast, compact, not cross-language)
  • Cross-language binary → MessagePack or CBOR
  • Embedded / no_std → postcard

Binary Data and repr(C)

For hardware diagnostics, parsing binary protocol data is common. Rust provides tools for safe, zero-copy binary data handling:

#![allow(unused)]
fn main() {
// --- #[repr(C)]: Predictable memory layout ---
// Ensures fields are laid out in declaration order with C padding rules.
// Essential for matching hardware register layouts and protocol headers.

#[repr(C)]
#[derive(Debug, Clone, Copy)]
struct IpmiHeader {
    rs_addr: u8,
    net_fn_lun: u8,
    checksum: u8,
    rq_addr: u8,
    rq_seq_lun: u8,
    cmd: u8,
}

// --- Safe binary parsing with manual deserialization ---
impl IpmiHeader {
    fn from_bytes(data: &[u8]) -> Option<Self> {
        if data.len() < size_of::<Self>() {
            return None;
        }
        Some(IpmiHeader {
            rs_addr:     data[0],
            net_fn_lun:  data[1],
            checksum:    data[2],
            rq_addr:     data[3],
            rq_seq_lun:  data[4],
            cmd:         data[5],
        })
    }

    fn net_fn(&self) -> u8 { self.net_fn_lun >> 2 }
    fn lun(&self)    -> u8 { self.net_fn_lun & 0x03 }
}

// --- Endianness-aware parsing ---
fn read_u16_le(data: &[u8], offset: usize) -> u16 {
    u16::from_le_bytes([data[offset], data[offset + 1]])
}

fn read_u32_be(data: &[u8], offset: usize) -> u32 {
    u32::from_be_bytes([
        data[offset], data[offset + 1],
        data[offset + 2], data[offset + 3],
    ])
}

// --- #[repr(C, packed)]: Remove padding (alignment = 1) ---
#[repr(C, packed)]
#[derive(Debug, Clone, Copy)]
struct PcieCapabilityHeader {
    cap_id: u8,        // Capability ID
    next_cap: u8,      // Pointer to next capability
    cap_reg: u16,      // Capability-specific register
}
// ⚠️ Packed structs: taking &field creates an unaligned reference — UB.
// Always copy fields out: let id = header.cap_id;  // OK (Copy)
// Never do: let r = &header.cap_reg;               // UB if unaligned
}

zerocopy and bytemuck — Safe Transmutation

Instead of unsafe transmute, use crates that verify layout safety at compile time:

#![allow(unused)]
fn main() {
// --- zerocopy: Compile-time checked zero-copy conversions ---
// Cargo.toml: zerocopy = { version = "0.8", features = ["derive"] }

use zerocopy::{FromBytes, IntoBytes, KnownLayout, Immutable};

#[derive(FromBytes, IntoBytes, KnownLayout, Immutable, Debug)]
#[repr(C)]
struct SensorReading {
    sensor_id: u16,
    flags: u8,
    _reserved: u8,
    value: u32,     // Fixed-point: actual = value / 1000.0
}

fn parse_sensor(raw: &[u8]) -> Option<&SensorReading> {
    // Safe zero-copy: verifies alignment and size AT COMPILE TIME
    SensorReading::ref_from_bytes(raw).ok()
    // Returns &SensorReading pointing INTO raw — no copy, no allocation
}

// --- bytemuck: Simple, battle-tested ---
// Cargo.toml: bytemuck = { version = "1", features = ["derive"] }

use bytemuck::{Pod, Zeroable};

#[derive(Pod, Zeroable, Clone, Copy, Debug)]
#[repr(C)]
struct GpuRegister {
    address: u32,
    value: u32,
}

fn cast_registers(data: &[u8]) -> &[GpuRegister] {
    // Safe cast: Pod guarantees all bit patterns are valid
    bytemuck::cast_slice(data)
}
}

When to use which:

ApproachSafetyOverheadUse When
Manual field-by-field parsing✅ SafeCopy fieldsSmall structs, complex layouts
zerocopy✅ SafeZero-copyLarge buffers, many reads, compile-time checks
bytemuck✅ SafeZero-copySimple Pod types, casting slices
unsafe { transmute() }❌ UnsafeZero-copyLast resort — avoid in application code

bytes::Bytes — Reference-Counted Buffers

The bytes crate (used by tokio, hyper, tonic) provides zero-copy byte buffers with reference counting — Bytes is to Vec<u8> what Arc<[u8]> is to owned slices:

use bytes::{Bytes, BytesMut, Buf, BufMut};

fn main() {
    // --- BytesMut: mutable buffer for building data ---
    let mut buf = BytesMut::with_capacity(1024);
    buf.put_u8(0x01);                    // Write a byte
    buf.put_u16(0x1234);                 // Write u16 (big-endian)
    buf.put_slice(b"hello");             // Write raw bytes
    buf.put(&b"world"[..]);              // Write from slice

    // Freeze into immutable Bytes (zero cost):
    let data: Bytes = buf.freeze();

    // --- Bytes: immutable, reference-counted, cloneable ---
    let data2 = data.clone();            // Cheap: increments refcount, NOT deep copy
    let slice = data.slice(3..8);        // Zero-copy sub-slice (shares buffer)

    // Read from Bytes using the Buf trait:
    let mut reader = &data[..];
    let byte = reader.get_u8();          // 0x01
    let short = reader.get_u16();        // 0x1234

    // Split without copying:
    let mut original = Bytes::from_static(b"HEADER\x00PAYLOAD");
    let header = original.split_to(6);   // header = "HEADER", original = "\x00PAYLOAD"

    println!("header: {:?}", &header[..]);
    println!("payload: {:?}", &original[1..]);
}

bytes vs Vec<u8>:

FeatureVec<u8>Bytes
Clone costO(n) deep copyO(1) refcount increment
Sub-slicingBorrows with lifetimeOwned, refcount-tracked
Thread safetyNot Sync (needs Arc)Send + Sync built in
MutabilityDirect &mutSplit into BytesMut first
EcosystemStandard librarytokio, hyper, tonic, axum

When to use bytes: Network protocols, packet parsing, any scenario where you receive a buffer and need to split it into parts that are processed by different components or threads. The zero-copy splitting is the killer feature.

Key Takeaways — Serialization & Binary Data

  • serde’s derive macros handle 90% of cases; use attributes (rename, skip, default) for the rest
  • Zero-copy deserialization (&'a str in structs) avoids allocation for read-heavy workloads
  • repr(C) + zerocopy/bytemuck for hardware register layouts; bytes::Bytes for reference-counted buffers

See also: Ch 9 — Error Handling for combining serde errors with thiserror. Ch 11 — Unsafe for repr(C) and FFI data layouts.

flowchart LR
    subgraph Input
        JSON["JSON"]
        TOML["TOML"]
        Bin["bincode"]
        MsgP["MessagePack"]
    end

    subgraph serde["serde data model"]
        Ser["Serialize"]
        De["Deserialize"]
    end

    subgraph Output
        Struct["Rust struct"]
        Enum["Rust enum"]
    end

    JSON --> De
    TOML --> De
    Bin --> De
    MsgP --> De
    De --> Struct
    De --> Enum
    Struct --> Ser
    Enum --> Ser
    Ser --> JSON
    Ser --> Bin

    style JSON fill:#e8f4f8,stroke:#2980b9,color:#000
    style TOML fill:#e8f4f8,stroke:#2980b9,color:#000
    style Bin fill:#e8f4f8,stroke:#2980b9,color:#000
    style MsgP fill:#e8f4f8,stroke:#2980b9,color:#000
    style Ser fill:#fef9e7,stroke:#f1c40f,color:#000
    style De fill:#fef9e7,stroke:#f1c40f,color:#000
    style Struct fill:#d4efdf,stroke:#27ae60,color:#000
    style Enum fill:#d4efdf,stroke:#27ae60,color:#000

Exercise: Custom serde Deserialization ★★★ (~45 min)

Design a HumanDuration wrapper that deserializes from human-readable strings like "30s", "5m", "2h" using a custom serde deserializer. It should also serialize back to the same format.

🔑 Solution
use serde::{Deserialize, Deserializer, Serialize, Serializer};
use std::fmt;

#[derive(Debug, Clone, PartialEq)]
struct HumanDuration(std::time::Duration);

impl HumanDuration {
    fn from_str(s: &str) -> Result<Self, String> {
        let s = s.trim();
        if s.is_empty() { return Err("empty duration string".into()); }

        let (num_str, suffix) = s.split_at(
            s.find(|c: char| !c.is_ascii_digit()).unwrap_or(s.len())
        );
        let value: u64 = num_str.parse()
            .map_err(|_| format!("invalid number: {num_str}"))?;

        let duration = match suffix {
            "s" | "sec"  => std::time::Duration::from_secs(value),
            "m" | "min"  => std::time::Duration::from_secs(value * 60),
            "h" | "hr"   => std::time::Duration::from_secs(value * 3600),
            "ms"         => std::time::Duration::from_millis(value),
            other        => return Err(format!("unknown suffix: {other}")),
        };
        Ok(HumanDuration(duration))
    }
}

impl fmt::Display for HumanDuration {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        let secs = self.0.as_secs();
        if secs == 0 {
            write!(f, "{}ms", self.0.as_millis())
        } else if secs % 3600 == 0 {
            write!(f, "{}h", secs / 3600)
        } else if secs % 60 == 0 {
            write!(f, "{}m", secs / 60)
        } else {
            write!(f, "{}s", secs)
        }
    }
}

impl Serialize for HumanDuration {
    fn serialize<S: Serializer>(&self, serializer: S) -> Result<S::Ok, S::Error> {
        serializer.serialize_str(&self.to_string())
    }
}

impl<'de> Deserialize<'de> for HumanDuration {
    fn deserialize<D: Deserializer<'de>>(deserializer: D) -> Result<Self, D::Error> {
        let s = String::deserialize(deserializer)?;
        HumanDuration::from_str(&s).map_err(serde::de::Error::custom)
    }
}

#[derive(Debug, Deserialize, Serialize)]
struct Config {
    timeout: HumanDuration,
    retry_interval: HumanDuration,
}

fn main() {
    let json = r#"{ "timeout": "30s", "retry_interval": "5m" }"#;
    let config: Config = serde_json::from_str(json).unwrap();

    assert_eq!(config.timeout.0, std::time::Duration::from_secs(30));
    assert_eq!(config.retry_interval.0, std::time::Duration::from_secs(300));

    let serialized = serde_json::to_string(&config).unwrap();
    assert!(serialized.contains("30s"));
    println!("Config: {serialized}");
}

12. Unsafe Rust — Controlled Danger 🔴

What you’ll learn:

  • The five unsafe superpowers and when each is needed
  • Writing sound abstractions: safe API, unsafe internals
  • FFI patterns for calling C from Rust (and back)
  • Common UB pitfalls and arena/slab allocator patterns

The Five Unsafe Superpowers

unsafe unlocks five operations that the compiler can’t verify:

#![allow(unused)]
fn main() {
// SAFETY: each operation is explained inline below.
unsafe {
    // 1. Dereference a raw pointer
    let ptr: *const i32 = &42;
    let value = *ptr; // Could be a dangling/null pointer

    // 2. Call an unsafe function
    let layout = std::alloc::Layout::new::<u64>();
    let mem = std::alloc::alloc(layout);

    // 3. Access a mutable static variable
    static mut COUNTER: u32 = 0;
    COUNTER += 1; // Data race if multiple threads access

    // 4. Implement an unsafe trait
    // unsafe impl Send for MyType {}

    // 5. Access fields of a union
    // union IntOrFloat { i: i32, f: f32 }
    // let u = IntOrFloat { i: 42 };
    // let f = u.f; // Reinterpret bits — could be garbage
}
}

Key principle: unsafe doesn’t turn off the borrow checker or type system. It only unlocks these five specific capabilities. All other Rust rules still apply.

Writing Sound Abstractions

The purpose of unsafe is to build safe abstractions around unsafe operations:

#![allow(unused)]
fn main() {
/// A fixed-capacity stack-allocated buffer.
/// All public methods are safe — the unsafe is encapsulated.
pub struct StackBuf<T, const N: usize> {
    data: [std::mem::MaybeUninit<T>; N],
    len: usize,
}

impl<T, const N: usize> StackBuf<T, N> {
    pub fn new() -> Self {
        StackBuf {
            // Each element is individually MaybeUninit — no unsafe needed.
            // `const { ... }` blocks (Rust 1.79+) let us repeat a non-Copy
            // const expression N times.
            data: [const { std::mem::MaybeUninit::uninit() }; N],
            len: 0,
        }
    }

    pub fn push(&mut self, value: T) -> Result<(), T> {
        if self.len >= N {
            return Err(value); // Buffer full — return value to caller
        }
        // SAFETY: len < N, so data[len] is within bounds.
        // We write a valid T into the MaybeUninit slot.
        self.data[self.len] = std::mem::MaybeUninit::new(value);
        self.len += 1;
        Ok(())
    }

    pub fn get(&self, index: usize) -> Option<&T> {
        if index < self.len {
            // SAFETY: index < len, and data[0..len] are all initialized.
            Some(unsafe { self.data[index].assume_init_ref() })
        } else {
            None
        }
    }
}

impl<T, const N: usize> Drop for StackBuf<T, N> {
    fn drop(&mut self) {
        // SAFETY: data[0..len] are initialized — drop them properly.
        for i in 0..self.len {
            unsafe { self.data[i].assume_init_drop(); }
        }
    }
}
}

The three rules of sound unsafe code:

  1. Document invariants — every // SAFETY: comment explains why the operation is valid
  2. Encapsulate — the unsafe is inside a safe API; users can’t trigger UB
  3. Minimize — only the smallest possible block is unsafe

FFI Patterns: Calling C from Rust

#![allow(unused)]
fn main() {
// Declare the C function signature:
extern "C" {
    fn strlen(s: *const std::ffi::c_char) -> usize;
    fn printf(format: *const std::ffi::c_char, ...) -> std::ffi::c_int;
}

// Safe wrapper:
fn safe_strlen(s: &str) -> usize {
    let c_string = std::ffi::CString::new(s).expect("string contains null byte");
    // SAFETY: c_string is a valid null-terminated string, alive for the call.
    unsafe { strlen(c_string.as_ptr()) }
}

// Calling Rust from C (export a function):
#[no_mangle]
pub extern "C" fn rust_add(a: i32, b: i32) -> i32 {
    a + b
}
}

Common FFI types:

RustCNotes
i32 / u32int32_t / uint32_tFixed-width, safe
*const T / *mut Tconst T* / T*Raw pointers
std::ffi::CStrconst char* (borrowed)Null-terminated, borrowed
std::ffi::CStringchar* (owned)Null-terminated, owned
std::ffi::c_voidvoidOpaque pointer target
Option<fn(...)>Nullable function pointerNone = NULL

Common UB Pitfalls

PitfallExampleWhy It’s UB
Null dereference*std::ptr::null::<i32>()Dereferencing null is always UB
Dangling pointerDereference after drop()Memory may be reused
Data raceTwo threads write to static mutUnsynchronized concurrent writes
Wrong assume_initMaybeUninit::<String>::uninit().assume_init()Reading uninitialized memory. Note: [const { MaybeUninit::uninit() }; N] (Rust 1.79+) is the safe way to create an array of MaybeUninit — no unsafe or assume_init needed (see StackBuf::new() above).
Aliasing violationCreating two &mut to same dataViolates Rust’s aliasing model
Invalid enum valuestd::mem::transmute::<u8, bool>(2)bool can only be 0 or 1

When to use unsafe in production:

  • FFI boundaries (calling C/C++ code)
  • Performance-critical inner loops (avoid bounds checks)
  • Building primitives (Vec, HashMap — these use unsafe internally)
  • Never in application logic if you can avoid it

Custom Allocators — Arena and Slab Patterns

In C, you’d write custom malloc() replacements for specific allocation patterns — arena allocators that free everything at once, slab allocators for fixed-size objects, or pool allocators for high-throughput systems. Rust provides the same power through the GlobalAlloc trait and allocator crates, with the added benefit of lifetime-scoped arenas that prevent use-after-free at compile time.

Arena Allocators — Bulk Allocation, Bulk Free

An arena allocates by bumping a pointer forward. Individual items can’t be freed — the entire arena is freed at once. This is perfect for request-scoped or frame-scoped allocations:

#![allow(unused)]
fn main() {
use bumpalo::Bump;

fn process_sensor_frame(raw_data: &[u8]) {
    // Create an arena for this frame's allocations
    let arena = Bump::new();

    // Allocate objects in the arena — ~2ns each (just a pointer bump)
    let header = arena.alloc(parse_header(raw_data));
    let readings: &mut [f32] = arena.alloc_slice_fill_default(header.sensor_count);

    for (i, chunk) in raw_data[header.payload_offset..].chunks(4).enumerate() {
        if i < readings.len() {
            readings[i] = f32::from_le_bytes(chunk.try_into().unwrap());
        }
    }

    // Use readings...
    let avg = readings.iter().sum::<f32>() / readings.len() as f32;
    println!("Frame avg: {avg:.2}");

    // `arena` drops here — ALL allocations freed at once in O(1)
    // No per-object destructor overhead, no fragmentation
}
fn parse_header(_: &[u8]) -> Header { Header { sensor_count: 4, payload_offset: 8 } }
struct Header { sensor_count: usize, payload_offset: usize }
}

Arena vs standard allocator:

AspectVec::new() / Box::new()Bump arena
Alloc speed~25ns (malloc)~2ns (pointer bump)
Free speedPer-object destructorO(1) bulk free
FragmentationYes (long-lived processes)None within arena
Lifetime safetyHeap — freed on DropArena reference — compile-time scoped
Use caseGeneral purposeRequest/frame/batch processing

typed-arena — Type-Safe Arena

When all arena objects are the same type, typed-arena provides a simpler API that returns references with the arena’s lifetime:

#![allow(unused)]
fn main() {
use typed_arena::Arena;

struct AstNode<'a> {
    value: i32,
    children: Vec<&'a AstNode<'a>>,
}

fn build_tree() {
    let arena: Arena<AstNode<'_>> = Arena::new();

    // Allocate nodes — returns &AstNode tied to arena's lifetime
    let root = arena.alloc(AstNode { value: 1, children: vec![] });
    let left = arena.alloc(AstNode { value: 2, children: vec![] });
    let right = arena.alloc(AstNode { value: 3, children: vec![] });

    // Build the tree — all references valid as long as `arena` lives
    // (Mutable access requires interior mutability for truly mutable trees)

    println!("Root: {}, Left: {}, Right: {}", root.value, left.value, right.value);

    // `arena` drops here — all nodes freed at once
}
}

Slab Allocators — Fixed-Size Object Pools

A slab allocator pre-allocates a pool of fixed-size slots. Objects are allocated and returned individually, but all slots are the same size — eliminating fragmentation and enabling O(1) alloc/free:

#![allow(unused)]
fn main() {
use slab::Slab;

struct Connection {
    id: u64,
    buffer: [u8; 1024],
    active: bool,
}

fn connection_pool_example() {
    // Pre-allocate a slab for connections
    let mut connections: Slab<Connection> = Slab::with_capacity(256);

    // Insert returns a key (usize index) — O(1)
    let key1 = connections.insert(Connection {
        id: 1001,
        buffer: [0; 1024],
        active: true,
    });

    let key2 = connections.insert(Connection {
        id: 1002,
        buffer: [0; 1024],
        active: true,
    });

    // Access by key — O(1)
    if let Some(conn) = connections.get_mut(key1) {
        conn.buffer[0..5].copy_from_slice(b"hello");
    }

    // Remove returns the value — O(1), slot is reused for next insert
    let removed = connections.remove(key2);
    assert_eq!(removed.id, 1002);

    // Next insert reuses the freed slot — no fragmentation
    let key3 = connections.insert(Connection {
        id: 1003,
        buffer: [0; 1024],
        active: true,
    });
    assert_eq!(key3, key2); // Same slot reused!
}
}

Implementing a Minimal Arena (for no_std)

For bare-metal environments where you can’t pull in bumpalo, here’s a minimal arena built on unsafe:

#![allow(unused)]
#![cfg_attr(not(test), no_std)]

fn main() {
use core::alloc::Layout;
use core::cell::{Cell, UnsafeCell};

/// A simple bump allocator backed by a fixed-size byte array.
/// Not thread-safe — use per-core or with a lock for multi-threaded contexts.
///
/// **Important**: Like `bumpalo`, this arena does NOT call destructors on
/// allocated items when the arena is dropped. Types with `Drop` impls will
/// leak their resources (file handles, sockets, etc.). Only allocate types
/// without meaningful `Drop` impls, or manually drop them before the arena.
pub struct FixedArena<const N: usize> {
    // UnsafeCell is REQUIRED here: we mutate `buf` through `&self`.
    // Without UnsafeCell, casting &self.buf to *mut u8 would be UB
    // (violates Rust's aliasing model — shared ref implies immutable).
    buf: UnsafeCell<[u8; N]>,
    offset: Cell<usize>, // Interior mutability for &self allocation
}

impl<const N: usize> FixedArena<N> {
    pub const fn new() -> Self {
        FixedArena {
            buf: UnsafeCell::new([0; N]),
            offset: Cell::new(0),
        }
    }

    /// Allocate a `T` in the arena. Returns `None` if out of space.
    pub fn alloc<T>(&self, value: T) -> Option<&mut T> {
        let layout = Layout::new::<T>();
        let current = self.offset.get();

        // Align up
        let aligned = (current + layout.align() - 1) & !(layout.align() - 1);
        let new_offset = aligned + layout.size();

        if new_offset > N {
            return None; // Arena full
        }

        self.offset.set(new_offset);

        // SAFETY:
        // - `aligned` is within `buf` bounds (checked above)
        // - Alignment is correct (aligned to T's requirement)
        // - No aliasing: each alloc returns a unique, non-overlapping region
        // - UnsafeCell grants permission to mutate through &self
        // - The arena outlives the returned reference (caller must ensure)
        let ptr = unsafe {
            let base = (self.buf.get() as *mut u8).add(aligned);
            let typed = base as *mut T;
            typed.write(value);
            &mut *typed
        };

        Some(ptr)
    }

    /// Reset the arena — invalidates all previous allocations.
    ///
    /// # Safety
    /// Caller must ensure no references to arena-allocated data exist.
    pub unsafe fn reset(&self) {
        self.offset.set(0);
    }

    pub fn used(&self) -> usize {
        self.offset.get()
    }

    pub fn remaining(&self) -> usize {
        N - self.offset.get()
    }
}
}

Choosing an Allocator Strategy

Note: The diagram below uses Mermaid syntax. It renders on GitHub and in tools that support Mermaid (mdBook with mermaid plugin, VS Code with Mermaid extension). In plain Markdown viewers, you’ll see the raw source.

graph TD
    A["What's your allocation pattern?"] --> B{All same type?}
    A --> I{"Environment?"}
    B -->|Yes| C{Need individual free?}
    B -->|No| D{Need individual free?}
    C -->|Yes| E["<b>Slab</b><br/>slab crate<br/>O(1) alloc + free<br/>Index-based access"]
    C -->|No| F["<b>typed-arena</b><br/>Bulk alloc, bulk free<br/>Lifetime-scoped refs"]
    D -->|Yes| G["<b>Standard allocator</b><br/>Box, Vec, etc.<br/>General-purpose malloc"]
    D -->|No| H["<b>Bump arena</b><br/>bumpalo crate<br/>~2ns alloc, O(1) bulk free"]
    
    I -->|no_std| J["FixedArena (custom)<br/>or embedded-alloc"]
    I -->|std| K["bumpalo / typed-arena / slab"]
    
    style E fill:#91e5a3,color:#000
    style F fill:#91e5a3,color:#000
    style G fill:#89CFF0,color:#000
    style H fill:#91e5a3,color:#000
    style J fill:#ffa07a,color:#000
    style K fill:#91e5a3,color:#000
C PatternRust EquivalentKey Advantage
Custom malloc() pool#[global_allocator] implType-safe, debuggable
obstack (GNU)bumpalo::BumpLifetime-scoped, no use-after-free
Kernel slab (kmem_cache)slab::Slab<T>Type-safe, index-based
Stack-allocated temp bufferFixedArena<N> (above)No heap, const constructible
alloca()[T; N] or SmallVecCompile-time sized, no UB

Cross-reference: For bare-metal allocator setup (#[global_allocator] with embedded-alloc), see the Rust Training for C Programmers, Chapter 15.1 “Global Allocator Setup” which covers the embedded-specific bootstrapping.

Key Takeaways — Unsafe Rust

  • Document invariants (SAFETY: comments), encapsulate behind safe APIs, minimize unsafe scope
  • [const { MaybeUninit::uninit() }; N] (Rust 1.79+) replaces the old assume_init anti-pattern
  • FFI requires extern "C", #[repr(C)], and careful null/lifetime handling
  • Arena and slab allocators trade general-purpose flexibility for allocation speed

See also: Ch 4 — PhantomData for variance and drop-check interactions with unsafe code. Ch 8 — Smart Pointers for Pin and self-referential types.


Exercise: Safe Wrapper around Unsafe ★★★ (~45 min)

Write a FixedVec<T, const N: usize> — a fixed-capacity, stack-allocated vector. Requirements:

  • push(&mut self, value: T) -> Result<(), T> returns Err(value) when full
  • pop(&mut self) -> Option<T> returns and removes the last element
  • as_slice(&self) -> &[T] borrows initialized elements
  • All public methods must be safe; all unsafe must be encapsulated with SAFETY: comments
  • Drop must clean up initialized elements
🔑 Solution
use std::mem::MaybeUninit;

pub struct FixedVec<T, const N: usize> {
    data: [MaybeUninit<T>; N],
    len: usize,
}

impl<T, const N: usize> FixedVec<T, N> {
    pub fn new() -> Self {
        FixedVec {
            data: [const { MaybeUninit::uninit() }; N],
            len: 0,
        }
    }

    pub fn push(&mut self, value: T) -> Result<(), T> {
        if self.len >= N { return Err(value); }
        // SAFETY: len < N, so data[len] is within bounds.
        self.data[self.len] = MaybeUninit::new(value);
        self.len += 1;
        Ok(())
    }

    pub fn pop(&mut self) -> Option<T> {
        if self.len == 0 { return None; }
        self.len -= 1;
        // SAFETY: data[len] was initialized (len was > 0 before decrement).
        Some(unsafe { self.data[self.len].assume_init_read() })
    }

    pub fn as_slice(&self) -> &[T] {
        // SAFETY: data[0..len] are all initialized, and MaybeUninit<T>
        // has the same layout as T.
        unsafe { std::slice::from_raw_parts(self.data.as_ptr() as *const T, self.len) }
    }

    pub fn len(&self) -> usize { self.len }
    pub fn is_empty(&self) -> bool { self.len == 0 }
}

impl<T, const N: usize> Drop for FixedVec<T, N> {
    fn drop(&mut self) {
        // SAFETY: data[0..len] are initialized — drop each one.
        for i in 0..self.len {
            unsafe { self.data[i].assume_init_drop(); }
        }
    }
}

fn main() {
    let mut v = FixedVec::<String, 4>::new();
    v.push("hello".into()).unwrap();
    v.push("world".into()).unwrap();
    assert_eq!(v.as_slice(), &["hello", "world"]);
    assert_eq!(v.pop(), Some("world".into()));
    assert_eq!(v.len(), 1);
}

13. Macros — Code That Writes Code 🟡

What you’ll learn:

  • Declarative macros (macro_rules!) with pattern matching and repetition
  • When macros are the right tool vs generics/traits
  • Procedural macros: derive, attribute, and function-like
  • Writing a custom derive macro with syn and quote

Declarative Macros (macro_rules!)

Macros match patterns on syntax and expand to code at compile time:

#![allow(unused)]
fn main() {
// A simple macro that creates a HashMap
macro_rules! hashmap {
    // Match: key => value pairs separated by commas
    ( $( $key:expr => $value:expr ),* $(,)? ) => {
        {
            let mut map = std::collections::HashMap::new();
            $( map.insert($key, $value); )*
            map
        }
    };
}

let scores = hashmap! {
    "Alice" => 95,
    "Bob" => 87,
    "Carol" => 92,
};
// Expands to:
// let mut map = HashMap::new();
// map.insert("Alice", 95);
// map.insert("Bob", 87);
// map.insert("Carol", 92);
// map
}

Macro fragment types:

FragmentMatchesExample
$x:exprAny expression42, a + b, foo()
$x:tyA typei32, Vec<String>
$x:identAn identifiermy_var, Config
$x:patA patternSome(x), _
$x:stmtA statementlet x = 5;
$x:ttA single token treeAnything (most flexible)
$x:literalA literal value42, "hello", true

Repetition: $( ... ),* means “zero or more, comma-separated”

#![allow(unused)]
fn main() {
// Generate test functions automatically
macro_rules! test_cases {
    ( $( $name:ident: $input:expr => $expected:expr ),* $(,)? ) => {
        $(
            #[test]
            fn $name() {
                assert_eq!(process($input), $expected);
            }
        )*
    };
}

test_cases! {
    test_empty: "" => "",
    test_hello: "hello" => "HELLO",
    test_trim: "  spaces  " => "SPACES",
}
// Generates three separate #[test] functions
}

When (Not) to Use Macros

Use macros when:

  • Reducing boilerplate that traits/generics can’t handle (variadic arguments, DRY test generation)
  • Creating DSLs (html!, sql!, vec!)
  • Conditional code generation (cfg!, compile_error!)

Don’t use macros when:

  • A function or generic would work (macros are harder to debug, autocomplete doesn’t help)
  • You need type checking inside the macro (macros operate on tokens, not types)
  • The pattern is used once or twice (not worth the abstraction cost)
#![allow(unused)]
fn main() {
// ❌ Unnecessary macro — a function works fine:
macro_rules! double {
    ($x:expr) => { $x * 2 };
}

// ✅ Just use a function:
fn double(x: i32) -> i32 { x * 2 }

// ✅ Good macro use — variadic, can't be a function:
macro_rules! println {
    ($($arg:tt)*) => { /* format string + args */ };
}
}

Procedural Macros Overview

Procedural macros are Rust functions that transform token streams. They require a separate crate with proc-macro = true:

#![allow(unused)]
fn main() {
// Three types of proc macros:

// 1. Derive macros — #[derive(MyTrait)]
// Generate trait implementations from struct definitions
#[derive(Debug, Clone, Serialize, Deserialize)]
struct Config {
    name: String,
    port: u16,
}

// 2. Attribute macros — #[my_attribute]
// Transform the annotated item
#[route(GET, "/api/users")]
async fn list_users() -> Json<Vec<User>> { /* ... */ }

// 3. Function-like macros — my_macro!(...)
// Custom syntax
let query = sql!(SELECT * FROM users WHERE id = ?);
}

Derive Macros in Practice

The most common proc macro type. Here’s how #[derive(Debug)] works conceptually:

#![allow(unused)]
fn main() {
// Input (your struct):
#[derive(Debug)]
struct Point {
    x: f64,
    y: f64,
}

// The derive macro generates:
impl std::fmt::Debug for Point {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        f.debug_struct("Point")
            .field("x", &self.x)
            .field("y", &self.y)
            .finish()
    }
}
}

Commonly used derive macros:

DeriveCrateWhat It Generates
Debugstdfmt::Debug impl (debug printing)
Clone, CopystdValue duplication
PartialEq, EqstdEquality comparison
HashstdHashing for HashMap keys
Serialize, DeserializeserdeJSON/YAML/etc. encoding
Errorthiserrorstd::error::Error + Display
ParserclapCLI argument parsing
Builderderive_builderBuilder pattern

Practical advice: Use derive macros liberally — they eliminate error-prone boilerplate. Writing your own proc macros is an advanced topic; use existing ones (serde, thiserror, clap) before building custom ones.

Macro Hygiene and $crate

Hygiene means that identifiers created inside a macro don’t collide with identifiers in the caller’s scope. Rust’s macro_rules! is partially hygienic:

macro_rules! make_var {
    () => {
        let x = 42; // This 'x' is in the MACRO's scope
    };
}

fn main() {
    let x = 10;
    make_var!();   // Creates a different 'x' (hygienic)
    println!("{x}"); // Prints 10, not 42 — macro's x doesn't leak
}

$crate: When writing macros in a library, use $crate to refer to your own crate — it resolves correctly regardless of how users import your crate:

#![allow(unused)]
fn main() {
// In my_diagnostics crate:

pub fn log_result(msg: &str) {
    println!("[diag] {msg}");
}

#[macro_export]
macro_rules! diag_log {
    ($($arg:tt)*) => {
        // ✅ $crate always resolves to my_diagnostics, even if the user
        // renamed the crate in their Cargo.toml
        $crate::log_result(&format!($($arg)*))
    };
}

// ❌ Without $crate:
// my_diagnostics::log_result(...)  ← breaks if user writes:
//   [dependencies]
//   diag = { package = "my_diagnostics", version = "1" }
}

Rule: Always use $crate:: in #[macro_export] macros. Never use your crate’s name directly.

Recursive Macros and tt Munching

Recursive macros process input one token at a time — a technique called tt munching (token-tree munching):

// Count the number of expressions passed to the macro
macro_rules! count {
    // Base case: no tokens left
    () => { 0usize };
    // Recursive case: consume one expression, count the rest
    ($head:expr $(, $tail:expr)* $(,)?) => {
        1usize + count!($($tail),*)
    };
}

fn main() {
    let n = count!("a", "b", "c", "d");
    assert_eq!(n, 4);

    // Works at compile time too:
    const N: usize = count!(1, 2, 3);
    assert_eq!(N, 3);
}
#![allow(unused)]
fn main() {
// Build a heterogeneous tuple from a list of expressions:
macro_rules! tuple_from {
    // Base: single element
    ($single:expr $(,)?) => { ($single,) };
    // Recursive: first element + rest
    ($head:expr, $($tail:expr),+ $(,)?) => {
        ($head, tuple_from!($($tail),+))
    };
}

let t = tuple_from!(1, "hello", 3.14, true);
// Expands to: (1, ("hello", (3.14, (true,))))
}

Fragment specifier subtleties:

FragmentGotcha
$x:exprGreedily parses — 1 + 2 is ONE expression, not three tokens
$x:tyGreedily parses — Vec<String> is one type; can’t be followed by + or <
$x:ttMatches exactly ONE token tree — most flexible, least checked
$x:identOnly plain identifiers — not paths like std::io
$x:patIn Rust 2021, matches A | B patterns; use $x:pat_param for single patterns

When to use tt: When you need to forward tokens to another macro without the parser constraining them. $($args:tt)* is the “accept everything” pattern (used by println!, format!, vec!).

Writing a Derive Macro with syn and quote

Derive macros live in a separate crate (proc-macro = true) and transform a token stream using syn (parse Rust) and quote (generate Rust):

# my_derive/Cargo.toml
[lib]
proc-macro = true

[dependencies]
syn = { version = "2", features = ["full"] }
quote = "1"
proc-macro2 = "1"
#![allow(unused)]
fn main() {
// my_derive/src/lib.rs
use proc_macro::TokenStream;
use quote::quote;
use syn::{parse_macro_input, DeriveInput};

/// Derive macro that generates a `describe()` method
/// returning the struct name and field names.
#[proc_macro_derive(Describe)]
pub fn derive_describe(input: TokenStream) -> TokenStream {
    let input = parse_macro_input!(input as DeriveInput);
    let name = &input.ident;
    let name_str = name.to_string();

    // Extract field names (only for structs with named fields)
    let fields = match &input.data {
        syn::Data::Struct(data) => {
            data.fields.iter()
                .filter_map(|f| f.ident.as_ref())
                .map(|id| id.to_string())
                .collect::<Vec<_>>()
        }
        _ => vec![],
    };

    let field_list = fields.join(", ");

    let expanded = quote! {
        impl #name {
            pub fn describe() -> String {
                format!("{} {{ {} }}", #name_str, #field_list)
            }
        }
    };

    TokenStream::from(expanded)
}
}
// In the application crate:
use my_derive::Describe;

#[derive(Describe)]
struct SensorReading {
    sensor_id: u16,
    value: f64,
    timestamp: u64,
}

fn main() {
    println!("{}", SensorReading::describe());
    // "SensorReading { sensor_id, value, timestamp }"
}

The workflow: TokenStream (raw tokens) → syn::parse (AST) → inspect/transform → quote! (generate tokens) → TokenStream (back to compiler).

CrateRoleKey types
proc-macroCompiler interfaceTokenStream
synParse Rust source into ASTDeriveInput, ItemFn, Type
quoteGenerate Rust tokens from templatesquote!{}, #variable interpolation
proc-macro2Bridge between syn/quote and proc-macroTokenStream, Span

Practical tip: Start by studying the source of a simple derive macro like thiserror or derive_more before writing your own. The cargo expand command (via cargo-expand) shows what any macro expands to — invaluable for debugging.

Key Takeaways — Macros

  • macro_rules! for simple code generation; proc macros (syn + quote) for complex derives
  • Prefer generics/traits over macros when possible — macros are harder to debug and maintain
  • $crate ensures hygiene; tt munching enables recursive pattern matching

See also: Ch 2 — Traits for when traits/generics beat macros. Ch 13 — Testing for testing macro-generated code.

flowchart LR
    A["Source code"] --> B["macro_rules!<br>pattern matching"]
    A --> C["#[derive(MyMacro)]<br>proc macro"]

    B --> D["Token expansion"]
    C --> E["syn: parse AST"]
    E --> F["Transform"]
    F --> G["quote!: generate tokens"]
    G --> D

    D --> H["Compiled code"]

    style A fill:#e8f4f8,stroke:#2980b9,color:#000
    style B fill:#d4efdf,stroke:#27ae60,color:#000
    style C fill:#fdebd0,stroke:#e67e22,color:#000
    style D fill:#fef9e7,stroke:#f1c40f,color:#000
    style E fill:#fdebd0,stroke:#e67e22,color:#000
    style F fill:#fdebd0,stroke:#e67e22,color:#000
    style G fill:#fdebd0,stroke:#e67e22,color:#000
    style H fill:#d4efdf,stroke:#27ae60,color:#000

Exercise: Declarative Macro — map! ★ (~15 min)

Write a map! macro that creates a HashMap from key-value pairs:

let m = map! {
    "host" => "localhost",
    "port" => "8080",
};
assert_eq!(m.get("host"), Some(&"localhost"));

Requirements: support trailing comma and empty invocation map!{}.

🔑 Solution
macro_rules! map {
    () => { std::collections::HashMap::new() };
    ( $( $key:expr => $val:expr ),+ $(,)? ) => {{
        let mut m = std::collections::HashMap::new();
        $( m.insert($key, $val); )+
        m
    }};
}

fn main() {
    let config = map! {
        "host" => "localhost",
        "port" => "8080",
        "timeout" => "30",
    };
    assert_eq!(config.len(), 3);
    assert_eq!(config["host"], "localhost");

    let empty: std::collections::HashMap<String, String> = map!();
    assert!(empty.is_empty());

    let scores = map! { 1 => 100, 2 => 200 };
    assert_eq!(scores[&1], 100);
}

14. Testing and Benchmarking Patterns 🟢

What you’ll learn:

  • Rust’s three test tiers: unit, integration, and doc tests
  • Property-based testing with proptest for discovering edge cases
  • Benchmarking with criterion for reliable performance measurement
  • Mocking strategies without heavyweight frameworks

Unit Tests, Integration Tests, Doc Tests

Rust has three testing tiers built into the language:

#![allow(unused)]
fn main() {
// --- Unit tests: in the same file as the code ---
pub fn factorial(n: u64) -> u64 {
    (1..=n).product()
}

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_factorial_zero() {
        // (1..=0).product() returns 1 — the multiplication identity for empty ranges
        assert_eq!(factorial(0), 1);
    }

    #[test]
    fn test_factorial_five() {
        assert_eq!(factorial(5), 120);
    }

    #[test]
    #[cfg(debug_assertions)] // overflow checks are only enabled in debug mode
    #[should_panic(expected = "overflow")]
    fn test_factorial_overflow() {
        // ⚠️ This test only passes in debug mode (overflow checks enabled).
        // In release mode (`cargo test --release`), u64 arithmetic wraps
        // silently and no panic occurs. Use `checked_mul` or the
        // `overflow-checks = true` profile setting for release-mode safety.
        factorial(100); // Should panic on overflow
    }

    #[test]
    fn test_with_result() -> Result<(), Box<dyn std::error::Error>> {
        // Tests can return Result — ? works inside!
        let value: u64 = "42".parse()?;
        assert_eq!(value, 42);
        Ok(())
    }
}
}
#![allow(unused)]
fn main() {
// --- Integration tests: in tests/ directory ---
// tests/integration_test.rs
// These test your crate's PUBLIC API only

use my_crate::factorial;

#[test]
fn test_factorial_from_outside() {
    assert_eq!(factorial(10), 3_628_800);
}
}
#![allow(unused)]
fn main() {
// --- Doc tests: in documentation comments ---
/// Computes the factorial of `n`.
///
/// # Examples
///
/// ```
/// use my_crate::factorial;
/// assert_eq!(factorial(5), 120);
/// ```
///
/// # Panics
///
/// Panics if the result overflows `u64`.
///
/// ```should_panic
/// my_crate::factorial(100);
/// ```
pub fn factorial(n: u64) -> u64 {
    (1..=n).product()
}
// Doc tests are compiled and run by `cargo test` — they keep examples honest.
}

Test Fixtures and Setup

#![allow(unused)]
fn main() {
#[cfg(test)]
mod tests {
    use super::*;

    // Shared setup — create a helper function
    fn setup_database() -> TestDb {
        let db = TestDb::new_in_memory();
        db.run_migrations();
        db.seed_test_data();
        db
    }

    #[test]
    fn test_user_creation() {
        let db = setup_database();
        let user = db.create_user("Alice", "[email protected]").unwrap();
        assert_eq!(user.name, "Alice");
    }

    #[test]
    fn test_user_deletion() {
        let db = setup_database();
        db.create_user("Bob", "[email protected]").unwrap();
        assert!(db.delete_user("Bob").is_ok());
        assert!(db.get_user("Bob").is_none());
    }

    // Cleanup with Drop (RAII):
    struct TempDir {
        path: std::path::PathBuf,
    }

    impl TempDir {
        fn new() -> Self {
            // Cargo.toml: rand = "0.8"
            let path = std::env::temp_dir().join(format!("test_{}", rand::random::<u32>()));
            std::fs::create_dir_all(&path).unwrap();
            TempDir { path }
        }
    }

    impl Drop for TempDir {
        fn drop(&mut self) {
            let _ = std::fs::remove_dir_all(&self.path);
        }
    }

    #[test]
    fn test_file_operations() {
        let dir = TempDir::new(); // Created
        std::fs::write(dir.path.join("test.txt"), "hello").unwrap();
        assert!(dir.path.join("test.txt").exists());
    } // dir dropped here → temp directory cleaned up
}
}

Property-Based Testing (proptest)

Instead of testing specific values, test properties that should always hold:

#![allow(unused)]
fn main() {
// Cargo.toml: proptest = "1"
use proptest::prelude::*;

fn reverse(v: &[i32]) -> Vec<i32> {
    v.iter().rev().cloned().collect()
}

proptest! {
    #[test]
    fn test_reverse_twice_is_identity(v in prop::collection::vec(any::<i32>(), 0..100)) {
        // Property: reversing twice gives back the original
        assert_eq!(reverse(&reverse(&v)), v);
    }

    #[test]
    fn test_reverse_preserves_length(v in prop::collection::vec(any::<i32>(), 0..100)) {
        assert_eq!(reverse(&v).len(), v.len());
    }

    #[test]
    fn test_sort_is_idempotent(mut v in prop::collection::vec(any::<i32>(), 0..100)) {
        v.sort();
        let sorted_once = v.clone();
        v.sort();
        assert_eq!(v, sorted_once); // Sorting twice = sorting once
    }

    #[test]
    fn test_parse_roundtrip(x in any::<f64>().prop_filter("finite", |x| x.is_finite())) {
        // Property: formatting then parsing gives back the same value
        let s = format!("{x}");
        let parsed: f64 = s.parse().unwrap();
        prop_assert!((x - parsed).abs() < f64::EPSILON);
    }
}
}

When to use proptest: When you’re testing a function with a large input space and want confidence it works for edge cases you didn’t think of. proptest generates hundreds of random inputs and shrinks failures to the minimal reproducing case.

Benchmarking with criterion

#![allow(unused)]
fn main() {
// Cargo.toml:
// [dev-dependencies]
// criterion = { version = "0.5", features = ["html_reports"] }
//
// [[bench]]
// name = "my_benchmarks"
// harness = false

// benches/my_benchmarks.rs
use criterion::{criterion_group, criterion_main, Criterion, black_box};

fn fibonacci(n: u64) -> u64 {
    match n {
        0 | 1 => n,
        _ => fibonacci(n - 1) + fibonacci(n - 2),
    }
}

fn bench_fibonacci(c: &mut Criterion) {
    c.bench_function("fibonacci 20", |b| {
        b.iter(|| fibonacci(black_box(20)))
    });

    // Compare different implementations:
    let mut group = c.benchmark_group("fibonacci_compare");
    for size in [10, 15, 20, 25] {
        group.bench_with_input(
            criterion::BenchmarkId::from_parameter(size),
            &size,
            |b, &size| b.iter(|| fibonacci(black_box(size))),
        );
    }
    group.finish();
}

criterion_group!(benches, bench_fibonacci);
criterion_main!(benches);

// Run: cargo bench
// Produces HTML reports in target/criterion/
}

Mocking Strategies without Frameworks

Rust’s trait system provides natural dependency injection — no mocking framework required:

#![allow(unused)]
fn main() {
// Define behavior as a trait
trait Clock {
    fn now(&self) -> std::time::Instant;
}

trait HttpClient {
    fn get(&self, url: &str) -> Result<String, String>;
}

// Production implementations
struct RealClock;
impl Clock for RealClock {
    fn now(&self) -> std::time::Instant { std::time::Instant::now() }
}

// Service depends on abstractions
struct CacheService<C: Clock, H: HttpClient> {
    clock: C,
    client: H,
    ttl: std::time::Duration,
}

impl<C: Clock, H: HttpClient> CacheService<C, H> {
    fn fetch(&self, url: &str) -> Result<String, String> {
        // Uses self.clock and self.client — injectable
        self.client.get(url)
    }
}

// Test with mock implementations — no framework needed!
#[cfg(test)]
mod tests {
    use super::*;

    struct MockClock {
        fixed_time: std::time::Instant,
    }
    impl Clock for MockClock {
        fn now(&self) -> std::time::Instant { self.fixed_time }
    }

    struct MockHttpClient {
        response: String,
    }
    impl HttpClient for MockHttpClient {
        fn get(&self, _url: &str) -> Result<String, String> {
            Ok(self.response.clone())
        }
    }

    #[test]
    fn test_cache_service() {
        let service = CacheService {
            clock: MockClock { fixed_time: std::time::Instant::now() },
            client: MockHttpClient { response: "cached data".into() },
            ttl: std::time::Duration::from_secs(300),
        };

        assert_eq!(service.fetch("http://example.com").unwrap(), "cached data");
    }
}
}

Test philosophy: Prefer real dependencies in integration tests, trait-based mocks in unit tests. Avoid mocking frameworks unless your dependency graph is complex — Rust’s trait generics handle most cases naturally.

Key Takeaways — Testing

  • Doc tests (///) double as documentation and regression tests — they’re compiled and run
  • proptest generates random inputs to find edge cases you’d never write manually
  • criterion provides statistically rigorous benchmarks with HTML reports
  • Mock via trait generics + test doubles, not mock frameworks

See also: Ch 12 — Macros for testing macro-generated code. Ch 14 — API Design for how module layout affects test organization.


Exercise: Property-Based Testing with proptest ★★ (~25 min)

Write a SortedVec<T: Ord> wrapper that maintains a sorted invariant. Use proptest to verify that:

  1. After any sequence of insertions, the internal vec is always sorted
  2. contains() agrees with the stdlib Vec::contains()
  3. The length equals the number of insertions
🔑 Solution
#[derive(Debug)]
struct SortedVec<T: Ord> {
    inner: Vec<T>,
}

impl<T: Ord> SortedVec<T> {
    fn new() -> Self { SortedVec { inner: Vec::new() } }

    fn insert(&mut self, value: T) {
        let pos = self.inner.binary_search(&value).unwrap_or_else(|p| p);
        self.inner.insert(pos, value);
    }

    fn contains(&self, value: &T) -> bool {
        self.inner.binary_search(value).is_ok()
    }

    fn len(&self) -> usize { self.inner.len() }
    fn as_slice(&self) -> &[T] { &self.inner }
}

#[cfg(test)]
mod tests {
    use super::*;
    use proptest::prelude::*;

    proptest! {
        #[test]
        fn always_sorted(values in proptest::collection::vec(-1000i32..1000, 0..100)) {
            let mut sv = SortedVec::new();
            for v in &values {
                sv.insert(*v);
            }
            for w in sv.as_slice().windows(2) {
                prop_assert!(w[0] <= w[1]);
            }
            prop_assert_eq!(sv.len(), values.len());
        }

        #[test]
        fn contains_matches_stdlib(values in proptest::collection::vec(0i32..50, 1..30)) {
            let mut sv = SortedVec::new();
            for v in &values {
                sv.insert(*v);
            }
            for v in &values {
                prop_assert!(sv.contains(v));
            }
            prop_assert!(!sv.contains(&9999));
        }
    }
}

15. Crate Architecture and API Design 🟡

What you’ll learn:

  • Module layout conventions and re-export strategies
  • The public API design checklist for polished crates
  • Ergonomic parameter patterns: impl Into, AsRef, Cow
  • “Parse, don’t validate” with TryFrom and validated types
  • Feature flags, conditional compilation, and workspace organization

Module Layout Conventions

my_crate/
├── Cargo.toml
├── src/
│   ├── lib.rs          # Crate root — re-exports and public API
│   ├── config.rs       # Feature module
│   ├── parser/         # Complex module with sub-modules
│   │   ├── mod.rs      # or parser.rs at parent level (Rust 2018+)
│   │   ├── lexer.rs
│   │   └── ast.rs
│   ├── error.rs        # Error types
│   └── utils.rs        # Internal helpers (pub(crate))
├── tests/
│   └── integration.rs  # Integration tests
├── benches/
│   └── perf.rs         # Benchmarks
└── examples/
    └── basic.rs        # cargo run --example basic
#![allow(unused)]
fn main() {
// lib.rs — curate your public API with re-exports:
mod config;
mod error;
mod parser;
mod utils;

// Re-export what users need:
pub use config::Config;
pub use error::Error;
pub use parser::Parser;

// Public types are at the crate root — users write:
// use my_crate::Config;
// NOT: use my_crate::config::Config;
}

Visibility modifiers:

ModifierVisible To
pubEveryone
pub(crate)This crate only
pub(super)Parent module
pub(in path)Specific ancestor module
(none)Current module and its children

Public API Design Checklist

  1. Accept references, return owned — fn process(input: &str) -> String
  2. Use impl Trait for parameters — fn read(r: impl Read) instead of fn read<R: Read>(r: R) for cleaner signatures
  3. Return Result, not panic! — let callers decide how to handle errors
  4. Implement standard traits — Debug, Display, Clone, Default, From/Into
  5. Make invalid states unrepresentable — use type states and newtypes
  6. Follow the builder pattern for complex configuration — with type-state if fields are required
  7. Seal traits you don’t want users to implement — pub trait Sealed: private::Sealed {}
  8. Mark types and functions #[must_use] — prevents silent discard of important Results, guards, or values. Apply to any type where ignoring the return value is almost certainly a bug:
    #![allow(unused)]
    fn main() {
    #[must_use = "dropping the guard immediately releases the lock"]
    pub struct LockGuard<'a, T> { /* ... */ }
    
    #[must_use]
    pub fn validate(input: &str) -> Result<ValidInput, ValidationError> { /* ... */ }
    }
#![allow(unused)]
fn main() {
// Sealed trait pattern — users can use but not implement:
mod private {
    pub trait Sealed {}
}

pub trait DatabaseDriver: private::Sealed {
    fn connect(&self, url: &str) -> Connection;
}

// Only types in THIS crate can implement Sealed → only we can implement DatabaseDriver
pub struct PostgresDriver;
impl private::Sealed for PostgresDriver {}
impl DatabaseDriver for PostgresDriver {
    fn connect(&self, url: &str) -> Connection { /* ... */ }
}
}

#[non_exhaustive] — mark public enums and structs so that adding variants or fields is not a breaking change. Downstream crates must use a wildcard arm (_ =>) in match statements, and cannot construct the type with struct literal syntax:

#![allow(unused)]
fn main() {
#[non_exhaustive]
pub enum DiagError {
    Timeout,
    HardwareFault,
    // Adding a new variant in a future release is NOT a semver break.
}
}

Ergonomic Parameter Patterns — impl Into, AsRef, Cow

One of Rust’s most impactful API patterns is accepting the most general type in function parameters, so callers don’t need repetitive .to_string(), &*s, or .as_ref() at every call site. This is the Rust-specific version of “be liberal in what you accept.”

impl Into<T> — Accept Anything Convertible

#![allow(unused)]
fn main() {
// ❌ Friction: callers must convert manually
fn connect(host: String, port: u16) -> Connection {
    // ...
}
connect("localhost".to_string(), 5432);  // Annoying .to_string()
connect(hostname.clone(), 5432);          // Unnecessary clone if we already have String

// ✅ Ergonomic: accept anything that converts to String
fn connect(host: impl Into<String>, port: u16) -> Connection {
    let host = host.into();  // Convert once, inside the function
    // ...
}
connect("localhost", 5432);     // &str — zero friction
connect(hostname, 5432);        // String — moved, no clone
}

This works because Rust’s From/Into trait pair provides blanket conversions. When you accept impl Into<T>, you’re saying: “give me anything that knows how to become a T.”

AsRef<T> — Borrow as a Reference

AsRef<T> is the borrowing counterpart to Into<T>. Use it when you only need to read the data, not take ownership:

#![allow(unused)]
fn main() {
use std::path::Path;

// ❌ Forces callers to convert to &Path
fn file_exists(path: &Path) -> bool {
    path.exists()
}
file_exists(Path::new("/tmp/test.txt"));  // Awkward

// ✅ Accept anything that can behave as a &Path
fn file_exists(path: impl AsRef<Path>) -> bool {
    path.as_ref().exists()
}
file_exists("/tmp/test.txt");                    // &str ✅
file_exists(String::from("/tmp/test.txt"));      // String ✅
file_exists(Path::new("/tmp/test.txt"));         // &Path ✅
file_exists(PathBuf::from("/tmp/test.txt"));     // PathBuf ✅

// Same pattern for string-like parameters:
fn log_message(msg: impl AsRef<str>) {
    println!("[LOG] {}", msg.as_ref());
}
log_message("hello");                    // &str ✅
log_message(String::from("hello"));      // String ✅
}

Cow<T> — Clone on Write

Cow<'a, T> (Clone on Write) delays allocation until mutation is needed. It holds either a borrowed &T or an owned T::Owned. This is perfect when most calls don’t need to modify the data:

#![allow(unused)]
fn main() {
use std::borrow::Cow;

/// Normalizes a diagnostic message — only allocates if changes are needed.
fn normalize_message(msg: &str) -> Cow<'_, str> {
    if msg.contains('\t') || msg.contains('\r') {
        // Must allocate — we need to modify the content
        Cow::Owned(msg.replace('\t', "    ").replace('\r', ""))
    } else {
        // No allocation — just borrow the original
        Cow::Borrowed(msg)
    }
}

// Most messages pass through without allocation:
let clean = normalize_message("All tests passed");          // Borrowed — free
let fixed = normalize_message("Error:\tfailed\r\n");        // Owned — allocated

// Cow<str> implements Deref<Target=str>, so it works like &str:
println!("{}", clean);
println!("{}", fixed.to_uppercase());
}

Quick Reference: Which to Use

Do you need ownership of the data inside the function?
├── YES → impl Into<T>
│         "Give me anything that can become a T"
└── NO  → Do you only need to read it?
     ├── YES → impl AsRef<T> or &T
     │         "Give me anything I can borrow as a &T"
     └── MAYBE (might need to modify sometimes?)
          └── Cow<'_, T>
              "Borrow if possible, clone only when you must"
PatternOwnershipAllocationWhen to use
&strBorrowedNeverSimple string params
impl AsRef<str>BorrowedNeverAccept String, &str, etc. — read only
impl Into<String>OwnedOn conversionAccept &str, String — will store/own
Cow<'_, str>EitherOnly if modifiedProcessing that usually doesn’t modify
&[u8] / impl AsRef<[u8]>BorrowedNeverByte-oriented APIs

Borrow<T> vs AsRef<T>: Both provide &T, but Borrow<T> additionally guarantees that Eq, Ord, and Hash are consistent between the original and borrowed form. This is why HashMap<String, V>::get() accepts &Q where String: Borrow<Q> — not AsRef. Use Borrow when the borrowed form is used as a lookup key; use AsRef for general “give me a reference” parameters.

Composing Conversions in APIs

#![allow(unused)]
fn main() {
/// A well-designed diagnostic API using ergonomic parameters:
pub struct DiagRunner {
    name: String,
    config_path: PathBuf,
    results: HashMap<String, TestResult>,
}

impl DiagRunner {
    /// Accept any string-like type for name, any path-like type for config.
    pub fn new(
        name: impl Into<String>,
        config_path: impl Into<PathBuf>,
    ) -> Self {
        DiagRunner {
            name: name.into(),
            config_path: config_path.into(),
        }
    }

    /// Accept any AsRef<str> for read-only lookup.
    pub fn get_result(&self, test_name: impl AsRef<str>) -> Option<&TestResult> {
        self.results.get(test_name.as_ref())
    }
}

// All of these work with zero caller friction:
let runner = DiagRunner::new("GPU Diag", "/etc/diag_tool/config.json");
let runner = DiagRunner::new(format!("Diag-{}", node_id), config_path);
let runner = DiagRunner::new(name_string, path_buf);
}

Case Study: Designing a Public Crate API — Before & After

A real-world example of evolving a stringly-typed internal API into an ergonomic, type-safe public API. Consider a configuration parser crate:

Before (stringly-typed, easy to misuse):

#![allow(unused)]
fn main() {
// ❌ All parameters are strings — no compile-time validation
pub fn parse_config(path: &str, format: &str, strict: bool) -> Result<Config, String> {
    // What formats are valid? "json"? "JSON"? "Json"?
    // Is path a file path or URL?
    // What does "strict" even mean?
    todo!()
}
}

After (type-safe, self-documenting):

#![allow(unused)]
fn main() {
use std::path::Path;

/// Supported configuration formats.
#[derive(Debug, Clone, Copy)]
#[non_exhaustive]  // Adding formats won't break downstream
pub enum Format {
    Json,
    Toml,
    Yaml,
}

/// Controls parsing strictness.
#[derive(Debug, Clone, Copy, Default)]
pub enum Strictness {
    /// Reject unknown fields (default for libraries)
    #[default]
    Strict,
    /// Ignore unknown fields (useful for forward-compatible configs)
    Lenient,
}

pub fn parse_config(
    path: &Path,          // Type-enforced: must be a filesystem path
    format: Format,       // Enum: impossible to pass invalid format
    strictness: Strictness,  // Named alternatives, not a bare bool
) -> Result<Config, ConfigError> {
    todo!()
}
}

What improved:

AspectBeforeAfter
Format validationRuntime string comparisonCompile-time enum
Path typeRaw &str (could be anything)&Path (filesystem-specific)
StrictnessMystery boolSelf-documenting enum
Error typeString (opaque)ConfigError (structured)
ExtensibilityBreaking changes#[non_exhaustive]

Rule of thumb: If you find yourself writing a match on string values, consider replacing the parameter with an enum. If a parameter is a boolean that isn’t obvious from context, use a two-variant enum instead.


Parse Don’t Validate — TryFrom and Validated Types

“Parse, don’t validate” is a principle that says: don’t check data and then pass around the raw unchecked form — instead, parse it into a type that can only exist if the data is valid. Rust’s TryFrom trait is the standard tool for this.

The Problem: Validation Without Enforcement

#![allow(unused)]
fn main() {
// ❌ Validate-then-use: nothing prevents using an invalid value after the check
fn process_port(port: u16) {
    if port == 0 || port > 65535 {
        panic!("Invalid port");           // We checked, but...
    }
    start_server(port);                    // What if someone calls start_server(0) directly?
}

// ❌ Stringly-typed: an email is just a String — any garbage gets through
fn send_email(to: String, body: String) {
    // Is `to` actually a valid email? We don't know.
    // Someone could pass "not-an-email" and we only find out at the SMTP server.
}
}

The Solution: Parse Into Validated Newtypes with TryFrom

use std::convert::TryFrom;
use std::fmt;

/// A validated TCP port number (1–65535).
/// If you have a `Port`, it is guaranteed valid.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct Port(u16);

impl TryFrom<u16> for Port {
    type Error = PortError;

    fn try_from(value: u16) -> Result<Self, Self::Error> {
        if value == 0 {
            Err(PortError::Zero)
        } else {
            Ok(Port(value))
        }
    }
}

impl Port {
    pub fn get(&self) -> u16 { self.0 }
}

#[derive(Debug)]
pub enum PortError {
    Zero,
    InvalidFormat,
}

impl fmt::Display for PortError {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        match self {
            PortError::Zero => write!(f, "port must be non-zero"),
            PortError::InvalidFormat => write!(f, "invalid port format"),
        }
    }
}

impl std::error::Error for PortError {}

// Now the type system enforces validity:
fn start_server(port: Port) {
    // No validation needed — Port can only be constructed via TryFrom,
    // which already verified it's valid.
    println!("Listening on port {}", port.get());
}

// Usage:
fn main() -> Result<(), Box<dyn std::error::Error>> {
    let port = Port::try_from(8080)?;   // ✅ Validated once at the boundary
    start_server(port);                  // No re-validation anywhere downstream

    let bad = Port::try_from(0);         // ❌ Err(PortError::Zero)
    Ok(())
}

Real-World Example: Validated IPMI Address

#![allow(unused)]
fn main() {
/// A validated IPMI slave address (0x20–0xFE, even only).
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct IpmiAddr(u8);

#[derive(Debug)]
pub enum IpmiAddrError {
    Odd(u8),
    OutOfRange(u8),
}

impl fmt::Display for IpmiAddrError {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        match self {
            IpmiAddrError::Odd(v) => write!(f, "IPMI address 0x{v:02X} must be even"),
            IpmiAddrError::OutOfRange(v) => {
                write!(f, "IPMI address 0x{v:02X} out of range (0x20..=0xFE)")
            }
        }
    }
}

impl TryFrom<u8> for IpmiAddr {
    type Error = IpmiAddrError;

    fn try_from(value: u8) -> Result<Self, Self::Error> {
        if value % 2 != 0 {
            Err(IpmiAddrError::Odd(value))
        } else if value < 0x20 || value > 0xFE {
            Err(IpmiAddrError::OutOfRange(value))
        } else {
            Ok(IpmiAddr(value))
        }
    }
}

impl IpmiAddr {
    pub fn get(&self) -> u8 { self.0 }
}

// Downstream code never needs to re-check:
fn send_ipmi_command(addr: IpmiAddr, cmd: u8, data: &[u8]) -> Result<Vec<u8>, IpmiError> {
    // addr.get() is guaranteed to be a valid, even IPMI address
    raw_ipmi_send(addr.get(), cmd, data)
}
}

Parsing Strings with FromStr

For types that are commonly parsed from text (CLI args, config files), implement FromStr:

#![allow(unused)]
fn main() {
use std::str::FromStr;

impl FromStr for Port {
    type Err = PortError;

    fn from_str(s: &str) -> Result<Self, Self::Err> {
        let n: u16 = s.parse().map_err(|_| PortError::InvalidFormat)?;
        Port::try_from(n)
    }
}

// Now works with .parse():
let port: Port = "8080".parse()?;   // Validates in one step

// And with clap CLI parsing:
// #[derive(Parser)]
// struct Args {
//     #[arg(short, long)]
//     port: Port,   // clap calls FromStr automatically
// }
}

TryFrom Chain for Complex Validation

#![allow(unused)]
fn main() {
// Stub types for this example — in production these would be in
// separate modules with their own TryFrom implementations.
}
#![allow(unused)]
fn main() {
struct Hostname(String);
impl TryFrom<String> for Hostname {
    type Error = String;
    fn try_from(s: String) -> Result<Self, String> { Ok(Hostname(s)) }
}
struct Timeout(u64);
impl TryFrom<u64> for Timeout {
    type Error = String;
    fn try_from(ms: u64) -> Result<Self, String> {
        if ms == 0 { Err("timeout must be > 0".into()) } else { Ok(Timeout(ms)) }
    }
}
struct RawConfig { host: String, port: u16, timeout_ms: u64 }
#[derive(Debug)]
enum ConfigError {
    InvalidHost(String),
    InvalidPort(PortError),
    InvalidTimeout(String),
}
impl From<std::io::Error> for ConfigError {
    fn from(e: std::io::Error) -> Self { ConfigError::InvalidHost(e.to_string()) }
}
impl From<serde_json::Error> for ConfigError {
    fn from(e: serde_json::Error) -> Self { ConfigError::InvalidHost(e.to_string()) }
}
/// A validated configuration that can only exist if all fields are valid.
pub struct ValidConfig {
    pub host: Hostname,
    pub port: Port,
    pub timeout_ms: Timeout,
}

impl TryFrom<RawConfig> for ValidConfig {
    type Error = ConfigError;

    fn try_from(raw: RawConfig) -> Result<Self, Self::Error> {
        Ok(ValidConfig {
            host: Hostname::try_from(raw.host)
                .map_err(ConfigError::InvalidHost)?,
            port: Port::try_from(raw.port)
                .map_err(ConfigError::InvalidPort)?,
            timeout_ms: Timeout::try_from(raw.timeout_ms)
                .map_err(ConfigError::InvalidTimeout)?,
        })
    }
}

// Parse once at the boundary, use the validated type everywhere:
fn load_config(path: &str) -> Result<ValidConfig, ConfigError> {
    let raw: RawConfig = serde_json::from_str(&std::fs::read_to_string(path)?)?;
    ValidConfig::try_from(raw)  // All validation happens here
}
}

Summary: Validate vs Parse

ApproachData checked?Compiler enforces validity?Re-validation needed?
Runtime checks (if/assert)✅❌Every function boundary
Validated newtype + TryFrom✅✅Never — type is proof

The rule: parse at the boundary, use validated types everywhere inside. Raw strings, integers, and byte slices enter your system, get parsed into validated types via TryFrom/FromStr, and from that point forward the type system guarantees they’re valid.

Feature Flags and Conditional Compilation

# Cargo.toml
[features]
default = ["json"]          # Enabled by default
json = ["dep:serde_json"]   # Enables JSON support
xml = ["dep:quick-xml"]     # Enables XML support
full = ["json", "xml"]      # Meta-feature: enables all

[dependencies]
serde = "1"
serde_json = { version = "1", optional = true }
quick-xml = { version = "0.31", optional = true }
#![allow(unused)]
fn main() {
// Conditional compilation based on features:
#[cfg(feature = "json")]
pub fn to_json<T: serde::Serialize>(value: &T) -> String {
    serde_json::to_string(value).unwrap()
}

#[cfg(feature = "xml")]
pub fn to_xml<T: serde::Serialize>(value: &T) -> String {
    quick_xml::se::to_string(value).unwrap()
}

// Compile error if a required feature isn't enabled:
#[cfg(not(any(feature = "json", feature = "xml")))]
compile_error!("At least one format feature (json, xml) must be enabled");
}

Best practices:

  • Keep default features minimal — users can opt in
  • Use dep: syntax (Rust 1.60+) for optional dependencies to avoid creating implicit features
  • Document features in your README and crate-level docs

Workspace Organization

For large projects, use a Cargo workspace to share dependencies and build artifacts:

# Root Cargo.toml
[workspace]
members = [
    "core",         # Shared types and traits
    "parser",       # Parsing library
    "server",       # Binary — the main application
    "client",       # Client library
    "cli",          # CLI binary
]

# Shared dependency versions:
[workspace.dependencies]
serde = { version = "1", features = ["derive"] }
tokio = { version = "1", features = ["full"] }
tracing = "0.1"

# In each member's Cargo.toml:
# [dependencies]
# serde = { workspace = true }

Benefits:

  • Single Cargo.lock — all crates use the same dependency versions
  • cargo test --workspace runs all tests
  • Shared build cache — compiling one crate benefits all
  • Clean dependency boundaries between components

.cargo/config.toml: Project-Level Configuration

The .cargo/config.toml file (at the workspace root or in $HOME/.cargo/) customizes Cargo behavior without modifying Cargo.toml:

# .cargo/config.toml

# Default target for this workspace
[build]
target = "x86_64-unknown-linux-gnu"

# Custom runner — e.g., run via QEMU for cross-compiled binaries
[target.aarch64-unknown-linux-gnu]
runner = "qemu-aarch64-static"
linker = "aarch64-linux-gnu-gcc"

# Cargo aliases — custom shortcut commands
[alias]
xt = "test --workspace --release"        # cargo xt = run all tests in release
ci = "clippy --workspace -- -D warnings" # cargo ci = lint with errors on warnings
cov = "llvm-cov --workspace"             # cargo cov = coverage (requires cargo-llvm-cov)

# Environment variables for build scripts
[env]
IPMI_LIB_PATH = "/usr/lib/bmc"

# Use a custom registry (for internal packages)
# [registries.internal]
# index = "https://gitlab.internal/crates/index"

Common configuration patterns:

SettingPurposeExample
[build] targetDefault compilation targetx86_64-unknown-linux-musl for static builds
[target.X] runnerHow to run the binary"qemu-aarch64-static" for cross-compiled
[target.X] linkerWhich linker to use"aarch64-linux-gnu-gcc"
[alias]Custom cargo subcommandsxt = "test --workspace"
[env]Build-time environment variablesLibrary paths, feature toggles
[net] offlinePrevent network accesstrue for air-gapped builds

Compile-Time Environment Variables: env!() and option_env!()

Rust can embed environment variables into the binary at compile time — useful for version strings, build metadata, and configuration:

#![allow(unused)]
fn main() {
// env!() — panics at compile time if the variable is missing
const VERSION: &str = env!("CARGO_PKG_VERSION"); // "0.1.0" from Cargo.toml
const PKG_NAME: &str = env!("CARGO_PKG_NAME");   // Crate name from Cargo.toml

// option_env!() — returns Option<&str>, doesn't panic if missing
const BUILD_SHA: Option<&str> = option_env!("GIT_SHA");
const BUILD_TIME: Option<&str> = option_env!("BUILD_TIMESTAMP");

fn print_version() {
    println!("{PKG_NAME} v{VERSION}");
    if let Some(sha) = BUILD_SHA {
        println!("  commit: {sha}");
    }
    if let Some(time) = BUILD_TIME {
        println!("  built:  {time}");
    }
}
}

Cargo automatically sets many useful environment variables:

VariableValueUse case
CARGO_PKG_VERSION"1.2.3"Version reporting
CARGO_PKG_NAME"diag_tool"Binary identification
CARGO_PKG_AUTHORSFrom Cargo.tomlAbout/help text
CARGO_MANIFEST_DIRAbsolute path to Cargo.tomlLocating test data files
OUT_DIRBuild output directorybuild.rs code generation target
TARGETTarget triplePlatform-specific logic in build.rs

You can set custom env vars from build.rs:

// build.rs
fn main() {
    println!("cargo::rustc-env=GIT_SHA={}", git_sha());
    println!("cargo::rustc-env=BUILD_TIMESTAMP={}", timestamp());
}

cfg_attr: Conditional Attributes

cfg_attr applies an attribute only when a condition is true. This is more targeted than #[cfg()], which includes/excludes entire items:

#![allow(unused)]
fn main() {
// Derive Serialize only when the "serde" feature is enabled:
#[cfg_attr(feature = "serde", derive(serde::Serialize, serde::Deserialize))]
#[derive(Debug, Clone)]
pub struct DiagResult {
    pub fc: u32,
    pub passed: bool,
    pub message: String,
}
// Without "serde" feature: no serde dependency needed at all
// With "serde" feature: DiagResult is serializable

// Conditional attribute for testing:
#[cfg_attr(test, derive(PartialEq))]  // Only derive PartialEq in test builds
pub struct LargeStruct { /* ... */ }

// Platform-specific function attributes:
#[cfg_attr(target_os = "linux", link_name = "ioctl")]
#[cfg_attr(target_os = "freebsd", link_name = "__ioctl")]
extern "C" fn platform_ioctl(fd: i32, request: u64) -> i32;
}
PatternWhat it does
#[cfg(feature = "x")]Include/exclude the entire item
#[cfg_attr(feature = "x", derive(Foo))]Add derive(Foo) only when feature “x” is on
#[cfg_attr(test, allow(unused))]Suppress warnings only in test builds
#[cfg_attr(doc, doc = "...")]Documentation visible only in cargo doc

cargo deny and cargo audit: Supply-Chain Security

# Install security audit tools
cargo install cargo-deny
cargo install cargo-audit

# Check for known vulnerabilities in dependencies
cargo audit

# Comprehensive checks: licenses, bans, advisories, sources
cargo deny check

Configure cargo deny with a deny.toml at the workspace root:

# deny.toml
[advisories]
vulnerability = "deny"      # Fail on known vulnerabilities
unmaintained = "warn"        # Warn on unmaintained crates

[licenses]
allow = ["MIT", "Apache-2.0", "BSD-2-Clause", "BSD-3-Clause"]
deny = ["GPL-3.0"]          # Reject copyleft licenses

[bans]
multiple-versions = "warn"  # Warn if multiple versions of same crate
deny = [
    { name = "openssl" },   # Force use of rustls instead
]

[sources]
allow-git = []              # No git dependencies in production
ToolPurposeWhen to run
cargo auditCheck for known CVEs in dependenciesCI pipeline, pre-release
cargo deny checkLicenses, bans, advisories, sourcesCI pipeline
cargo deny check licensesLicense compliance onlyBefore open-sourcing
cargo deny check bansPrevent specific cratesEnforce architecture decisions

Doc Tests: Tests Inside Documentation

Rust doc comments (///) can contain code blocks that are compiled and run as tests:

#![allow(unused)]
fn main() {
/// Parses a diagnostic fault code from a string.
///
/// # Examples
///
/// ```
/// use my_crate::parse_fc;
///
/// let fc = parse_fc("FC:12345").unwrap();
/// assert_eq!(fc, 12345);
/// ```
///
/// Invalid input returns an error:
///
/// ```
/// use my_crate::parse_fc;
///
/// assert!(parse_fc("not-a-fc").is_err());
/// ```
pub fn parse_fc(input: &str) -> Result<u32, ParseError> {
    input.strip_prefix("FC:")
        .ok_or(ParseError::MissingPrefix)?
        .parse()
        .map_err(ParseError::InvalidNumber)
}
}
cargo test --doc  # Run only doc tests
cargo test        # Runs unit + integration + doc tests

Module-level documentation uses //! at the top of a file:

#![allow(unused)]
fn main() {
//! # Diagnostic Framework
//!
//! This crate provides the core diagnostic execution engine.
//! It supports running diagnostic tests, collecting results,
//! and reporting to the BMC via IPMI.
//!
//! ## Quick Start
//!
//! ```no_run
//! use diag_framework::Framework;
//!
//! let mut fw = Framework::new("config.json")?;
//! fw.run_all_tests()?;
//! ```
}

Benchmarking with Criterion

Full coverage: See the Benchmarking with criterion section in Chapter 13 (Testing and Benchmarking Patterns) for complete criterion setup, API examples, and a comparison table vs cargo bench. Below is a quick-reference for architecture-specific usage.

When benchmarking your crate’s public API, place benchmarks in benches/ and keep them focused on the hot path — typically parsers, serializers, or validation boundaries:

cargo bench                  # Run all benchmarks
cargo bench -- parse_config  # Run specific benchmark
# Results in target/criterion/ with HTML reports

Key Takeaways — Architecture & API Design

  • Accept the most general type (impl Into, impl AsRef, Cow); return the most specific
  • Parse Don’t Validate: use TryFrom to create types that are valid by construction
  • #[non_exhaustive] on public enums prevents breaking changes when adding variants
  • #[must_use] catches silent discards of important values

See also: Ch 9 — Error Handling for error type design in public APIs. Ch 13 — Testing for testing your crate’s public API.


Exercise: Crate API Refactoring ★★ (~30 min)

Refactor the following “stringly-typed” API into one that uses TryFrom, newtypes, and builder pattern:

// BEFORE: Easy to misuse
fn create_server(host: &str, port: &str, max_conn: &str) -> Server { ... }

Design a ServerConfig with validated types Host, Port (1–65535), and MaxConnections (1–10000) that reject invalid values at parse time.

🔑 Solution
#[derive(Debug, Clone)]
struct Host(String);

impl TryFrom<&str> for Host {
    type Error = String;
    fn try_from(s: &str) -> Result<Self, String> {
        if s.is_empty() { return Err("host cannot be empty".into()); }
        if s.contains(' ') { return Err("host cannot contain spaces".into()); }
        Ok(Host(s.to_string()))
    }
}

#[derive(Debug, Clone, Copy)]
struct Port(u16);

impl TryFrom<u16> for Port {
    type Error = String;
    fn try_from(p: u16) -> Result<Self, String> {
        if p == 0 { return Err("port must be >= 1".into()); }
        Ok(Port(p))
    }
}

#[derive(Debug, Clone, Copy)]
struct MaxConnections(u32);

impl TryFrom<u32> for MaxConnections {
    type Error = String;
    fn try_from(n: u32) -> Result<Self, String> {
        if n == 0 || n > 10_000 {
            return Err(format!("max_connections must be 1–10000, got {n}"));
        }
        Ok(MaxConnections(n))
    }
}

#[derive(Debug)]
struct ServerConfig {
    host: Host,
    port: Port,
    max_connections: MaxConnections,
}

impl ServerConfig {
    fn new(host: Host, port: Port, max_connections: MaxConnections) -> Self {
        ServerConfig { host, port, max_connections }
    }
}

fn main() {
    let config = ServerConfig::new(
        Host::try_from("localhost").unwrap(),
        Port::try_from(8080).unwrap(),
        MaxConnections::try_from(100).unwrap(),
    );
    println!("{config:?}");

    // Invalid values caught at parse time:
    assert!(Host::try_from("").is_err());
    assert!(Port::try_from(0).is_err());
    assert!(MaxConnections::try_from(99999).is_err());
}

16. Async/Await Essentials 🔴

What you’ll learn:

  • How Rust’s Future trait differs from Go’s goroutines and Python’s asyncio
  • Tokio quick-start: spawning tasks, join!, and runtime configuration
  • Common async pitfalls and how to fix them
  • When to offload blocking work with spawn_blocking

Futures, Runtimes, and async fn

Rust’s async model is fundamentally different from Go’s goroutines or Python’s asyncio. Understanding three concepts is enough to get started:

  1. A Future is a lazy state machine — calling async fn doesn’t execute anything; it returns a Future that must be polled.
  2. You need a runtime to poll futures — tokio, async-std, or smol. The standard library defines Future but provides no runtime.
  3. async fn is sugar — the compiler transforms it into a state machine that implements Future.
#![allow(unused)]
fn main() {
// A Future is just a trait:
pub trait Future {
    type Output;
    fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output>;
}

// async fn desugars to:
// fn fetch_data(url: &str) -> impl Future<Output = Result<Vec<u8>, Error>>
async fn fetch_data(url: &str) -> Result<Vec<u8>, reqwest::Error> {
    let response = reqwest::get(url).await?;  // .await yields until ready
    let bytes = response.bytes().await?;
    Ok(bytes.to_vec())
}
}

Tokio Quick Start

# Cargo.toml
[dependencies]
tokio = { version = "1", features = ["full"] }
use tokio::time::{sleep, Duration};
use tokio::task;

#[tokio::main]
async fn main() {
    // Spawn concurrent tasks (like lightweight threads):
    let handle_a = task::spawn(async {
        sleep(Duration::from_millis(100)).await;
        "task A done"
    });

    let handle_b = task::spawn(async {
        sleep(Duration::from_millis(50)).await;
        "task B done"
    });

    // .await both — they run concurrently, not sequentially:
    let (a, b) = tokio::join!(handle_a, handle_b);
    println!("{}, {}", a.unwrap(), b.unwrap());
}

Async Common Pitfalls

PitfallWhy It HappensFix
Blocking in asyncstd::thread::sleep or CPU work blocks the executorUse tokio::task::spawn_blocking or rayon
Send bound errorsFuture held across .await contains !Send type (e.g., Rc, MutexGuard)Restructure to drop non-Send values before .await
Future not polledCalling async fn without .await or spawning — nothing happensAlways .await or tokio::spawn the returned future
Holding MutexGuard across .awaitstd::sync::MutexGuard is !Send; async tasks may resume on different threadUse tokio::sync::Mutex or drop the guard before .await
Accidental sequential executionlet a = foo().await; let b = bar().await; runs sequentiallyUse tokio::join! or tokio::spawn for concurrency
#![allow(unused)]
fn main() {
// ❌ Blocking the async executor:
async fn bad() {
    std::thread::sleep(std::time::Duration::from_secs(5)); // Blocks entire thread!
}

// ✅ Offload blocking work:
async fn good() {
    tokio::task::spawn_blocking(|| {
        std::thread::sleep(std::time::Duration::from_secs(5)); // Runs on blocking pool
    }).await.unwrap();
}
}

Comprehensive async coverage: For Stream, select!, cancellation safety, structured concurrency, and tower middleware, see our dedicated Async Rust Training guide. This section covers just enough to read and write basic async code.

Spawning and Structured Concurrency

Tokio’s spawn creates a new asynchronous task — similar to thread::spawn but much lighter:

use tokio::task;
use tokio::time::{sleep, Duration};

#[tokio::main]
async fn main() {
    // Spawn three concurrent tasks
    let h1 = task::spawn(async {
        sleep(Duration::from_millis(200)).await;
        "fetched user profile"
    });

    let h2 = task::spawn(async {
        sleep(Duration::from_millis(100)).await;
        "fetched order history"
    });

    let h3 = task::spawn(async {
        sleep(Duration::from_millis(150)).await;
        "fetched recommendations"
    });

    // Wait for all three concurrently (not sequentially!)
    let (r1, r2, r3) = tokio::join!(h1, h2, h3);
    println!("{}", r1.unwrap());
    println!("{}", r2.unwrap());
    println!("{}", r3.unwrap());
}

join! vs try_join! vs select!:

MacroBehaviorUse when
join!Waits for ALL futuresAll tasks must complete
try_join!Waits for all, short-circuits on first ErrTasks return Result
select!Returns when FIRST future completesTimeouts, cancellation
use tokio::time::{timeout, Duration};

async fn fetch_with_timeout() -> Result<String, Box<dyn std::error::Error>> {
    let result = timeout(Duration::from_secs(5), async {
        // Simulate slow network call
        tokio::time::sleep(Duration::from_millis(100)).await;
        Ok::<_, Box<dyn std::error::Error>>("data".to_string())
    }).await??; // First ? unwraps Elapsed, second ? unwraps inner Result

    Ok(result)
}

Send Bounds and Why Futures Must Be Send

When you tokio::spawn a future, it may resume on a different OS thread. This means the future must be Send. Common pitfalls:

use std::rc::Rc;

async fn not_send() {
    let rc = Rc::new(42); // Rc is !Send
    tokio::time::sleep(std::time::Duration::from_millis(10)).await;
    println!("{}", rc); // rc is held across .await — future is !Send
}

// Fix 1: Drop before .await
async fn fixed_drop() {
    let data = {
        let rc = Rc::new(42);
        *rc // Copy the value out
    }; // rc dropped here
    tokio::time::sleep(std::time::Duration::from_millis(10)).await;
    println!("{}", data); // Just an i32, which is Send
}

// Fix 2: Use Arc instead of Rc
async fn fixed_arc() {
    let arc = std::sync::Arc::new(42); // Arc is Send
    tokio::time::sleep(std::time::Duration::from_millis(10)).await;
    println!("{}", arc); // ✅ Future is Send
}

Comprehensive async coverage: For Stream, select!, cancellation safety, structured concurrency, and tower middleware, see our dedicated Async Rust Training guide. This section covers just enough to read and write basic async code.

See also: Ch 5 — Channels for synchronous channels. Ch 6 — Concurrency for OS threads vs async tasks.

Key Takeaways — Async

  • async fn returns a lazy Future — nothing runs until you .await or spawn it
  • Use tokio::task::spawn_blocking for CPU-heavy or blocking work inside async contexts
  • Don’t hold std::sync::MutexGuard across .await — use tokio::sync::Mutex instead
  • Futures must be Send when spawned — drop !Send types before .await points

Exercise: Concurrent Fetcher with Timeout ★★ (~25 min)

Write an async function fetch_all that spawns three tokio::spawn tasks, each simulating a network call with tokio::time::sleep. Join all three with tokio::try_join! wrapped in tokio::time::timeout(Duration::from_secs(5), ...). Return Result<Vec<String>, ...> or an error if any task fails or the deadline expires.

🔑 Solution
use tokio::time::{sleep, timeout, Duration};

async fn fake_fetch(name: &'static str, delay_ms: u64) -> Result<String, String> {
    sleep(Duration::from_millis(delay_ms)).await;
    Ok(format!("{name}: OK"))
}

async fn fetch_all() -> Result<Vec<String>, Box<dyn std::error::Error>> {
    let deadline = Duration::from_secs(5);

    let (a, b, c) = timeout(deadline, async {
        let h1 = tokio::spawn(fake_fetch("svc-a", 100));
        let h2 = tokio::spawn(fake_fetch("svc-b", 200));
        let h3 = tokio::spawn(fake_fetch("svc-c", 150));
        tokio::try_join!(h1, h2, h3)
    })
    .await??;

    Ok(vec![a?, b?, c?])
}

#[tokio::main]
async fn main() {
    let results = fetch_all().await.unwrap();
    for r in &results {
        println!("{r}");
    }
}

Exercises

Exercise 1: Type-Safe State Machine ★★ (~30 min)

Build a traffic light state machine using the type-state pattern. The light must transition Red → Green → Yellow → Red and no other order should be possible.

🔑 Solution
use std::marker::PhantomData;

struct Red;
struct Green;
struct Yellow;

struct TrafficLight<State> {
    _state: PhantomData<State>,
}

impl TrafficLight<Red> {
    fn new() -> Self {
        println!("🔴 Red — STOP");
        TrafficLight { _state: PhantomData }
    }

    fn go(self) -> TrafficLight<Green> {
        println!("🟢 Green — GO");
        TrafficLight { _state: PhantomData }
    }
}

impl TrafficLight<Green> {
    fn caution(self) -> TrafficLight<Yellow> {
        println!("🟡 Yellow — CAUTION");
        TrafficLight { _state: PhantomData }
    }
}

impl TrafficLight<Yellow> {
    fn stop(self) -> TrafficLight<Red> {
        println!("🔴 Red — STOP");
        TrafficLight { _state: PhantomData }
    }
}

fn main() {
    let light = TrafficLight::new(); // Red
    let light = light.go();          // Green
    let light = light.caution();     // Yellow
    let light = light.stop();        // Red

    // light.caution(); // ❌ Compile error: no method `caution` on Red
    // TrafficLight::new().stop(); // ❌ Compile error: no method `stop` on Red
}

Key takeaway: Invalid transitions are compile errors, not runtime panics.


Exercise 2: Unit-of-Measure with PhantomData ★★ (~30 min)

Extend the unit-of-measure pattern from Ch4 to support:

  • Meters, Seconds, Kilograms
  • Addition of same units
  • Multiplication: Meters * Meters = SquareMeters
  • Division: Meters / Seconds = MetersPerSecond
🔑 Solution
use std::marker::PhantomData;
use std::ops::{Add, Mul, Div};

#[derive(Clone, Copy)]
struct Meters;
#[derive(Clone, Copy)]
struct Seconds;
#[derive(Clone, Copy)]
struct Kilograms;
#[derive(Clone, Copy)]
struct SquareMeters;
#[derive(Clone, Copy)]
struct MetersPerSecond;

#[derive(Debug, Clone, Copy)]
struct Qty<U> {
    value: f64,
    _unit: PhantomData<U>,
}

impl<U> Qty<U> {
    fn new(v: f64) -> Self { Qty { value: v, _unit: PhantomData } }
}

impl<U> Add for Qty<U> {
    type Output = Qty<U>;
    fn add(self, rhs: Self) -> Self::Output { Qty::new(self.value + rhs.value) }
}

impl Mul<Qty<Meters>> for Qty<Meters> {
    type Output = Qty<SquareMeters>;
    fn mul(self, rhs: Qty<Meters>) -> Qty<SquareMeters> {
        Qty::new(self.value * rhs.value)
    }
}

impl Div<Qty<Seconds>> for Qty<Meters> {
    type Output = Qty<MetersPerSecond>;
    fn div(self, rhs: Qty<Seconds>) -> Qty<MetersPerSecond> {
        Qty::new(self.value / rhs.value)
    }
}

fn main() {
    let width = Qty::<Meters>::new(5.0);
    let height = Qty::<Meters>::new(3.0);
    let area = width * height; // Qty<SquareMeters>
    println!("Area: {:.1} m²", area.value);

    let dist = Qty::<Meters>::new(100.0);
    let time = Qty::<Seconds>::new(9.58);
    let speed = dist / time;
    println!("Speed: {:.2} m/s", speed.value);

    let sum = width + height; // Same unit ✅
    println!("Sum: {:.1} m", sum.value);

    // let bad = width + time; // ❌ Compile error: can't add Meters + Seconds
}

Exercise 3: Channel-Based Worker Pool ★★★ (~45 min)

Build a worker pool using channels where:

  • A dispatcher sends Job structs through a channel
  • N workers consume jobs and send results back
  • Use crossbeam-channel (or std::sync::mpsc if crossbeam is unavailable)
🔑 Solution
use std::sync::mpsc;
use std::thread;

struct Job {
    id: u64,
    data: String,
}

struct JobResult {
    job_id: u64,
    output: String,
    worker_id: usize,
}

fn worker_pool(jobs: Vec<Job>, num_workers: usize) -> Vec<JobResult> {
    let (job_tx, job_rx) = mpsc::channel::<Job>();
    let (result_tx, result_rx) = mpsc::channel::<JobResult>();

    // Wrap receiver in Arc<Mutex> for sharing among workers
    let job_rx = std::sync::Arc::new(std::sync::Mutex::new(job_rx));

    // Spawn workers
    let mut handles = Vec::new();
    for worker_id in 0..num_workers {
        let job_rx = job_rx.clone();
        let result_tx = result_tx.clone();
        handles.push(thread::spawn(move || {
            loop {
                // Lock, receive, unlock — short critical section
                let job = {
                    let rx = job_rx.lock().unwrap();
                    rx.recv() // Blocks until a job or channel closes
                };
                match job {
                    Ok(job) => {
                        let output = format!("processed '{}' by worker {worker_id}", job.data);
                        result_tx.send(JobResult {
                            job_id: job.id,
                            output,
                            worker_id,
                        }).unwrap();
                    }
                    Err(_) => break, // Channel closed — exit
                }
            }
        }));
    }
    drop(result_tx); // Drop our copy so result channel closes when workers finish

    // Dispatch jobs
    let num_jobs = jobs.len();
    for job in jobs {
        job_tx.send(job).unwrap();
    }
    drop(job_tx); // Close the job channel — workers will exit after draining

    // Collect results
    let mut results = Vec::new();
    for result in result_rx {
        results.push(result);
    }
    assert_eq!(results.len(), num_jobs);

    for h in handles { h.join().unwrap(); }
    results
}

fn main() {
    let jobs: Vec<Job> = (0..20).map(|i| Job {
        id: i,
        data: format!("task-{i}"),
    }).collect();

    let results = worker_pool(jobs, 4);
    for r in &results {
        println!("[worker {}] job {}: {}", r.worker_id, r.job_id, r.output);
    }
}

Exercise 4: Higher-Order Combinator Pipeline ★★ (~25 min)

Create a Pipeline struct that chains transformations. It should support .pipe(f) to add a transformation and .execute(input) to run the full chain.

🔑 Solution
struct Pipeline<T> {
    transforms: Vec<Box<dyn Fn(T) -> T>>,
}

impl<T: 'static> Pipeline<T> {
    fn new() -> Self {
        Pipeline { transforms: Vec::new() }
    }

    fn pipe(mut self, f: impl Fn(T) -> T + 'static) -> Self {
        self.transforms.push(Box::new(f));
        self
    }

    fn execute(self, input: T) -> T {
        self.transforms.into_iter().fold(input, |val, f| f(val))
    }
}

fn main() {
    let result = Pipeline::new()
        .pipe(|s: String| s.trim().to_string())
        .pipe(|s| s.to_uppercase())
        .pipe(|s| format!(">>> {s} <<<"))
        .execute("  hello world  ".to_string());

    println!("{result}"); // >>> HELLO WORLD <<<

    // Numeric pipeline:
    let result = Pipeline::new()
        .pipe(|x: i32| x * 2)
        .pipe(|x| x + 10)
        .pipe(|x| x * x)
        .execute(5);

    println!("{result}"); // (5*2 + 10)^2 = 400
}

Bonus: Generic pipeline that changes type between stages would use a different design — each .pipe() returns a Pipeline with a different output type (this requires more advanced generic plumbing).


Exercise 5: Error Hierarchy with thiserror ★★ (~30 min)

Design an error type hierarchy for a file-processing application that can fail during I/O, parsing (JSON and CSV), and validation. Use thiserror and demonstrate ? propagation.

🔑 Solution
use thiserror::Error;

#[derive(Error, Debug)]
pub enum AppError {
    #[error("I/O error: {0}")]
    Io(#[from] std::io::Error),

    #[error("JSON parse error: {0}")]
    Json(#[from] serde_json::Error),

    #[error("CSV error at line {line}: {message}")]
    Csv { line: usize, message: String },

    #[error("validation error: {field} — {reason}")]
    Validation { field: String, reason: String },
}

fn read_file(path: &str) -> Result<String, AppError> {
    Ok(std::fs::read_to_string(path)?) // io::Error → AppError::Io via #[from]
}

fn parse_json(content: &str) -> Result<serde_json::Value, AppError> {
    Ok(serde_json::from_str(content)?) // serde_json::Error → AppError::Json
}

fn validate_name(value: &serde_json::Value) -> Result<String, AppError> {
    let name = value.get("name")
        .and_then(|v| v.as_str())
        .ok_or_else(|| AppError::Validation {
            field: "name".into(),
            reason: "must be a non-null string".into(),
        })?;

    if name.is_empty() {
        return Err(AppError::Validation {
            field: "name".into(),
            reason: "must not be empty".into(),
        });
    }

    Ok(name.to_string())
}

fn process_file(path: &str) -> Result<String, AppError> {
    let content = read_file(path)?;
    let json = parse_json(&content)?;
    let name = validate_name(&json)?;
    Ok(name)
}

fn main() {
    match process_file("config.json") {
        Ok(name) => println!("Name: {name}"),
        Err(e) => eprintln!("Error: {e}"),
    }
}

Exercise 6: Generic Trait with Associated Types ★★★ (~40 min)

Design a Repository<T> trait with associated Error and Id types. Implement it for an in-memory store and demonstrate compile-time type safety.

🔑 Solution
use std::collections::HashMap;

trait Repository {
    type Item;
    type Id;
    type Error;

    fn get(&self, id: &Self::Id) -> Result<Option<&Self::Item>, Self::Error>;
    fn insert(&mut self, item: Self::Item) -> Result<Self::Id, Self::Error>;
    fn delete(&mut self, id: &Self::Id) -> Result<bool, Self::Error>;
}

#[derive(Debug, Clone)]
struct User {
    name: String,
    email: String,
}

struct InMemoryUserRepo {
    data: HashMap<u64, User>,
    next_id: u64,
}

impl InMemoryUserRepo {
    fn new() -> Self {
        InMemoryUserRepo { data: HashMap::new(), next_id: 1 }
    }
}

// Error type is Infallible — in-memory ops never fail
impl Repository for InMemoryUserRepo {
    type Item = User;
    type Id = u64;
    type Error = std::convert::Infallible;

    fn get(&self, id: &u64) -> Result<Option<&User>, Self::Error> {
        Ok(self.data.get(id))
    }

    fn insert(&mut self, item: User) -> Result<u64, Self::Error> {
        let id = self.next_id;
        self.next_id += 1;
        self.data.insert(id, item);
        Ok(id)
    }

    fn delete(&mut self, id: &u64) -> Result<bool, Self::Error> {
        Ok(self.data.remove(id).is_some())
    }
}

// Generic function works with ANY repository:
fn create_and_fetch<R: Repository>(repo: &mut R, item: R::Item) -> Result<(), R::Error>
where
    R::Item: std::fmt::Debug,
    R::Id: std::fmt::Debug,
{
    let id = repo.insert(item)?;
    println!("Inserted with id: {id:?}");
    let retrieved = repo.get(&id)?;
    println!("Retrieved: {retrieved:?}");
    Ok(())
}

fn main() {
    let mut repo = InMemoryUserRepo::new();
    create_and_fetch(&mut repo, User {
        name: "Alice".into(),
        email: "[email protected]".into(),
    }).unwrap();
}

Exercise 7: Safe Wrapper around Unsafe (Ch11) ★★★ (~45 min)

Write a FixedVec<T, const N: usize> — a fixed-capacity, stack-allocated vector. Requirements:

  • push(&mut self, value: T) -> Result<(), T> returns Err(value) when full
  • pop(&mut self) -> Option<T> returns and removes the last element
  • as_slice(&self) -> &[T] borrows initialized elements
  • All public methods must be safe; all unsafe must be encapsulated with SAFETY: comments
  • Drop must clean up initialized elements

Hint: Use MaybeUninit<T> and [const { MaybeUninit::uninit() }; N].

🔑 Solution
use std::mem::MaybeUninit;

pub struct FixedVec<T, const N: usize> {
    data: [MaybeUninit<T>; N],
    len: usize,
}

impl<T, const N: usize> FixedVec<T, N> {
    pub fn new() -> Self {
        FixedVec {
            data: [const { MaybeUninit::uninit() }; N],
            len: 0,
        }
    }

    pub fn push(&mut self, value: T) -> Result<(), T> {
        if self.len >= N { return Err(value); }
        // SAFETY: len < N, so data[len] is within bounds.
        self.data[self.len] = MaybeUninit::new(value);
        self.len += 1;
        Ok(())
    }

    pub fn pop(&mut self) -> Option<T> {
        if self.len == 0 { return None; }
        self.len -= 1;
        // SAFETY: data[len] was initialized (len was > 0 before decrement).
        Some(unsafe { self.data[self.len].assume_init_read() })
    }

    pub fn as_slice(&self) -> &[T] {
        // SAFETY: data[0..len] are all initialized, and MaybeUninit<T>
        // has the same layout as T.
        unsafe { std::slice::from_raw_parts(self.data.as_ptr() as *const T, self.len) }
    }

    pub fn len(&self) -> usize { self.len }
    pub fn is_empty(&self) -> bool { self.len == 0 }
}

impl<T, const N: usize> Drop for FixedVec<T, N> {
    fn drop(&mut self) {
        // SAFETY: data[0..len] are initialized — drop each one.
        for i in 0..self.len {
            unsafe { self.data[i].assume_init_drop(); }
        }
    }
}

fn main() {
    let mut v = FixedVec::<String, 4>::new();
    v.push("hello".into()).unwrap();
    v.push("world".into()).unwrap();
    assert_eq!(v.as_slice(), &["hello", "world"]);
    assert_eq!(v.pop(), Some("world".into()));
    assert_eq!(v.len(), 1);
    // Drop cleans up remaining "hello"
}

Exercise 8: Declarative Macro — map! (Ch12) ★ (~15 min)

Write a map! macro that creates a HashMap from key-value pairs, similar to vec![]:

#![allow(unused)]
fn main() {
let m = map! {
    "host" => "localhost",
    "port" => "8080",
};
assert_eq!(m.get("host"), Some(&"localhost"));
assert_eq!(m.len(), 2);
}

Requirements:

  • Support trailing comma
  • Support empty invocation map!{}
  • Work with any types that implement Into<K> and Into<V> for maximum flexibility
🔑 Solution
macro_rules! map {
    // Empty case
    () => {
        std::collections::HashMap::new()
    };
    // One or more key => value pairs (trailing comma optional)
    ( $( $key:expr => $val:expr ),+ $(,)? ) => {{
        let mut m = std::collections::HashMap::new();
        $( m.insert($key, $val); )+
        m
    }};
}

fn main() {
    // Basic usage:
    let config = map! {
        "host" => "localhost",
        "port" => "8080",
        "timeout" => "30",
    };
    assert_eq!(config.len(), 3);
    assert_eq!(config["host"], "localhost");

    // Empty map:
    let empty: std::collections::HashMap<String, String> = map!();
    assert!(empty.is_empty());

    // Different types:
    let scores = map! {
        1 => 100,
        2 => 200,
    };
    assert_eq!(scores[&1], 100);
}

Exercise 9: Custom serde Deserialization (Ch10) ★★★ (~45 min)

Design a Duration wrapper that deserializes from human-readable strings like "30s", "5m", "2h" using a custom serde deserializer. The struct should also serialize back to the same format.

🔑 Solution
use serde::{Deserialize, Deserializer, Serialize, Serializer};
use std::fmt;

#[derive(Debug, Clone, PartialEq)]
struct HumanDuration(std::time::Duration);

impl HumanDuration {
    fn from_str(s: &str) -> Result<Self, String> {
        let s = s.trim();
        if s.is_empty() { return Err("empty duration string".into()); }

        let (num_str, suffix) = s.split_at(
            s.find(|c: char| !c.is_ascii_digit()).unwrap_or(s.len())
        );
        let value: u64 = num_str.parse()
            .map_err(|_| format!("invalid number: {num_str}"))?;

        let duration = match suffix {
            "s" | "sec"  => std::time::Duration::from_secs(value),
            "m" | "min"  => std::time::Duration::from_secs(value * 60),
            "h" | "hr"   => std::time::Duration::from_secs(value * 3600),
            "ms"         => std::time::Duration::from_millis(value),
            other        => return Err(format!("unknown suffix: {other}")),
        };
        Ok(HumanDuration(duration))
    }
}

impl fmt::Display for HumanDuration {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        let secs = self.0.as_secs();
        if secs == 0 {
            write!(f, "{}ms", self.0.as_millis())
        } else if secs % 3600 == 0 {
            write!(f, "{}h", secs / 3600)
        } else if secs % 60 == 0 {
            write!(f, "{}m", secs / 60)
        } else {
            write!(f, "{}s", secs)
        }
    }
}

impl Serialize for HumanDuration {
    fn serialize<S: Serializer>(&self, serializer: S) -> Result<S::Ok, S::Error> {
        serializer.serialize_str(&self.to_string())
    }
}

impl<'de> Deserialize<'de> for HumanDuration {
    fn deserialize<D: Deserializer<'de>>(deserializer: D) -> Result<Self, D::Error> {
        let s = String::deserialize(deserializer)?;
        HumanDuration::from_str(&s).map_err(serde::de::Error::custom)
    }
}

#[derive(Debug, Deserialize, Serialize)]
struct Config {
    timeout: HumanDuration,
    retry_interval: HumanDuration,
}

fn main() {
    let json = r#"{ "timeout": "30s", "retry_interval": "5m" }"#;
    let config: Config = serde_json::from_str(json).unwrap();

    assert_eq!(config.timeout.0, std::time::Duration::from_secs(30));
    assert_eq!(config.retry_interval.0, std::time::Duration::from_secs(300));

    // Round-trips correctly:
    let serialized = serde_json::to_string(&config).unwrap();
    assert!(serialized.contains("30s"));
    assert!(serialized.contains("5m"));
    println!("Config: {serialized}");
}

Exercise 10 — Concurrent Fetcher with Timeout ★★ (~25 min)

Write an async function fetch_all that spawns three tokio::spawn tasks, each simulating a network call with tokio::time::sleep. Join all three with tokio::try_join! wrapped in tokio::time::timeout(Duration::from_secs(5), ...). Return Result<Vec<String>, ...> or an error if any task fails or the deadline expires.

Learning goals: tokio::spawn, try_join!, timeout, error propagation across task boundaries.

Hint

Each spawned task returns Result<String, _>. try_join! unwraps all three. Wrap the whole try_join! in timeout() — the Elapsed error means you hit the deadline.

Solution
use tokio::time::{sleep, timeout, Duration};

async fn fake_fetch(name: &'static str, delay_ms: u64) -> Result<String, String> {
    sleep(Duration::from_millis(delay_ms)).await;
    Ok(format!("{name}: OK"))
}

async fn fetch_all() -> Result<Vec<String>, Box<dyn std::error::Error>> {
    let deadline = Duration::from_secs(5);

    let (a, b, c) = timeout(deadline, async {
        let h1 = tokio::spawn(fake_fetch("svc-a", 100));
        let h2 = tokio::spawn(fake_fetch("svc-b", 200));
        let h3 = tokio::spawn(fake_fetch("svc-c", 150));
        tokio::try_join!(h1, h2, h3)
    })
    .await??; // first ? = timeout, second ? = join

    Ok(vec![a?, b?, c?]) // unwrap inner Results
}

#[tokio::main]
async fn main() {
    let results = fetch_all().await.unwrap();
    for r in &results {
        println!("{r}");
    }
}

Exercise 11 — Async Channel Pipeline ★★★ (~40 min)

Build a producer → transformer → consumer pipeline using tokio::sync::mpsc:

  1. Producer: sends integers 1..=20 into channel A (capacity 4).
  2. Transformer: reads from channel A, squares each value, sends into channel B.
  3. Consumer: reads from channel B, collects into a Vec<u64>, returns it.

All three stages run as concurrent tokio::spawn tasks. Use bounded channels to demonstrate back-pressure. Assert the final vec equals [1, 4, 9, ..., 400].

Learning goals: mpsc::channel, bounded back-pressure, tokio::spawn with move closures, graceful shutdown via channel close.

Solution
use tokio::sync::mpsc;

#[tokio::main]
async fn main() {
    let (tx_a, mut rx_a) = mpsc::channel::<u64>(4); // bounded — back-pressure
    let (tx_b, mut rx_b) = mpsc::channel::<u64>(4);

    // Producer
    let producer = tokio::spawn(async move {
        for i in 1..=20u64 {
            tx_a.send(i).await.unwrap();
        }
        // tx_a dropped here → channel A closes
    });

    // Transformer
    let transformer = tokio::spawn(async move {
        while let Some(val) = rx_a.recv().await {
            tx_b.send(val * val).await.unwrap();
        }
        // tx_b dropped here → channel B closes
    });

    // Consumer
    let consumer = tokio::spawn(async move {
        let mut results = Vec::new();
        while let Some(val) = rx_b.recv().await {
            results.push(val);
        }
        results
    });

    producer.await.unwrap();
    transformer.await.unwrap();
    let results = consumer.await.unwrap();

    let expected: Vec<u64> = (1..=20).map(|x: u64| x * x).collect();
    assert_eq!(results, expected);
    println!("Pipeline complete: {results:?}");
}

Quick Reference Card

Pattern Decision Guide

Need type safety for primitives?
└── Newtype pattern (Ch3)

Need compile-time state enforcement?
└── Type-state pattern (Ch3)

Need a "tag" with no runtime data?
└── PhantomData (Ch4)

Need to break Rc/Arc reference cycles?
└── Weak<T> / sync::Weak<T> (Ch8)

Need to wait for a condition without busy-looping?
└── Condvar + Mutex (Ch6)

Need to handle "one of N types"?
├── Known closed set → Enum
├── Open set, hot path → Generics
├── Open set, cold path → dyn Trait
└── Completely unknown types → Any + TypeId (Ch2)

Need shared state across threads?
├── Simple counter/flag → Atomics
├── Short critical section → Mutex
├── Read-heavy → RwLock
├── Lazy one-time init → OnceLock / LazyLock (Ch6)
└── Complex state → Actor + Channels

Need to parallelize computation?
├── Collection processing → rayon::par_iter
├── Background task → thread::spawn
└── Borrow local data → thread::scope

Need async I/O or concurrent networking?
├── Basic → tokio + async/await (Ch15)
└── Advanced (streams, middleware) → see Async Rust Training

Need error handling?
├── Library → thiserror (#[derive(Error)])
└── Application → anyhow (Result<T>)

Need to prevent a value from being moved?
└── Pin<T> (Ch8) — required for Futures, self-referential types

Trait Bounds Cheat Sheet

BoundMeaning
T: CloneCan be duplicated
T: SendCan be moved to another thread
T: Sync&T can be shared between threads
T: 'staticContains no non-static references
T: SizedSize known at compile time (default)
T: ?SizedSize may not be known ([T], dyn Trait)
T: UnpinSafe to move after pinning
T: DefaultHas a default value
T: Into<U>Can be converted to U
T: AsRef<U>Can be borrowed as &U
T: Deref<Target = U>Auto-derefs to &U
F: Fn(A) -> BCallable, borrows state immutably
F: FnMut(A) -> BCallable, may mutate state
F: FnOnce(A) -> BCallable exactly once, may consume state

Lifetime Elision Rules

The compiler inserts lifetimes automatically in three cases (so you don’t have to):

#![allow(unused)]
fn main() {
// Rule 1: Each reference parameter gets its own lifetime
// fn foo(x: &str, y: &str)  →  fn foo<'a, 'b>(x: &'a str, y: &'b str)

// Rule 2: If there's exactly ONE input lifetime, it's used for all outputs
// fn foo(x: &str) -> &str   →  fn foo<'a>(x: &'a str) -> &'a str

// Rule 3: If one parameter is &self or &mut self, its lifetime is used
// fn foo(&self, x: &str) -> &str  →  fn foo<'a>(&'a self, x: &str) -> &'a str
}

When you MUST write explicit lifetimes:

  • Multiple input references and a reference output (compiler can’t guess which input)
  • Struct fields that hold references: struct Ref<'a> { data: &'a str }
  • 'static bounds when you need data without borrowed references

Common Derive Traits

#![allow(unused)]
fn main() {
#[derive(
    Debug,          // {:?} formatting
    Clone,          // .clone()
    Copy,           // Implicit copy (only for simple types)
    PartialEq, Eq,  // == comparison
    PartialOrd, Ord, // < > comparison + sorting
    Hash,           // HashMap/HashSet key
    Default,        // Type::default()
)]
struct MyType { /* ... */ }
}

Module Visibility Quick Reference

pub           → visible everywhere
pub(crate)    → visible within the crate
pub(super)    → visible to parent module
pub(in path)  → visible within a specific path
(nothing)     → private to current module + children

Further Reading

ResourceWhy
Rust Design PatternsCatalog of idiomatic patterns and anti-patterns
Rust API GuidelinesOfficial checklist for polished public APIs
Rust Atomics and LocksMara Bos’s deep dive into concurrency primitives
The RustonomiconOfficial guide to unsafe Rust and dark corners
Error Handling in RustAndrew Gallant’s comprehensive guide
Jon Gjengset — Crust of Rust seriesDeep dives into iterators, lifetimes, channels, etc.
Effective Rust35 specific ways to improve your Rust code

End of Rust Patterns & Engineering How-Tos

Capstone Project: Type-Safe Task Scheduler

This project integrates patterns from across the book into a single, production-style system. You’ll build a type-safe, concurrent task scheduler that uses generics, traits, typestate, channels, error handling, and testing.

Estimated time: 4–6 hours | Difficulty: ★★★

What you’ll practice:

  • Generics and trait bounds (Ch 1–2)
  • Typestate pattern for task lifecycle (Ch 3)
  • PhantomData for zero-cost state markers (Ch 4)
  • Channels for worker communication (Ch 5)
  • Concurrency with scoped threads (Ch 6)
  • Error handling with thiserror (Ch 9)
  • Testing with property-based tests (Ch 13)
  • API design with TryFrom and validated types (Ch 14)

The Problem

Build a task scheduler where:

  1. Tasks have a typed lifecycle: Pending → Running → Completed (or Failed)
  2. Workers pull tasks from a channel, execute them, and report results
  3. The scheduler manages task submission, worker coordination, and result collection
  4. Invalid state transitions are compile-time errors
stateDiagram-v2
    [*] --> Pending: scheduler.submit(task)
    Pending --> Running: worker picks up task
    Running --> Completed: task succeeds
    Running --> Failed: task returns Err
    Completed --> [*]: scheduler.results()
    Failed --> [*]: scheduler.results()

    Pending --> Pending: ❌ can't execute directly
    Completed --> Running: ❌ can't re-run

Step 1: Define the Task Types

Start with the typestate markers and a generic Task:

#![allow(unused)]
fn main() {
use std::marker::PhantomData;

// --- State markers (zero-sized) ---
struct Pending;
struct Running;
struct Completed;
struct Failed;

// --- Task ID (newtype for type safety) ---
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
struct TaskId(u64);

// --- The Task struct, parameterized by lifecycle state ---
struct Task<State, R> {
    id: TaskId,
    name: String,
    _state: PhantomData<State>,
    _result: PhantomData<R>,
}
}

Your job: Implement state transitions so that:

  • Task<Pending, R> can transition to Task<Running, R> (via start())
  • Task<Running, R> can transition to Task<Completed, R> or Task<Failed, R>
  • No other transitions compile
💡 Hint

Each transition method should consume self and return the new state:

#![allow(unused)]
fn main() {
impl<R> Task<Pending, R> {
    fn start(self) -> Task<Running, R> {
        Task {
            id: self.id,
            name: self.name,
            _state: PhantomData,
            _result: PhantomData,
        }
    }
}
}

Step 2: Define the Work Function

Tasks need a function to execute. Use a boxed closure:

#![allow(unused)]
fn main() {
struct WorkItem<R: Send + 'static> {
    id: TaskId,
    name: String,
    work: Box<dyn FnOnce() -> Result<R, String> + Send>,
}
}

Your job: Implement WorkItem::new() that accepts a task name and closure. Add a TaskId generator (simple atomic counter or mutex-protected counter).

Step 3: Error Handling

Define the scheduler’s error types using thiserror:

use thiserror::Error;

#[derive(Error, Debug)]
pub enum SchedulerError {
    #[error("scheduler is shut down")]
    ShutDown,

    #[error("task {0:?} failed: {1}")]
    TaskFailed(TaskId, String),

    #[error("channel send error")]
    ChannelError(#[from] std::sync::mpsc::SendError<()>),

    #[error("worker panicked")]
    WorkerPanic,
}

Step 4: The Scheduler

Build the scheduler using channels (Ch 5) and scoped threads (Ch 6):

#![allow(unused)]
fn main() {
use std::sync::mpsc;

struct Scheduler<R: Send + 'static> {
    sender: Option<mpsc::Sender<WorkItem<R>>>,
    results: mpsc::Receiver<TaskResult<R>>,
    num_workers: usize,
}

struct TaskResult<R> {
    id: TaskId,
    name: String,
    outcome: Result<R, String>,
}
}

Your job: Implement:

  • Scheduler::new(num_workers: usize) -> Self — creates channels and spawns workers
  • Scheduler::submit(&self, item: WorkItem<R>) -> Result<TaskId, SchedulerError>
  • Scheduler::shutdown(self) -> Vec<TaskResult<R>> — drops the sender, joins workers, collects results
💡 Hint — Worker loop
#![allow(unused)]
fn main() {
fn worker_loop<R: Send + 'static>(
    rx: std::sync::Arc<std::sync::Mutex<mpsc::Receiver<WorkItem<R>>>>,
    result_tx: mpsc::Sender<TaskResult<R>>,
    worker_id: usize,
) {
    loop {
        let item = {
            let rx = rx.lock().unwrap();
            rx.recv()
        };
        match item {
            Ok(work_item) => {
                let outcome = (work_item.work)();
                let _ = result_tx.send(TaskResult {
                    id: work_item.id,
                    name: work_item.name,
                    outcome,
                });
            }
            Err(_) => break, // Channel closed
        }
    }
}
}

Step 5: Integration Test

Write tests that verify:

  1. Happy path: Submit 10 tasks, shut down, verify all 10 results are Ok
  2. Error handling: Submit tasks that fail, verify TaskResult.outcome is Err
  3. Empty scheduler: Create and immediately shut down — no panics
  4. Property test (bonus): Use proptest to verify that for any N tasks (1..100), the scheduler always returns exactly N results
#![allow(unused)]
fn main() {
#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn happy_path() {
        let scheduler = Scheduler::<String>::new(4);

        for i in 0..10 {
            let item = WorkItem::new(
                format!("task-{i}"),
                move || Ok(format!("result-{i}")),
            );
            scheduler.submit(item).unwrap();
        }

        let results = scheduler.shutdown();
        assert_eq!(results.len(), 10);
        for r in &results {
            assert!(r.outcome.is_ok());
        }
    }

    #[test]
    fn handles_failures() {
        let scheduler = Scheduler::<String>::new(2);

        scheduler.submit(WorkItem::new("good", || Ok("ok".into()))).unwrap();
        scheduler.submit(WorkItem::new("bad", || Err("boom".into()))).unwrap();

        let results = scheduler.shutdown();
        assert_eq!(results.len(), 2);

        let failures: Vec<_> = results.iter()
            .filter(|r| r.outcome.is_err())
            .collect();
        assert_eq!(failures.len(), 1);
    }
}
}

Step 6: Put It All Together

Here’s the main() that demonstrates the full system:

fn main() {
    let scheduler = Scheduler::<String>::new(4);

    // Submit tasks with varying workloads
    for i in 0..20 {
        let item = WorkItem::new(
            format!("compute-{i}"),
            move || {
                // Simulate work
                std::thread::sleep(std::time::Duration::from_millis(10));
                if i % 7 == 0 {
                    Err(format!("task {i} hit a simulated error"))
                } else {
                    Ok(format!("task {i} completed with value {}", i * i))
                }
            },
        );
        // NOTE: .unwrap() is used for brevity — handle SendError in production.
        scheduler.submit(item).unwrap();
    }

    println!("All tasks submitted. Shutting down...");
    let results = scheduler.shutdown();

    let (ok, err): (Vec<_>, Vec<_>) = results.iter()
        .partition(|r| r.outcome.is_ok());

    println!("\n✅ Succeeded: {}", ok.len());
    for r in &ok {
        println!("  {} → {}", r.name, r.outcome.as_ref().unwrap());
    }

    println!("\n❌ Failed: {}", err.len());
    for r in &err {
        println!("  {} → {}", r.name, r.outcome.as_ref().unwrap_err());
    }
}

Evaluation Criteria

CriterionTarget
Type safetyInvalid state transitions don’t compile
ConcurrencyWorkers run in parallel, no data races
Error handlingAll failures captured in TaskResult, no panics
TestingAt least 3 tests; bonus for proptest
Code organizationClean module structure, public API uses validated types
DocumentationKey types have doc comments explaining invariants

Extension Ideas

Once the basic scheduler works, try these enhancements:

  1. Priority queue: Add a Priority newtype (1–10) and process higher-priority tasks first
  2. Retry policy: Failed tasks retry up to N times before being marked permanently failed
  3. Cancellation: Add a cancel(TaskId) method that removes pending tasks
  4. Async version: Port to tokio::spawn with tokio::sync::mpsc channels (Ch 15)
  5. Metrics: Track per-worker task counts, average execution time, and failure rates