423 Advanced

423: Procedural Macro Introduction

Functional Programming

Tutorial Video

Text description (accessibility)

This video demonstrates the "423: Procedural Macro Introduction" functional Rust example. Difficulty level: Advanced. Key concepts covered: Functional Programming. `macro_rules!` handles syntactic patterns but cannot inspect type information or generate identifiers dynamically based on field names. Key difference from OCaml: 1. **Token vs. AST**: Rust proc macros receive raw token streams; OCaml PPX receives the parsed AST. Rust is more flexible but requires manual parsing; OCaml's AST is structured but verbose.

Tutorial

The Problem

macro_rules! handles syntactic patterns but cannot inspect type information or generate identifiers dynamically based on field names. Procedural macros (proc macros) operate on the full Rust token stream at compile time as external Rust programs: they receive a TokenStream, parse it using syn, and emit generated code using quote. This enables #[derive(Serialize)] to generate different code for each struct's specific field names and types — impossible with macro_rules!.

Proc macros power the entire Rust ecosystem's most powerful abstractions: serde, tokio::main, actix::web::get, clap::Parser, and thousands of derive macros.

🎯 Learning Outcomes

• Understand the three types of proc macros: derive, attribute, function-like

• Learn the proc macro development model: separate crate, TokenStream in/out

• See conceptually what a #[derive(MyDebug)] proc macro generates for Point

• Understand how syn parses token streams into AST nodes and quote! generates code

• Learn why proc macros must be in a separate crate with proc-macro = true

Code Example

#![allow(clippy::all)]
//! Procedural Macro Introduction
//!
//! Understanding proc macros without implementing them.

/// Proc macros operate on token streams.
/// This example shows the concepts, not actual proc macro code.

/// Example: what a derive macro generates.
/// #[derive(MyDebug)] on Point would generate something like:
pub struct Point {
    pub x: i32,
    pub y: i32,
}

impl std::fmt::Debug for Point {
    fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
        f.debug_struct("Point")
            .field("x", &self.x)
            .field("y", &self.y)
            .finish()
    }
}

/// Example: what an attribute macro might do.
/// #[log_calls] on a function adds logging.
pub fn example_function(x: i32) -> i32 {
    // Imagine: println!("Entering example_function");
    let result = x * 2;
    // Imagine: println!("Exiting example_function");
    result
}

/// Three types of proc macros:
/// 1. Derive macros: #[derive(Trait)]
/// 2. Attribute macros: #[attribute]
/// 3. Function-like macros: my_macro!(...)

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_debug_format() {
        let p = Point { x: 1, y: 2 };
        let s = format!("{:?}", p);
        assert!(s.contains("Point"));
        assert!(s.contains("x: 1"));
    }

    #[test]
    fn test_example_function() {
        assert_eq!(example_function(5), 10);
    }

    #[test]
    fn test_point_fields() {
        let p = Point { x: 3, y: 4 };
        assert_eq!(p.x, 3);
        assert_eq!(p.y, 4);
    }

    #[test]
    fn test_debug_struct() {
        let p = Point { x: 0, y: 0 };
        assert_eq!(format!("{:?}", p), "Point { x: 0, y: 0 }");
    }

    #[test]
    fn test_pretty_debug() {
        let p = Point { x: 1, y: 2 };
        let s = format!("{:#?}", p);
        assert!(s.contains("Point"));
    }
}

(* Proc macro concepts in OCaml via ppx *)

(* ppx_deriving example — what a custom derive would generate *)
(* Without ppx, we write the boilerplate manually *)

type describable = {
  type_name: string;
  field_names: string list;
  field_values: string list;
}

(* Simulate what #[derive(Describe)] would generate *)
let make_describable tn fns fvs = { type_name=tn; field_names=fns; field_values=fvs }

let describe d =
  Printf.printf "%s { " d.type_name;
  List.iter2 (fun name value ->
    Printf.printf "%s: %s, " name value
  ) d.field_names d.field_values;
  Printf.printf "}\n"

type person = { name: string; age: int }

(* Generated by hypothetical #[derive(Describe)] *)
let describe_person p =
  describe (make_describable "Person"
    ["name"; "age"]
    [p.name; string_of_int p.age])

let () =
  describe_person {name = "Alice"; age = 30}

Key Differences

Token vs. AST: Rust proc macros receive raw token streams; OCaml PPX receives the parsed AST. Rust is more flexible but requires manual parsing; OCaml's AST is structured but verbose.

Separate crate: Rust proc macros must live in a separate crate with proc-macro = true; OCaml PPX plugins are separate executables configured in dune.

syn/quote ecosystem: Rust uses syn for parsing and quote! for generation; OCaml uses Ppxlib for AST traversal and Ast_builder for generation.

Error reporting: Rust proc macros use compile_error! macro or syn::Error::to_compile_error(); OCaml uses Location.error_extensionf for positioned errors.

OCaml Approach

OCaml's PPX framework is the direct equivalent: a PPX plugin is a standalone OCaml program that receives the parsed OCaml AST (as Parsetree values), transforms it, and returns the modified AST. ppx_deriving is OCaml's derive equivalent. The AST is richer than Rust's token stream (already parsed) but requires knowledge of OCaml's Parsetree module. Dune integrates PPX as build-time preprocessors.

Full Source

#![allow(clippy::all)]
//! Procedural Macro Introduction
//!
//! Understanding proc macros without implementing them.

/// Proc macros operate on token streams.
/// This example shows the concepts, not actual proc macro code.

/// Example: what a derive macro generates.
/// #[derive(MyDebug)] on Point would generate something like:
pub struct Point {
    pub x: i32,
    pub y: i32,
}

impl std::fmt::Debug for Point {
    fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
        f.debug_struct("Point")
            .field("x", &self.x)
            .field("y", &self.y)
            .finish()
    }
}

/// Example: what an attribute macro might do.
/// #[log_calls] on a function adds logging.
pub fn example_function(x: i32) -> i32 {
    // Imagine: println!("Entering example_function");
    let result = x * 2;
    // Imagine: println!("Exiting example_function");
    result
}

/// Three types of proc macros:
/// 1. Derive macros: #[derive(Trait)]
/// 2. Attribute macros: #[attribute]
/// 3. Function-like macros: my_macro!(...)

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_debug_format() {
        let p = Point { x: 1, y: 2 };
        let s = format!("{:?}", p);
        assert!(s.contains("Point"));
        assert!(s.contains("x: 1"));
    }

    #[test]
    fn test_example_function() {
        assert_eq!(example_function(5), 10);
    }

    #[test]
    fn test_point_fields() {
        let p = Point { x: 3, y: 4 };
        assert_eq!(p.x, 3);
        assert_eq!(p.y, 4);
    }

    #[test]
    fn test_debug_struct() {
        let p = Point { x: 0, y: 0 };
        assert_eq!(format!("{:?}", p), "Point { x: 0, y: 0 }");
    }

    #[test]
    fn test_pretty_debug() {
        let p = Point { x: 1, y: 2 };
        let s = format!("{:#?}", p);
        assert!(s.contains("Point"));
    }
}

(* Proc macro concepts in OCaml via ppx *)

(* ppx_deriving example — what a custom derive would generate *)
(* Without ppx, we write the boilerplate manually *)

type describable = {
  type_name: string;
  field_names: string list;
  field_values: string list;
}

(* Simulate what #[derive(Describe)] would generate *)
let make_describable tn fns fvs = { type_name=tn; field_names=fns; field_values=fvs }

let describe d =
  Printf.printf "%s { " d.type_name;
  List.iter2 (fun name value ->
    Printf.printf "%s: %s, " name value
  ) d.field_names d.field_values;
  Printf.printf "}\n"

type person = { name: string; age: int }

(* Generated by hypothetical #[derive(Describe)] *)
let describe_person p =
  describe (make_describable "Person"
    ["name"; "age"]
    [p.name; string_of_int p.age])

let () =
  describe_person {name = "Alice"; age = 30}

✓ Tests Rust test suite

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_debug_format() {
        let p = Point { x: 1, y: 2 };
        let s = format!("{:?}", p);
        assert!(s.contains("Point"));
        assert!(s.contains("x: 1"));
    }

    #[test]
    fn test_example_function() {
        assert_eq!(example_function(5), 10);
    }

    #[test]
    fn test_point_fields() {
        let p = Point { x: 3, y: 4 };
        assert_eq!(p.x, 3);
        assert_eq!(p.y, 4);
    }

    #[test]
    fn test_debug_struct() {
        let p = Point { x: 0, y: 0 };
        assert_eq!(format!("{:?}", p), "Point { x: 0, y: 0 }");
    }

    #[test]
    fn test_pretty_debug() {
        let p = Point { x: 1, y: 2 };
        let s = format!("{:#?}", p);
        assert!(s.contains("Point"));
    }
}

Deep Comparison

OCaml vs Rust: proc macro intro

See example.rs and example.ml for side-by-side implementations.

Key Points

Rust macros operate at compile time

OCaml uses ppx for similar metaprogramming

Both languages support powerful code generation

Rust's macro_rules! is built into the language

OCaml's approach requires external tooling

Exercises

Understand the output: Take the Point struct and Debug impl in src/lib.rs. Trace through what syn would parse (struct name, field names, field types) and how quote! would produce the impl Debug for Point block.

Attribute macro sketch: Write a // TODO: proc-macro function that takes fn add(a: i32, b: i32) -> i32 and describes in comments what the #[log_calls] attribute macro would need to generate — including the logging calls and the original function body.

Research project: Find three proc macros you use regularly (tokio::main, serde::Deserialize, clap::Parser). For each, describe in a code comment what token stream input they receive and what code they generate.

Open Source Repos

functional-rust

View the source for this example on GitHub — OCaml and Rust side by side in the repo.

Rust