760 Fundamental

760-serde-derive-concept — Serde Derive Concept

Functional Programming

Tutorial Video

Text description (accessibility)

This video demonstrates the "760-serde-derive-concept — Serde Derive Concept" functional Rust example. Difficulty level: Fundamental. Key concepts covered: Functional Programming. `#[derive(Serialize, Deserialize)]` is the most-used derive macro in the Rust ecosystem — the `serde` crate has been downloaded billions of times. Key difference from OCaml: 1. **Unified abstraction**: Rust's `serde::Serializer` trait enables one `#[derive(Serialize)]` to work with all formats; OCaml requires a separate `[@@deriving ...]` per format.

Tutorial

The Problem

#[derive(Serialize, Deserialize)] is the most-used derive macro in the Rust ecosystem — the serde crate has been downloaded billions of times. Understanding what the macro generates demystifies serde's design: it calls into a Serializer or Deserializer trait whose concrete implementation handles the format (JSON, TOML, MessagePack). This separation means your types are format-agnostic; the format is injected at the call site.

🎯 Learning Outcomes

• Understand the Visitor pattern that serde uses under the hood

• See what code #[derive(Serialize)] generates for a simple Point { x: i32, y: i32 } struct

• Implement a minimal Serializer trait with struct and primitive methods

• See how a JsonSerializer implements the trait to produce JSON output

• Understand why serde's trait-based design enables zero-cost abstraction across formats

Code Example

// You write:
#[derive(Serialize)]
struct Point { x: i32, y: i32 }

// The macro generates:
impl Serialize for Point {
    fn serialize<S: Serializer>(&self, serializer: &mut S) {
        serializer.serialize_struct_start("Point", 2);
        serializer.serialize_field("x");
        self.x.serialize(serializer);
        serializer.serialize_field("y");
        self.y.serialize(serializer);
        serializer.serialize_struct_end();
    }
}

(* You write: *)
type point = { x: int; y: int } [@@deriving yojson]

(* The ppx generates: *)
let point_to_yojson p =
  `Assoc [
    ("x", `Int p.x);
    ("y", `Int p.y)
  ]

Key Differences

Unified abstraction: Rust's serde::Serializer trait enables one #[derive(Serialize)] to work with all formats; OCaml requires a separate [@@deriving ...] per format.

Visitor pattern: serde uses the Visitor pattern to drive deserialization; OCaml's derivers generate direct recursive descent parsers.

Zero cost: Both generate code equivalent to hand-written serialization; no runtime dispatch is needed.

Human-readable formats: Rust's serde handles both human-readable (JSON) and binary formats uniformly; OCaml uses sexp for human-readable and Bin_prot for binary.

OCaml Approach

OCaml's ppx_sexp_conv generates sexp_of_t and t_of_sexp functions from a [@@deriving sexp] attribute — analogous to #[derive(Serialize, Deserialize)]. Bin_prot generates bin_write_t / bin_read_t pairs. ppx_yojson_conv generates JSON serializers. Unlike Rust's unified Serializer trait, OCaml uses separate functions per format, generated by separate ppx packages.

Full Source

#![allow(clippy::all)]
//! # Serde Derive Concept
//!
//! Understanding how derive macros generate serialization code.

/// Simulated derive output for a struct
///
/// Given:
/// ```ignore
/// #[derive(Serialize)]
/// struct Point { x: i32, y: i32 }
/// ```
///
/// The derive macro generates something like this implementation.

// Manual implementation showing what #[derive(Serialize)] would generate

/// A simple output trait (simulating serde's Serializer)
pub trait Serializer {
    fn serialize_i32(&mut self, v: i32);
    fn serialize_str(&mut self, v: &str);
    fn serialize_struct_start(&mut self, name: &str, len: usize);
    fn serialize_field(&mut self, name: &str);
    fn serialize_struct_end(&mut self);
}

/// Our serialize trait
pub trait Serialize {
    fn serialize<S: Serializer>(&self, serializer: &mut S);
}

// Primitive implementations
impl Serialize for i32 {
    fn serialize<S: Serializer>(&self, serializer: &mut S) {
        serializer.serialize_i32(*self);
    }
}

impl Serialize for String {
    fn serialize<S: Serializer>(&self, serializer: &mut S) {
        serializer.serialize_str(self);
    }
}

impl Serialize for &str {
    fn serialize<S: Serializer>(&self, serializer: &mut S) {
        serializer.serialize_str(self);
    }
}

/// Example struct
#[derive(Debug, PartialEq)]
pub struct Point {
    pub x: i32,
    pub y: i32,
}

/// What #[derive(Serialize)] would generate for Point
impl Serialize for Point {
    fn serialize<S: Serializer>(&self, serializer: &mut S) {
        serializer.serialize_struct_start("Point", 2);
        serializer.serialize_field("x");
        self.x.serialize(serializer);
        serializer.serialize_field("y");
        self.y.serialize(serializer);
        serializer.serialize_struct_end();
    }
}

/// Another example struct
#[derive(Debug, PartialEq)]
pub struct Person {
    pub name: String,
    pub age: i32,
}

impl Serialize for Person {
    fn serialize<S: Serializer>(&self, serializer: &mut S) {
        serializer.serialize_struct_start("Person", 2);
        serializer.serialize_field("name");
        self.name.serialize(serializer);
        serializer.serialize_field("age");
        self.age.serialize(serializer);
        serializer.serialize_struct_end();
    }
}

// A JSON-like serializer for testing
pub struct JsonSerializer {
    output: String,
    first_field: bool,
}

impl JsonSerializer {
    pub fn new() -> Self {
        JsonSerializer {
            output: String::new(),
            first_field: true,
        }
    }

    pub fn into_string(self) -> String {
        self.output
    }
}

impl Default for JsonSerializer {
    fn default() -> Self {
        Self::new()
    }
}

impl Serializer for JsonSerializer {
    fn serialize_i32(&mut self, v: i32) {
        self.output.push_str(&v.to_string());
    }

    fn serialize_str(&mut self, v: &str) {
        self.output.push('"');
        self.output.push_str(v);
        self.output.push('"');
    }

    fn serialize_struct_start(&mut self, _name: &str, _len: usize) {
        self.output.push('{');
        self.first_field = true;
    }

    fn serialize_field(&mut self, name: &str) {
        if !self.first_field {
            self.output.push_str(", ");
        }
        self.first_field = false;
        self.output.push('"');
        self.output.push_str(name);
        self.output.push_str("\": ");
    }

    fn serialize_struct_end(&mut self) {
        self.output.push('}');
    }
}

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_serialize_point() {
        let point = Point { x: 10, y: 20 };
        let mut ser = JsonSerializer::new();
        point.serialize(&mut ser);
        assert_eq!(ser.into_string(), r#"{"x": 10, "y": 20}"#);
    }

    #[test]
    fn test_serialize_person() {
        let person = Person {
            name: "Alice".to_string(),
            age: 30,
        };
        let mut ser = JsonSerializer::new();
        person.serialize(&mut ser);
        assert_eq!(ser.into_string(), r#"{"name": "Alice", "age": 30}"#);
    }

    #[test]
    fn test_serialize_i32() {
        let mut ser = JsonSerializer::new();
        42i32.serialize(&mut ser);
        assert_eq!(ser.into_string(), "42");
    }

    #[test]
    fn test_serialize_string() {
        let mut ser = JsonSerializer::new();
        "hello".serialize(&mut ser);
        assert_eq!(ser.into_string(), "\"hello\"");
    }
}

(* Derive-based serialization concept in OCaml
   We simulate what [@@deriving] does by writing the expansion manually,
   then compare with the PPX-generated version (commented). *)

(* ---------- The "trait" (module type) ---------- *)
module type SERIALIZABLE = sig
  type t
  val fields : string list           (* field names in order *)
  val to_assoc : t -> (string * string) list
  val of_assoc : (string * string) list -> t option
end

(* ---------- Generic serializer using the module ---------- *)
module JsonLike (S : SERIALIZABLE) = struct
  let serialize v =
    let pairs = S.to_assoc v in
    let inner =
      List.map (fun (k, v) -> Printf.sprintf "%S: %S" k v) pairs
      |> String.concat ", "
    in
    Printf.sprintf "{ %s }" inner

  let deserialize s =
    (* Toy parser: strip braces, split on ", ", split each on ": " *)
    let s = String.trim s in
    let s = String.sub s 1 (String.length s - 2) |> String.trim in
    let pairs =
      String.split_on_char ',' s
      |> List.filter_map (fun pair ->
        match String.split_on_char ':' (String.trim pair) with
        | [k; v] ->
          let unquote x =
            let x = String.trim x in
            if String.length x >= 2 && x.[0] = '"'
            then String.sub x 1 (String.length x - 2)
            else x
          in
          Some (unquote k, unquote v)
        | _ -> None)
    in
    S.of_assoc pairs
end

(* ---------- Domain type ---------- *)
(* What [@@deriving serialize] would generate: *)
type color = { r: int; g: int; b: int }

(* Manual "derive" expansion *)
module ColorSerializable : SERIALIZABLE with type t = color = struct
  type t = color
  let fields = ["r"; "g"; "b"]
  let to_assoc c = [
    ("r", string_of_int c.r);
    ("g", string_of_int c.g);
    ("b", string_of_int c.b);
  ]
  let of_assoc pairs =
    let find k = List.assoc_opt k pairs in
    match find "r", find "g", find "b" with
    | Some r, Some g, Some b ->
      (try Some { r = int_of_string r; g = int_of_string g; b = int_of_string b }
       with Failure _ -> None)
    | _ -> None
end

module ColorJson = JsonLike(ColorSerializable)

let () =
  let red = { r = 255; g = 0; b = 0 } in
  let encoded = ColorJson.serialize red in
  Printf.printf "Serialized: %s\n" encoded;
  match ColorJson.deserialize encoded with
  | Some c -> Printf.printf "Deserialized: r=%d g=%d b=%d\n" c.r c.g c.b
  | None   -> Printf.printf "Failed to deserialize\n"

✓ Tests Rust test suite

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_serialize_point() {
        let point = Point { x: 10, y: 20 };
        let mut ser = JsonSerializer::new();
        point.serialize(&mut ser);
        assert_eq!(ser.into_string(), r#"{"x": 10, "y": 20}"#);
    }

    #[test]
    fn test_serialize_person() {
        let person = Person {
            name: "Alice".to_string(),
            age: 30,
        };
        let mut ser = JsonSerializer::new();
        person.serialize(&mut ser);
        assert_eq!(ser.into_string(), r#"{"name": "Alice", "age": 30}"#);
    }

    #[test]
    fn test_serialize_i32() {
        let mut ser = JsonSerializer::new();
        42i32.serialize(&mut ser);
        assert_eq!(ser.into_string(), "42");
    }

    #[test]
    fn test_serialize_string() {
        let mut ser = JsonSerializer::new();
        "hello".serialize(&mut ser);
        assert_eq!(ser.into_string(), "\"hello\"");
    }
}

Deep Comparison

OCaml vs Rust: Serde Derive Concept

What Derive Generates

Rust

// You write:
#[derive(Serialize)]
struct Point { x: i32, y: i32 }

// The macro generates:
impl Serialize for Point {
    fn serialize<S: Serializer>(&self, serializer: &mut S) {
        serializer.serialize_struct_start("Point", 2);
        serializer.serialize_field("x");
        self.x.serialize(serializer);
        serializer.serialize_field("y");
        self.y.serialize(serializer);
        serializer.serialize_struct_end();
    }
}

OCaml (ppx_deriving)

(* You write: *)
type point = { x: int; y: int } [@@deriving yojson]

(* The ppx generates: *)
let point_to_yojson p =
  `Assoc [
    ("x", `Int p.x);
    ("y", `Int p.y)
  ]

Serializer Trait Pattern

Rust

pub trait Serializer {
    fn serialize_i32(&mut self, v: i32);
    fn serialize_str(&mut self, v: &str);
    fn serialize_struct_start(&mut self, name: &str, len: usize);
    fn serialize_field(&mut self, name: &str);
    fn serialize_struct_end(&mut self);
}

Key Differences

Aspect	OCaml	Rust
Macro system	PPX	Proc macros
Type-driven	ppx_deriving	serde derive
Output format	Specific (`yojson`)	Generic (Serializer trait)
Format agnostic	Multiple ppx needed	Single `Serialize` impl

Exercises

Implement a TomlSerializer that produces TOML output for simple structs (key = value pairs), and verify it produces correct output for Point { x: 1, y: 2 }.

Add serialize_vec to the Serializer trait and implement it for JsonSerializer as a JSON array, then implement Serialize for Vec<T: Serialize>.

Write the "generated" Deserialize implementation for Point — what the visitor-based deserialization looks like for a two-field struct.

Open Source Repos

functional-rust

View the source for this example on GitHub — OCaml and Rust side by side in the repo.

Rust