763 Fundamental

763-json-format-from-scratch — JSON Format From Scratch

Functional Programming

Tutorial Video

Text description (accessibility)

This video demonstrates the "763-json-format-from-scratch — JSON Format From Scratch" functional Rust example. Difficulty level: Fundamental. Key concepts covered: Functional Programming. JSON (JavaScript Object Notation) was introduced in 2001 and is now the universal data interchange format. Key difference from OCaml: 1. **Recursion**: Both represent JSON as recursive algebraic data types (enums in Rust, variants in OCaml); the recursive `to_string` pattern is identical.

Tutorial

The Problem

JSON (JavaScript Object Notation) was introduced in 2001 and is now the universal data interchange format. Building a JSON serializer from scratch teaches you about recursive data structures, string escaping, number formatting, and the performance trade-offs in text-based formats. Understanding JSON's structure also helps when working with serde's JSON support and when debugging serialization issues in production systems.

🎯 Learning Outcomes

• Represent JSON as a recursive JsonValue enum: Null, Bool, Number, String, Array, Object

• Implement to_json(&self) -> String for compact output

• Implement to_json_pretty(&self, indent: usize) -> String with configurable indentation

• Handle string escaping: ", \, newlines, control characters

• Understand JSON number formatting: integers vs. floats, scientific notation edge cases

Code Example

pub enum JsonValue {
    Null,
    Bool(bool),
    Number(f64),
    String(String),
    Array(Vec<JsonValue>),
    Object(Vec<(String, JsonValue)>),
}

type json =
  | `Null
  | `Bool of bool
  | `Float of float
  | `String of string
  | `List of json list
  | `Assoc of (string * json) list

Key Differences

Recursion: Both represent JSON as recursive algebraic data types (enums in Rust, variants in OCaml); the recursive to_string pattern is identical.

Object ordering: Rust's example uses Vec<(String, JsonValue)> to preserve insertion order; OCaml's Yojson uses an association list with the same property.

Number handling: JSON numbers are IEEE 754 doubles; both languages face the same 1.0 vs 1 formatting challenge.

Performance: Production JSON libraries (Rust's simd-json, OCaml's jsonaf) use SIMD for bulk scanning; this from-scratch version prioritizes clarity.

OCaml Approach

OCaml's Yojson library represents JSON as a variant type similar to JsonValue. Encoding uses Yojson.Safe.to_string and Yojson.Safe.pretty_to_string. Custom serialization uses the to_basic function to convert from library types. OCaml's Jsonaf (Jane Street) provides a high-performance alternative. Both OCaml JSON libraries handle Unicode and number formatting edge cases that this from-scratch example omits.

Full Source

#![allow(clippy::all)]
//! # JSON Format From Scratch
//!
//! Building a simple JSON serializer without serde.

/// JSON value representation
#[derive(Debug, Clone, PartialEq)]
pub enum JsonValue {
    Null,
    Bool(bool),
    Number(f64),
    String(String),
    Array(Vec<JsonValue>),
    Object(Vec<(String, JsonValue)>),
}

impl JsonValue {
    /// Serialize to JSON string
    pub fn to_json(&self) -> String {
        match self {
            JsonValue::Null => "null".to_string(),
            JsonValue::Bool(b) => b.to_string(),
            JsonValue::Number(n) => {
                if n.fract() == 0.0 && n.abs() < 1e15 {
                    format!("{:.0}", n)
                } else {
                    n.to_string()
                }
            }
            JsonValue::String(s) => format!("\"{}\"", escape_json_string(s)),
            JsonValue::Array(arr) => {
                let items: Vec<String> = arr.iter().map(|v| v.to_json()).collect();
                format!("[{}]", items.join(", "))
            }
            JsonValue::Object(obj) => {
                let pairs: Vec<String> = obj
                    .iter()
                    .map(|(k, v)| format!("\"{}\": {}", escape_json_string(k), v.to_json()))
                    .collect();
                format!("{{{}}}", pairs.join(", "))
            }
        }
    }

    /// Pretty print with indentation
    pub fn to_json_pretty(&self, indent: usize) -> String {
        self.to_json_indent(0, indent)
    }

    fn to_json_indent(&self, level: usize, indent: usize) -> String {
        let prefix = " ".repeat(level * indent);
        let inner_prefix = " ".repeat((level + 1) * indent);

        match self {
            JsonValue::Array(arr) if arr.is_empty() => "[]".to_string(),
            JsonValue::Array(arr) => {
                let items: Vec<String> = arr
                    .iter()
                    .map(|v| format!("{}{}", inner_prefix, v.to_json_indent(level + 1, indent)))
                    .collect();
                format!("[\n{}\n{}]", items.join(",\n"), prefix)
            }
            JsonValue::Object(obj) if obj.is_empty() => "{}".to_string(),
            JsonValue::Object(obj) => {
                let pairs: Vec<String> = obj
                    .iter()
                    .map(|(k, v)| {
                        format!(
                            "{}\"{}\": {}",
                            inner_prefix,
                            escape_json_string(k),
                            v.to_json_indent(level + 1, indent)
                        )
                    })
                    .collect();
                format!("{{\n{}\n{}}}", pairs.join(",\n"), prefix)
            }
            _ => self.to_json(),
        }
    }
}

/// Escape special characters in JSON strings
fn escape_json_string(s: &str) -> String {
    let mut result = String::new();
    for c in s.chars() {
        match c {
            '"' => result.push_str("\\\""),
            '\\' => result.push_str("\\\\"),
            '\n' => result.push_str("\\n"),
            '\r' => result.push_str("\\r"),
            '\t' => result.push_str("\\t"),
            c if c.is_control() => result.push_str(&format!("\\u{:04x}", c as u32)),
            c => result.push(c),
        }
    }
    result
}

/// Trait for converting to JSON
pub trait ToJson {
    fn to_json_value(&self) -> JsonValue;
}

impl ToJson for bool {
    fn to_json_value(&self) -> JsonValue {
        JsonValue::Bool(*self)
    }
}

impl ToJson for i32 {
    fn to_json_value(&self) -> JsonValue {
        JsonValue::Number(*self as f64)
    }
}

impl ToJson for f64 {
    fn to_json_value(&self) -> JsonValue {
        JsonValue::Number(*self)
    }
}

impl ToJson for String {
    fn to_json_value(&self) -> JsonValue {
        JsonValue::String(self.clone())
    }
}

impl ToJson for &str {
    fn to_json_value(&self) -> JsonValue {
        JsonValue::String(self.to_string())
    }
}

impl<T: ToJson> ToJson for Vec<T> {
    fn to_json_value(&self) -> JsonValue {
        JsonValue::Array(self.iter().map(|v| v.to_json_value()).collect())
    }
}

impl<T: ToJson> ToJson for Option<T> {
    fn to_json_value(&self) -> JsonValue {
        match self {
            Some(v) => v.to_json_value(),
            None => JsonValue::Null,
        }
    }
}

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_null() {
        assert_eq!(JsonValue::Null.to_json(), "null");
    }

    #[test]
    fn test_bool() {
        assert_eq!(JsonValue::Bool(true).to_json(), "true");
        assert_eq!(JsonValue::Bool(false).to_json(), "false");
    }

    #[test]
    fn test_number() {
        assert_eq!(JsonValue::Number(42.0).to_json(), "42");
        assert_eq!(JsonValue::Number(3.14).to_json(), "3.14");
    }

    #[test]
    fn test_string() {
        assert_eq!(
            JsonValue::String("hello".to_string()).to_json(),
            "\"hello\""
        );
    }

    #[test]
    fn test_string_escape() {
        assert_eq!(
            JsonValue::String("a\"b".to_string()).to_json(),
            "\"a\\\"b\""
        );
        assert_eq!(JsonValue::String("a\nb".to_string()).to_json(), "\"a\\nb\"");
    }

    #[test]
    fn test_array() {
        let arr = JsonValue::Array(vec![
            JsonValue::Number(1.0),
            JsonValue::Number(2.0),
            JsonValue::Number(3.0),
        ]);
        assert_eq!(arr.to_json(), "[1, 2, 3]");
    }

    #[test]
    fn test_object() {
        let obj = JsonValue::Object(vec![
            ("a".to_string(), JsonValue::Number(1.0)),
            ("b".to_string(), JsonValue::Number(2.0)),
        ]);
        assert_eq!(obj.to_json(), r#"{"a": 1, "b": 2}"#);
    }

    #[test]
    fn test_trait() {
        assert_eq!(42i32.to_json_value().to_json(), "42");
        assert_eq!("hello".to_json_value().to_json(), "\"hello\"");
    }

    #[test]
    fn test_option() {
        let some: Option<i32> = Some(42);
        let none: Option<i32> = None;
        assert_eq!(some.to_json_value().to_json(), "42");
        assert_eq!(none.to_json_value().to_json(), "null");
    }
}

(* JSON-like format built from scratch in OCaml *)

(* ── Value type ─────────────────────────────────────────────────────────────── *)
type json =
  | JNull
  | JBool   of bool
  | JInt    of int
  | JFloat  of float
  | JString of string
  | JArray  of json list
  | JObject of (string * json) list

(* ── Serializer ──────────────────────────────────────────────────────────────── *)
let rec to_string = function
  | JNull       -> "null"
  | JBool true  -> "true"
  | JBool false -> "false"
  | JInt n      -> string_of_int n
  | JFloat f    -> Printf.sprintf "%g" f
  | JString s   -> Printf.sprintf "%S" s
  | JArray arr  ->
    let items = String.concat ", " (List.map to_string arr) in
    Printf.sprintf "[%s]" items
  | JObject obj ->
    let pairs =
      List.map (fun (k, v) -> Printf.sprintf "%S: %s" k (to_string v)) obj
      |> String.concat ", "
    in
    Printf.sprintf "{%s}" pairs

(* ── Minimal parser ──────────────────────────────────────────────────────────── *)
let parse_error msg = failwith ("JSON parse error: " ^ msg)

let skip_ws s pos =
  while !pos < String.length s && (s.[!pos] = ' ' || s.[!pos] = '\n' || s.[!pos] = '\t') do
    incr pos
  done

let parse_string s pos =
  (* pos points to opening '"' *)
  incr pos;
  let buf = Buffer.create 16 in
  let continue = ref true in
  while !continue do
    if !pos >= String.length s then parse_error "unterminated string";
    let c = s.[!pos] in
    incr pos;
    if c = '"' then continue := false
    else if c = '\\' then begin
      if !pos >= String.length s then parse_error "escape at EOF";
      let esc = s.[!pos] in incr pos;
      Buffer.add_char buf (match esc with
        | 'n' -> '\n' | 't' -> '\t' | '"' -> '"' | '\\' -> '\\'
        | c -> c)
    end else Buffer.add_char buf c
  done;
  Buffer.contents buf

let rec parse_value s pos =
  skip_ws s pos;
  if !pos >= String.length s then parse_error "unexpected EOF";
  match s.[!pos] with
  | '"' -> JString (parse_string s pos)
  | 't' -> pos := !pos + 4; JBool true
  | 'f' -> pos := !pos + 5; JBool false
  | 'n' -> pos := !pos + 4; JNull
  | '[' ->
    incr pos; skip_ws s pos;
    if s.[!pos] = ']' then (incr pos; JArray [])
    else begin
      let items = ref [] in
      let go = ref true in
      while !go do
        items := parse_value s pos :: !items;
        skip_ws s pos;
        if !pos < String.length s && s.[!pos] = ',' then incr pos
        else go := false
      done;
      skip_ws s pos;
      if s.[!pos] <> ']' then parse_error "expected ']'";
      incr pos;
      JArray (List.rev !items)
    end
  | '{' ->
    incr pos; skip_ws s pos;
    if s.[!pos] = '}' then (incr pos; JObject [])
    else begin
      let pairs = ref [] in
      let go = ref true in
      while !go do
        skip_ws s pos;
        let k = parse_string s pos in
        skip_ws s pos;
        if s.[!pos] <> ':' then parse_error "expected ':'";
        incr pos;
        let v = parse_value s pos in
        pairs := (k, v) :: !pairs;
        skip_ws s pos;
        if !pos < String.length s && s.[!pos] = ',' then incr pos
        else go := false
      done;
      skip_ws s pos;
      if s.[!pos] <> '}' then parse_error "expected '}'";
      incr pos;
      JObject (List.rev !pairs)
    end
  | c when c >= '0' && c <= '9' || c = '-' ->
    let start = !pos in
    while !pos < String.length s &&
          (let ch = s.[!pos] in
           (ch >= '0' && ch <= '9') || ch = '.' || ch = '-' || ch = 'e' || ch = 'E')
    do incr pos done;
    let tok = String.sub s start (!pos - start) in
    (try JInt (int_of_string tok)
     with _ -> try JFloat (float_of_string tok)
     with _ -> parse_error ("bad number: " ^ tok))
  | c -> parse_error (Printf.sprintf "unexpected char '%c'" c)

let parse s =
  let pos = ref 0 in
  parse_value s pos

let () =
  let v = JObject [
    ("name", JString "Alice");
    ("age", JInt 30);
    ("scores", JArray [JInt 95; JInt 87; JInt 100]);
    ("active", JBool true);
    ("address", JNull);
  ] in
  let s = to_string v in
  Printf.printf "Serialized:\n%s\n\n" s;
  let v2 = parse s in
  Printf.printf "Re-serialized:\n%s\n" (to_string v2)

✓ Tests Rust test suite

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_null() {
        assert_eq!(JsonValue::Null.to_json(), "null");
    }

    #[test]
    fn test_bool() {
        assert_eq!(JsonValue::Bool(true).to_json(), "true");
        assert_eq!(JsonValue::Bool(false).to_json(), "false");
    }

    #[test]
    fn test_number() {
        assert_eq!(JsonValue::Number(42.0).to_json(), "42");
        assert_eq!(JsonValue::Number(3.14).to_json(), "3.14");
    }

    #[test]
    fn test_string() {
        assert_eq!(
            JsonValue::String("hello".to_string()).to_json(),
            "\"hello\""
        );
    }

    #[test]
    fn test_string_escape() {
        assert_eq!(
            JsonValue::String("a\"b".to_string()).to_json(),
            "\"a\\\"b\""
        );
        assert_eq!(JsonValue::String("a\nb".to_string()).to_json(), "\"a\\nb\"");
    }

    #[test]
    fn test_array() {
        let arr = JsonValue::Array(vec![
            JsonValue::Number(1.0),
            JsonValue::Number(2.0),
            JsonValue::Number(3.0),
        ]);
        assert_eq!(arr.to_json(), "[1, 2, 3]");
    }

    #[test]
    fn test_object() {
        let obj = JsonValue::Object(vec![
            ("a".to_string(), JsonValue::Number(1.0)),
            ("b".to_string(), JsonValue::Number(2.0)),
        ]);
        assert_eq!(obj.to_json(), r#"{"a": 1, "b": 2}"#);
    }

    #[test]
    fn test_trait() {
        assert_eq!(42i32.to_json_value().to_json(), "42");
        assert_eq!("hello".to_json_value().to_json(), "\"hello\"");
    }

    #[test]
    fn test_option() {
        let some: Option<i32> = Some(42);
        let none: Option<i32> = None;
        assert_eq!(some.to_json_value().to_json(), "42");
        assert_eq!(none.to_json_value().to_json(), "null");
    }
}

Deep Comparison

OCaml vs Rust: JSON Format From Scratch

JSON Value Type

Rust

pub enum JsonValue {
    Null,
    Bool(bool),
    Number(f64),
    String(String),
    Array(Vec<JsonValue>),
    Object(Vec<(String, JsonValue)>),
}

OCaml (Yojson-like)

type json =
  | `Null
  | `Bool of bool
  | `Float of float
  | `String of string
  | `List of json list
  | `Assoc of (string * json) list

Serialization

Rust

impl JsonValue {
    pub fn to_json(&self) -> String {
        match self {
            JsonValue::Null => "null".to_string(),
            JsonValue::Bool(b) => b.to_string(),
            JsonValue::Number(n) => n.to_string(),
            JsonValue::String(s) => format!("\"{}\"", escape(s)),
            JsonValue::Array(arr) => format!("[{}]", 
                arr.iter().map(|v| v.to_json()).collect::<Vec<_>>().join(", ")),
            JsonValue::Object(obj) => format!("{{{}}}",
                obj.iter().map(|(k, v)| format!("\"{}\": {}", k, v.to_json()))
                    .collect::<Vec<_>>().join(", ")),
        }
    }
}

OCaml

let rec to_string = function
  | `Null -> "null"
  | `Bool b -> string_of_bool b
  | `Float f -> string_of_float f
  | `String s -> Printf.sprintf "\"%s\"" (escape s)
  | `List lst -> Printf.sprintf "[%s]" 
      (String.concat ", " (List.map to_string lst))
  | `Assoc pairs -> Printf.sprintf "{%s}"
      (String.concat ", " (List.map (fun (k, v) -> 
        Printf.sprintf "\"%s\": %s" k (to_string v)) pairs))

Key Differences

Aspect	OCaml	Rust
Variant syntax	Polymorphic variants	Enum
String format	`Printf.sprintf`	`format!`
List join	`String.concat`	`.join()`
Escape handling	Manual function	Manual function

Exercises

Implement from_json(s: &str) -> Result<JsonValue, ParseError> — a simple recursive descent JSON parser for the types in the JsonValue enum.

Add merge_objects that combines two JsonValue::Object values, with the second's values overriding the first's for duplicate keys.

Implement json_path_get(root: &JsonValue, path: &str) -> Option<&JsonValue> that navigates a dot-separated path like "users.0.name" through a JSON tree.

Open Source Repos

functional-rust

View the source for this example on GitHub — OCaml and Rust side by side in the repo.

Rust