757 Fundamental

757-golden-file-tests — Golden File Tests

Functional Programming

Tutorial Video

Text description (accessibility)

This video demonstrates the "757-golden-file-tests — Golden File Tests" functional Rust example. Difficulty level: Fundamental. Key concepts covered: Functional Programming. Snapshot testing and golden file testing are closely related: both compare actual output against stored expected output. Key difference from OCaml: 1. **Inline vs file**: OCaml's `expect_test` stores expected output inline; Rust's golden file approach uses separate files (closer to the compiler testing tradition).

Tutorial

The Problem

Snapshot testing and golden file testing are closely related: both compare actual output against stored expected output. Golden files differ in that the expected output is stored as human-readable files you commit to version control, making output changes visible in code review. This pattern is standard in compilers (test output for each source file), documentation generators, and code formatters. It makes regressions visible as diffs rather than assertion failures.

🎯 Learning Outcomes

• Store expected output in tests/golden/ files committed to version control

• Compare rendered output against golden files with informative diff-like error messages

• Use an UPDATE_GOLDEN=1 environment variable to regenerate golden files

• Test complex transformations (Markdown-to-HTML, JSON formatting) against stable golden output

• Understand the golden file review workflow: generate, review diff, commit

Code Example

pub fn assert_golden(name: &str, actual: &str, golden_dir: &Path) {
    let golden_path = golden_dir.join(format!("{}.golden", name));
    
    if !golden_path.exists() {
        fs::write(&golden_path, actual)?;
        return;
    }
    
    let expected = fs::read_to_string(&golden_path)?;
    assert_eq!(actual, expected, "Golden test '{}' failed!", name);
}

let%expect_test "markdown rendering" =
  let output = render_markdown "# Title" in
  print_string output;
  [%expect {| <h1>Title</h1> |}]

Key Differences

Inline vs file: OCaml's expect_test stores expected output inline; Rust's golden file approach uses separate files (closer to the compiler testing tradition).

Review workflow: Both use a "promote/update" workflow; Rust uses UPDATE_GOLDEN=1 env var; OCaml uses dune promote.

Diff output: OCaml's Alcotest.check string provides colored line diffs; this example shows the first differing line; the insta crate provides full colored diffs.

Granularity: Golden files are per-transformation; expect_test is per-assertion within a test function.

OCaml Approach

OCaml's compiler test suite uses golden files extensively — each .ml test file has a corresponding .expected file. dune runtest and dune promote manage the workflow. The expect_test framework (Jane Street) is an inline variant. For standalone golden tests, OCaml uses read_file, runs the transformation, and calls assert_equal ~pp_diff with Alcotest for pretty diff output on failure.

Full Source

#![allow(clippy::all)]
//! # Golden File Tests
//!
//! Testing by comparing output against known-good "golden" files.

use std::fs;
use std::path::Path;

/// Render a markdown document
pub fn render_markdown(input: &str) -> String {
    let mut output = String::new();
    for line in input.lines() {
        if line.starts_with("# ") {
            output.push_str(&format!("<h1>{}</h1>\n", &line[2..]));
        } else if line.starts_with("## ") {
            output.push_str(&format!("<h2>{}</h2>\n", &line[3..]));
        } else if line.starts_with("- ") {
            output.push_str(&format!("<li>{}</li>\n", &line[2..]));
        } else if line.is_empty() {
            output.push_str("<br/>\n");
        } else {
            output.push_str(&format!("<p>{}</p>\n", line));
        }
    }
    output
}

/// Pretty print JSON
pub fn pretty_json(input: &str) -> String {
    // Simple formatter - just add indentation
    let mut output = String::new();
    let mut indent = 0;
    let mut in_string = false;

    for ch in input.chars() {
        if ch == '"' && !in_string {
            in_string = true;
            output.push(ch);
        } else if ch == '"' && in_string {
            in_string = false;
            output.push(ch);
        } else if in_string {
            output.push(ch);
        } else {
            match ch {
                '{' | '[' => {
                    output.push(ch);
                    output.push('\n');
                    indent += 2;
                    output.push_str(&" ".repeat(indent));
                }
                '}' | ']' => {
                    output.push('\n');
                    indent = indent.saturating_sub(2);
                    output.push_str(&" ".repeat(indent));
                    output.push(ch);
                }
                ',' => {
                    output.push(ch);
                    output.push('\n');
                    output.push_str(&" ".repeat(indent));
                }
                ':' => {
                    output.push_str(": ");
                }
                ' ' | '\n' | '\t' | '\r' => {}
                _ => output.push(ch),
            }
        }
    }
    output
}

/// Assert output matches golden file
pub fn assert_golden(name: &str, actual: &str, golden_dir: &Path) {
    let golden_path = golden_dir.join(format!("{}.golden", name));

    if !golden_path.exists() {
        fs::write(&golden_path, actual).expect("failed to write golden file");
        println!("Created golden file: {:?}", golden_path);
        return;
    }

    let expected = fs::read_to_string(&golden_path).expect("failed to read golden file");

    if actual != expected {
        let diff_path = golden_dir.join(format!("{}.actual", name));
        fs::write(&diff_path, actual).expect("failed to write actual file");
        panic!(
            "Golden test '{}' failed!\nExpected:\n{}\nActual:\n{}\nActual saved to: {:?}",
            name, expected, actual, diff_path
        );
    }
}

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_render_markdown_headers() {
        let input = "# Title\n## Subtitle";
        let output = render_markdown(input);
        assert!(output.contains("<h1>Title</h1>"));
        assert!(output.contains("<h2>Subtitle</h2>"));
    }

    #[test]
    fn test_render_markdown_list() {
        let input = "- Item 1\n- Item 2";
        let output = render_markdown(input);
        assert!(output.contains("<li>Item 1</li>"));
        assert!(output.contains("<li>Item 2</li>"));
    }

    #[test]
    fn test_render_markdown_paragraph() {
        let input = "Hello world";
        let output = render_markdown(input);
        assert!(output.contains("<p>Hello world</p>"));
    }

    #[test]
    fn test_pretty_json_object() {
        let input = r#"{"a":1,"b":2}"#;
        let output = pretty_json(input);
        assert!(output.contains("\"a\": 1"));
        assert!(output.contains("\"b\": 2"));
    }

    #[test]
    fn test_pretty_json_array() {
        let input = r#"[1,2,3]"#;
        let output = pretty_json(input);
        assert!(output.contains("1"));
        assert!(output.contains("2"));
        assert!(output.contains("3"));
    }

    #[test]
    fn test_pretty_json_nested() {
        let input = r#"{"outer":{"inner":42}}"#;
        let output = pretty_json(input);
        assert!(output.contains("\"inner\": 42"));
    }
}

(* 757: Golden File Tests — OCaml *)

(* Code under test: renders an AST to a string *)
type expr =
  | Num   of int
  | Add   of expr * expr
  | Mul   of expr * expr
  | Var   of string

let rec render = function
  | Num n       -> string_of_int n
  | Var s       -> s
  | Add (a, b)  -> Printf.sprintf "(%s + %s)" (render a) (render b)
  | Mul (a, b)  -> Printf.sprintf "(%s * %s)" (render a) (render b)

(* Golden file infrastructure *)
let read_file path =
  try
    let ic = open_in path in
    let n = in_channel_length ic in
    let s = Bytes.create n in
    really_input ic s 0 n;
    close_in ic;
    Some (Bytes.to_string s)
  with Sys_error _ -> None

let write_file path content =
  let oc = open_out path in
  output_string oc content;
  close_out oc

let golden_dir = "tests/golden"

let assert_golden name actual =
  let path = Printf.sprintf "%s/%s.txt" golden_dir name in
  let update = (try Sys.getenv "UPDATE_GOLDEN" = "1" with Not_found -> false) in
  match read_file path, update with
  | None, _ | _, true ->
    (try Unix.mkdir golden_dir 0o755 with Unix.Unix_error _ -> ());
    write_file path actual;
    Printf.printf "[golden:%s] %s\n" name (if update then "Updated" else "Created")
  | Some expected, false ->
    if actual = expected
    then Printf.printf "[golden:%s] OK\n" name
    else begin
      Printf.printf "[golden:%s] MISMATCH\n" name;
      Printf.printf "Expected:\n%s\nActual:\n%s\n" expected actual;
      failwith "golden test failed"
    end

let () =
  let expr = Add (Mul (Num 2, Var "x"), Num 3) in
  assert_golden "expr_render" (render expr);
  let complex = Mul (Add (Var "a", Var "b"), Add (Num 1, Num 2)) in
  assert_golden "complex_expr" (render complex);
  Printf.printf "Golden tests done!\n"

✓ Tests Rust test suite

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_render_markdown_headers() {
        let input = "# Title\n## Subtitle";
        let output = render_markdown(input);
        assert!(output.contains("<h1>Title</h1>"));
        assert!(output.contains("<h2>Subtitle</h2>"));
    }

    #[test]
    fn test_render_markdown_list() {
        let input = "- Item 1\n- Item 2";
        let output = render_markdown(input);
        assert!(output.contains("<li>Item 1</li>"));
        assert!(output.contains("<li>Item 2</li>"));
    }

    #[test]
    fn test_render_markdown_paragraph() {
        let input = "Hello world";
        let output = render_markdown(input);
        assert!(output.contains("<p>Hello world</p>"));
    }

    #[test]
    fn test_pretty_json_object() {
        let input = r#"{"a":1,"b":2}"#;
        let output = pretty_json(input);
        assert!(output.contains("\"a\": 1"));
        assert!(output.contains("\"b\": 2"));
    }

    #[test]
    fn test_pretty_json_array() {
        let input = r#"[1,2,3]"#;
        let output = pretty_json(input);
        assert!(output.contains("1"));
        assert!(output.contains("2"));
        assert!(output.contains("3"));
    }

    #[test]
    fn test_pretty_json_nested() {
        let input = r#"{"outer":{"inner":42}}"#;
        let output = pretty_json(input);
        assert!(output.contains("\"inner\": 42"));
    }
}

Deep Comparison

OCaml vs Rust: Golden File Tests

Golden File Assertion

Rust

pub fn assert_golden(name: &str, actual: &str, golden_dir: &Path) {
    let golden_path = golden_dir.join(format!("{}.golden", name));
    
    if !golden_path.exists() {
        fs::write(&golden_path, actual)?;
        return;
    }
    
    let expected = fs::read_to_string(&golden_path)?;
    assert_eq!(actual, expected, "Golden test '{}' failed!", name);
}

OCaml (expect_test)

let%expect_test "markdown rendering" =
  let output = render_markdown "# Title" in
  print_string output;
  [%expect {| <h1>Title</h1> |}]

Updating Golden Files

Rust

# Typically delete .golden files and re-run tests
rm tests/golden/*.golden
cargo test

OCaml

dune runtest --auto-promote

Key Differences

Aspect	OCaml	Rust
Storage	Inline in source	External .golden files
Update	`--auto-promote`	Delete and regenerate
Library	expect_test (ppx)	Manual or insta crate
Diff display	Built-in	Manual

Exercises

Add a render_csv_to_markdown_table function and create a golden file for its output. Write a test that regenerates it with UPDATE_GOLDEN=1 cargo test.

Implement assert_golden_json that normalizes JSON whitespace before comparing, so formatting changes don't cause spurious golden file failures.

Write a script scripts/update_golden.sh that runs all tests with UPDATE_GOLDEN=1 and prints the number of golden files updated.

Open Source Repos

functional-rust

View the source for this example on GitHub — OCaml and Rust side by side in the repo.

Rust