285 Intermediate

285: Building Custom Iterator Adapters

Functional Programming

Tutorial Video

Text description (accessibility)

This video demonstrates the "285: Building Custom Iterator Adapters" functional Rust example. Difficulty level: Intermediate. Key concepts covered: Functional Programming. The standard library's `map`, `filter`, and `zip` adapters cover many cases, but domain-specific transformations often need their own adapters — a rate limiter that throttles output, a deduplicator that removes consecutive duplicates, or a strider that yields every nth element. Key difference from OCaml: 1. **State storage**: Rust adapters store state in struct fields; OCaml adapters capture state in closure variables.

Tutorial

The Problem

The standard library's map, filter, and zip adapters cover many cases, but domain-specific transformations often need their own adapters — a rate limiter that throttles output, a deduplicator that removes consecutive duplicates, or a strider that yields every nth element. Building custom adapters in Rust follows the same pattern as the standard library: wrap an inner iterator in a struct and implement Iterator on it, making the adapter composable with the entire ecosystem.

🎯 Learning Outcomes

• Understand the adapter pattern: wrap I: Iterator in a struct, implement Iterator on the wrapper

• Build a EveryNth adapter that yields every nth element from any iterator

• Recognize that custom adapters gain the full standard library API for free

• Use generic struct parameters <I: Iterator> to make adapters work with any iterator source

Code Example

struct EveryNth<I> { inner: I, n: usize, count: usize }

impl<I: Iterator> Iterator for EveryNth<I> {
    type Item = I::Item;
    
    fn next(&mut self) -> Option<I::Item> {
        loop {
            let item = self.inner.next()?;
            let emit = self.count % self.n == 0;
            self.count += 1;
            if emit { return Some(item); }
        }
    }
}

let every_nth n seq =
  Seq.unfold (fun (i, rest) ->
    let rec skip_to k s =
      if k = 0 then
        match Seq.uncons s with
        | Some (v, rest') -> Some (v, (n-1, rest'))
        | None -> None
      else
        match Seq.uncons s with
        | Some (_, rest') -> skip_to (k-1) rest'
        | None -> None
    in
    skip_to i rest
  ) (0, seq)

Key Differences

State storage: Rust adapters store state in struct fields; OCaml adapters capture state in closure variables.

Type system integration: Rust adapters implement the Iterator trait and gain all standard adapters; OCaml functions on Seq.t gain all Seq module functions.

Zero-cost: Rust's struct-based adapters are monomorphized and inlined by the compiler; OCaml's functional adapters use closures with potential allocation.

Ecosystem integration: Custom Rust adapters work with Rayon parallel iterators if they also implement the right parallel traits.

OCaml Approach

OCaml's Seq module allows custom adapters as functions returning new sequences. Since Seq.t is just unit -> node, a custom adapter is simply a function that wraps the original sequence:

let every_nth n seq =
  let rec go i s () = match s () with
    | Seq.Nil -> Seq.Nil
    | Seq.Cons (x, rest) ->
      if i mod n = 0 then Seq.Cons (x, go (i+1) rest)
      else go (i+1) rest ()
  in go 0 seq

Both approaches create composable, lazy transformations.

Full Source

#![allow(clippy::all)]
//! # Building Custom Iterator Adapters
//!
//! Custom adapters wrap an iterator in a struct and implement `Iterator` on it.
//! This is the same pattern used by `map`, `filter`, and `zip` in the standard library.

/// Yields every nth element starting from the first
pub struct EveryNth<I> {
    inner: I,
    n: usize,
    count: usize,
}

impl<I: Iterator> EveryNth<I> {
    pub fn new(inner: I, n: usize) -> Self {
        assert!(n > 0, "n must be positive");
        EveryNth { inner, n, count: 0 }
    }
}

impl<I: Iterator> Iterator for EveryNth<I> {
    type Item = I::Item;

    fn next(&mut self) -> Option<I::Item> {
        loop {
            let item = self.inner.next()?;
            let emit = self.count % self.n == 0;
            self.count += 1;
            if emit {
                return Some(item);
            }
        }
    }
}

/// Yields adjacent pairs (a, b) as a sliding window of 2
pub struct Pairs<I: Iterator> {
    inner: I,
    prev: Option<I::Item>,
}

impl<I: Iterator> Pairs<I>
where
    I::Item: Clone,
{
    pub fn new(mut inner: I) -> Self {
        let prev = inner.next();
        Pairs { inner, prev }
    }
}

impl<I: Iterator> Iterator for Pairs<I>
where
    I::Item: Clone,
{
    type Item = (I::Item, I::Item);

    fn next(&mut self) -> Option<Self::Item> {
        let next = self.inner.next()?;
        let prev = self.prev.replace(next.clone())?;
        Some((prev, next))
    }
}

/// Adapter that applies a function to each element, keeping only Some results
pub struct FilterMapWith<I, F> {
    inner: I,
    f: F,
}

impl<I, F, B> FilterMapWith<I, F>
where
    I: Iterator,
    F: FnMut(I::Item) -> Option<B>,
{
    pub fn new(inner: I, f: F) -> Self {
        FilterMapWith { inner, f }
    }
}

impl<I, F, B> Iterator for FilterMapWith<I, F>
where
    I: Iterator,
    F: FnMut(I::Item) -> Option<B>,
{
    type Item = B;

    fn next(&mut self) -> Option<B> {
        loop {
            match (self.f)(self.inner.next()?) {
                Some(b) => return Some(b),
                None => continue,
            }
        }
    }
}

/// Extension trait to add our adapters to all iterators
pub trait IteratorExt: Iterator + Sized {
    fn every_nth(self, n: usize) -> EveryNth<Self> {
        EveryNth::new(self, n)
    }

    fn pairs(self) -> Pairs<Self>
    where
        Self::Item: Clone,
    {
        Pairs::new(self)
    }

    fn filter_map_with<F, B>(self, f: F) -> FilterMapWith<Self, F>
    where
        F: FnMut(Self::Item) -> Option<B>,
    {
        FilterMapWith::new(self, f)
    }
}

impl<I: Iterator> IteratorExt for I {}

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_every_nth_3() {
        let result: Vec<i32> = (0..9).every_nth(3).collect();
        assert_eq!(result, vec![0, 3, 6]);
    }

    #[test]
    fn test_every_nth_2() {
        let result: Vec<i32> = (0..10).every_nth(2).collect();
        assert_eq!(result, vec![0, 2, 4, 6, 8]);
    }

    #[test]
    fn test_pairs() {
        let result: Vec<(i32, i32)> = [1i32, 2, 3, 4].iter().copied().pairs().collect();
        assert_eq!(result, vec![(1, 2), (2, 3), (3, 4)]);
    }

    #[test]
    fn test_pairs_single_element() {
        let result: Vec<(i32, i32)> = [1i32].iter().copied().pairs().collect();
        assert!(result.is_empty());
    }

    #[test]
    fn test_every_nth_1_identity() {
        let result: Vec<i32> = [1i32, 2, 3].iter().copied().every_nth(1).collect();
        assert_eq!(result, vec![1, 2, 3]);
    }

    #[test]
    fn test_chain_adapters() {
        let result: Vec<(i32, i32)> = (0i32..20).every_nth(2).pairs().collect();
        assert_eq!(
            result,
            vec![
                (0, 2),
                (2, 4),
                (4, 6),
                (6, 8),
                (8, 10),
                (10, 12),
                (12, 14),
                (14, 16),
                (16, 18)
            ]
        );
    }

    #[test]
    fn test_filter_map_with() {
        let result: Vec<i32> = (0..10)
            .filter_map_with(|x| if x % 2 == 0 { Some(x * 10) } else { None })
            .collect();
        assert_eq!(result, vec![0, 20, 40, 60, 80]);
    }

    #[test]
    fn test_with_standard_adapters() {
        let result: Vec<i32> = (0..20).filter(|x| x % 2 == 0).every_nth(2).collect();
        assert_eq!(result, vec![0, 4, 8, 12, 16]);
    }
}

(* 285. Building custom iterator adapters - OCaml *)

(* Custom adapter: every_nth -- yield every nth element *)
let every_nth n seq =
  Seq.unfold (fun (i, rest) ->
    let rec skip_to k s =
      if k = 0 then
        match Seq.uncons s with
        | Some (v, rest') -> Some (v, (n-1, rest'))
        | None -> None
      else
        match Seq.uncons s with
        | Some (_, rest') -> skip_to (k-1) rest'
        | None -> None
    in
    skip_to i rest
  ) (0, seq)

(* Custom adapter: running_window -- sliding running tuple *)
let pairs seq =
  Seq.unfold (fun s ->
    match Seq.uncons s with
    | None -> None
    | Some (a, rest) ->
      match Seq.uncons rest with
      | None -> None
      | Some (b, _) -> Some ((a, b), Seq.drop 1 s)
  ) seq

let () =
  let nums = List.to_seq [1; 2; 3; 4; 5; 6; 7; 8; 9; 10] in
  let thirds = every_nth 3 nums in
  Printf.printf "Every 3rd: %s\n"
    (String.concat ", " (List.map string_of_int (List.of_seq thirds)));

  let data = List.to_seq [10; 20; 30; 40; 50] in
  let adjacent = pairs data in
  List.iter (fun (a, b) -> Printf.printf "(%d,%d) " a b) (List.of_seq adjacent);
  print_newline ()

✓ Tests Rust test suite

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_every_nth_3() {
        let result: Vec<i32> = (0..9).every_nth(3).collect();
        assert_eq!(result, vec![0, 3, 6]);
    }

    #[test]
    fn test_every_nth_2() {
        let result: Vec<i32> = (0..10).every_nth(2).collect();
        assert_eq!(result, vec![0, 2, 4, 6, 8]);
    }

    #[test]
    fn test_pairs() {
        let result: Vec<(i32, i32)> = [1i32, 2, 3, 4].iter().copied().pairs().collect();
        assert_eq!(result, vec![(1, 2), (2, 3), (3, 4)]);
    }

    #[test]
    fn test_pairs_single_element() {
        let result: Vec<(i32, i32)> = [1i32].iter().copied().pairs().collect();
        assert!(result.is_empty());
    }

    #[test]
    fn test_every_nth_1_identity() {
        let result: Vec<i32> = [1i32, 2, 3].iter().copied().every_nth(1).collect();
        assert_eq!(result, vec![1, 2, 3]);
    }

    #[test]
    fn test_chain_adapters() {
        let result: Vec<(i32, i32)> = (0i32..20).every_nth(2).pairs().collect();
        assert_eq!(
            result,
            vec![
                (0, 2),
                (2, 4),
                (4, 6),
                (6, 8),
                (8, 10),
                (10, 12),
                (12, 14),
                (14, 16),
                (16, 18)
            ]
        );
    }

    #[test]
    fn test_filter_map_with() {
        let result: Vec<i32> = (0..10)
            .filter_map_with(|x| if x % 2 == 0 { Some(x * 10) } else { None })
            .collect();
        assert_eq!(result, vec![0, 20, 40, 60, 80]);
    }

    #[test]
    fn test_with_standard_adapters() {
        let result: Vec<i32> = (0..20).filter(|x| x % 2 == 0).every_nth(2).collect();
        assert_eq!(result, vec![0, 4, 8, 12, 16]);
    }
}

Deep Comparison

OCaml vs Rust: Iterator Adapter Pattern

Pattern 1: Every Nth Element

OCaml

let every_nth n seq =
  Seq.unfold (fun (i, rest) ->
    let rec skip_to k s =
      if k = 0 then
        match Seq.uncons s with
        | Some (v, rest') -> Some (v, (n-1, rest'))
        | None -> None
      else
        match Seq.uncons s with
        | Some (_, rest') -> skip_to (k-1) rest'
        | None -> None
    in
    skip_to i rest
  ) (0, seq)

Rust

struct EveryNth<I> { inner: I, n: usize, count: usize }

impl<I: Iterator> Iterator for EveryNth<I> {
    type Item = I::Item;
    
    fn next(&mut self) -> Option<I::Item> {
        loop {
            let item = self.inner.next()?;
            let emit = self.count % self.n == 0;
            self.count += 1;
            if emit { return Some(item); }
        }
    }
}

Pattern 2: Sliding Window Pairs

OCaml

let pairs seq =
  Seq.unfold (fun s ->
    match Seq.uncons s with
    | None -> None
    | Some (a, rest) ->
      match Seq.uncons rest with
      | None -> None
      | Some (b, _) -> Some ((a, b), Seq.drop 1 s)
  ) seq

Rust

struct Pairs<I: Iterator> { inner: I, prev: Option<I::Item> }

impl<I: Iterator> Iterator for Pairs<I> 
where I::Item: Clone 
{
    type Item = (I::Item, I::Item);
    
    fn next(&mut self) -> Option<Self::Item> {
        let next = self.inner.next()?;
        let prev = self.prev.replace(next.clone())?;
        Some((prev, next))
    }
}

Pattern 3: Extension Trait

Rust

trait IteratorExt: Iterator + Sized {
    fn every_nth(self, n: usize) -> EveryNth<Self> {
        EveryNth::new(self, n)
    }
    fn pairs(self) -> Pairs<Self> where Self::Item: Clone {
        Pairs::new(self)
    }
}
impl<I: Iterator> IteratorExt for I {}

// Now usable on any iterator:
let result = (0..20).every_nth(3).pairs().collect();

Key Differences

Aspect	OCaml	Rust
Adapter type	Function over `Seq`	Struct + `impl Iterator`
Extension method	Module or `\|>` pipeline	Extension trait
Composability	`Seq` functions via `\|>`	Method chaining
Type signature	`'a Seq.t -> 'b Seq.t`	`Adapter<I> where I: Iterator`
Lazy evaluation	`Seq` is lazy	All iterators lazy

Exercises

Build a Deduplicate<I> adapter that yields consecutive elements only when they differ from the previous one (run-length deduplication).

Build a Buffered<I> adapter that collects elements into batches of size N and yields Vec<T> per batch.

Build a TimeoutIterator<I> adapter that stops yielding elements after a Duration has elapsed (simulated with an iteration count).

Open Source Repos

functional-rust

View the source for this example on GitHub — OCaml and Rust side by side in the repo.

Rust