# BMP Streaming Decoder API Proposal

## Executive Summary

This proposal introduces a streaming/resumable decoding API for the BMP decoder in the `image` crate. The API enables incremental decoding of BMP images without requiring all data to be available upfront, which is essential for network streaming, progressive rendering, and memory-constrained environments.

## Motivation

### Current Limitations

The existing BMP decoder requires:
1. **Complete decoding upfront**: All image data must be decoded into a complete output buffer before any pixels are accessible
2. **Blocking I/O**: Uses `read_exact()` which blocks until all requested data is available
3. **Memory overhead**: Large images require allocating the entire decoded buffer upfront (width × height × channels bytes)

**Note**: While the input data must be available via a `BufRead + Seek` reader (which may buffer network data), the primary limitation is that decoding happens all-at-once rather than incrementally. The streaming API enables row-by-row decoding, reducing peak memory usage from O(image_size) to O(row_width).

### Use Cases Requiring Streaming

1. **Network Streaming**: Decode BMP images as they download over HTTP/network connections
2. **Progressive Rendering**: Display image rows as they become available for better UX
3. **Memory-Constrained Devices**: Process images row-by-row to minimize memory footprint
4. **Large Image Processing**: Handle multi-gigabyte BMPs without exhausting RAM
5. **Embedded Systems**: Real-time image processing with limited buffering

### Real-World Example: Chromium's Blink decoders

Chromium's Blink rendering engine implements incremental BMP decoding in `BmpImageReader`. The decoder supports:
- Incremental metadata reading
- Row-by-row pixel decoding
- Resumable state after partial reads
- Progressive rendering as data arrives over network

This proposal brings similar capabilities to Rust's `image` crate.

## Design Principles

### 1. Backward Compatibility
- **Zero breaking changes**: All existing APIs remain unchanged
- **Opt-in**: Streaming mode is explicitly enabled, not default
- **Drop-in replacement**: Existing code continues working without modifications

### 2. Minimal API Surface
- Introduce only essential new methods
- Leverage existing `BufRead + Seek` trait bounds
- Avoid complexity where possible

### 3. Composability
- Methods work independently but compose well
- Support both imperative (`try_read_row()`) and iterator (`into_rows()`) patterns
- Enable custom processing pipelines

### 4. Error Handling Philosophy
- **"Needs more data" has its specific error**: Define a new `ImageError::InsufficientData` to report the case that more data is needed to complete decoding.
- **Avoids `UnexpectedEof` overloading**: Don't reuse existing error types for normal streaming states
- **Clear semantics**: `Result<Option<T>>` pattern cleanly separates three outcomes:
- `Ok(Some(value))` - operation succeeded with data
- `Ok(None)` - operation succeeded but no more data to decode
- `Err(ImageError::InsufficientData)` - there is missing data to complete decoding. This is a recoverable error
- `Err(error)` - operation failed due to invalid data or I/O error

### 5. Async and Coroutine Compatibility
- **Non-blocking API contract**: The `Result<Option<T>>` return type allows callers to:
- Call the method when data is available
- Get (newly defined) `ImageError::InsufficientData` when more data is needed
- Yield control back to event loop/async runtime
- Resume later when more data arrives
- **Event-driven friendly**: No assumptions about data availability; caller controls when to retry
- **Future async support**: API signature is compatible with async/await patterns:
- Can be wrapped in async functions that `await` on data availability
- No assigning double significate to existing IO errors by defining a new ImageError enum value for the case in which we are missing data to continue decoding.
- No breaking changes needed when adding async reader support

### 6. BMP-specific API
- Designing a format-agnostic API is explicitly a non-goal
- Rationale: row-by-row BMP decoding requires a different API than progressive JPG/JXL decoding (or streaming decoding of _interlaced_ PNG images).
- In the future (once we gain experience and confidence in format-specific APIs) we may want to revisit this and consider designing a format-agnostic API (if one is possible)

## API Design

### 1. State Management

#### New Field: `streaming_state`

pub struct BmpDecoder<R: BufRead + Seek> {
    // ... existing fields ...
    streaming_state: Option<DecoderState>,
}

**Rationale:**
- `Option<DecoderState>` allows backward compatibility (None = traditional mode)
- Explicitly tracks decode progress through all phases
- Enables resumption after partial reads

#### New Enum: `DecoderState` (internal)

#[derive(Clone, Debug)]
enum DecoderState {  // Note: not pub - internal implementation detail
    Initial,
    ReadingFileHeader { bytes_read: usize, buffer: Vec<u8> },
    ReadingDibHeaderSize { bytes_read: usize, buffer: Vec<u8> },
    ReadingDibHeader { header_size: u32, bytes_read: usize, buffer: Vec<u8> },    
    ReadingIccProfile { offset: u64, size: u32, bytes_read: usize, buffer: Vec<u8> },
    ReadingPalette { entries_read: u32, total_entries: u32, bytes_per_color: usize, buffer: Vec<u8> },
    ReadingBitmasks { bytes_read: usize, total_bytes: usize, buffer: Vec<u8> },
    ReadyForPixelData { current_row: i32, row_byte_length: usize },
    ReadingPixelRow { row_index: i32, bytes_read: usize, row_buffer: Vec<u8> },
    Complete,
}

**Visibility:**
- `DecoderState` is private (internal implementation detail)
- Users never directly access or construct state values
- State is managed automatically by public methods (`try_read_metadata()`, `read_row()`)
- Keeps API surface minimal and allows internal state machine changes without breaking compatibility

**Rationale:**
- **Explicit states**: Each decode phase is clearly represented
- **Resumable**: Stores partial progress (bytes_read, buffers) to resume after data arrival
- **Type-safe**: Impossible to be in invalid state combinations
- **Buffer storage**: Each state owns necessary buffers, avoiding separate allocations

#### New Constructor: `new_streaming()`

pub fn new_streaming(reader: R) -> ImageResult<BmpDecoder<R>>

**Returns:**
- `Ok(BmpDecoder)` - Decoder initialized in streaming mode, ready for `try_read_metadata()`
- `Err(_)` - Reader initialization failed

**Rationale:**
- **Explicit opt-in**: Different constructor makes streaming mode choice visible at call site
- **No automatic metadata loading**: Unlike `new()`, doesn't call `read_metadata()` internally
- **Initialized state**: Sets `streaming_state = Some(DecoderState::Initial)` automatically
- **Same reader requirements**: Takes same `R: BufRead + Seek` as traditional constructor
- **Avoiding breaking changes**: Clients using old constructors expect blocking IO, but the new constructor shouldn't block.

#### Enabling Streaming Mode

To use the streaming API, create a decoder with `new_streaming()` instead of `new()`:

// Traditional mode (auto-loads metadata on construction)
let decoder = BmpDecoder::new(reader)?;

// Streaming mode (opt-in, manual metadata loading)
let mut decoder = BmpDecoder::new_streaming(reader)?;

// Now use streaming methods
decoder.try_read_metadata()?;
decoder.read_row(&mut buffer)?;

**Rationale:**
- **Explicit opt-in**: Streaming mode must be explicitly enabled via constructor choice
- **Backward compatible**: Existing `new()` continues to work unchanged
- **Clear separation**: Different constructor signals different behavior and expectations
- **No manual state setup**: Constructor handles `streaming_state` initialization internally

**State Transitions:**

Initial 
  → ReadingFileHeader (14 bytes)
  → ReadingDibHeaderSize (4 bytes)
  → ReadingDibHeader (variable size, extracts ICC metadata if V5)
  → ReadingBitmasks? (optional 12-16 bytes, carries ICC metadata)
  → ReadingPalette? (optional, variable size, carries ICC metadata)
  → ReadyForPixelData (reads pixel rows, carries ICC metadata)
  → ReadingIccProfile? (optional, after ALL pixel data, for V5 with embedded profiles)
  → Complete

### 2. Metadata Reading

#### New Method: `try_read_metadata()`

pub fn try_read_metadata(&mut self) -> ImageResult<Option<()>>

**Returns:**
- `Ok(Some(()))` - Metadata fully loaded, ready for pixel reading
- `Err(ImageError::InsufficientData)` - there is missing data to complete decoding. This is a recoverable error
- `Err(_)` - Actual decode error (invalid format, I/O failure, etc.)

**Rationale:**
- **Non-blocking pattern**: `Option<T>` is idiomatic Rust for "may need more attempts"
- **Composable**: Easy to integrate into async loops or event-driven systems
- **Clear contract**: Three distinct outcomes, no ambiguity
- **No error variant pollution**: "Needs more data" returns its specific error `ImageError::InsufficientData` instead of giving a new meaning to an already existing error.

**Why `Result<Option<()>>` instead of just `Result<bool>` or a new error variant?**
- `Option<()>` is more idiomatic than `bool` for "incomplete/complete" semantics
- Matches patterns in other Rust libraries (e.g., `Iterator::next()`)
- Clearer intent: `Some(())` = "got something", `None` = "nothing yet"
- Avoids polluting error enums with non-error states
- Reserves `Err(_)` exclusively for actual failures

**Example Usage:**

// Event-driven processing
loop {
    match decoder.try_read_metadata()? {
        Some(()) => break, // Metadata complete
        None => {
            // Wait for more data
            wait_for_data(&mut reader)?;
        }
    }
}

### 3. Pixel Data Reading

#### New Method: `try_read_row()`

pub fn try_read_row(&mut self, buf: &mut [u8]) -> ImageResult<Option<usize>>

**Parameters:**
- `buf`: Pre-allocated buffer of size `width * channels`
- Use existing `ImageDecoder` trait methods to determine size:
- `dimensions()` returns `(width, height)`
- `color_type()` or derived methods provide channel count
- Row size = `width * num_channels`

**Returns:**
- `Ok(Some(row_index))` - Row successfully decoded, returns which row (0-based)
- `Ok(None)` - operation succeeded but no more data to decode
- `Err(ImageError::InsufficientData)` - there is missing data to complete decoding. This is a recoverable error
- `Err(_)` - Decode error

**Note**: In the case that all rows have already been decoded and try_next_row is called again, it is expected to return a DecoderError as the caller shouldn't call for more rows than the image has.

**Row Ordering:**
- **Streaming mode**: Rows returned in file storage order without automatic flipping
- Bottom-up BMPs: row 0 is bottom of image, incrementing upward
- Top-down BMPs: row 0 is top of image, incrementing downward
- Use `is_top_down()` to determine orientation and flip if needed for display
- **Important difference from traditional decoder**: The non-streaming `BmpDecoder::new()` automatically flips bottom-up images to top-down order. The streaming API returns rows as stored in the file to enable incremental processing.

**Rationale:**
- **Memory efficiency**: Caller controls allocation, can reuse same buffer
- **Progress tracking**: Returns row index for progress bars/status updates
- **Explicit termination**: `None` clearly indicates completion
- **Buffer validation**: Checks size matches expected row width
- **Streaming-compatible**: Cannot pre-flip all rows; must return as read from file

**Why require pre-allocated buffer?**
- Allows buffer reuse across rows (zero-allocation loop)
- Caller can use stack allocation for small images
- More flexible than forcing heap allocation

**Example Usage:**

use image::codecs::bmp::BmpDecoder;
use std::fs::File;
use std::io::BufReader;

let file = File::open("image.bmp")?;
let reader = BufReader::new(file);
let mut decoder = BmpDecoder::new_streaming(reader)?;

// Load metadata first
decoder.try_read_metadata()?;

// Get dimensions to allocate buffer
let (width, height) = decoder.dimensions();
let channels = 3; // RGB - or use color_type() to determine
let row_size = width as usize * channels;
let mut row_buffer = vec![0u8; row_size];

// Read all rows
while let Some(row_idx) = decoder.try_read_row(&mut row_buffer)? {
    // Process row (your custom logic here)
    process_row(row_idx, &row_buffer);
    // row_buffer is reused, no allocations in loop
}

#### New Method: `is_top_down()`

pub fn is_top_down(&self) -> bool

**Returns:**
- `true` if rows are stored top-down in the file (negative height in BMP header)
- `false` if rows are stored bottom-up (standard BMP orientation).

**Behavior difference:**
- **In streaming mode** (`new_streaming()`): Returns the actual file storage orientation
- **In traditional mode** (`new()`): Always returns `true` because decoder normalizes to top-down during `new()`

**Rationale:**
- **Essential for streaming**: Callers need to know orientation to correctly display or process rows
- **Enables correct rendering**: Caller can flip rows for display if needed
- **Documents behavioral difference**: Makes explicit that streaming mode preserves file order

#### New Method: `into_rows()` (Not required for Chromium integration)

pub fn into_rows(&mut self) -> BmpRowIterator<'_, R>

**Returns:** Iterator yielding `ImageResult<Vec<u8>>`

**Rationale:**
- **Ergonomic**: Natural Rust idiom for sequential processing
- **Composable**: Works with iterator adapters (`map`, `filter`, `take`, etc.)
- **Convenience**: Handles allocation automatically for simple cases
- **Fallible iteration**: Errors propagate naturally through iterator

**Why allocate internally?**
- Many use cases don't need buffer reuse
- Simpler API for common case
- Users needing zero-copy can use `try_read_row()` directly

**Why provide both `try_read_row()` and `into_rows()`?**
- **`try_read_row()`**: Minimal API for performance-critical code (zero allocations, caller-controlled buffers)
- **`into_rows()`**: Ergonomic API for typical usage (iterator pattern, automatic allocation)
- **Different use cases**: Performance vs convenience tradeoff left to caller
- **Precedent**: Similar to `BufRead::read_line()` vs `BufRead::lines()` in std

**Example Usage:**

// Simple iteration
for row in decoder.into_rows() {
    let pixels = row?;
    display_row(&pixels);
}

// With iterator adapters
let processed: Vec<_> = decoder.into_rows()
    .map(|r| r.map(|pixels| apply_filter(&pixels)))
    .collect::<ImageResult<_>>()?;

### 4. Supporting Type

#### New Struct: `BmpRowIterator` (Not required for Chromium integration)

pub struct BmpRowIterator<'a, R: BufRead + Seek> {
    decoder: &'a mut BmpDecoder<R>,
    row_buffer: Vec<u8>,
}

impl<'a, R: BufRead + Seek> Iterator for BmpRowIterator<'a, R> {
    type Item = ImageResult<Vec<u8>>;
    // ...
}

**Rationale:**
- **Standard trait**: Implements `Iterator` for Rust ecosystem compatibility
- **Owned buffers**: Each iteration returns owned `Vec<u8>` for safety
- **Lazy evaluation**: Rows decoded on-demand, not all upfront
- **Error handling**: Yields `Result` to handle I/O errors gracefully