# Hybrid Storage Pools

## Use Case
As a ceph Storage admin, I want to configure a Hybrid Storage Pool to improve read performance at a lower cost.

## What is a Hybrid Storage Pool?
Hybrid storage is a combination of two different storage tiers. For example SSD and HDD

This helps to improve the read performance of cluster by placing, for example, 1 copy of data on the higher performance tier (SSD or NVME) and 2 replicated copies on lower cost tier (HDDs).

## Rook

### API Changes

Update `PoolSpec` API to include option to setup Hybrid Storage Pool

type ReplicatedSpec struct {
	...
	// HybridStorage represents hybrid storage tier settings
	HybridStorage HybridStorageSpec `json:"hybridStorage,omitempty"`
}

// HybridStorageSpec reprsents the settings for hybrid storage pool
type HybridStorageSpec struct {
	// Enable hybrid storage tiering
	// +optional
	Enabled bool `json:"enabled,omitempty"`
	// PrimaryDeviceClass represents high performance tier (for example SSD or NVME) for Primary OSD
	// +optional
	PrimaryDeviceClass string `json:"primaryDeviceClass,omitempty"`
	// SecondaryDeviceClass represents low performance tier (for example HDDs) for remaining OSDs
	// +optional
	SecondaryDeviceClass string `json:"secondaryDeviceClass,omitempty"`
}

## API Validation:
If user creates `HybridStorageSpec.PrimaryDeviceClass = "ssd"`, then rook should validate whether `ssd` device class is available in the crush map or not. Same goes for `SecondaryDeviceClass` values as well.

## Setup
Rook would create the two step crush rule for hybrid storage pool if user creates CephBlockPool CR with `HybridStorageSpec.Enabled = true`

Two step crush rule:
1. First step: choose the primary OSD (firstn 1) in the high performance device class, “ssd” in the example
2. Second step: choose the rest of the OSDs (firstn -1) in the low performance device class, “root-hdd” in the example

# Hybrid storage policy
rule hybrid {
  ruleset 2
  type replicated
  min_size 1
  max_size 10
  step take default class ssd
  step chooseleaf firstn 1 type host
  step emit
  step take default class hdd
  step chooseleaf firstn -1 type host
  step emit
}