### Lecture 09 - Device and file management

**I/O manager** - is used for external device management for input and output, such as: hard drives, keyboard, screens and network connection. It can be formulated as a an interface between the devices and the rest of the OS. It's tasks are outputting commnads to external devices, interrupt processing and error handling
![[Pasted image 20240218224800.png]]
![[Pasted image 20240219104812.png|inlR|400]]
Each device has it's own transfer/data rate in (bit/s || byte/s) for example:

| Device | Transfer rate or data rate |
| ---- | ---- |
| Keyboard | approx. 10 bytes/s |
| Mouse | approx. 100 bytes/s |
| Laser printer | approx. 100 KByte/s |
| USB3 | 5Gbps |
| ISA bus | 16 MB/s |
| Fast Ethernet | 12.5 MB/s |
| Serial Attached SCSI | 750 MB/s |
| Tape | approx. 320 MB/s |
| PCI Express | 32 Gbps |
| IDE hard drives with ATA | approx. 130 MB/s |
| Hard drives with S-ATA | Up to almost 2 GB/s |
| Bus communication system AMD HyperTransport or Intel QPI | GBit/s to over 100 GBit/s |
**Driver** - are device specific program parts, they are usually implemented in their own modules by the device manufacturer, and it's most important task is to run initialisation and announcement of the device, to do a data transfer to an from a device, translate logical programming model to device specific requirement as well as buffering of data, interrupt processing and coordination of concurrent addresses.

There are various **types of devices** that can interact with computer system, most notable are:
- **Serial access devices** - or serial devices no need for addressing information is necessary, reading and writing goes character by character and examples are *mouse slow line printer*
- **Character-oriented devices** - information is stored and read in individual characters, data is read in streams and characters are note addressable, examples are *printer, mouse and network cards*
- **Block oriented devices** - are optional and information is stored and read in blocks of at least 512 bytes, they also addressable and accessable such examples are *hard drives, tapes and etc*
- **Random access devices** - or direct access are where information addressing is required such devices are disk storages and are addressed via track numer, sector number or cylinder number
- Other devices such as clock timers

**Hard drive** - consist of plastic or aluminuim discs whic hare covered with an extremely thin magnetic layer, it also has read/write heads that are equipped with a small electromagnet. Data is store in blocks of fixed size sectors so it can be found. Arrangement of the discs is in cylinders, since each disk is lying on top of the other and all tracks with the same number on which the read/write heads can position at the same time
![[Pasted image 20240219115316.png|inlR|400]]![[Pasted image 20240219115350.png|inlR|300]]

Disks are divided in tracks, cylinders, which are physical sectors it can be represented as a pice of cake. All tracks in the circle are called cylinders
![[Pasted image 20240219115510.png]]
Common plate diametar is between 3.25 or 2.5 inches or smaller, with 2 or 8 plates. The capacity of these are 39 sectors with 761 cylinders and 8 heads with 512 byte sectors, which results in: $39 \times 761 \times 8 \times 512 = 121,565,184$ bytes it can grow to capacity to several TB and it has massively improved last 20 years 200 fields but the access time has only stayed the same.

The operating system knows logical sectors in the number order which are labels are LSN (Logical Sector Number). The translation from LSN into physical address is required, the number that needs to be translated is quadruple form of sequence from 1 to N uniquely separated by come (g, z, h, s) which are:
g - drive
z - cylinder number
h - number of the read write head
s - sector nymber

**Scheduling Strategy** for a disk driver must be set so that process of read/write jobs is as optimal and fair as possible, known strategies are:
- FCFS (First Come Firs Serve) - which is simple and fair strategy which executes all orders in the order in which they arrive
- SSTF (Shortes Seek Time First) - which prioritieses the execution of jobs that are closer in proximity to the current read/write head position, it avoids the head movement and as the jobs arrive always the closes will be picked over the one that is further away, which can result into job starvation.

**Devices in Unix** - are assigned "special file" so that they can be treated as one. For example directory `/dev` extract names depending on unix derivative, and OS is able to make system calls like open, read, write. Special files can be divided into:
- **Block special file** like `/dev/hd1` (hard drive)
- **Character special file** like `/dev/tty` (screen, keyboard etc)
Example command `cp file /dev/lp` copies a file called `file` directly to the printer

#### File system

**File** - are an abstract mechanism for storing and retrieving information, the part of the OS that deals with files and their organisation is called file management, since specific info needs to managed for each file. Files an have name and content, as well as type of file, length in bytes, block that make up the file, access rights to the file, statistic data like data of last change and creation and etc

**File systems** - store and manage the files, they are often organized in an hierarchy and have file groups (directiries) with management information and regular files. Structuring is done with subdirectories, and each subdirectory contains entries from other or normal files.
![[Pasted image 20240219122524.png]]
Operating that can be performed on files are:
- generating a catalog entry
- delete the entries and file content
- copy files
- delete file
- change file name or content
- transmit files to another directory

Files can be edited block by block or character by character, since each file with content has a list of blocks it occupies, the catalog contains references to the blocks that make up that file. The file system also manages one **pseudo file** consisting of all free blocks of data and one **pseudo file** with a list of all unreliable blocks

#### Unix Classification of file systems

![[Pasted image 20240219122914.png]]
Unix uses an arbitrarily deep hierarchical file
![[Pasted image 20240219122950.png|inlR|400]]

The root of the tree is called **root** or **root directory** and could be abbreviated as **/**. Bellow are home directiries of individual users located under **/usr**. The inner nodes of the three are subdirectories and the leaves of the tree are actual files. Devices are also treated like "special files" , and certain conventions have become established for the organisation of the Unix tree. Among the children of the root dir there are usually subdirectories with de following meaning:
- am: System files (binary)
- dev: Device files (devices "special files")
- lib: Libraries
- usr: User files
External file systems or physical data carriers such as hard disk CDROM, floppy can contain independent file systems, at any point we can command **mount** and hook devices or other independent file system into the unix file tree, after that there is no longer any distinction between files on the hard drive and other mounted disks, with **unmount** the subdirectory is removed.

Following is the example of hard drive file system layout
![[Pasted image 20240219124816.png]]
**MBR** - master boot recrod on sector 0 to boot the computer system,
**Partition Table** - which lies at end of MBR, and it devide disks into several partition
**Boot block** - is read and executed block at startup
**Super block** - contains management information about the file system
**Free blocks** like bitmap - indicates the free blocks of the file system
**Root directory** - contains the contents of the file system
**i-nodes** - are entities in the table of contents of the file system

![[Pasted image 20240219125103.png]]
**NFS** or **Network File System** - when we connect file system from several computer to form a logically related file system over NFS protocol
![[Pasted image 20240219125155.png]]
#### Windows file system manager
**Volume manager** - which manages disks at the logical level, provides logical drives for application, and does change, reparation if possible dynamically without system reboot
![[Pasted image 20240219125313.png|inlR|400]]
**Windows Cluster** - is file system block addressable in Windows, it consist of one or more sectors, and uses FAT-X which indicates number of bits used for cluster addressing which then determines the size of the file system.
![[Pasted image 20240219125457.png|inlR|400]]

| file system | Bits for Cluster index | Number Clusters | Supported Cluster sizes | Maximum File system size |
| ---- | ---- | ---- | ---- | ---- |
| FAT-12 | 12 bits | $2_{12}= 4,096$ | 512 bytes – 8 KiB | 32 MiB |
| FAT-16 | 16 bits | $2_{16}= 65,536$ | 512 bytes – 64 KiB | 4 GiB (GibiByte) |
| FAT-32 | 32 bits, but only 28 used | $2_{28}$ | 512 bytes – 32 KiB | 8 TiB (TebiByte) limited to 32 GiB |
| NTFS | 64 bits | $2_{64}$ | 512 bytes – 64 KiB | 16 EiB (ExbiByte) limited to 256 TiB |

#### Storage Systems

**Terms**
DAS - Direct Attached Storage = conventional storage system
SAN - High speed network between servers and subsystems
NAS - Network attached storage, NAS systems are file servers not hard drive servers.
**RAID Systems** - are Redundant Array of Independent Disks or originally Redundant Array of Inexpresive Disks are multiple disk storages that are very common today, they are several small disks that are used as a large one virtual managed disk. They are also used to improve performance and to increase Resiliency, there are couple of different variants of it but 2 main aspects are **data striping** and **data redundancy**

**Data striping** - when logical disk area is divided into stripes that extend across several disks, it used to improve performance through parallel reading/writing
![[Pasted image 20240219130539.png|inlR|400]]
**Data redundancy** - when additional (redundant) data is stored to improve reliability, system test data for error correction getting Error Correction Codes (ECC). Example could be simple parity bit, where parity bit is 0 if the length of 1-bit is an even number, otherwise 1
![[Pasted image 20240219130853.png|inlR|400]]

**Terminology:**
- **strip** - is unit of logaically contiguous data stored on a single disk
- **stripe** - the set of trips spanning across all the disks
- **strip depth/ strip size** - is amount of data in one strip
- **stripe width** - number of data disks in the array
- **stripe size** - (strip-depth) $\times$ (stripe-width)
![[Pasted image 20240219131100.png|inlR|400]]
**RAID 0 system** - stripes are distributed across multiple disks in an array, and stip size significantly affects performance spanning from 32KB, 64KB, 128KB. Distribution is done either by the OS or a RAID controller. By Higher I/O throughput fast variant but not fail-safe. Block level striping without redundancy or Error Correction Codes, it improves read and write performance without fault tolerance.
![[Pasted image 20240219183102.png|inlR|200]]

**RAID 1 system** - OS or RAID controller makes reflection(mirroring) of data , full redundancy of data, making data more fail-safe but without any improvement in write performance. Has no stripping, can tolerate failure up to half of disks, but it improves read.
![[Pasted image 20240219183114.png|inlR|200]]
**RAID 10 (RAID 1 + 0) system** - stripping and mirroring and combination of RAID-0 and RAID-1, it is very fast with improved read and write, and fail safe since it makes mirroring
![[Pasted image 20240219183126.png|inlR|400]]
**RAID 2 system** - makes bitwise stripping and supplementation of test data for error correction. All data disks are used for every I/O access, and uses dedicated Hamming Code for ECC, if disk fails it can be reconstructed
![[Pasted image 20240219183148.png|inlR|300]]
**RAID 3 system** - makes byte level striping, and all data disks from 0 - 2 are used for every I/O access. Test data are being summarised and saved on dedicated disk, with simple parity bits as ECC, and reductio in cost for 1:1 mirroring as no longer as relevant today. Failure in disk doesn't result in runtime
![[Pasted image 20240219183155.png|inlR|300]]
**RAID 4 system** - is similar to RAID-3 only it does block-level striping, and includes parallel execution of various I/O operation which results in better performance
![[Pasted image 20240219183159.png|inlR|300]]
**RAID 5 system** - in contrast to RAID-4 the parity sections are distributed across the participating disks which makes even disk utilisation. It also has optimised reading performance, even if one disk is lost there is no data loss, but if more than 2 disks are lost it results in failure. RAID-5 is not suitable for high transaction loads, more for many small concurrent transactions write operations
![[Pasted image 20240219183211.png|inlR|300]]
**RAID 6 system** - Like RAID-5 with redundant test data on another disk so that even the failure of two disks has no effect. Reading performance is the same as RAID-5, but writing performance is worse than RAID-5

![[Pasted image 20240219183218.png|inlR|300]]

RAID level comparison computes array characteristic given the disk capacity, the number of disks and the array type it is evaluated as following
![[Pasted image 20240219183459.png|inlR|300]]

NAS - Network attached storage, it can be also mass storage device connected to local area network (LAN) only disadvantage is that is burden on the network. It is configurable and can provide storage space on network, one or more hard drives can be connected with power supply and network interface, the access is regulated in NAS rather then centrally and data backup is done using RAID.
![[Pasted image 20240219183608.png|inlR|300]]

Storage Area Network (SAN) - is own network between servers and storage resources, storage can be treated virtually as a single hard drive
![[Pasted image 20240219183811.png|inlR|400]]
![[Pasted image 20240219183839.png|inlR|400]]
![[Pasted image 20240219183851.png]]

| NAS | SAN |
| ---- | ---- |
| TCP/IP Networks: Ethernet, FDDI, ATM | Typically Fiber Channel network |
| TCP/IP and NFS/CIFS/HTTP | Encapsulated SCSI |
| File level access to storage resources.resources | Block level access to storage resources.resources |
| Identifies data by file name and byte offsets, transfers file data or file meta-data and handles security, user authentication, file locking | Addresses data by disk block number and transfers raw disk blocks |
| The management of the file system resides with the NAS device | The servers manage the file system |

> [!Summary]
> Device and file management are separate software layers in the kernel
> Hard drives are the most important external storage medium
> Files and file system (FAT, NTFS, Unix file system) as an abstraction
> RAID systems are used for secure data storage, RAID-1, RAID-10 and RAID-5 most commonly used
> NAS or SAN are storage systems that are widely used on network.