# Storagenode write tp vs hash/disk sync
This test creates a new piece store (`pieces.NewStore`) and tries to write as many 2.3 Mb files as possible.
Variables:
* test is executed both with SHA-256 hashing (current) and BLAKE (planned)
* test is executed both with/without disc sync before the rename of the commit
2.3 Mb is chosen because the 64 Mb block size and 29 EC parameter results with 2.3 Mb piece size.
## Results
TLDR;
* Using BLAKE3 we can achieve significant better write thoughput (4x)
* but if disk is slow (good old HDD), we won't see the benfit: we are busy with sync, cpu is not a bottleneck
On a real physical server machine (which also runs other workloads):
### Phyisical server machine (spinning disk!!!):
**Note: This machine also had other production workloads (like a storagenode) at the same time**
```
name time/op
ReadWrite/sync-SHA256-8 80.0ms ± 0%
ReadWrite/nosync-SHA256-8 6.61ms ±15%
ReadWrite/sync-BLAKE3-8 79.1ms ± 4%
ReadWrite/nosync-BLAKE3-8 2.18ms ± 8%
name speed
ReadWrite/sync-SHA256-8 29.0MB/s ± 0%
ReadWrite/nosync-SHA256-8 355MB/s ±14%
ReadWrite/sync-BLAKE3-8 29.3MB/s ± 4%
ReadWrite/nosync-BLAKE3-8 1.07GB/s ± 9%
```
### Local machine (tmpfs!!! no sync cost):
This shows that Blake have 4x better potential.
```
name time/op
ReadWrite/sync-SHA256-12 4.64ms ± 1%
ReadWrite/nosync-SHA256-12 4.62ms ± 0%
ReadWrite/sync-BLAKE3-12 1.01ms ± 1%
ReadWrite/nosync-BLAKE3-12 1.00ms ± 2%
name speed
ReadWrite/sync-SHA256-12 500MB/s ± 1%
ReadWrite/nosync-SHA256-12 502MB/s ± 0%
ReadWrite/sync-BLAKE3-12 2.30GB/s ± 1%
ReadWrite/nosync-BLAKE3-12 2.32GB/s ± 2%
```
### Cloud machine (GCE, root disk is *new balanced persistent disk*):
```
name time/op
ReadWrite/sync-SHA256-8 19.0ms ±40%
ReadWrite/nosync-SHA256-8 9.53ms ±27%
ReadWrite/sync-BLAKE3-8 16.1ms ± 0%
ReadWrite/nosync-BLAKE3-8 4.94ms ±86%
name speed
ReadWrite/sync-SHA256-8 127MB/s ±31%
ReadWrite/nosync-SHA256-8 250MB/s ±24%
ReadWrite/sync-BLAKE3-8 144MB/s ± 0%
ReadWrite/nosync-BLAKE3-8 560MB/s ±67%
```
## Test method
```
cd storagenode/pieces/b
CGO_ENABLED=0 go test -c
```
And the binary is used to execute tests:
```
./b.test -test.v -test.benchtime=1000x -test.bench 'BenchmarkReadWrite' | tee -a server.txt
go install golang.org/x/perf/cmd/benchstat@latest
benchstat server.txt
```
Note: we need to use counter based (`1000x`) and not timer based limit to write the same amount of bytes. Results of tests with time based limit couldn't be compared as one test may write significant more bytes.
Used main commit:
```
commit f35b4163f98de6c8a9032a738c12ce3766a2314d (origin/main, origin/HEAD)
Author: NickolaiYurchenko
Date: Mon Aug 22 17:47:43 2022 +0300
web/satellite: multiple passphrase notification added
```
But it also includes the piece hash patches:
https://github.com/elek/storj/commit/9bd26c777c272e152d7b90ec6703d93d484219c4
## Reference
```
curl https://gist.githubusercontent.com/elek/7b1eb55b37d2988534bb05ce98f7f6e0/raw/hw.sh | tee | sudo bash
```
### Local machine
```
VENDOR
Manufacturer: ASUS
Product Name: System Product Name
MEMORY
Size: 32 GB
Non-Volatile Size: None
Volatile Size: 32 GB
Cache Size: None
Logical Size: None
Size: 32 GB
Non-Volatile Size: None
Volatile Size: 32 GB
Cache Size: None
Logical Size: None
CPUs
Version: 12th Gen Intel(R) Core(TM) i5-12400
Max Speed: 4400 MHz
Core Enabled: 6
Thread Count: 12
DISKS
NAME SIZE TYPE FSTYPE MOUNTPOINT VENDOR MODEL
sda 3.6T disk ATA WDC WD40EFRX-68N32N0
└─sda1 3.6T part linux_raid_member
└─md127 3.6T raid1 ext4
nvme0n1 931.5G disk Samsung SSD 980 PRO 1TB
├─nvme0n1p1 100M part vfat /boot
├─nvme0n1p2 16M part
├─nvme0n1p3 243.4G part ntfs
├─nvme0n1p4 611M part ntfs
└─nvme0n1p5 687.4G part crypto_LUKS
└─root 687.4G crypt ext4 /
```
### Server machine
```
VENDOR
Manufacturer: FUJITSU
Product Name:
MEMORY
Size: 16384 MB
Size: 16384 MB
CPUs
Version: Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz
Max Speed: 8300 MHz
Core Enabled: 4
Thread Count: 8
DISKS
NAME SIZE TYPE FSTYPE MOUNTPOINT VENDOR MODEL
sda 3.7T disk ATA TOSHIBA_MG04ACA400EY
├─sda1 3.7T part linux_raid_member
│ └─md0 3.7T raid1 ext4 /
└─sda2 1M part
sdb 3.7T disk ATA TOSHIBA_MG04ACA400EY
├─sdb1 3.7T part linux_raid_member
│ └─md0 3.7T raid1 ext4 /
└─sdb2 1M part
```
### Cloud machine
* GCE e2-standard-8
* Balanced persistent disk
```
VENDOR
Manufacturer: Google
Product Name: Google Compute Engine
MEMORY
Size: 16 GB
Size: 16 GB
CPUs
Version: Not Specified
Max Speed: 2000 MHz
Version: Not Specified
Max Speed: 2000 MHz
Version: Not Specified
Max Speed: 2000 MHz
Version: Not Specified
Max Speed: 2000 MHz
Version: Not Specified
Max Speed: 2000 MHz
Version: Not Specified
Max Speed: 2000 MHz
Version: Not Specified
Max Speed: 2000 MHz
Version: Not Specified
Max Speed: 2000 MHz
DISKS
NAME SIZE TYPE FSTYPE MOUNTPOINT VENDOR MODEL
sda 200G disk Google PersistentDisk
├─sda1 199.9G part ext4 /
├─sda14 3M part
└─sda15 124M part vfat /boot/efi
```