# NiM - The NixOS Machine

* hosted on **Hetzner Dedicated**
* runing **NixOS**
* encrypred ZFS
* ephemeral root FS

## Resources

* https://github.com/nix-community/nixos-install-scripts/tree/master/hosters/hetzner-dedicated
* https://gist.github.com/bitonic/78529d3dd007d779d60651db076a321a

## Setup

See: https://www.hetzner.com/dedicated-rootserver/

* 6 Core (12 Thread)
* 64 ECC RAM
* 2 x 512GB NVMe
* ~~BIOS~~ UEFI [^2]

### Partitioning:

`/boot`: `ext2`, mirrored

**zpools**

* rpool
* local
* nix
* root
* system
* journald
* data
* data
* _app1_
* _app2_
* ...

# zpool: -O for FS options, -o for pool options
zpool create \
    -o ashift=12 \
    -R /mnt \
    -O mountpoint=none \
    -O canmount=off \
    -O xattr=sa \
    -O acltype=posix \
    -O atime=off \
    -f \
    rpool mirror \
    /dev/... \
    /dev/...

zfs create -o mountpoint=none rpool/local
zfs create -o mountpoint=none rpool/system
zfs create -o mountpoint=none rpool/data

# Create root FS and take an empty snapshot

zfs create -o mountpoint=legacy rpool/local/nix
zfs create -o mountpoint=legacy rpool/data/data
zfs create -o mountpoint=legacy rpool/system/root
zfs snapshot rpool/system/root@blank

zfs create -o reservation=1G -o mountpoint=none rpool/reserved

For a notebook

see https://openzfs.github.io/openzfs-docs/Getting%20Started/NixOS/Root%20on%20ZFS.html

zpool create \
    -o ashift=12 \
    -o autotrim=on \
    -R /mnt \
    -O acltype=posixacl \
    -O canmount=off \
    -O compression=on \
    -O dnodesize=auto \
    -O normalization=formD \
    -O relatime=on \
    -O xattr=sa \
    -O mountpoint=none \
    rpool /dev/xxx
  
zfs create \
    -o canmount=off \
    -o mountpoint=none \
    -o encryption=on \
    -o keylocation=prompt \
    -o keyformat=passphrase \
    rpool/enc

zfs create -o canmount=off rpool/enc/local
zfs create -o canmount=off rpool/enc/system
zfs create -o canmount=off rpool/enc/user

# Root FS
zfs create -o canmount=on -o mountpoint=/ rpool/enc/system/root
zfs snapshot rpool/enc/system/root@blank

# Homedir
zfs create -o canmount=on -o mountpoint=/home rpool/enc/user/home

# System dirs
zfs create -o canmount=on -o mountpoint=/var/log rpool/enc/system/log

# Boot stuff
mkdir /mnt/boot
mount /dev/vda1 /mnt/boot

## ZFS

* https://nixos.wiki/wiki/ZFS
* See [Unlock encrypted zfs via ssh on boot](https://nixos.wiki/wiki/ZFS#Unlock_encrypted_zfs_via_ssh_on_boot)
* https://grahamc.com/blog/nixos-on-zfs
* Ephemeral root FS: https://grahamc.com/blog/erase-your-darlings

### Dataset properties

* `compression=lz4`
* Specifying `on` instead of `lz4` or another specific algorithm will always pick the best available compression algorithm. [^1]
* For wherever `/var` lives:
* `xattr=sa` for Journald
* `acltype=posixacl` also for Journald
* `relatime=on` for reduced stress on SSDs

The following is a list of dataset properties which are often useful, but do have drawbacks:

* `atime=off` disables if a file's access time is updated when the file is read. This can result in significant performance gains, but might confuse some software like mailers.
* Nix doesn't use `atime`, so setting `atime=off` on the `/nix` FS is fine

NixOS requires (as of 2020-04-11) `mountpoint=legacy` for all datasets. NixOS does not yet have tooling to require implicitly created ZFS mounts to settle before booting, and `mountpoint=legacy` plus explicit mount points in hardware-configuration.nix will ensure all your datasets are mounted at the right time

## Hetzner Dedicated

https://jappie.me/the-nix-mutli-monolith-machine-nmmm.html

[^1]: https://grahamc.com/blog/nixos-on-zfs
[^2]: https://docs.hetzner.com/robot/dedicated-server/operating-systems/uefi