Skip to main content

Storage Configuration

During the configuration of a profile, our NixOS installation scripts will automatically configure the storage setup.

There are several potential scenarios:

  1. With a single disk (common with laptops): the disk will be split up into EFI System Partition, Swap and Main Data partition. The Main Data will be formatted as the ZFS rpool.
  2. With multiple homogenous disks (not very common): each disk will have a mirrored EFI System Partition, striped Swap and Main Data partition. The Main Data partitions will be combined together into a single rpool using mirroring or striping. Mirroring will be used if the data is critical, striping will be used if they are not.
  3. With multiple heterogenous disks (common with desktops): this is where there can be a set of fast disks and a set of slow disks, usually a set of fast but small SSDs and a set of slow but large HDDs. In this scenario, the EFI System Partition, Swap are placed on the SSDs in the same way as the homogenous disks setup. However 2 additional partitions are created for the SSDs: ZIL SLOG Parititon and L2ARC Partition. The set of ZIL SLOG Partition will be mirrored or striped depending on the main data disks, while the set of L2ARC Partition will be striped. The entire HDD set will be combined together into a single rpool using mirroring or striping. Mirroring will be used if the data is critical, striping will be used if they are not. If data disks are mirrored, then the ZIL SLOG Partition set should be mirrored. If the data disks are not mirrored, then the ZIL SLOG Partition set should be striped.

In most of our laptop configurations given to our developers, we use configuration 1. Some bigger desktops will use configuration 3. Configuration 2. is more likely for specialized unattended computers performing functions in our data center, most likely data-storage and backup functionality.

Take note that the EFI System Partition does not get auto-mirrored. This partition will be mounted as the /boot directory which will contain all the relevant stage-1 boot software (stage-2 is post-mounting of the root filesystem, and executes systemd at the very end). Since this partition is only changed upon nixos-rebuild, our rebuild scripts will auto-copy to all other mirrored disks.

Furthermore the usage of Swap is only needed for operator platforms. Autonomous platforms should generally not require Swap. This is because their memory capacity would be sized specifically for their usecase. Operator platforms on the other hand may need to do lots of different unknown operations, and the Swap gives memory capacity flexibility.

However the Swap currently is not encrypted, and therefore it is subject to potential data leakage if the swap partition was to be extracted out from sudden power loss. Even during a regular shutdown, the Swap partition is not cleared. In the future we plan to implement encryption for the Swap partition as well, and develop a fully secure platform for all employees of the company.

ZFS​

We use ZFS as our root filesystem because of:

  • Data Integrity: ZFS ensures data integrity by using checksums to detect and correct silent data corruption.
  • Storage Pooling: It allows for pooling of various storage devices, making it easy to manage storage resources.
  • Snapshots and Clones: ZFS provides efficient snapshot and cloning capabilities, aiding in data backup and recovery efforts.
  • RAID Configurations: Native support for multiple RAID levels, facilitating robust data redundancy and fault tolerance.
  • Scalability: Capable of scaling with large storage capacities and high-performance I/O operations.
  • Data Compression: Offers transparent data compression, optimizing storage utilization and performance.
  • Copy-on-Write: Each write operation is transactional, which prevents data corruption during unexpected shutdowns.
  • Deduplication: Supports deduplication to eliminate duplicate copies of repeating data, conserving storage space.

Our ZFS root filesystem is always fully encrypted at rest to ensure the security of the data stored on disk. This encryption protects against unauthorized access and secures sensitive information by making it unreadable without the appropriate key. The encryption process utilizes advanced cryptographic algorithms to convert data into an encoded format, which can only be decrypted and accessed with a unique key. By ensuring that all data written to disk is encrypted, we mitigate the risk of data breaches and unauthorized data retrieval, maintaining the confidentiality and integrity of the stored information.

ZFS RPool Password​

During the initial installation/provisioning, the rpool password will be prompted for and set.

To change the rpool password, use:

zfs change-key rpool

Take note that platforms that have a rpool password cannot auto-boot all the way to systemd without user intervention at the stage-1 boot.

This means if you are intending the platform to be fully autonomous, the rpool password will need to be auto-filled at stage-1 boot. Doing such a thing will require enabling stage-1 boot SSH or KVM (if you also want to be able to do autonomous pre-boot configuration of the BIOS).