Skip to main content

Storage Architecture

Purpose

This page documents the physical disks, Proxmox storage pools, logical-volume layout, filesystem mounts, and intended storage roles within the Bytek homelab.

The storage architecture separates:

  • Proxmox and application operating-system disks.
  • Large user and application data.
  • Local Proxmox backups.
  • Temporary snapshots and rollback points.

The main design goal is to prevent application data, operating-system disks, and backups from competing for the same physical storage role.

Storage Overview

Storage Tier Approximate Capacity Proxmox Storage Primary Purpose
System NVMe 1 TB local-lvm Operating systems and infrastructure services
User-data NVMe 4 TB user-data Large application and user-data disks
Backup HDD 4 TB pve-backup Proxmox VZDump backups

Physical Disk Inventory

Linux Device Approximate Usable Size Device Type Current Role
/dev/nvme1n1 931.5 GiB NVMe SSD Proxmox operating system and local-lvm
/dev/nvme0n1 3.6 TiB NVMe SSD user-data LVM-thin storage
/dev/sda 3.6 TiB WDC HDD Dedicated Proxmox backup storage

Linux device names such as /dev/sda can change between boots or hardware changes.

Permanent references should use:

  • Filesystem UUIDs for mounted filesystems.
  • /dev/disk/by-id/ paths for physical-device operations.
  • Proxmox storage IDs for normal VM and backup management.

System NVMe

Physical Device

Setting Value
Linux device /dev/nvme1n1
Approximate capacity 931.5 GiB
Partition table GPT
Primary volume group pve

Partition Layout

Partition Approximate Size Purpose
/dev/nvme1n1p1 1007 KiB Boot support partition
/dev/nvme1n1p2 1 GiB EFI system partition
/dev/nvme1n1p3 930.5 GiB Proxmox LVM physical volume

Logical Volumes

Logical Volume Approximate Size Purpose
pve-root 96 GiB Proxmox root filesystem
pve-swap 8 GiB Proxmox swap
pve-data 794.3 GiB Proxmox LVM-thin pool

Proxmox Storage

The pve-data thin pool appears in Proxmox as:

  • Storage ID: local-lvm
  • Storage type: LVM-thin

Intended Use

The system NVMe stores:

  • Proxmox VE.
  • Infrastructure VM and LXC operating-system disks.
  • Application operating-system disks.
  • EFI disks.
  • Small configuration volumes.
  • Temporary Proxmox snapshots.
  • Docker application configuration stored inside service VMs.

Current Workloads

The system NVMe currently stores operating-system disks for services such as:

  • WireGuard.
  • Pi-hole.
  • Uptime Kuma.
  • Traefik.
  • Authentik.
  • Nextcloud.
  • Vikunja.
  • BookStack.

Storage Policy

Use local-lvm for:

  • VM and LXC operating-system disks.
  • Infrastructure services.
  • Small persistent application datasets.
  • VM configuration disks.
  • Temporary snapshots.

Avoid using local-lvm for:

  • Large Nextcloud user libraries.
  • Large Plex media libraries.
  • Large game-server libraries.
  • Backup archives.
  • Long-term media collections.

User-Data NVMe

Physical Device

Setting Value
Linux device /dev/nvme0n1
Approximate capacity 3.6 TiB
Device purpose Large application and user data
LVM volume group lvm1
Thin pool data

LVM-Thin Layout

The user-data NVMe contains:

  • lvm1-data_tmeta
  • lvm1-data_tdata
  • lvm1-data

These components represent:

  • Thin-pool metadata.
  • Thin-pool data.
  • The visible LVM-thin pool.

Proxmox Storage

The thin pool appears in Proxmox as:

  • Storage ID: user-data
  • Storage type: LVM-thin
  • Volume group: lvm1
  • Thin pool: data
  • Configured chunk size: 256 KiB

Intended Use

The user-data NVMe stores:

  • Nextcloud user files.
  • Large application-data disks.
  • Future game-server data.
  • Future media-storage testing.
  • Other user-generated content.

Current Major Workload

Nextcloud has a separate virtual data disk on user-data.

Setting Value
Nominal virtual-disk size 500 GB
Guest filesystem ext4
Guest mount point /mnt/nextcloud-data
Nextcloud data directory /mnt/nextcloud-data/ncdata

The Nextcloud operating system remains on local-lvm, while user files remain on user-data.

This separation makes it easier to:

  • Expand the user-data disk.
  • Monitor data growth.
  • Keep the operating-system disk smaller.
  • Apply different backup decisions to application and data disks.
  • Migrate data disks independently in the future.

Thin Provisioning

The local-lvm and user-data storage pools use LVM-thin provisioning.

A thin-provisioned virtual disk does not immediately consume its full nominal capacity.

For example, a 500 GB virtual disk initially consumes only the physical blocks that have actually been written.

Important Consequence

The sum of all nominal virtual-disk sizes may exceed the physical capacity of the thin pool.

This is safe only when actual usage is monitored.

If the thin pool reaches full physical allocation, affected VMs may experience:

  • Write failures.
  • Filesystem corruption.
  • Application outages.
  • Database failures.
  • Guest crashes.

Thin-Pool Monitoring

Review Proxmox storage usage using:

pvesm status

Review LVM-thin usage using:

lvs -a -o vg_name,lv_name,segtype,lv_attr,lv_size,data_percent,metadata_percent

Monitor both:

  • Data percent
  • Metadata percent

Suggested operational thresholds:

Utilization Response
Below 70 percent Normal monitoring
70 to 84 percent Review growth and available capacity
85 percent or higher Immediate capacity planning
Near 100 percent Critical risk of storage failure

These thresholds are operational recommendations and should be adjusted based on growth rate and workload importance.

Backup HDD

Physical Device

Setting Value
Linux device /dev/sda
Partition /dev/sda1
Approximate usable capacity 3.6 TiB
Filesystem ext4
Partition table GPT
Stable WWN 0x5000cca097c44d7e

Permanent Mount

Setting Value
Mount point /mnt/pve/pve-backup
Proxmox storage ID pve-backup
Proxmox content type VZDump backup files
Mount method Filesystem UUID through /etc/fstab

The filesystem UUID is used instead of /dev/sda1 because Linux device letters can change.

Mount Options

The backup filesystem uses:

  • Read-write mounting.
  • noatime to reduce unnecessary metadata writes.

Mount-Point Protection

The Proxmox storage uses mount-point validation.

This prevents Proxmox from treating the empty directory /mnt/pve/pve-backup as valid storage when the HDD is not mounted.

Without this protection, a failed HDD mount could cause backups to be written to the Proxmox root filesystem.

That could fill the root filesystem and disrupt the hypervisor.

Backup-HDD Intended Use

The backup HDD stores:

  • Scheduled VM backups.
  • Scheduled LXC backups.
  • Manual baseline backups.
  • Upgrade rollback backups.
  • Restore-test source archives.

The backup HDD must remain dedicated to backups.

Prohibited Backup-HDD Uses

Do not use the backup HDD for:

  • Active VM disks.
  • Active LXC root filesystems.
  • Nextcloud user files.
  • Plex media.
  • Steam game-server data.
  • Docker application data.
  • Download staging.
  • General file storage.
  • Temporary application caches.

Mixing active workloads with backups would:

  • Increase disk contention.
  • Increase wear.
  • Reduce available retention.
  • Expand the failure impact.
  • Make recovery planning less predictable.

Storage Placement Rules

Workload Preferred Storage
Infrastructure OS disk local-lvm
Application OS disk local-lvm
Nextcloud user data user-data
Vikunja database and files Current VM OS disk unless growth requires separation
BookStack database and files Current VM OS disk unless growth requires separation
Plex configuration and metadata local-lvm or a dedicated application disk
Plex media Future dedicated media storage
Game-server executable files Dedicated game-server disk or user-data
Game world and save data user-data with backup enabled
Proxmox backup archives pve-backup
Short-term snapshots Same thin pool as the source disk

Future Plex Storage

The RTX 2060 Plex VM should not use the backup HDD for media.

  • Plex operating system on local-lvm.
  • Plex configuration and metadata on NVMe-backed storage.
  • Plex transcode directory on temporary fast storage.
  • Movies and television content on future dedicated media storage.
  • Music content on future shared media storage if Navidrome is deployed.

A temporary media disk may be created on user-data for initial testing.

Long-term media storage should ideally use:

  • A dedicated HDD.
  • A mirrored HDD pair.
  • A dedicated storage pool.
  • A NAS or storage server.

Media capacity planning must account for continued Nextcloud growth on user-data.

Future Game-Server Storage

A dedicated game-server VM should use:

  • Operating-system disk on local-lvm.
  • Game data or world-save disk on user-data.
  • Backup enabled for unique save data.
  • Application-level save backups where supported.

Steam-installed server binaries can usually be downloaded again.

Unique game-world data, configuration files, allowlists, bans, and mods require backup.

Storage Expansion Principles

Before adding physical storage, decide whether the new disk is intended for:

  • Operating-system workloads.
  • Application data.
  • Media.
  • Backups.
  • Redundancy.
  • Offsite synchronization.

Avoid adding unrelated physical disks into an existing unmirrored LVM volume group merely to increase capacity.

Safer future expansion patterns include:

  1. Create a new independent Proxmox storage pool.
  2. Move selected VM disks to the new pool.
  3. Replace a smaller disk with a larger disk.
  4. Add mirrored storage for critical data.
  5. Add a dedicated media-storage pool.
  6. Add a dedicated Proxmox Backup Server.
  7. Add an encrypted offsite backup target.

Failure-Domain Considerations

System NVMe Failure

Expected impact:

  • Proxmox root filesystem may be lost.
  • Infrastructure VM and LXC operating-system disks may be lost.
  • The user-data NVMe and backup HDD may remain physically intact.

Recovery would require:

  • Proxmox reinstallation or system-disk restoration.
  • Reconnection of existing storage.
  • VM and LXC restoration from pve-backup.

User-Data NVMe Failure

Expected impact:

  • Nextcloud user-data disk may be lost.
  • Other large application-data disks may be lost.
  • VM operating-system disks on local-lvm may continue working.
  • Backups on the HDD may remain available.

Backup HDD Failure

Expected impact:

  • Live services continue running.
  • New backup jobs fail.
  • Existing local recovery points are lost.
  • There is no current offsite replacement copy.

Complete Host Failure

Expected impact:

  • All running services become unavailable.
  • The internal backup HDD may be inaccessible until moved or the host is repaired.
  • A severe electrical or physical event could affect all three storage devices.

This is the primary reason a future offsite or separate-host backup is required.

Snapshot Policy

Proxmox snapshots are used as short-term rollback points.

  • Before a major operating-system upgrade.
  • Before changing Authentik integration.
  • Before changing Traefik routing or middleware.
  • Before changing a Docker stack.
  • Before database migrations.
  • Before a major application upgrade.
  • Before enabling GPU passthrough.

Snapshots are not backups because:

  • Snapshots remain on the same storage.
  • A physical disk failure destroys the snapshot with the live disk.
  • Long-lived snapshots consume thin-pool capacity.
  • Snapshot growth can create unexpected storage pressure.

Delete unnecessary snapshots after the associated change has been validated.

Avoid snapshots with RAM unless restoring the exact running state is required.

Storage Validation

Confirm Proxmox Storage

Use:

pvesm status

Expected storage IDs include:

  • local
  • local-lvm
  • user-data
  • pve-backup

Confirm Backup Mount

Use:

findmnt /mnt/pve/pve-backup

The source should be /dev/sda1.

Confirm Backup Capacity

Use:

df -h /mnt/pve/pve-backup

Confirm Physical Devices

Use:

lsblk -e7 -o NAME,PATH,SIZE,TYPE,FSTYPE,PTTYPE,MOUNTPOINTS,MODEL,SERIAL,WWN

Confirm Thin-Pool Utilization

Use:

lvs -a -o vg_name,lv_name,segtype,lv_attr,lv_size,data_percent,metadata_percent

Confirm Storage Configuration

Use:

cat /etc/pve/storage.cfg

Do not store credentials or secrets in the documentation when recording command output.

Disk-Health Monitoring

The backup HDD should be monitored using SMART.

Stable backup-HDD identifier:

/dev/disk/by-id/wwn-0x5000cca097c44d7e

Useful checks:

  • smartctl -H /dev/disk/by-id/wwn-0x5000cca097c44d7e
  • smartctl -A /dev/disk/by-id/wwn-0x5000cca097c44d7e
  • smartctl -x /dev/disk/by-id/wwn-0x5000cca097c44d7e

Important attributes include:

  • Reallocated sector count.
  • Current pending sector count.
  • Offline uncorrectable sector count.
  • UDMA CRC error count.
  • Temperature.
  • Power-on hours.
  • SMART error log.

Pending or uncorrectable sectors require immediate review.

NVMe health should also be reviewed periodically using the appropriate NVMe SMART tools.

Capacity Review Checklist

Perform a storage review when:

  • Nextcloud user data grows significantly.
  • Plex media is introduced.
  • A game-server data disk is created.
  • Backup retention consumes unexpected space.
  • Thin-pool utilization exceeds 70 percent.
  • Snapshot count increases.
  • A disk reports SMART warnings.
  • Another physical disk is being considered.

Review:

  1. Physical free capacity.
  2. Thin-pool data utilization.
  3. Thin-pool metadata utilization.
  4. Snapshot count.
  5. Backup-HDD free space.
  6. Nextcloud data growth.
  7. Media-storage requirements.
  8. Restore-capacity requirements.
  9. Backup-retention effectiveness.

Current Limitations

The current storage architecture does not provide disk redundancy.

The system NVMe, user-data NVMe, and backup HDD are independent single devices.

The architecture protects against some logical failures through backups, but does not provide uninterrupted operation after a physical disk failure.

The local backup HDD is in the same:

  • Physical server.
  • Building.
  • Power environment.
  • Administrative security boundary.

Future priorities include:

  • Dedicated media storage.
  • A third backup copy.
  • Offsite encrypted backups.
  • Separate Proxmox Backup Server hardware.
  • Storage redundancy for critical user data.

Document Control

  • Owner: Bryan Gagne-Plante
  • Last verified: YYYY-MM-DD
  • System storage: 1 TB NVMe on local-lvm
  • Application-data storage: 4 TB NVMe on user-data
  • Backup storage: 4 TB HDD on pve-backup
  • Storage redundancy: None
  • Offsite backup: Not configured
  • Recovery tested: Partial
  • Known limitations: Single-device storage tiers and one physical host

No comments to display

Back to top