Skip to content

ZFS: per-snapshot USED badly understates what destroying a set of snapshots reclaims; use usedbysnapshots and a dry-run range destroy

TL;DR.

zfs list -t snapshot USED shows only blocks unique to each snapshot, so three snapshots showing 1.38G + 7M + 136M actually pinned 12.1G (the blocks they shared). Check zfs list -o usedbysnapshots <ds> and zfs destroy -nv ds@first%last (works without root; range must be oldest%newest or it says 'could not find any snapshots to destroy').

Context: Ubuntu 20.04 ZFS-on-root host (rpool/ROOT/ubuntu), root at 96% full. Three rollback snapshots were three months old. zfs list -H -t snapshot -o name,used rpool/ROOT/ubuntu showed 1.38G, 7.38M and 136M, which suggested destroying them would free about 1.5G.

In fact zfs list -H -o usedbysnapshots rpool/ROOT/ubuntu said 12.1G. Per-snapshot USED only counts blocks referenced by that snapshot alone. Blocks shared by two or more snapshots (and no longer in the live dataset) show up in no snapshot's USED, but are freed when all of those snapshots go. On this host the growth came from a workload that rewrites large files in place: Plex's SQLite library DB (~650M) and blobs DB (1.1G), plus its 3-daily dated DB backups, which also lived on the root dataset. Every rewrite left the old blocks pinned by all the snapshots.

How to measure:

  • zfs list -o name,usedbysnapshots,usedbydataset,usedbychildren <ds> gives the dataset-level total.
  • zfs destroy -nv <ds>@<oldest>%<newest> dry-runs the whole range and prints would reclaim 12.1G. On ZFS 0.8 (focal) it ran fine as an unprivileged user without sudo.
  • Gotcha: the range must run oldest%newest. Reversed, it fails with could not find any snapshots to destroy; check snapshot names., which reads like a typo in the name.

Takeaways: never judge snapshot reclaim from per-snapshot USED. Keep long-lived rollback snapshots off datasets that hold rewrite-heavy DBs, or destroy them promptly. Point app DB backups (Plex: Settings > Scheduled Tasks > backup directory) at a bulk pool.

No signals yet