Using mergerfs to increase your virtual storage

What happens if you have multiple disks or partitions that you’d like to use for a media project and you don’t want to lose any of your existing data, but you’d like to have everything located or mounted under one drive. That’s where mergerfs can come to your rescue!

mergerfs is a union filesystem geared towards simplifying storage and management of files across numerous commodity storage devices.

You will need to grab the latest RPM from their github page here. The releases for Fedora have fc and the version number in the name. For example here is the version for Fedora 31:

mergerfs-2.29.0-1.fc31.x86_64.rpm

Installing and configuring mergerfs

Install the mergerfs package that you’ve downloaded using sudo:

$ sudo dnf install mergerfs-2.29.0-1.fc31.x86_64.rpm

You will now be able to mount multiple disks as one drive. This comes in handy if you have a media server and you’d like all of your media files to show up under one location. If you upload new files to your system, you can copy them to your mergerfs directory and mergerfs will automatically copy them to which ever drive has enough free space available.

Here is an example to make it easier to understand:

$ df -hT | grep disk
/dev/sdb1      ext4      23M  386K 21M 2% /disk1
/dev/sdc1      ext4      44M  1.1M 40M 3% /disk2

$ ls -l /disk1/Videos/
total 1
-rw-r--r--. 1 curt curt 0 Mar 8 17:17 Our Wedding.mkv

$ ls -l /disk2/Videos/
total 2
-rw-r--r--. 1 curt curt 0 Mar 8 17:17 Baby's first Xmas.mkv
-rw-rw-r--. 1 curt curt 0 Mar 8 17:21 Halloween hijinks.mkv

In this example there are two disks mounted as disk1 and disk2. Both drives have a Videos directory with existing files.

Now we’re going to mount those drives using mergerfs to make them appear as one larger drive.

$ sudo mergerfs -o defaults,allow_other,use_ino,category.create=mfs,moveonenospc=true,minfreespace=1M /disk1:/disk2 /media

The mergerfs man page is quite extensive and complex so we’ll break down the options that were specified.

  • defaults: This will use the default settings unless specified.
  • allow_other: allows users besides sudo or root to see the filesystem.
  • use_ino: Causes mergerfs to supply file/directory inodes rather than libfuse. While not a default it is recommended it be enabled so that linked files share the same inode value.
  • category.create=mfs: Spreads files out across your drives based on available space.
  • moveonenospc=true: If enabled, if writing fails, a scan will be done looking for the drive with the most free space.
  • minfreespace=1M: The minimum space value used.
  • disk1: First hard drive.
  • disk2: Second hard drive.
  • /media: The directory folder where the drives are mounted.

Here is what it looks like:

$ df -hT | grep disk 
/dev/sdb1  ext4           23M      386K 21M 2% /disk1 
/dev/sdc1  ext4           44M      1.1M 40M 3% /disk2 

$ df -hT | grep media 
1:2        fuse.mergerfs  66M      1.4M 60M 3% /media 

You can see that the mergerfs mount now shows a total capacity of 66M which is the combined total of the two hard drives.

Using mergerfs

Continuing with the example:

There is a 30Mb video called Baby’s second Xmas.mkv. Let’s copy it to the /media folder which is the mergerfs mount.

$ ls -lh "Baby's second Xmas.mkv"
-rw-rw-r--. 1 curt curt 30M Apr 20 08:45 Baby's second Xmas.mkv
$ cp "Baby's second Xmas.mkv" /media/Videos/

Here is the end result:

$ df -hT | grep disk
/dev/sdb1  ext4          23M 386K 21M 2% /disk1
/dev/sdc1  ext4          44M 31M 9.8M 76% /disk2

$ df -hT | grep media
1:2        fuse.mergerfs 66M 31M 30M 51% /media

You can see from the disk space utilization that mergerfs automatically copied the file to disk2 because disk1 did not have enough free space.

Here is a breakdown of all of the files:

$ ls -l /disk1/Videos/
total 1
-rw-r--r--. 1 curt curt 0 Mar 8 17:17 Our Wedding.mkv

$ ls -l /disk2/Videos/
total 30003
-rw-r--r--. 1 curt curt 0 Mar 8 17:17 Baby's first Xmas.mkv
-rw-rw-r--. 1 curt curt 30720000 Apr 20 08:47 Baby's second Xmas.mkv
-rw-rw-r--. 1 curt curt 0 Mar 8 17:21 Halloween hijinks.mkv

$ ls -l /media/Videos/
total 30004
-rw-r--r--. 1 curt curt 0 Mar 8 17:17 Baby's first Xmas.mkv
-rw-rw-r--. 1 curt curt 30720000 Apr 20 08:47 Baby's second Xmas.mkv
-rw-rw-r--. 1 curt curt 0 Mar 8 17:21 Halloween hijinks.mkv
-rw-r--r--. 1 curt curt 0 Mar 8 17:17 Our Wedding.mkv

When you copy files to your mergerfs mount, it will always copy the files to the hard disk that has enough free space. If none of the drives in the pool have enough free space, then you won’t be able to copy them.

Fedora Project community For System Administrators Using Software

10 Comments

  1. Mike

    Nice article as I wasn’t aware of mergerfs, but someone is going to be really mad when they find out that their Wedding video and Baby’s First Christmas are empty files 😉

    • Why would their files be empty?

      No matter what storage system you have you need backup.

  2. Ph0zzy

    Does it have something to do with silverblue? I mean it would be nice to be able to install overlay packages without reboot.

    • ondrej

      This is not possible, as the system is readonly, it cant be changed on the fly. You can always use toolbox (containers) or virtualization.

  3. Vernon Van Steenkist

    Great Article. A couple of questions:

    How is this different from regular unionfs?

    Do you take any file system performance hit?

    Where does the mea data get stored?

    Thanks

    • The mergerfs docs are very thorough. I recommend checking them out.

      How is this different from regular unionfs?

      As in unionfs-fuse? 1) It’s still maintained. 2) It offers a lot more functionality.

      Do you take any file system performance hit?

      Yes. Naturally. There is an additional layer so it will take a hit. The amount depends greatly on the usage patterns and system.

      Where does the mea data get stored?

      What metadata?

  4. Mx

    Use lvm

    • rfrr

      exactly

    • Cara

      lvm and mergerfs serve different needs and lvm has the drawback of data spanning volumes, a failure that spans multiple volumes wouldmean data loss if any of the volumes it spans goes down, whereas with mergerfs you only lose whats on one volume and theres no data spanning multiple volumes. you cal also add in-use data disks to a mergerfs pool. …additionally you can use LVM with mergerfs since they work on fundamentally different levels
      https://www.teknophiles.com/2018/02/19/disk-pooling-in-linux-with-mergerfs/

  5. Everything is very open with a really clear description of the challenges.
    It waas truly informative. Your site is useful.

    Many thanks for sharing!

Comments are Closed

The opinions expressed on this website are those of each author, not of the author's employer or of Red Hat. Fedora Magazine aspires to publish all content under a Creative Commons license but may not be able to do so in all cases. You are responsible for ensuring that you have the necessary permission to reuse any work on this site. The Fedora logo is a trademark of Red Hat, Inc. Terms and Conditions