Unifi Controller on Core OS with Terraform (Part 2: External Block mount)

tl;dr The code for the complete Unifi setup is available in the niels-s/unifi-terraform-example repo

This post is part of a small series, go and read the previous post to setup the basics

Create DO Volume

In this post we will setup an external block volume mount so we can store the data of the Unifi Controller. This gives us some more flexibility.

resource "digitalocean_volume" "unifi_controller_data" {
  region                  = digitalocean_droplet.unifi_controller.region
  name                    = "unifi_controller_data"
  size                    = 20
  initial_filesystem_type = "xfs"
  description             = "Store the data of the Unifi Controller"
}

resource "digitalocean_volume_attachment" "unifi_controller" {
  droplet_id = digitalocean_droplet.unifi_controller.id
  volume_id  = digitalocean_volume.unifi_controller_data.id
}

The critical part here is the digitalocean_volume_attachment resource. This resource, like the name mentions, makes sure the volume you created is attached to the droplet.

Mounting the volume on the host

Unfortunately, when you use a CoreOS host on Digital Ocean, the volumes aren’t mounted automatically (unlike with other distro’s), so we need to take care of that ourselves. So we need to write a little Ignition configuration

data "ignition_filesystem" "unifi_controller_data_mount" {
  name = "unifi_controller_data_mount"

  mount {
    device          = "/dev/disk/by-id/scsi-0DO_Volume_sdb"
    wipe_filesystem = false
    format          = "xfs"
  }
}

data "ignition_systemd_unit" "unifi_controller_data_unit" {
  name    = "mnt-unifi_controller_data.mount"
  enabled = true
  content = <<-CONFIG
    [Unit]
    Description = Unifi Controller Data Mount

    [Mount]
    What=/dev/disk/by-id/scsi-0DO_Volume_sdb
    Where=/mnt/unifi_controller_data
    Options=defaults,discard,noatime
    Type=xfs

    [Install]
    WantedBy = multi-user.target
  CONFIG
}

Luckily I didn’t need to invent the whole configuration myself, but Digital Ocean is so nice to provide you with an example config which you can find when you browse to Volumes > More > Config instructions.

Be careful though I spend quite some time searching why the host couldn’t mount the volume, and I found out the path of the device shared by Digital Ocean doesn’t match the actual path. The device path is used in the configuration above for ignition_filesystem.mount.device or ignition_systemd_unit.content.mount.what.

When you want to rename the mount, that’s perfectly fine, but make sure the ignition_systemd_unit.name and the mount path match otherwise CoreOS won’t be able to boot.

Debug tip: if the Droplet fails to boot, you can always get access through the Digital Ocean UI. Go to your Droplet > Access > Launch Console. The console immediately tells you if there is an issue. You can use journalctl -xe and systemctl to inspect the host.

For a little more information on mounting volumes with Ignition and Terraform you can check this blog which helped me figure it out: Mount a volume using Ignition and Terraform

Update the Ignition Config

Only adding the previous resources won’t work, you need to add them to the ignition_config resource otherwise the resources won’t be used

data "ignition_config" "unifi_controller" {
  ...

  filesystems = [
    data.ignition_filesystem.unifi_controller_data_mount.rendered
  ]

  systemd = [
    ...
    data.ignition_systemd_unit.unifi_controller_data_unit.rendered
  ]
}

When you apply these changes, you should see your volume mounted at /mnt/unifi_controller_data. Also, verify the size of your volume (df -h). I noticed the volume mounted in my case, but it wasn’t using its full size (20Gb). I fixed it manually for now by following the guidelines increase-size by Digital Ocean.

Note

After a week or so, I ran into issues with the MongoDB database. It had corrupted files. The database couldn’t start anymore and was trapped in a restart cycle consuming all the CPU. Luckily I configured auto backups so I could start over with a clean slate. But the logs did mention the WiredTiger Storage Engine, which prefers to run on XFS instead of EXT4. So I changed the filesystem type to XFS instead.

The code for the complete Unifi setup is available in the niels-s/unifi-terraform-example repo , the changes of this post can be found in this commit

This post is part of a small series, go and read the next post to setup a nginx proxy