bgenc.net/content/posts/2023.04.25.self-hosted-backups-with-minio-kopia-tailscale.md

title	date	toc	images	tags
Self Hosted Backups with Minio, Kopia, and Tailscale	2023-04-25T00:11:31-04:00	false		homelab

I've been struggling with my local backup setup for a while now. I use Kopia for backups, which is really good, and I have a custom built NAS where I can store backups. That's all good so far, but how do I get the backups from my desktop to my NAS?

My first attempt was to set up Kopia to do backups with SSH access, which Kopia does support. But when I decided to also limit how much of the server Kopia could access, I started to hit issues. You can set up the OpenSSH server limit certain users to SFTP only with the ForceCommand internal-sftp setting, and a ChrootDirectory option can let you limit them into specific folders too. But I kept hitting issues while setting this up, with the server refusing connections whenever the limit is active. While I'm sure there's an answer to why I was failing to set this up, I came up with an easier solution: Minio.

Minio is an S3 compatible, self hosted block storage service. It is generally meant to be used in clusters, but there's nothing to stop you from putting it on a single device! You do lose a few features like file locking, but most features still work. Kopia has S3 support, so it should work with Minio.

To set up Minio, I put it in a docker-compose.yml like this:

minio:
  image: minio/minio
  command: minio server /data --console-address ":9001"
  restart: always
  volumes:
    - minio-data:/data
  ports:
    - "9000:9000"
    - "9001:9001"
  env_file:
    - .minio.env

Then in .minio.env, I enter the root username and password:

MINIO_ROOT_USER=... 
MINIO_ROOT_PASSWORD=...

A docker compose up -d, and minio was running!

I hit a minor issue though: the minio server was running with HTTP not HTTPS, so no encryption. This is not a big deal because the connection is only local, it's literally 2 computers sitting in a room, connected with wires to each other over a network switch. And Kopia does have a setting to allow HTTP connections. And I could create a self-signed certificate and tell Kopia to use that, but dealing with self-signed certificates can be a little annoying.

Now, I've also been looking for excused to play around with Tailscale. Tailscale is a mesh VPN software that lets you connect devices securely, while still allowing them to communicate peer-to-peer directly (when possible). I recently set up all my devices with Tailscale to make it easier for myself to access my home network remotely. But Tailscale also comes with a lot of cool additional features. One of these is the "MagicDNS", which automatically assigns "hostname.network.ts.net" domain names to devices on your Tailscale network. And another feature allows you to generate real TLS certificates for your MagicDNS domains. This is really cool because the generated certificates are "real", they are not self-signed certificates and you don't need anything special for browsers and other tools to accept them.

So putting these together, I enabled MagicDNS and HTTPS certificates for my network. Then, I generated my certificates with sudo tailscale cert --cert-file public.crt --key-file private.key hostname.network.ts.net, and put those certificates into a certs folder. Next, I adjusted my docker-compose file to make Minio use these certificates:

minio:
  image: minio/minio
  command: minio server /data --console-address ":9001" --certs-dir /certs
  restart: always
  volumes:
    - minio-data:/data
    - ./certs:/certs
  ports:
    - "9000:9000"
    - "9001:9001"
  env_file:
    - .minio.env

Note the added --certs-dir /certs in the command, and the extra mount under volumes.

And that's about it! I rebuilt the container with docker compose up -d minio, then navigated to https://hostname.network.ts.net:9001 on a browser on a Tailscale connected computer. And boom! HTTPS protected Minio console. While the URL suggests it could be public, these domains are local to your Tailscale network unless you explicitly expose them.

Next, I created a bucket named backup to house my backups. Then, I created an access key. Minio allows you to restrict what a client can and can't do with an access key by defining a policy. I restricted this access key to only access my backups bucket with this policy:

{
 "Version": "2012-10-17",
 "Statement": [
  {
   "Effect": "Allow",
   "Action": [
    "s3:*"
   ],
   "Resource": [
    "arn:aws:s3:::backup/*",
    "arn:aws:s3:::backup"
   ]
  },
  {
   "Effect": "Allow",
   "Action": [
    "s3:ListAllMyBuckets"
   ],
   "Resource": [
    "arn:aws:s3:::*"
   ]
  }
 ]
}

There is 2 statements in the policy, first allowing all access to the backup bucket only. The second statement allows the key to check what buckets are available. I'm not sure if I could have restricted that further as well, but I'm happy with how strict this is already.

All that's left is to point Kopia at https://hostname.network.ts.net:9000, enter the access key, and let it back things up.