From c93b48bb7478d95f39f36e2395c2d6ee25cad14c Mon Sep 17 00:00:00 2001 From: Kaan Barmore-Genc Date: Tue, 25 Apr 2023 00:56:05 -0400 Subject: [PATCH] Add "self hosted backups with Minio, Kopia, and Tailscale" --- content/img/2023-04-25.tailscale.png | 3 + ...sted-backups-with-minio-kopia-tailscale.md | 138 ++++++++++++++++++ 2 files changed, 141 insertions(+) create mode 100644 content/img/2023-04-25.tailscale.png create mode 100644 content/posts/2023.04.25.self-hosted-backups-with-minio-kopia-tailscale.md diff --git a/content/img/2023-04-25.tailscale.png b/content/img/2023-04-25.tailscale.png new file mode 100644 index 0000000..f4aba4e --- /dev/null +++ b/content/img/2023-04-25.tailscale.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fcd75bd3a362146581c204d22f7d83461fc4d2c6543e7aa0ec6d79b61ebe4cd6 +size 15384 diff --git a/content/posts/2023.04.25.self-hosted-backups-with-minio-kopia-tailscale.md b/content/posts/2023.04.25.self-hosted-backups-with-minio-kopia-tailscale.md new file mode 100644 index 0000000..ae76114 --- /dev/null +++ b/content/posts/2023.04.25.self-hosted-backups-with-minio-kopia-tailscale.md @@ -0,0 +1,138 @@ +--- +title: "Self Hosted Backups with Minio, Kopia, and Tailscale" +date: 2023-04-25T00:11:31-04:00 +toc: false +images: +tags: + - homelab +--- + +I've been struggling with my local backup setup for a while now. +I use [Kopia for backups](/2022.05.29.my-new-backup-kopia/), which is really good, +and I have a [custom built NAS](/raid/) where I can store backups. +That's all good so far, but how do I get the backups from my desktop to my NAS? + +My first attempt was to set up Kopia to do backups with SSH access, which Kopia does support. +But when I decided to also limit how much of the server Kopia could access, I started to hit issues. +You can set up the OpenSSH server limit certain users to SFTP only with the `ForceCommand internal-sftp` setting, and a `ChrootDirectory` option can let you +limit them into specific folders too. But I kept hitting issues while setting this up, with the server +refusing connections whenever the limit is active. While I'm sure there's an answer to why I was failing to set this up, +I came up with an easier solution: Minio. + +Minio is an S3 compatible, self hosted block storage service. +It is generally meant to be used in clusters, but there's nothing to stop you from putting it on a single device! +You do lose a few features like file locking, but most features still work. +Kopia has S3 support, so it should work with Minio. + +To set up Minio, I put it in a `docker-compose.yml` like this: + +```yml +minio: + image: minio/minio + command: minio server /data --console-address ":9001" + restart: always + volumes: + - minio-data:/data + ports: + - "9000:9000" + - "9001:9001" + env_file: + - .minio.env +``` + +Then in `.minio.env`, I enter the root username and password: + +``` +MINIO_ROOT_USER=... +MINIO_ROOT_PASSWORD=... +``` + +A `docker compose up -d`, and minio was running! + +I hit a minor issue though: the minio server was running with HTTP not HTTPS, so no encryption. +This is not a big deal because the connection is only local, +it's literally 2 computers sitting in a room, connected with wires to each other over a network switch. +And Kopia does have a setting to allow HTTP connections. +And I could create a self-signed certificate and tell Kopia to use that, +but dealing with self-signed certificates can be a little annoying. + +Now, I've also been looking for excused to play around with Tailscale. Tailscale +is a mesh VPN software that lets you connect devices securely, while still +allowing them to communicate peer-to-peer directly (when possible). I recently +set up all my devices with Tailscale to make it easier for myself to access my +home network remotely. +But Tailscale also comes with a lot of cool additional features. +One of these is the "MagicDNS", which automatically assigns "hostname.network.ts.net" domain names +to devices on your Tailscale network. And another feature allows you to generate real TLS +certificates for your MagicDNS domains. This is really cool because the generated certificates are "real", +they are not self-signed certificates and you don't need anything special for browsers and other tools to accept them. + +![A web page with the contents: HTTPS Certificates. Beta. Allow users to provision HTTPS cerificates for their devices. Learn More. Below is a button labeled "Disable HTTPS".](/img/2023-04-25.tailscale.png) + +So putting these together, I enabled MagicDNS and HTTPS certificates for my network. Then, +I generated my certificates with `sudo tailscale cert --cert-file public.crt --key-file private.key hostname.network.ts.net`, +and put those certificates into a `certs` folder. +Next, I adjusted my `docker-compose` file to make Minio use these certificates: + +```yml +minio: + image: minio/minio + command: minio server /data --console-address ":9001" --certs-dir /certs + restart: always + volumes: + - minio-data:/data + - ./certs:/certs + ports: + - "9000:9000" + - "9001:9001" + env_file: + - .minio.env +``` + +Note the added `--certs-dir /certs` in the command, and the extra mount under volumes. + +And that's about it! I rebuilt the container with `docker compose up -d minio`, +then navigated to `https://hostname.network.ts.net:9001` on a browser on a +Tailscale connected computer. And boom! HTTPS protected Minio console. While the +URL suggests it could be public, these domains are local to your Tailscale +network unless you explicitly expose them. + +Next, I created a bucket named `backup` to house my backups. Then, I created an +access key. Minio allows you to restrict what a client can and can't do with an +access key by defining a policy. I restricted this access key to only access my +backups bucket with this policy: + +```json +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": [ + "s3:*" + ], + "Resource": [ + "arn:aws:s3:::backup/*", + "arn:aws:s3:::backup" + ] + }, + { + "Effect": "Allow", + "Action": [ + "s3:ListAllMyBuckets" + ], + "Resource": [ + "arn:aws:s3:::*" + ] + } + ] +} +``` + +There is 2 statements in the policy, first allowing all access to the backup +bucket only. The second statement allows the key to check what buckets are +available. I'm not sure if I could have restricted that further as well, but I'm +happy with how strict this is already. + +All that's left is to point Kopia at `https://hostname.network.ts.net:9000`, +enter the access key, and let it back things up.