139 lines
5.2 KiB
Markdown
139 lines
5.2 KiB
Markdown
---
|
|
title: "Self Hosted Backups with Minio, Kopia, and Tailscale"
|
|
date: 2023-04-25T00:11:31-04:00
|
|
toc: false
|
|
images:
|
|
tags:
|
|
- homelab
|
|
---
|
|
|
|
I've been struggling with my local backup setup for a while now.
|
|
I use [Kopia for backups](/2022.05.29.my-new-backup-kopia/), which is really good,
|
|
and I have a [custom built NAS](/raid/) where I can store backups.
|
|
That's all good so far, but how do I get the backups from my desktop to my NAS?
|
|
|
|
My first attempt was to set up Kopia to do backups with SSH access, which Kopia does support.
|
|
But when I decided to also limit how much of the server Kopia could access, I started to hit issues.
|
|
You can set up the OpenSSH server limit certain users to SFTP only with the `ForceCommand internal-sftp` setting, and a `ChrootDirectory` option can let you
|
|
limit them into specific folders too. But I kept hitting issues while setting this up, with the server
|
|
refusing connections whenever the limit is active. While I'm sure there's an answer to why I was failing to set this up,
|
|
I came up with an easier solution: Minio.
|
|
|
|
Minio is an S3 compatible, self hosted block storage service.
|
|
It is generally meant to be used in clusters, but there's nothing to stop you from putting it on a single device!
|
|
You do lose a few features like file locking, but most features still work.
|
|
Kopia has S3 support, so it should work with Minio.
|
|
|
|
To set up Minio, I put it in a `docker-compose.yml` like this:
|
|
|
|
```yml
|
|
minio:
|
|
image: minio/minio
|
|
command: minio server /data --console-address ":9001"
|
|
restart: always
|
|
volumes:
|
|
- minio-data:/data
|
|
ports:
|
|
- "9000:9000"
|
|
- "9001:9001"
|
|
env_file:
|
|
- .minio.env
|
|
```
|
|
|
|
Then in `.minio.env`, I enter the root username and password:
|
|
|
|
```
|
|
MINIO_ROOT_USER=...
|
|
MINIO_ROOT_PASSWORD=...
|
|
```
|
|
|
|
A `docker compose up -d`, and minio was running!
|
|
|
|
I hit a minor issue though: the minio server was running with HTTP not HTTPS, so no encryption.
|
|
This is not a big deal because the connection is only local,
|
|
it's literally 2 computers sitting in a room, connected with wires to each other over a network switch.
|
|
And Kopia does have a setting to allow HTTP connections.
|
|
And I could create a self-signed certificate and tell Kopia to use that,
|
|
but dealing with self-signed certificates can be a little annoying.
|
|
|
|
Now, I've also been looking for excused to play around with Tailscale. Tailscale
|
|
is a mesh VPN software that lets you connect devices securely, while still
|
|
allowing them to communicate peer-to-peer directly (when possible). I recently
|
|
set up all my devices with Tailscale to make it easier for myself to access my
|
|
home network remotely.
|
|
But Tailscale also comes with a lot of cool additional features.
|
|
One of these is the "MagicDNS", which automatically assigns "hostname.network.ts.net" domain names
|
|
to devices on your Tailscale network. And another feature allows you to generate real TLS
|
|
certificates for your MagicDNS domains. This is really cool because the generated certificates are "real",
|
|
they are not self-signed certificates and you don't need anything special for browsers and other tools to accept them.
|
|
|
|
|
|
|
|
So putting these together, I enabled MagicDNS and HTTPS certificates for my network. Then,
|
|
I generated my certificates with `sudo tailscale cert --cert-file public.crt --key-file private.key hostname.network.ts.net`,
|
|
and put those certificates into a `certs` folder.
|
|
Next, I adjusted my `docker-compose` file to make Minio use these certificates:
|
|
|
|
```yml
|
|
minio:
|
|
image: minio/minio
|
|
command: minio server /data --console-address ":9001" --certs-dir /certs
|
|
restart: always
|
|
volumes:
|
|
- minio-data:/data
|
|
- ./certs:/certs
|
|
ports:
|
|
- "9000:9000"
|
|
- "9001:9001"
|
|
env_file:
|
|
- .minio.env
|
|
```
|
|
|
|
Note the added `--certs-dir /certs` in the command, and the extra mount under volumes.
|
|
|
|
And that's about it! I rebuilt the container with `docker compose up -d minio`,
|
|
then navigated to `https://hostname.network.ts.net:9001` on a browser on a
|
|
Tailscale connected computer. And boom! HTTPS protected Minio console. While the
|
|
URL suggests it could be public, these domains are local to your Tailscale
|
|
network unless you explicitly expose them.
|
|
|
|
Next, I created a bucket named `backup` to house my backups. Then, I created an
|
|
access key. Minio allows you to restrict what a client can and can't do with an
|
|
access key by defining a policy. I restricted this access key to only access my
|
|
backups bucket with this policy:
|
|
|
|
```json
|
|
{
|
|
"Version": "2012-10-17",
|
|
"Statement": [
|
|
{
|
|
"Effect": "Allow",
|
|
"Action": [
|
|
"s3:*"
|
|
],
|
|
"Resource": [
|
|
"arn:aws:s3:::backup/*",
|
|
"arn:aws:s3:::backup"
|
|
]
|
|
},
|
|
{
|
|
"Effect": "Allow",
|
|
"Action": [
|
|
"s3:ListAllMyBuckets"
|
|
],
|
|
"Resource": [
|
|
"arn:aws:s3:::*"
|
|
]
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
There is 2 statements in the policy, first allowing all access to the backup
|
|
bucket only. The second statement allows the key to check what buckets are
|
|
available. I'm not sure if I could have restricted that further as well, but I'm
|
|
happy with how strict this is already.
|
|
|
|
All that's left is to point Kopia at `https://hostname.network.ts.net:9000`,
|
|
enter the access key, and let it back things up.
|