Death of a Cluster

After enthusiastically building my homelab to spread out over 3 machines via Docker Swarm with the data replicated via GlusterFS, I finally had enough and pulled the plug. My homelab is no longer a cluster, and now purrs happily as a single-node server.

Why? Because it gave me hell.

The whole idea of that setup was to make the homelab more resilient. Instead, it became a ticking time bomb. Docker Swarm, even with its best intentions, screwed up graceful shutdowns of my MySQL containers. The machine gets restarted due to an update, electricity flickers, etc. all things can happen and normally it's fine. What's not fine is Docker Swarm trying to instantly revive my MySQL server whilst GlusterFS trying to sync it from the failed server. I'm not sure whose fault is this, but these two can't coexist. I got tired of restoring from backups and losing data (and facing a pissed off wife because her website is down).

Home Assistant was also another failure. All sorts of docker networking issues when multiple device and port types are involved. I got tired of trying to get Apple Home recognising Home Assistant as a hub, no matter how many times I tried. Yes, I do have a firewall on my UniFi Cloud Gateway Ultra, but that's the least of my issues.

Trying to host a pihole on three nodes with syncing between them over docker swarm overlay? A hell on its own.

A lot of things worked well: Keepalived switching the welcoming IP between nodes was great, ability to keep websites up to patch the nodes was amazing, learning how all of these tied up was a great experience.

But it was very tiring to keep these 3 nodes alive. Gluster always was problematic, and even gave a "split brain" issue which is one of the worst things you can get with it. Docker Swarm was a great choice over Kubernetes, but still wasn't playing well with containers and GlusterFS.

Another reason is, I got tired of trying to self-host new things. Other than hosting a few Ghost websites to avoid paying monthly subscription fees, nothing stuck. I tried note apps like Siyuan, tried home automation, tried recipe books (I might bring back KitchenOwl, though), implemented my own authentication with Authelia, all sort of shit I pulled, nothing ended up being permanent. In the end I only ended up using the blogs and VaultWarden (which is purely because I can't easily share passwords between Apple accounts). Everything else, I went back to iCloud.

Also I have a Synology NAS, so no TrueNAS or any other solution needed. I also don't use Sonarr or media streaming through it.

All in all, went back to basics with a single node. Will see how far it'll take me.