(Justin)

Tech nerd from Sweden

  • 0 Posts
  • 127 Comments
Joined 2 years ago
cake
Cake day: June 10th, 2023

help-circle



  • Yup, same here. being able to skip all the networking and DNS hassle and have it automated for you is so nice.

    Having databases fully managed with cnpg is AMAZING

    I just have renovate set to auto update my argocd, so everything just runs itself with zero issues. Only the occasional stateful container that has breaking changes in a minor version.

    If something OOMs or crashes, it all just self heals, I never need to worry about it. I don’t have any HPAs (nor cluster scaling obv), though I do have some HA stuff set up just to reduce restart times and help keep the databases happy.

    The main issue with Kubernetes is that a lot of self-hosted software makes bad design decisions that actively make kubernetes harder, eg sqlite instead of postgres and secrets stored in large config files. The other big issue is that documentation only supports docker compose and not kubernetes 90% of the time so you have to know how to write yaml and read documentation.

    Moving my hass from a statefulset to kubevirt sounds tempting. Did you have better reliability/ergonomics? I have been looking into moving my Hass automation to NodeRed, so that I can GitOps it all, since NodeRed supports git syncing.

















  • Of the services OP is asking about, I’ve only run Lemmy, but I will say that running fediverse services are quite advanced, which is exactly what k8s is made for - Running advanced web applications.

    I’m firmly on the “k8s at any scale” team. If you can figure out how to run the k3s install command and are willing to look at some yaml documentation, you will have a much easier time setting up database and networking, running backups, porting your infrastructure to other providers, and maintaining everything, than with legacy control panels or docker compose. The main reason why Docker Compose is so much more accessible for self-hosters is because of the quantity of noob-focused documentation for Docker Compose, But learning either system requires learning the same concepts of containers, IP adresses, storage, etc. Docker Compose also has some disk and networking shortcuts for single-server workloads, but they also have their downsides (what is a macvlan?).

    The main reason why I think Kubernetes is critical for this specific workload is the number of production-critical databases that OP will need to run. OP will be running something like 4-8 postgres databases, with high uptime and 100% durability requirements. Trying to do that manually with Docker compose just isn’t feasible unless you’re willing to code. Kubernetes makes all of that automated with CNPG. See how easy it is to create a database and have automated backups to S3 with Kubernetes

    The biggest challenge for kubernetes is probably that the smaller applications don’t come with example configs for Kubernetes. I only see mastodon having one officially. Still, I’ve provided my config for Lemmy, and there are docker containers available for Friendica and mbin (though docker isn’t officially supported for these two). I’m happy to help give yaml examples for the installation of the applications.