Written on 2021-11-10
This post is a summary of my experiences running Kubernetes for personal and side projects. I’ve tried several approaches: starting with Kubeadm on Scaleway, switching to k3s for simplicity, and eventually moving to a bare-metal cluster on Hetzner. I’ll go over the setups, what worked and what didn’t, and some practical lessons about storage, automation, and cost.
At the time, I didn’t need a full Kubernetes cluster, just a few lightweight applications:
When I set up this cluster, there were no easy solutions; only Kubeadm was available, and it was somewhat challenging to configure. My setup:
Scaleway is a solid provider and worked well overall. Occasionally, I had to reset the master or a node due to system unresponsiveness. The NFS provisioner was unstable and required frequent restarts. Despite my efforts, I couldn’t fully resolve these issues.
After the first year, renewing the internal certificate was more difficult than expected. In the second year, I missed a step during the renewal, which broke communication between the master and nodes, making the K8S API unusable.
As my needs grew, I spent too much time managing Kubeadm. I added:
Scaleway remained affordable, and k3s was attractive, easy to use and much simpler than Kubeadm. The infrastructure was similar (I reinstalled the OS on every machine):
The main difference was switching to Longhorn for storage, which was much faster and more reliable than the NFS Provisioner. I kept an external NFS node for long-term backups.
Absolutely, blazing fast, and I rarely needed to SSH into any nodes (usually just to run htop and leave).
However, Scaleway announced a price increase of about 70%. Paying 90 EUR for a side project was no longer feasible.
My requirements increased further:
Hetzner auctions are fantastic. I now rent a single server with an i7 and 64GB RAM for less than 40 EUR! Plus, I have a hardware transcoder (Intel GPU) for the media service, enabling HEVC and 4K video transcoding without heavy CPU usage, leaving plenty of resources for other apps.
Definitely! It’s much cheaper and far more powerful. The only downside is the lack of redundancy, but I set up CRON jobs to back up all sensitive data to other providers (SCW Object Storage).
You can check out the current apps running on the cluster at k8s.fruitice.fr