Ansible on inherent site

State of Homelab 2024

Mon, 27 Jan 2025 22:38:48 +0200

When operating a homelab there is always something new to do or something waiting to be fixed. This post’s aim is to detail the state of my homelab at the end of 2024 and beginning of 2025.

Router

The previous post was about virtualizing pfSense inside ProxMox but this setup is no longer in use. In its place, there is a new system running OPNSense on bare-metal. This was done for a few reasons, the main one being that I wanted to move away from pfSense. More than that, there was a circular dependency where the ProxMox host had to be up in order for the router to function but in order for the cluster nodes to talk to each other, the router needed to be online. In practice this was not a huge issue but it bothered me since it made experimenting more difficult because recreating the cluster meant taking down the router as well. I’ve lost a few things, mainly pfblocker-ng and native tailscale integration but they’re not as big of a loss as I had anticipated. In the end I think it was worth it, over the course of the past few years the company making pfSense has acted against the best interest of pfSense users and I did not feel like using their software anymore.

Virtualization

I’m still running ProxMox, although now it is a 3-node cluster. I’ve spent a pretty significant amount of time tweaking the bootstrap process from a fresh install to having the entire homelab operational. Somewhere along the line I tried making it so non-cluster operation of multiple hosts was supported but gave up on that eventually. Each of the nodes has a 250GB SSD used as a boot drive and for node-local storage along with a 2TB HDD used as a Ceph OSD. Since the last post about using ansible and terraform with ProxMox, I’ve figured out the process of adding tools to the debian image in the virtual machine template. There is also the option to deploy using GitLab CI so just making committing and pushing to the repo is enough to create a new virtual machine. The k3s virtual machine terraform configuration was improved significantly but I’ve not adopted modules yet which seems to be the next step for organization-related improvements. I also run all software directly on top of debian virtual machines without containers to practice writing ansible. At work, our team runs everything in Docker so it could be a good idea to start moving to something similar at home for the sake of practice.

Kubernetes

The kubernetes cluster got a significant update. I’ve adopted FluxCD to a greater degree, figured out secrets storage and a pretty good structure for the YAML files. The k3s cluster has 3 control plane nodes and 3 worker nodes, one per virtualization host. I’m using longhorn for storage with plans to try and use the Ceph cluster as an additional storage class. The networking setup consists of MetalLB to have an IP in the subnet and then run ingress-nginx as the ingress controller on that IP. No plans for checking out Gateway API have been made at current but it is a good and necessary step forward in the ecosystem, I plan to move to it after things are a bit less in flux. The DNS is manual right now due to the low amount of services but I’ve tested external-dns with PiHole and I’m aware of the OPNSense webhook option, it remains to be seen what I’ll end up using. The setup needs a bit more tender love and care but it’s running well and doing what I need it to. I have plans to move more stuff to the cluster after the foundation is solid and not just have my own demo applications in there. The thought has crossed my mind to completely do away with ProxMox and then run Talos Linux bare metal on the systems but I’d be losing out on some flexibility which I don’t want to commit to right now. All in all, kubernetes is still pretty fun and the 1.30 and later releases have made good quality of life improvements.

NAS

Since I upgraded my desktop, the old one’s internals were transplanted to a different case are now a second NAS. I’m saving up to get a big external drive so I can have an extra copy of the current main NAS so I can freely mess with them to have a solid backup solution. The changes in TrueNAS Scale regarding applications has left me waiting for things to settle down before I commit to a final setup. Overall nothing exciting on this front.

Hardware

I run mostly used office desktops and a basic managed switch. I have 3 Dell Optiplex machines: 2nd gen, 3rd gen and 4th gen Intel i5 CPUs with RAM sizes ranging from 16 to 32GB each. For storage they have a 250GB SSD and a 2TB HDD as mentioned earlier. The router is a Fujitsu equivalent desktop with a 4th gen Intel Pentium and 8GB of RAM as well as a 500GB SSD. My rackmount NAS is an i5 4570 with 16GB of RAM and 4x4TB HDDs in a 3-drive ZFS RAID-Z1 plus one hot spare. My backup NAS is an i7 4770 with 32GB of RAM and 2x8TB HDDs in a 2-way ZFS mirror. The main network switch is a Netgear GS724TV4, just a layer 2 managed gigabit switch.

Conclusion

I wanted to write all of this down to have a reference for myself when I inevitably change something and it might help someone else as well. Also the blog is back(?) after more than a 2 year silence, hopefully I will not neglect it for that long again.

Ansible, Terraform and Proxmox

Fri, 24 Jun 2022 23:24:57 +0300

In the continuing quest to improve my homelab I’ve written a few ansible playbooks as well as terraform configuration to automate a lot of tasks. The initial setup of proxmox is handled by ansible, the provisioning of virtual machines is handled by terraform and the installation of k3s on the VMs is done by ansible.

Proxmox Setup with Ansible

Before getting to use all the shiny devops with a selfhosted solution, we need to do a bit of setup first. This isn’t perfect since it’s just ansible running a bash script but it gets the job done. The other issue,, is that a lot of cloud images don’t include qemu-guest-agent and other packages by default. I have a few ideas how to solve that (check out this TODO.md for more info) but haven’t tested out anything yet. Since qemu-guest-agent and things like nfs-common are required on most VMs, a lot of redundant package installation happens but such is life for now. With that out of the way, let’s start with how I configure proxmox.

Host

While proxmox has a lot of things ready to go out of the box, a bit of configuration is always required and even more so in this case. The goal of the following steps is to turn our hosts into capable, cloud-like, targets for provisioning resources using terraform.

Repository

Initially the enterprise repository which we don’t have a license to access is enabled. To fix this, I delete the file in /etc/apt/sources.list.d that contains it and add a file with the no-subscription repository. This might not be necessary since the lae.proxmox ansible role can do essentially the same thing and many more. Here are the ansible tasks:

    - name: remove enterprise repo
      file:
        path: /etc/apt/sources.list.d/pve-enterprise.list
        state: absent

    - name: add no-subscription repo
      copy:
        src: pve-no-sub.list
        dest: /etc/apt/sources.list.d/pve-no-subscription.list
        mode: '0644'
        owner: root
        group: root

Template VM

The terraform provider for proxmox can work with cloud-init which is a great vendor-agnostic way of configuring the basics (username, password, IP addressing, DNS, SSH keys) in a virtualized environment. This is achieved by having a virtual disk or virtual cd-rom that mounts inside the VM and provides this information to the guest.

VM Image

Since software is not always compatible with the latest version of a linux distribution, in this case debian, I keep the last two versions around. The first step is to download the images:

    - name: get debian 10 cloud image
      get_url:
        url: https://cloud.debian.org/images/cloud/buster/20211011-792/debian-10-generic-amd64-20211011-792.qcow2
        dest: /root/debian-10-openstack-amd64.qcow2
        checksum: sha512:f3dac13104b4e28eb62c46cbbb1e30fc9c792834500f9101d477c19c258c31ff04850933ee0b3e63236eca38c854447d95a0ab45163c4a3fccf05f9dd95601a5
        mode: '0644'
        owner: root
        group: root

    - name: get debian 11 cloud image
      get_url:
        url: https://cloud.debian.org/images/cloud/bullseye/20220121-894/debian-11-generic-amd64-20220121-894.qcow2
        dest: /root/debian-11-openstack-amd64.qcow2
        checksum: sha512:0948dc56b4834a7755e4eae7d5532e138b90484a949161ffe9fc6894c7a14b1bd32ebf96fa3f3d03d498fe7ee125f8014f31bfab5825915a93de9330df814f7b
        mode: '0644'
        owner: root
        group: root

Template Script

As mentioned in the beginning I’d like to automate this with ansible but this is what I have for now:

if [ -z "$1" ]; then
    VMID=9001
else
    VMID="$1"
fi

if [ -z "$2" ]; then
    VMNAME="debian-tmpl"
else
    VMNAME="$2"
fi

VMEXISTS="$(qm list | grep $VMID)"

if [ "$VMEXISTS" ]; then
    echo "VM with ${VMID} already exists"
else
    qm create "${VMID}" -name "${VMNAME}" -memory 1024 -net0 virtio,bridge=vmbr0 -cores 1 -sockets 1 -cpu cputype=kvm64 -description "debbie image" -kvm 1 -numa 1
    qm importdisk "${VMID}" debian-10-openstack-amd64.qcow2 local-zfs
    qm set "${VMID}" -scsihw virtio-scsi-pci -virtio0 local-zfs:vm-"${VMID}"-disk-0
    qm set "${VMID}" -serial0 socket
    qm set "${VMID}" -boot c -bootdisk virtio0
#   qm set "${VMID}" -agent 1 #disabled since VM does not initially have qga installed
    qm set "${VMID}" -hotplug disk,network,usb,memory,cpu
    qm set "${VMID}" -vcpus 1
    qm set "${VMID}" -vga qxl
    qm set "${VMID}" -name "${VMNAME}"
    qm set "${VMID}" -ide2 local-zfs:cloudinit
    qm template "${VMID}"
fi

This works well enough for first-time setup and with a few additions could become idempotent. The above script and its debian 11 equivalent are copied over to the proxmox hosts and executed by the following ansible tasks:

    - name: get debian 10 template vm script
      copy:
        src: setup-debian-10-cloudinit.sh
        dest: /root/setup-debian-10-template.sh
        mode: '0755'
        owner: root
        group: root

    - name: run debian 10 template vm script
      shell:
        cmd: /root/setup-debian-10-template.sh

    - name: get debian 11 template vm script
      copy:
        src: setup-debian-11-cloudinit.sh
        dest: /root/setup-debian-11-template.sh
        mode: '0755'
        owner: root
        group: root

    - name: run debian 11 template vm script
      shell:
        cmd: /root/setup-debian-11-template.sh

Terraform Setup

With proxmox ready, let’s move on to the terraform setup. If you haven’t used it before, terraform is a declarative way of describing our infrastructure. It can manage virtual machines, DNS records, object storage buckets, kubernetes resources and many more things with the appropriate providers.

Provider

Providers in terraform are plugins that enable terraform to manage resources that it doesn’t support out of the box. In this case, we want to manage proxmox virtual machines and containers but it could be other things like a router running pfsense or vyos. In previous versions, the provider would need to be installed manually but nowadays we only need to create a main.tf with the provider listed and upon running terraform init, the provider will be downloaded automatically.

Basic `main.tf`

Let’s create that file then. Open your favorite editor, paste the following and replace the IP as well as the password with the ones applicable to your setup:

terraform {
	required_providers {
		proxmox = {
			source = "Telmate/proxmox"
			version = ">=2.9.5"
		}
	}
}

provider "proxmox" {
	alias = "pve"
	pm_tls_insecure = true
	pm_api_url = "https://192.168.70.1:8006/api2/json" #Replace IP
	pm_password = "password123" #Replace password
	pm_user = "root@pam"
}

Save the file and then run terraform init. You should see the following output (the version might be different):

Initializing the backend...

Initializing provider plugins...
- Finding telmate/proxmox versions matching ">= 2.9.5"...
- Installing telmate/proxmox v2.9.10...
- Installed telmate/proxmox v2.9.10 (self-signed, key ID A9EBBE091B35AFCE)

Partner and community providers are signed by their developers.
If you'd like to know more about provider signing, you can read about it here:
https://www.terraform.io/docs/cli/plugins/signing.html

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

And ta-da! We’ve installed the provider and are ready to start creating resources.

Creating a test VM

The above file won’t do anything so we’ll declare a virtual machine and then tell terraform to create it. The example below is specific to my setup so you probably want to modify it before using but here it is:

terraform {
	required_providers {
		proxmox = {
			source = "Telmate/proxmox"
			version = ">=2.9.5"
		}
	}
}

provider "proxmox" {
	alias = "pve"
	pm_tls_insecure = true
	pm_api_url = "https://192.168.70.1:8006/api2/json" #Replace IP
	pm_password = "password123" #Replace password
	pm_user = "root@pam"
}

resource "proxmox_vm_qemu" "proxmox_test_vm" {
	provider = proxmox.pve
	name = "deb-10-test"
	target_node = "pve"

	/* change to debian-templ for 11 */
	clone = "debian-tmpl"
	os_type = "cloud-init"
	cores = 2
	sockets = 1
	cpu = "host"
	memory = 2048
	scsihw = "virtio-scsi-pci"
	bootdisk = "virtio0"
	agent = 0

	disk {
		size = "30G"
		type = "virtio"
		storage = "local-zfs"
	}

	network {
		model = "virtio"
		bridge = "vmbr0"
	}

	ipconfig0 = "ip=192.168.50.25/16,gw=192.168.0.1"

	/*workstation SSH key*/
	sshkeys = "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5jzKi37jm3517bqThbw+7LR/GXm3qC6Az5F+ZUa36vYM7Ygk2K5bWcFIL2YUCrkL5jfSsvoowONjCAxyuoyxtW4MJxnQLyq4u4yDsRC7YvBPAKZUYaHwnbkCfDs5a75dEFOoDxCA0DY2GrhqzBndaTcCfl0fZ4vN+9LcKOb1dSKiHeHvsh35YNtwntbL21meo+hiycUEgGwNe9/4kxKpdGTr7HvbeX2Fjm/UZBZIJKVcGop/3gCHXYnKH+OY5zc8cmt9Jg4CIwEqrSKeOX0bE8LSPRpVRXH4v8OcMaMei/HQejlH8NBwybEdJ4mhl8vHaFEjDbIWoOujmiRQF2263 angle@puddle"

	/*required otherwise the state always appears modified*/
	lifecycle {
		ignore_changes = [
			cipassword,
			network,
			desc,
		]
	}
}

After saving the file, we can run terraform plan -out create-test-vm.plan to calculate the changes required and save them to a file. Following that, we can run terraform apply create-test-vm.plan to apply the changes to our infrastructure. Some output will appear while the template is cloned, the VM settings are adjusted and the cloud-init configuration is put into the drive. In the end, you should have a brand new VM created by terraform!

Creating VMs for k3s

The above example is realistic but what if we need 5 similar VMs that could be used as kubernetes nodes? Conveniently, terraform provides a count variable that we can use to slightly differ settings where needed. Here is the resource that provisions the virtual machines that I run a k3s cluster on:

resource "proxmox_vm_qemu" "proxmox_vm_k3s" {
	provider = proxmox.pve
	count = 5
	name = "deb-k3s-${count.index + 1}"
	target_node = "pve"

	clone = "debian-tmpl"
	os_type = "cloud-init"
	cores = 2
	sockets = 1
	cpu = "host"
	memory = 3084
	scsihw = "virtio-scsi-pci"
	bootdisk = "virtio0"
	agent = 0
	onboot = true

	disk {
		size = "30G"
		type = "virtio"
		storage = "local-zfs"
	}

	network {
		model = "virtio"
		bridge = "vmbr0"
	}

	ipconfig0 = "ip=192.168.15.5${count.index + 1}/16,gw=192.168.0.1"

	sshkeys = "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5jzKi37jm3517bqThbw+7LR/GXm3qC6Az5F+ZUa36vYM7Ygk2K5bWcFIL2YUCrkL5jfSsvoowONjCAxyuoyxtW4MJxnQLyq4u4yDsRC7YvBPAKZUYaHwnbkCfDs5a75dEFOoDxCA0DY2GrhqzBndaTcCfl0fZ4vN+9LcKOb1dSKiHeHvsh35YNtwntbL21meo+hiycUEgGwNe9/4kxKpdGTr7HvbeX2Fjm/UZBZIJKVcGop/3gCHXYnKH+OY5zc8cmt9Jg4CIwEqrSKeOX0bE8LSPRpVRXH4v8OcMaMei/HQejlH8NBwybEdJ4mhl8vHaFEjDbIWoOujmiRQF2263 angle@puddle"

	lifecycle {
		ignore_changes = [
			cipassword,
			network,
			desc,
		]
	}
}

There aren’t many changes from the previous example, the name and the IP address use the index in order to avoid 5 VMs having the same name and IP address and that’s it!

Conclusion

Hopefully this gives you inspiration for some awesome things you can do to automate your infrastructure and make working in a virtualized environment easier. Thank you for reading.

Reinstalling Gentoo

Wed, 10 Mar 2021 00:54:04 +0200

Why

This gentoo install has lasted a good long couple of years. There is nothing really wrong with it, the SSD it’s installed on is a bit full and it’s a good opportunity to make a change. However I’m looking to shake things up a bit. The big change is installing on ZFS this time. It’s fairly well-known that I’m a big fan of ZFS, my storage server has been using it in one form or another sinc forever and all of my home infrastructure directly or indirectly depends on it. However it is not the only change being made, executing the install using ansible will be a good improvement too. One of the issues with gentoo, to people that have been using it for a while, is the seeming non-repeatability of the install. A script can get really close however certain steps are not as easy to get right and everything is highly procedural instead of declarative or idempotent way. The other changes will include moving to a system with as few, as I view them, harmful or unnecessary componenets. Stuff like dbus, eudev, elogind, pulseaudio, systray and a few other things that I’m in the process of nailing down will be (hopefully) gone.

How

Ansible

First off, the ansible playbook. There are a few different ones floating around but are outdated or don’t fit my needs. Of course that doesn’t mean I can’t examine the pre-existing solutions, I will absolutely take inspiration from solutions that smarter people have come up with.

ZFS

The most important decision will be the layout. I know I’ll be using docker and libvirt so adjustments to /var/lib are needed. Also, the system needs to be bootable and usable to some extent even if the ZFS kernel module doesn’t get loaded. There are 2 different solutions to this, either go with EXT4/XFS or just build ZFS into the kernel. I’m still undecided about the route I’ll go with but remain open to experimenting and seeing which one is better.

Cruft removal

Several components are impacted here. The most interesting one I’m looking forward to installing is libudev-zero to seemingly satisfy any dependencies that require (e)udev while in reality using busybox mdev. This gets rid of some dependency problems with libinput/evdev and anything else that has a hard dependency on a single device manager implementation without source code patches to those projects. It also give me the ability to use input classes in xorg instead of hardcoding device paths which is something that was somewhat annoying last time I tried switching away from eudev. As for the rest, I’m testing out what can be effortlessly removed and what requires significant changes to one thing or another on my laptop. We’ll see what I end up with, I’ll update this post after it’s up and running.

Conclusion

I’ve enjoyed my gentoo system for multiple years at this point but I want to make things better so I’ll be reinstalling and starting from a clean slate. Switching the filesystem to the one my storage server relies on will be pretty sweet (I might also look into some pre or post package install hooks). The installation process, and hopefully the full system setup, will be described using ansible to make it more easily repeatable And finally, any piece of software I don’t like or use will be removed or replaced and incompatibilites fixed along the way. I look forward to a better gentoo system in 2021.

Homelab Current Form

Sun, 07 Feb 2021 14:35:32 +0200

The first part of the software changes I made was covered in the first part where I explained how and why I started going down this path.

Software choices

This whole journey was about making my home infrastructure better. Part of that was about having a way to more easily create the foundation on which kubernetes would run on, as well as describing my setup as code. Proxmox is a beast in its own right. Features include being able to manage kvm virtual machines and lxc containers, an API that can be used to interact with it programmatically and most importantly being open source. While researching what automation tools could interact with proxmox, I found a community-made terraform provider as well as a set of two ansible community modules (kvm and lxc). After thinking about it a little bit, I wanted to use terraform in a non-cloud environment but also gather some knowledge about it so that’s what I decided to use. I can have my provisioning requirements in a file that terraform understands and store it in git which is exactly what I was going for. Ansible is still in the picture since it is used to set up and configure the debian environment inside the virtual machines. Now that the hypervisor part is covered, let’s move on to running services.

Making it store is harder than making it run

As mentioned in the first part about software changes, I was learning kubernetes and now I could not only wipe and recreate the cluster itself but also the entire virtual machines that it was running on. Some time after starting to use proxmox with terraform I attempted to use rancher to manage kubernetes but ended up ditching it due to various problems with running even basic stuff on it (very likely that it’s a case of PEBCAK, I don’t think rancher is terrible or anything like that).

The problem

However the problem of shared storage continued to taunt me for the following months. There was seemingly nothing that a simple fella could do to use a nice and simple NAS running truenas core as storage that is able to be dynamically provisioned for use by kubernetes. Now, I hear you, “what about nfs-client-provisioner”, someone less familiar with this pile of madness might exclaim. Indeed it does exist and barely work except the helm chart for it is deprecated and it does not work with kubernetes version 1.20 and later since it does not seem to be using CSI drivers.

I really tried

Months of furious and frustrating testing ensued. Not only was I trying to get applications running on kubernetes, I was also fighting with the unexpectedly complex tast of using the storage server I had available.

glusterfs

I’ve gone through basically any and all commonly suggested options for dynamically provisioned storage. Prette early on I tried glusterfs with heketi by making 3 LXC containers and mounting an nfs share to each one that would serve as the brick in the gluster volume. Suffice it to say that it didn’t work and things were getting out of hand.

longhorn

After a bit more research and I found out that longhorn, another project by the authors of k3s, used iscsi to communicate with kubernetes. That could work I thought, except there was no coherent example of how to use it without using targetd for iscsi. Longhorn was starting to look more appealing, I could just put the virtual drives of the VMs on an nfs share, run longhorn inside the VMs to pool all their storage together and call it good enough. No, I could not give up yet. A sub-optimal solution would do if there was no other way but I was convinced something more was out there. My patience and hope were running out but not empty yet.

success

That all changed in early January where during my nearly daily search for possible storage solutions I hadn’t tried, I found out about democratic-csi. This was it. Made to be used with freenas, truenas as well as DIY ZFS setups. The silver bullet was here. Just a few minutes of reading about it and writing my configs, a short(read: long) helm command later and…success! The test pod was using the newly created storageclass that was backed by the nfs share on truenas. After that I rushed to deploy my standard set of gitea, droneci and minecraft to test it out and it was working for real without a hitch.

Conclusion

This was a long journey and it isn’t coming to a close any time soon. The next step is to continuous deployment but that’s an issue for another time. To conclude as briefly as possible, I’ve no put most of the critical pieces of my homelab in a single git repo that I can use to recreate almost all of it from scratch with minimal manual intervention (there are still a few quirks like not having dynamic inventory for ansible and having to manually copy IPs but I can put that aside for now). Despite the imperfections like not having dynamic inventory for ansible or having secrets stored unencrypted in git, I’m very happy with the setup is working. If you missed them, make sure to check out the hardware updates and the first part about software changes for a better of what I’ve been up to.

Homelab Evolved

Sun, 07 Feb 2021 13:53:10 +0200

The hardware side of the changes made to my homelab were covered in a previous post where I also alluded to some software changes.

What came before

This is the first part explaining more in-depth the issues that I had and how I dealt with them. Let’s set the stage first, my homelab was working fine in many ways. It was nice and stable debian install running kvm virtual machines using qemu and libvirt with virt-manager to perform the simpler start/stop tasks. There was really nothing wrong with it, I was exploring different software, running some services on the local network and even hosting a public website on it.

Good enough is not good enough

However as someone interested in infrastructure a few things were bothering me. Sure, I was using docker-compose with traefik for my public website and services. Yes, I was using ansible to manage the configuration of most virtual machines. But “good enough” and “I guess it works” doesn’t cut it. On the upside, I really liked having ZFS snapshots on a 2x2TB mirror. The ability to go back to a point in time where things were not broken had saved me a couple times when I inevitably broke one thing or deleted a file that could be replaced but would take a while. My backups were also on that ZFS mirror, on a different dataset of course, and that also very useful. I knew I wanted ZFS and that there was much room for improvement in regards to handling how I was running services. The single docker-compose virtual machine for the public stuff and another one for local stuff were mostly adequate but the manual management was not. Not to mention that and all my precious stuff was on one box which is well below ideal.

Taking the step

One day I decided it was time to move up in the world. Despite using docker and docker-compose for years I had never dived into kubernetes because it seemed difficult and complex. New syntax, new system with its own architecture and internal structure, new workflow, new everything. I decided to set up 3 VMs and played with default k8s as well as k3s on them. Dipping my toes into it was pretty fun and since nothing depended on it I was free to wipe it and start over. The same issue kept cropping up, having to manually install debian thrice and set up kubernetes just to delete all of it and redo it was again, less than ideal Initially I started writing a playbook before finding out that someone smarter had already gone down this path. Setting up k3s-ansible was easy so there was at least kubernetes setup automation but the whole thing was too thrown together there was no shared storage. However, I was becoming more familiar with kubernetes and learning the concepts as well as how to write yaml manifests.

Conclusion

At some point it became too bothersome, a lot of manual work was involved and basic stuff like copying over ssh keys was also done mostly manually so I had to move on. While all of this was going on, I had started planning what hardware updates I would get which you can read about here: Hardware Updates 2020

Ansible on inherent site

State of Homelab 2024

Router

Virtualization

Kubernetes

NAS

Hardware

Conclusion

Ansible, Terraform and Proxmox

Proxmox Setup with Ansible

Host

Repository

Template VM

VM Image

Template Script

Terraform Setup

Provider

Basic main.tf

Creating a test VM

Creating VMs for k3s

Conclusion

Reinstalling Gentoo

Why

How

Ansible

ZFS

Cruft removal

Conclusion

Homelab Current Form

Software choices

Making it store is harder than making it run

The problem

I really tried

glusterfs

longhorn

success

Conclusion

Homelab Evolved

What came before

Good enough is not good enough

Taking the step

Conclusion

Basic `main.tf`