In the past I wrote about how I setup my home-server but it became near unmaintainable. I had multiple issues with raid assembling wrong and nearly breaking all my large data and many configurations had issues. Most of this (more like everything) is 100% my fault but I don't have time all day fixing my home-server every other day.
So I wanted to fix it once and for all and well... 2025 and multi billion euro heavy company's happened and now hardware, even used, is unaffordable. So no used server hardware for me (For now, maybe that sh*t finally crashes and hardware is cheaper again, hopefully).
I looked into how I can improve the situation the most, and first was just cleaning, adding SATA and cable management and other basic stuff. After that I knew I need a new OS and setup else the Container would keep on breaking. I decided on Proxmox, open source, payed but free in the community version. Normal consumer can not afford the subscription version because primary business software aka. expensive af.
From here most went well. I installed Proxmox, checked out how everything worked, put my 4 SSD's in a zfs zRAID1 (not storage) then added it as folder and created many MANY Directory Mappings, fixed permission so unprivileged containers can work with them, then added them to my LXC by mapping them in the config file directly and finally added the Directory Mappings to my VM's and mount them with fstab on boot. Here I needed to re assign owner in the VM to the non-root default VM-Server-User. After checking I can see changes on both ends I used docker and Portainer to setup everything I needed. I even could reuse most of my old docker-compose files. I tried to install Nextcloud-AIO again and again saw why I hate Nextcloud often so much. This AIO thing is so bad it is painful. Only works with HTTPS which in a Home-setup is NOT NEEDED and just causes issues, hell it causes issues even in business environments. Enough ranting.
I installed stuff like pi-hole directly in a LXC container for less overhead and easy SEPARATE management of it to prevent DNS issues once and for real. Some stuff is only really usable with docker which is not recommended in LXC but for simple stuff like Nginx Proxy Manager it works like a VM. Create an Ubuntu or Debian container, update, install docker, install with compose the needed app and only the app. Done. Keep it simple with CORE utils like DNS or reverse Proxy, trust me.
ZFS has quite a big overhead and sending files over FTP to some folders can really use A LOT OF RAM. And I mean a lot. Like 24 GB just for 200 GB backup files. The system slows down the IO-Speed to keep the VM from overflowing the ZFS stuff but wow.
Only anoying thing is updating. On a basic Ubuntu-Server a simple batch script can jump around and execute all update commands one after the other. With this setup I would need to find a way to enter VM's and LXC containers and update them without losing context or script flow and state, exit them and even somehow use API's of Portainer to trigger re-pulls. AKA. Auto-Update is not easily possible with Proxmox. I saw setups online with services all over the place and jobs timed everywhere and so on. Just update everything once a month and be happy it dose not break for no reason after you looked at it funny.
Is this ideal or the best setup? No! The virtiofs use with VM's and folder mounting in Container has a good amount of overhead, how stuff is run, if in a VM or LXC container could have been better selected, scripts to manage everything and even the docker-compose's them self could have been cleaned up, but it works. Everything is done clean, with in spec of Proxmox and there recommendations, to some extend, no out of spec jank and all monitor-able and backup-able. I like it far more then the old jank setup with Ubuntu-Server. I might not be a fan of the way some projects like Ubuntu-Server, Docker, Nextcloud or Proxmox are run but all I use is OpenSource, dose not phone anywhere I don't tell it to and works how I want it to. And also: Having EXTREMELY EASY AND AUTOMATIC separate IP addresses for different types of container and VM's I made is brilliant. Not even the need to static IP them. Tell the router to give the MAC-Address a static IP and the Container or VM will use it no questions asked!