I just spent a good chunk of today migrating some services onto new docker containers in Proxmox LXCs.
As I was updating my network diagram, I was struck by just how many services, hosts, and LXCs I’m running, so counted everything up.
- 116 docker containers
- Running on 25 docker hosts
- 50 are the same on each docker host - Watchtower and Portainer agent
- 38 Proxmox LXCs (19 are docker hosts)
- 8 physical servers
- 7 VLANs
- 5 SSIDs
- 2 NASes
So, it got me wondering about the size of other people’s homelabs. What are your stats?
You must log in or register to comment.
Dude, are you living in your company’s server room?
Lol - not quite. It sounds like a lot, but all of this runs on a couple of HP DL360s, a handful of Raspberry Pis, a nettop box, and a couple of consumer NASes.
“i swear it’s not a lot”
Goes on the describe an infrastructure setup comparable to most medium sized businesses
I love this community!
Well, to be fair, I do use my homelab to play with stuff I may or may not want to use at work. I don’t need PEAP auth for wireless, with a separate RADIUS server and Postgres database. But I have it. 😉
How do people get to so many Docker containers before moving to Kubernetes? I only have 76 containers across 68 pods and that’s far too much for me to manage in Docker.
Honestly, anything not mission critical (network/internet and home automation, mainly) gets auto-updated by Watchtower. I have Watchtower set to pull latest images of everything on a weekly basis, and specific containers that are set to monitor only. Every Saturday morning, I check the Slack channel for notifications of containers that need controlled updating.
Not really doing much docker, but a lot of LXC - everything scripted with ansible. I define basic container metadata in a yaml parsed by a custom inventory plugin - and that is sufficient for deploying a container before doing provisioning in it.
- 8 Hosts (6 physical/local, 2 VPS/remote)
- 72 Docker containers
- Pi-hole (3 of them, 2 local, 1 on a VPS)
- Orbital-sync (keeps the pi-holes synced up)
- Searxng (search engine)
- Kutt (URL shortener)
- LenPaste (Pastebin-like)
- Ladder (paywall bypass)
- Squoosh (Image converter, runs fully in browser but I like hosting it anyway)
- Paperless-ng (Document management)
- CryptPad (Secure E2EE office colaboration)
- Immich (Google Photos replacement)
- Audiobookplayer (Audiobook player)
- Calibre (Ebook management)
- NextCloud (Don’t honestly use this one much these days)
- VaultWarden (Password/2FA/PassKey management)
- Memos (Like Google Keep)
- typehere (A simple scratchpad that stores in browser memory)
- librechat (Kind of like chatgpt except self-hosted and able to use your own models/api keys)
- Stable Diffusion (AI image generator)
- JellyFin (Video streaming)
- Matrix (E2EE Secure Chat provider)
- IRC (oldschool chat service)
- FireFlyIII (finance management)
- ActualBudget (another finance thing)
- TimeTagger (Time tracking/invoicing)
- Firefox Sync (Use my own server to handle syncing between browsers)
- LibreSpeed (A few instances, to speed testing my connection to the servers)
- Probably others I can’t think of right now
Most of these I use at least regularly, quite a few I use constantly.
I can’t imagine living without Searxng, VaultWarden, Immich, JellyFin, and CryptPad.
I also wouldn’t want to go back to using the free ad-supported services out there for things like memos, kutt, and lenpaste.
Also librechat I think is underappreciated. Even just using it for GPT with an api key is infinitely better for your privacy than using the free chatgpt service that collects/owns all your data.
But it’s also great for using gpt4 to generate an image prompt, sending it through a prompt refiner, and then sending it to Stable Diffusion to generate an image, all via a single self-hosted interface.
Audiobookplayer
Which program do you mean with this? I’m only familiar eith audiobookshelf
Ah yeah that’s the one, sorry
A single SFF desktop setup in a Node306. 2700x, 32 GB RAM, Arc A380, some WD reds.
- Homeassistant & associated packages for esphome and Zwave stuff
- Jellyfin
- *arr suite + transmission
- yacht
- uptimekuma
- paperless
- immich
- authelia with OIDC SSO for containers where possible
- traefik for reverse proxy
- Nexcloud
- valheim server
- boinc in the winter
- syncthing for phone sync
- more services for keeping up the others
Soon a pihole to come.
I want to expand my smart home setup. My project this spring is integrating my smart gas and electric meters into homeassistant. We are completely stripping the house so I am wiring up everything with KNX with a nee Zwave devices where needed. Greatly expanding the smartish home.
I also have to set up a proper network. Right now I am using my Proximus Internet Box from the ISP which admittedly is pretty customizable.
Love this! Lot of similarity to what I use - Authelia’s awesome, especially paired with a free push 2FA like Duo.
- *art suite + transmission
*arr suite?
It’s not much, but I’ve got a little LG netbook with an Atom CPU and 2GB RAM running Pi-hole and Syncthing.
My starting point (with this incarnation of my homelab) was my Asrock ION330 nettop box. Then I discovered Raspberry Pis. Then I decided I needed a couple of HP DL360s. RIP my power bill.
One day when I’m all growed up I want to have a better setup. For now I’ve got what I absolutely need.
Yep - fair enough. Admittedly, my homelab is as much for professional development as it is home use, but pretty much everything gets used all the time.
How many W are you pulling, on the average? Or kWh per year.
For reference: Using dual E5-2630L, DL360/380G8 uses around 130-150 watts average unless something is spiking.
With a couple Cisco routers, 4 HP server, adds about 150 dollars to my monthly bill. This wouldn’t be possible in Europe.
My current supplier rate is about 0.6 EUR/kWh. I make some 1/2 to 2/3 of my power myself, for a price that’s less than half of that.
make some 1/2 to 2/3 of my power myself I’d have to :) That’s .66c US per. Mine is .11-12 US / .10 EUR. Mine is 6 times cheaper. `Merica
Insert rant about our power is probably a large percentage of coal and gas (cheap + super bad)
Good question. According to my UPS, I’m pulling about 173Wh for everything except my pair of HP DL360s. Those each have a couple of 480W PSUs in them, but they’re nowhere near running at full tilt, so I can’t be sure. I really should get some power measurement going…
You’re probably drawing about 400-450 W.
Yeah, seems about right. I’m planning on buying a 32RU rack in the new year - will fit it out with power monitoring PDUs while I’m at it.
- 3 DL360G8 Esxi (86Ghz/512GB RAM)
- 1 DL380G8 TrueNAS
- 1 DL360G7 Veeam
- Dell n5070 Extended PVE SophosnUTM
- 48 Port Catalyst rack switch
- Cisco 2921
- Fibre Channel / iSCSI
50+ VMs and containers:
- VMware ESXi, vCenter, VMware Log Insight, VMware OPS
- DMVPN to remote locations like a desk switch at work and family member houses
- Sophos UTM
- Active Directory for my home computers
- hybrid sync to MS Entra (Azure Active Directory) with Entra Connect
- hybrid Exchange on Premise and Exchange online
- Active Directory for management network
- Security Onion VMs for IDS
- Network monitoring like Elastiflow, PRTG
- Docker, gitlab, OpenSalt / Saltstack
- Trellix ePO for AV
- Nessus vuln scanners
- Team Awareness Kit (TAK) server
- Active Directory Certificate Services
- Home media applications
These things are mostly to maintain familiarity and documentation development. I write off the cost of electricity as continuing education and professional development. More enterprise than some enterprises.
Love it! I’m gonna grab a 32RU rack soon. Got most of my stuff in a small ~14RU wall cabinet right now. I was originally aiming for low power everything - RasPis, etc. But I’ve since bought a couple DL360s, and you just can’t beat the sheer grunt factor, especially when paired with Proxmox.
I have a very modest 7 docker containers on a vm on my gaming rig and I have a raspberry pi for my DNS server. Honestly my setup is quite scuffed (in comparison to yours), but it does what I need it to do
Dang, how does your isp feel about that many machines talking out to the internet, have they made you pay for business plans yet?
Lol - I’m on unlimited 1Gbps fibre here. So far, they haven’t raised any concerns.
That’s awesome, best of luck it stays that way!
What are you running in docker
There’s a lot not worth mentioning, but broadly…
- Home automation
- Home Assistant
- esphome
- Node-RED
- MQTT
- Frigate
- Homelab/management
- 2 x Pi-hole (plus supporting services - Cloudflare tunnel, for example)
- Grafana
- Prometheus
- Shellinabox
- Forgejo (git)
- Netbox
- VScode
- Media/entertainment
- 2 x Sonarr
- 3 x Radarr
- Calibre
- Piped
- Minecraft
- other supporting *arrs
- Data
- Paperless-ngx
- Immich
- Social
- Lemmy
- Mastodon
That’s a lot. Impressive
- Home automation
I don’t have a homelab ( space contrains ) but I do have 2 vps that I use to host in total 13 docker containers, mail server and an xmpp server.
Edit: My lemmy server is also hosted on them.
What I’m more interesting in is what is it that you selfhost to have so many docker containers?
Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I’ve seen in this thread:
Fewer Letters More Letters AP WiFi Access Point DNS Domain Name Service/System ESXi VMWare virtual machine hypervisor Git Popular version control system, primarily for code HTTP Hypertext Transfer Protocol, the Web LVM (Linux) Logical Volume Manager for filesystem mapping LXC Linux Containers MQTT Message Queue Telemetry Transport point-to-point networking NAS Network-Attached Storage NUC Next Unit of Computing brand of Intel small computers PSU Power Supply Unit PiHole Network-wide ad-blocker (DNS sinkhole) Plex Brand of media server package PoE Power over Ethernet RAID Redundant Array of Independent Disks for mass storage SSO Single Sign-On Unifi Ubiquiti WiFi hardware brand VPN Virtual Private Network VPS Virtual Private Server (opposed to shared hosting) ZFS Solaris/Linux filesystem focusing on data integrity nginx Popular HTTP server
20 acronyms in this thread; the most compressed thread commented on today has 3 acronyms.
[Thread #370 for this sub, first seen 24th Dec 2023, 07:35] [FAQ] [Full list] [Contact] [Source code]
Mine’s pretty moderate in comparison to yours lol
- 2 cloud VPSes
- 2 physical locations
- 4 physical servers
- ~20-30 docker containers across the servers
- 3 VMs
- 3 managed switches
- 5 VLANs (2 with internet access)
- 2 SSIDs
2 Raspberry Pi 4 with a few services running (some directly, some via docker): pihole, pialert, gitlab plantuml, munin, restic rest server, jupyter instance, airsonic-advanced. And an old synology NAS which serves as document and media server
I’m able to get a lot of gear secondhand through my job, so I’ve got:
One 2u Intel server running proxmox in a ‘cluster’ (circa 2013ish. Added RAM and upgraded the CPU/storage.)
One Intel nuc with an i7-7th gen as the other host in the cluster - only one VM is set to fail over between the two if needed.
VMs:
- Plex
- 2x PiHoles (one of these is the failover VM) (these also have a few docker containers like Uptime Kuma.)
- Windows arr box (I know it’s blasphemy but I felt more comfortable doing that stuff in windows)
- anything else I want to mess with because the server really doesn’t run that hard.
Network:
- Sonicwall TZ 300 (incl a perpetual VPN license)
- Unifi 24 port switch (it’s gigabit and POE but doesn’t output enough power for the…)
- single Unifi AP.
All acquired over the last couple years for the low low price of “it was going into the trash anyway”
Nice! There’s nothing better than finding new life for old tech.
