Selfhosting

I’ve been selfhosting lots of services for years now. It’s been quite a journey to get here. Along the way I’ve taught myself many different aspects of hosting my own server.
Be it networking, (Linux) command line kung fu, low level hardware diagnostics or docker, just to name a few.

There are so many valuable resources online, that I decided to create a small post about the ones I found most useful. And since I’ve done a presentation on this subject for my colleagues, I thought this serves as a nice reference for the people who’ve attended.

I’ll break things down in a few key subjects, and provide links to resources for further reading,

• What is selfhosting?
• Hardware
  • Power consumption
  • ECC RAM
  • Hard disks
  • Powering on the server
• Infrastructure
• Software
  • Operating System
    • TrueNAS
    • UnRAID
    • Proxmox
  • Installation method
  • Security
    • Two factor authentication
    • Fail2Ban
    • Geoblocking
  • Some recommendations
    • Vaultwarden
    • Jellyfin
    • Apache Guacamole
    • Immich
    • Nextcloud
• More resources

1. What is selfhosting?

First off let’s settle what the term “selfhosting” actually means, and what it means within the context of this post.

Selfhosting is a term that was coined when cloud services started becoming main stream. It simply means to host your own services in a network that you control.
Within the context of this post, selfhosting is referred to as hosting services from your private home network. Served from a residential internet connection.
For example, running a server within you homes local LAN and being able to access those services outside of that LAN.

All the concepts discussed in this post can be achieved with consumer grade hardware (for example your ISP’s provided modem/router). And most of the tools that are used are FOSS (Free and open-source software), commercial but free for personal use or are very affordable.

Getting into selfhosting doesn’t necessarily require any subscriptions or paid services to get up and running. The biggest hurdle would be configuring everything and expanding your knowledge in order to do this properly.

2. Hardware

You can selfhost on almost anything. May it be a Raspberry Pi zero powered by an army of potato batteries. An old banged up laptop. A NAS build by one of the big brands or just a tower/rack you’ve build yourself.

Making the choice of hardware that suits the job, is an important first step of getting your server online.

2.1 Power consumption

Running a server 24/7 can cost a pretty penny. This can be a concern for places where energy has become more expensive in the recent past.

The obvious choice is to use a low power SBC (Single Board Computer, like a Raspberry Pi) to do the job. However, the available performance on such boards might be lacking for your use-case. Not to mention an ARM based platform has some limitations on what you’re able to run. There’s an excellent thread over on Tweakers where people dive deep into performant and energy efficient setups.

If you decide to do your build with standard PC components, consider using a PicoPSU Instead of a standard ATX power supply. A PicoPSU can be used by an external power supply (just like the ones you would use for a laptop, you can also use one of those old power bricks you have lying around). These tend to be more power efficient, especially when using lower amounts of power.

2.2 ECC RAM

ECC stands for Error Correction Code. Just like other types of fault tolerant memory/storage, ECC RAM uses parity to detect potential bit-flips inside the volatile memory and can correct/recover when data is corrupted.

Most consumer hardware doesn’t use ECC memory. And not all consumer platforms (the motherboard and CPU) support it. ECC used to be viewed as non essential for consumers, and only needed for mission critical corporate environments. Linus Torvalds has a different view on the matter though.

If ECC is a must have, and you don’t want to pay big bucks for enterprise hardware, you can opt for an AMD based platform. AMD unofficially supports ECC memory modules on consumer hardware. Intel only supports ECC on enterprise grade hardware.

Do note that DDR5 has some ECC capabilities on the RAM module. This is not the same as traditional ECC memory, and only ensures data integrity on the RAM module. The value of this is debatable, most problems arise when data is in transit (for example being transmitted through PCB motherboard traces).

And as a final consideration; the filesystem ZFS is heavily reliant on system memory. A lot of people argue ECC is a must when using ZFS.

There’s no clear answer on which to choose. It depends on your needs, requirements and budget. So go out and read on the subject to make an informed decision.

2.3 Hard disks

Although hard disk technology is quite old by now, they still have a place in your selfhosted server. Of course, SSDs are much faster and more energy efficient. But when your capacity requirements are in the Terabytes of data, it might be costly to completely abandon “spinning rust”.

When you decide to go with hard disks for (some of) your storage requirements, it’s important to note that not all of them are created equally.
Our main concern when picking drives, is to always avoid disks based on SMR (Shingled Magnetic Recording). I will not go in detail about SMR, but the TLDR of it is that they are slow and when used in RAID environments perform very badly. When a drive crashes, you’ll need to rebuild your RAID array and you want to happen this as quick as possible. While at risk of losing your data in this process.

Although not widely available, HAMR (Heat Assisted Magnetic Recording) and MAMR (Microwave Assisted Magnetic Recording) are expected to have the same downsides.
So when picking up drives for your new server, always pick CMR (Conventional Magnetic Recording) drives where possible.

If you want to minimize the power used by traditional hard drives, you could opt to have them “spin down” when they’re idle. This will put the disks in a standby mode; the RPM will ramp down in order to save power. Thus accessing data on the spun down disks will take longer since the disks need to spin up.
Some say that ramping up and down the RPM on the disks might cause more wear compared to just having them run at full speed continuously. I’ve never seen any factual evidence to this though, so take that at it’s face value.

Finally we can consider having a tiered storage architecture. Instead of writing data directly to hard disks, we can put a “cache” (for example a SSD) in front of it. This has some benefits, such as; decreased latency, increased throughput and avoiding/postponing spinning up hard drives.

2.4 Powering on the server

To conserve some power, you might choose to not have your server running 24/7. You might even want it to be turned off most of the time, and only have it run when you actually use your services.

Besides pressing the power button every time to access your server, there are other more convenient ways.

I’ve always found using WoL (Wake On LAN) the most convenient way of turning on the server. WoL is an ethernet standard that allows you to sent a specifically crafted packet to your device. When the device receives this packet (and is configured to allow this, mainly some BIOS/UEFI configuration is needed) it will turn on. This works from various power states, such as S3 (sleep) or S5 (soft off).
The side note here is that your ethernet device will always be “on” to some degree, to be able to listen for WoL packets. Therefore having a minimum impact on the power your system uses.

Some services have integrated WoL support (for example Guacamole). When accessing those services, it can be configured to first fire off a WoL packet.

Instead of using WoL, you could also choose to go for so called smartplugs. Smartplugs are hardware power sockets that have a network connection. With your smartphone you can simply turn the smartplug on or off.
You configure your server in such a way (see BIOS/UEFI settings) that it will power on when the power is restored.

The downside of choosing smartplugs is that you have another (iOT) device in your network. Additionally the integration offered by some services with WoL would be lacking when using smartplugs. And of course those plugs themselves are using some amount of power.

3. Infrastructure

Exposing your server online comes with quite extensive risks.
If you’re not sure on how to secure things properly, go for the safer choices. Accessing your network through a VPN might be your best option (and I do not mean the worthless VPN services you can get for a monthly subscription, host your own!). Tailscale is a very easy and convenient way to setup a VPN network and is worth looking into. Of course trusty OpenVPN or Wireguard are also up for the task.

If you do decide to directly expose your server online, take into consideration that without any extra effort, people could find your home IP address through DNS records.

There are ways of setting up a public domain that is not directly linked to your home IP.
Cloudflare has awesome free tools to do just that.
Below you find a basic introduction to setting up Cloudflare tunnels, which in my opinion is currently the best way to achieve this;

4. Software

The software is what makes the whole thing go “vroom”! In this chapter we will look into different aspects of software that we can run, Look at the down- and up-sides of each, and just have a general overview of little gems that can be found out there.

4.1 Operating System

The first thing we would need to decide on is which OS (Operating System) we would like to use. Since we are running a server/NAS system, I have picked three popular OS´s specifically designed for this use-case. Of course no one will hold you back from running it all on plain old Windows, a vanilla Linux distribution or a proprietary NAS operating system.

4.1.1 TrueNAS

Originally started under a different name; FreeNAS. The project has been around for about 2 decades now and has become quite mature.
The OS comes in multiple flavors, including TrueNAS Scale and TrueNAS Core that are suitable for home use. The main difference between these two is that Scale is based on Linux (Debian) and Core is based on FreeBSD. There’s much more to write about the differences between these two. There are many resources online for you to read up on. So more research is required to pick the right one.

TrueNAS uses ZFS as it´s filesystem of choice. ZFS is very reliable but can be a bit daunting for a beginner.

TrueNAS is very flexible and stable but has a somewhat steep learning curve. Be prepared to go down a rabbit hole should you decide to use TrueNAS as your OS.

I would consider TrueNAS for the more advanced users. When you do not have any experience with Linux/FreeBSD and troubleshooting those systems (which often require magic in the terminal), or ZFS is completely new to you, this might not be the right choice.

All the versions of TrueNAS are free for personal use.

4.1.2 UnRAID

UnRAID is the more beginner friendly choice of OS. It’s quite easy to set up and the community around it is large and active.

The main benefit over TrueNAS is it’s flexibility in storage. Having a bunch of mismatched disks in an array is no problem on UnRAID (in contrast to TrueNAS where this would have some downsides). Expanding the array with more disks after the array has been created, is more flexible (depending on the setup) compared to TrueNAS.

Using docker on UnRAID is a breeze although the simplicity can be limiting. The built-in VM capabilities are sufficient for most home users.

All in all, depending on your level of experience, this might be a good choice when diving into selfhosting for the very first time.

Unlike TrueNAS, UnRAID is paid.

4.1.3 Proxmox VE

The final OS of choice is Proxmox VE. This OS is quite different than the other two, since it’s main focus is virtualization (the V in VE).

If your goal is to run virtual machines, this OS might be the right choice. Proxmox is a type 1 Hypervisor. This means Proxmox runs directly on the hardware like an OS would. This has performance benefits over type 2 Hypervisors, which would run on top of an OS. An example of a type 2 Hypervisor is installing VirtualBox inside an operating system.

It’s possible and not uncommon to combine both Proxmox and run for example TrueNAS inside a VM. With this setup you can have best of both worlds.

4.2 Installation method

So we have chosen our OS and we are ready to start setting up some software. The question is how we are going to install them?

Below you find a couple of choices. All of them are valid depending on your use case, but each have their own strengths.

• Bare metal
  This is a reference to installing software the traditional way; straight on the OS. This has the least amount of overhead and might be the right choice if performance is absolutely critical.
  However it has clear downsides. For example managing dependencies and the different versions required for different services you want to run can be challenging.
  Furthermore it’s not as portable or easily backed up as the other choices.
  
• Virtual machine(s)
  Most of you will have heard of VMs, they’re around for decades now.
  A VM is nicely isolated from the host it runs on. This can provide security boundaries for parts of your software stack. One of the downsides is the additional overhead it brings.
  
• Containers
  A container is, from my point of view, a middle ground between the choices above. Since it’s not completely virtualized, the performance impact of running containers are a lot smaller compared to VMs.
  In contrast to installing everything on bare metal, containers are more easily backed up (or at least you can orchestrate where everything for your services is stored easier) and easier to port to a new system.

4.3 Security

There are a lot of things to consider when it comes to the security aspects of your setup. In this chapter I will focus on some of the software tools you can use to harden your security.

4.3.1 Two factor authentication

When you are running all kinds of services, they all have their own authentication implementation (if you’re lucky, some have none whatsoever). Some implementations can be better than others, but what most of them will lack is two factor authentication (2FA). There are a few open source projects that have jumped into this gap. Such as Authelia, Keycloak and Authentik,

If you use a reverse proxy such as Traefik to expose your services, it’s quite easy to hook these 2FA implementations in front of those services. When accessing a service normally just protected by a login, it will first have you authenticate on the authentication service before letting you access the requested service.

It’s even possible to set up Single Sign On (SSO) through these authentication services. You can use protocols such as OpenID Connect (OIDC) to authenticate in the services that provides integration with OIDC. This can simplify your authentication flow.

Instead of using Time-based one-time passwords (TOTP), you can use a service like Duo Security to push 2FA requests to the associated app. Duo Security can be used freely with limitations; you’ll get a certain amount of credits to use for phone/sms based authentication. The use of push notifications through the app is free.

4.3.2 Fail2Ban or CrowdSec

Fail2Ban is a piece of software that can block IPs based on log entries of your services. You can have it detect failed login attempts for a certain service, by pointing Fail2Ban to the logfile and having it look for a pattern/regex.
If the block criteria have been met, Fail2Ban will block the matched IP on firewall level. This can be a great tool to protect against brute force attempts.

CrowdSec is a similair open source project but uses the community to strengthen it’s capabilities.

4.3.3 Geoblocking

When you expose your services online, do you really need to expose it to the entire internet? Maybe we can restrict access based on location? Yup this is entirely possible. Depending on your setup there are ways to block entire countries from accessing any of your services by using the IP address.

If you use Cloudflare for example, you can specify countries to block. If you run your own reverse proxy such as Traefik than there are plugins/middleware to do the same.

4.4 Some recommendations

Doing this for a while now, there are some tools I appreciate a lot, and others less so. There are tools that are built very well but are part of the infrastructure and therefore not in the spotlight (looking at you Traefik and Authelia),

Not considering those, I want to give you my top 5 at this moment (it’s always changing :)).

4.4.1 Vaultwarden

A password manager written in rust which is perfect for selfhosted environments, It’s compatible with the Bitwarden API and can use the associated apps and plugins.

4.4.2 Jellyfin

A very powerful media server. It can stream all your media files through the web or compatible mobile applications. It supports transcoding, is completely free and just works!

4.4.3 Apache Guacamole

If you’d like to access the computers in your home network than Guacamole is made for you.
It’s a Remote Desktop/Shell portal that let’s you access any PC through popular protocols such a as SSH, RDP and VNC. It can even send a WoL packet before trying to attempt a connection.

4.4.4 Immich

I’m a strong believer that your photos shouldn’t end up in the cloud somewhere and that you need to take care of your own storage.

Immich is a very capable frontend to provide access to this storage. It has awesome features such as facial recognition built-in. It is under very active development however and the motto “Move fast and break things” seems to apply.

4.4.5 Nextcloud

Of course a home server is not complete without cloud storage. Nextcloud is just that, and even more. It can replace your Dropbox or OneDrive for example. It also has chat and video meeting capabilities. It supports end to end encryption and lots more features you will need out of a collaboration tool.

5. More resources

As you might expect by now I could go on and on about this subject, but at some point this post has to end. So to sent you off, I’d like to finish with some very valuable resources for selfhosting out there.