Are there any viable ARM Servers available in a homelab price range? Every time I look I only see the 2000$ enterprise 2U servers. I have outgrown the Raspberry Pis but still love the idea of moving my homelab to ARM
The fastest ARM-based computers that are not more expensive than better Intel or AMD based computers, which means that a complete fully-equipped computer with DRAM and SSD should not exceed the $200 to $250 range (where computers with Intel N100 or with older Zen 3 Ryzen 5 mobile CPUs can be found), are the computers with CPUs having Cortex-A76 cores (like the smartphones from 2019).
In this class, there are many models with RK3588, having e.g. dual 2.5 Gb/s Ethernet ports (with the possibility of attaching more Ethernet NICs on USB 3 or on PCIe M.2 adapters) and supporting PCIe 3 x4 M.2 SSDs and/or eMMC (and the attachment of more SSDs on USB). (The model that I like most is NanoPC-T6, which exploits best all the interfaces of RK3588, but without adding things that should better be added externally, only when they are needed, like the additional USB hub present in many other models.)
A cheaper option, but with much slower peripheral interfaces, is the new Raspberry Pi 5 model.
Nevertheless, a homelab server with an ARM CPU makes sense only for developing Aarch64 applications.
For just doing the job there are many small and cheap fanless computers with Intel N100 (4 E-cores) and 4 to 8 2.5 Gb/s Ethernet ports (typically sold on Amazon as firewall appliances). For only a few dollars extra it is possible to find much faster cheap small computers (from companies like Minisforum or Beelink) with older AMD Zen 3 CPUs, like the 6-core Ryzen 5 5600H.
For a higher price of $500 to $600, there are small computers with AMD Ryzen 9 7940HS, which can support e.g. dual M.2 PCIe 4 x4 SSDs and SATA SSDs, and dual 10 Gb/s Ethernet NICs (on Thunderbolt), dual 2.5 Gb/s ports on the MB + many other 2.5 Gb/s ports on USB, while being faster than big and expensive servers from some years ago.
NanoPC-T6 looks great. One of my concerns is about the software that the company is providing. For example, how can I trust that the OS images of this company don't contain any spyware?
The software cannot be trusted, but it is easy to replace everything but the Linux kernel with another Linux distribution, including with one compiled from sources (like Gentoo). This should be doable just by following the generic installation instructions for Aarch64 of that distribution.
The recompilation of the Linux kernel may be more difficult, because the right configuration file and modules must be selected before doing it, but it should be possible as most support for RK3588 is included in the mainstream kernel. Also U-Boot (the boot loader that loads the Linux kernel) should be recompilable from sources.
The hardware is more trustworthy than that of Intel or AMD computers, because it comes with the complete schematics and the technical reference for RK3588 is much more complete than for any Intel or AMD CPU.
A hardware backdoor could have been implemented only in the Ethernet interface of RK3588, but that is not used in NanoPC-T6, which uses Realtek Ethernet interfaces on PCIe lanes. Any hardware backdoor in those would have required a close and secret cooperation between a major Taiwanese company and a major mainland Chinese company, which is unbelievable.
I'm a noob at this, but at a first glance it looks difficult to get all the peripherals and hardware acceleration working when compiling from sources. Anyway, I am really keen on getting one.
The hardware acceleration that is problematic is that of the Arm Mali GPU and of the video codecs.
This is normally provided in all Arm SoCs by binary blobs. Nevertheless, at least for the Mali GPU there is a reverse-engineered driver in the Linux kernel, which might be usable with RK3588.
In any case, this hardware acceleration is the same in all RK3588 boards, regardless of the vendor, it is not specific to NanoPC-T6.
> how can I trust that the OS images of this company don't contain any spyware?
Software from board manufacturers shouldn't be used anyway, if not because in a few years it is often discontinued and not updated anymore because they're pushing newer models. Thankfully we have Armbian and Dietpi which are the distros of choice for all boards that don't run major PC oriented distros (and a nice alternative for those that do).
The number of boards supported by these two distros is astonishing:
Intel had in development a "rugged NUC" with the Atom-branded equivalent of the Intel N100 CPU, which supported ECC and which was expected to have a low price.
Unfortunately, I assume that this product has been canceled a few months ago, when Intel sold their NUC business to ASUS.
There are a few small computers that support ECC and which use obsolete Intel Tiger Lake or Tremont-core-based Intel Elkhart Lake CPUs, but those CPUs are a dead end, being slow and supporting instruction sets that are different from the current mainline Intel CPUs, so I would not recommend any of them.
The best remaining choice depends on which is more important, the size and the power consumption or the price of the server.
For very small size and low power consumption I am not aware of any good solution at a reasonable price, because even when some of the Arm CPU SoCs or Intel or AMD mobile CPUs support ECC, I have not seen any such computer board that includes the ECC support. There are some industrial computers with ECC, but those are expensive for what they offer.
If only the cost is the problem, and second-hand servers are avoided because the server to be bought is intended to be used for many years, then a server with desktop Intel or AMD CPUs must be used. The MBs with the Intel W480 chipset are expensive, so the cheapest solution is to use one of the AM5 MBs that specify ECC memory support, e.g. from ASUS or ASRock Rack, together with one of the cheaper Ryzen 7000.
Another option is an older AM4 MB, like the Mini-ITX ASRock Rack X570D4I-2T ($400 due to including dual 10 Gb/s Ethernet ports), which has the advantage of using cheaper older Ryzen 5000 CPUs, with cheaper DDR4 ECC memory, so the total system cost would be reasonable.
The only disadvantage of the desktop Ryzen CPUs when used as servers is that, even if they have excellent energy efficiency when they are actually running programs, they have a relatively high idle power consumption, because only the cores are shut down when doing nothing, while the I/O die has a permanent consumption around 20 W or more. Therefore one must choose between the low idle power consumption of a few watts of the laptop CPUs and the ECC memory support of the desktop CPUs.
Because in my home lab most servers alternate between times when they are used intensively with times when they stay idle for hours or days, except for one server that is connected permanently to the Internet, all the others are used with Wake-on-LAN, so they are shut down when idle, for negligible power consumption.
Rk3588-based are gaining traction. Upstream support for uboot and linux has been improving rapidly.
I bought orange pi 5+ to replace my x86-based odroid-h2+. Will start the installation in the next few weeks. I have seen enough anecdotes people running NixOS on this chip, so feel pretty optimistic about it.
Why? Cost, performance, and power consumption should be in the same ballpark as x86, but support for x86 is still significantly better. There's nothing magical about running a workload on ARM. The math obviously changes when you have thousands of machines.
How solid does it run? Is it able to make use of all what is currently making the Mac Studio so interesting for LLMs? And does it handle all the power management as it should? These are honest questions, as I'm in the need of a new workstation and eying a Mac Studio even if I have never used any Apple products before (with the exception for the iPods).
You can also just run macOS and use Tart, Docker, or UDM and run VMs/containers for your services. Asahi isn't bad either, but for some people it's a bit more hassle to set it up and maintain versus macOS.
