Building a Raspberry Pi Compute Module 4 NAS

May 13, 2022 at 04:17

A few months ago, my venerable NETGEAR ReadyNAS NV+ died. Not sure what happened; it just shut off one day and refused to turn back on again.

For a while now, I’ve wanted to build my own NAS. I don’t like proprietary hardware that depends on a manufacturer for OS updates. But I also didn’t want to build a large, unsightly server, and burn more electricity on something that would be running 24/7. Granted, my NAS uses spinning-rust hard disk drives, which likely draw much more power than most low-power mainboards and CPUs, so maybe the power consumption concern is a bit overblown.

The 4th generation of the Raspberry Pi hardware has a single PCIe lane. In the retail model, this is wired to a USB3 chip. The Compute Module 4, however, allows board designers to do whatever they want with it. I’d hoped for a while that someone would build a board with at least four SATA ports. I was initially excited about a group called Wire Trustee, but they abandoned their hardware plans to instead build a VPN.

Then came Axzez, with their Interceptor carrier board, which was exactly what I was looking for, with five SATA ports.

Performance-wise, there are certainly better SBCs for this task. But I’m comfortable with the Raspberry Pi, and see it as a stable, long-term platform to build on.

What follows is a simple description of my build, and how it worked out.

Parts List

  • Axzez Interceptor carrier board
  • Axzez Interceptor board adapter (adapts the board for a mini-ITX form factor)
  • CM4 module (I ended up with a 4GB RAM/16GB eMMC/WiFi module, as that was all I could find available without having to wait the better part of a year for it to ship; ideally I wanted an 8GB RAM model, and didn’t care if it had WiFi or not)
  • Fractal Design Node 304 chassis
  • Seasonic SSP-300SFG power supply (SFX form factor)
  • Mains cable for the power supply (Seasonic doesn’t bundle one)
  • 4-pin Molex to dual 15-pin SATA power cable (if you have more than three hard drives)
  • SATA data cables (one for each drive)
  • CR2032 battery (the Interceptor board has a battery-backed clock)
  • Hard drives (I already had these from my previous NAS)

Verification and OS Setup

To start, I wanted to ensure the parts actually worked. I downloaded the latest 64-bit Raspberry Pi OS (minimal) image, and wrote it to a USB flash drive. Axzez provides their own OS image, which includes a patched kernel with support for the included 4-port ethernet switch. I don’t need this, so I preferred to stick with a more stock OS.

I mounted the boot partition, and configured things for headless operation, including connecting to a WiFi network (I wasn’t near an ethernet cable while doing this), enabling SSH, and creating a default user. I also added enable_uart=1 to /boot/config.txt so I can hook up the serial console; this could be useful if the board doesn’t boot for some reason.

I pulled the cover off the chassis, unpacked the power supply, and seated the CM4 module into the carrier board. I plugged the ATX power connector into the board, attached the power LED and power switch wires from the chassis to the board, and plugged the USB flash drive into one of the board’s USB ports. I plugged in the power supply and flipped the switch.

Everything turned on by itself, with the power LED glowing blue. I waited a few minutes, and tried to ssh to raspberrypi.local, surprised when I was greeted with a password prompt. Signing in presented me with a pretty standard-looking Raspberry Pi OS system.

Booting from USB is fine, but I wanted the OS on the internal eMMC storage. So I copied the OS image over from my laptop, and then used dd to write it to /dev/mmcblk0. I repeated the same steps as before to enable headless operation. In addition, since I’m not using Axzez’s OS image, there are a few settings I needed to add to /boot/config.txt:

1enable_uart=1
2dtparam=i2c_vc=on
3dtoverlay=i2c-rtc,rv3028,i2c0,addr=0x52

The first isn’t required, but could be useful. The second enables the VideoCore’s I2C bus (not sure if this is required, but the Axzez OS does this). The third loads an overlay that sets up the RTC (real-time clock) chip included on the board.

After that, I power-cycled after pulling the USB flash drive. Again I waited, and ssh’ed into the board. A quick grep through dmesg confirmed that the RTC setup worked:

1rtc-rv3028 0-0052: registered as rtc0
2rtc-rv3028 0-0052: hctosys: unable to read the hardware clock

(The error is because the clock has never been written to; after the next clean reboot the clock should have been initialized with the time at shutdown.)

While I was here, with everything disassembled and easily accessible, I figured it’d be a good idea to update the OS and install most of the software I’ll need later. So:

1sudo apt update
2sudo apt dist-upgrade
3sudo apt install neovim mdadm lvm2 smartmontools ifplugd nfs-kernel-server
4sudo apt purge vim-tiny

In my case that didn’t upgrade too much, as the OS image I had downloaded was fairly recent. But I rebooted at this point, just to ensure things still come up properly after installing new packages.

I use neovim as my primary text editor, so it made sense to install it here. mdadm and lvm2 are needed for managing my hard disk arrays. smartmontools monitors disk drive self-reported health stats. ifplugd is a system daemon that automatically configures wired network interfaces when ethernet cables are plugged in (or unplugged). nfs-kernel-server provides an NFS server, which is how I primarily access files on my NAS across the network. You might also want to install samba (another file sharing protocol used by Windows computers).

I also modified /etc/hostname to set the hostname I wanted (and changed the raspberrypi entry in /etc/hosts), and added an IP reservation in my router for the ethernet port’s MAC address to ensure it always gets the same IP address (more or less required for NFS mounts if I don’t want it to be a pain).

This is also a good time to run dpkg-reconfigure tzdata to set the desired time zone, and dpkg-reconfigure locales to set the correct locale.

At this point I noticed that occasionally the board wouldn’t reboot cleanly (via sudo reboot). Sometimes (I discovered this by watching output on the serial console), the kernel would panic on reboot, requring me to power-cycle the power supply to continue. Annoying, but not a showstopper.

Hardware Assembly

Now that I’d verified everything works, it was time to put the hardware together. The manual supplied with the chassis recommended removing the disk brackets before starting, and then installing the mainboard first, followed by the power supply, and then finally the disks.

Mainboard

The mainboard was relatively straighforward to install. First screw the Interceptor board onto the board adapter, with the ports flush against the side of the adapter. Then attach the board adapter to the chassis itself, after screwing the four metal standoffs into the chassis. Orient the board adapter so the ports face out the back of the chassis. Don’t attach the board adapter to the chassis first, or it will be really difficult to screw the board into the adapter later.

If you disconnected them, now’s a good time to reconnect the wires for the power switch and power LED.

Also insert the coin-sized battery into the mainboard.

Power Supply

Next comes the power supply. The power supply I chose came with an adapter bracket to adapt the SFX form factor to a case accepting an ATX power supply. I chose SFX because they are often smaller and quieter. After attaching the bracket, mount the power supply in the chassis, with the power supply’s fan facing downward toward the opening in the bottom of the chassis.

The chassis has a power cable extension that you can now plug into the power supply.

From here you can connect the 24-pin ATX power connector to the mainboard.

Now that they won’t get in the way, you can connect the three chassis fans to the fan connectors on the mainboard.

Attach the 4-pin Molex to dual SATA power adapter to the appropriate power supply lead. This power supply only gives us three SATA power connectors; if you have four or five drives, you’ll need the extra adapter.

Drives

Finally it’s time to connect the hard drives. The chassis will hold up to six drives, though the mainboard only has ports for five of them. I only had four drives at the time of my build, so I plugged four SATA data cables into the first four SATA ports on the mainboard.

The drives themselves were fairly straighforward to attach to the drive rails. I had to move one of the rubber grommets to one of the extra holes. I was only able to line up three screw holes on the drives I have, but that’s fine.

After placing the drive rails back into the chassis, I routed power and data cables to each, and tried to bundle the excess cable as well as I could.

Wrapping it Up

Before closing up the chassis, I plugged in power and ethernet, and flipped the power switch on the side of the power supply. To my relief, everything came up properly, as expected, including all three fans.

I used mdadm and vgchange to manually set up and mount my drive array to ensure everything was working properly (later I’d write a script to do it on boot).

I deleted /etc/wpa_supplicant/wpa_supplicant.conf to disable WiFi, since ethernet was working fine.

Finally, I closed up the chassis and replaced the thumb screws.

Benchmarking

I feel like I’d be somewhat remiss if I didn’t post some performance numbers. Take this with many many many grains of salt. There are some oddities in my setup: my RAID5 array is actually two RAID5 arrays, joined into one logical partition using LVM2. I did a lazy dd test for this, to an ext4 filesystem on the RAID5+LVM2 volume, like so:

1dd if=/dev/zero of=bench bs=4M count=10000 status=progress oflag=direct

This gave me sustained write speeds of 137 MB/s.

When reading it back, via:

1dd if=bench of=/dev/null bs=4M status=progress iflag=direct

… I get sustained read speeds of 394 MB/s.

I know this isn’t particularly scientific, and I’m sure there are flaws in this benchmarking process. I don’t particularly care; I’m merely just trying to get a vague idea as to whether or not things are working reasonably well. These speeds are more than sufficient for my purposes. I did not test any read/write scenarios more complicated than this, and I didn’t do any performance tuning whatsoever.

Final Thoughts

Overall I’m really happy with how this build turned out. Right now I am just manually managing the storage and network shares on the box, though I have considered installing something like Open Media Vault.

There are a few minor annoyances and omissions that would be nice for Axzez to implement in a future board revision:

  1. The fans don’t appear to be controllable or monitorable by the OS. The board supports three-wire fans, which should at least allow us to monitor fan speed (helpful both for control feedback as well as alerting if a fan has died). Controlling the speed could be done using the CM4’s builtin PWM hardware (maybe? though I believe there are only two channels), or by adding an external fan controller chip.
  2. There isn’t a way to flash the CM4 bootloader or modify the CM4’s boot settings. With a CM4 I/O devkit, you can place a jumper on the board to put it into usbboot mode, which allows you to do all this from a USB-connected computer, but the Interceptor board doesn’t provide this ability.
  3. I didn’t need to make use of the 4-port ethernet switch, but it would be great if Axzez could work to get the drivers for it upstreamed so it doesn’t require a custom kernel. At the very least, a DKMS package would be sufficient. To their credit, they set things up so single-port operation works just fine if you don’t use a patched kernel, which is fine for my purposes.
  4. It would be a nice addition to expose another pair of pins so I could connect the chassis’ HDD activity LED as well.
  5. Documenting how to enable the RTC chip on a vanilla Raspberry Pi OS would be nice (it wasn’t hard to figure out; I just read through their kernel patch).

While the Interceptor certainly isn’t meant to be a general-purpose board, exposing some of the unused GPIO pins could also be really useful (for example, I could probably build my own fan controller daughter board, or set up the HDD activity LED trigger myself). I’ve heard that the two 40-pin FFC connectors expose an I2C and two GPIOs each, but it’s not the most convenient thing to work with. (Having said that, I’ll take what I can get!)

One thing that I am incredibly happy with is that the board works with the stock Raspberry Pi OS. The last thing I want is to be stuck on an old version of the OS after the manufacturer loses interest in supporting it, which is pretty much inevitable for most products, and is the situation I ended up in with my old retail NAS. With the exception of the ethernet switch, everything on this board has upstream support, and the switch gracefully degrades to a single usable ethernet port when booted with the stock Raspberry Pi OS kernel.

As for the rest of the hardware:

  1. The chassis I chose doesn’t make it possible to access the drives without opening it up. I could have chosen a chassis that has hot-swap bays, but I personally don’t need this, so I didn’t bother. I’m actually not sure if the SATA controller on the board supports hot-swap.
  2. This chassis has USB3 ports on the front. Because the CM4’s single PCIe lane is taken up by the SATA ports and ethernet switch, it does not have USB3 capability. Presumably I could cut the USB3 connector off the chassis’ cable and use the ports as USB2 ports (the main board has a pin header for extra USB2 ports), but this isn’t a priority for me to set up.
  3. There’s currently no back panel on the chassis around all the ports. It’s not that big a deal, though I’m afraid of dust getting drawn in through there.
  4. I haven’t measured the power consumption of the power supply I chose. I think it should be reasonably efficient at the sub-100W power draw expected while idle, but it’s likely not as efficient as a smaller supply would be. I debated going the “pico PSU” route, but decided to stick with something more traditional for now.

I’m very pleased that Axzez decided to design and build this carrier board. I’ve been waiting for something like this ever since the CM4 was released, and the board did not disappoint.