I just figured I’d document the (rough) configuration I have for my Ten64 that I use as a router.
Working:
- Ethernet ports
- NVMe SSD
- Rebooting
- USB Ports
- Upstream kernel
Not tested:
- Any WiFi/other internal devices not mentioned
- Front panel LED’s, buttons, etc.
- Fan controls or internal GPIO
- SIM/SD slot
Didn’t work about 2 years ago, but might now:
- SFP+
When I originally installed it, I simply grabbed the NixOS aarch64 installer, booted it up UEFI from a flash drive, modified the kernel command line to ignore the IOMMU issue (arm-smmu.disable_bypass=n), and it booted into a command line.
You will need to format the disk GPT, add the two partitions (see hardware-configuration.nix below for ideas), and replace the disk UUID’s with the ones you have. I have /boot at 1GB, and that’s okay but remember to clean up old generations as the kernels quickly fill the partition otherwise.
For my hardware-configuration.nix:
# Do not modify this file! It was generated by ‘nixos-generate-config’
# and may be overwritten by future invocations. Please make changes
# to /etc/nixos/configuration.nix instead.
{ config, lib, pkgs, modulesPath, ... }:
{
imports =
[ (modulesPath + "/installer/scan/not-detected.nix")
];
boot.initrd.availableKernelModules = [ "nvme" "usb_storage" ];
boot.initrd.kernelModules = [ ];
boot.kernelModules = [ ];
boot.extraModulePackages = [ ];
fileSystems."/" =
{ device = "/dev/disk/by-uuid/badae59e-a3b4-4499-aa2c-050c29051f3f";
fsType = "ext4";
};
fileSystems."/boot" =
{ device = "/dev/disk/by-uuid/A695-045B";
fsType = "vfat";
};
swapDevices = [ ];
powerManagement.cpuFreqGovernor = lib.mkDefault "ondemand";
}
As for the configuration.nix, you may have some changes to make but here’s my template:
{ config, pkgs, ... }:
{
imports =
[
./hardware-configuration.nix
];
boot.kernelPackages = pkgs.linuxPackages_latest;
boot.kernelParams = [ "arm-smmu.disable_bypass=n" ];
boot.loader.grub.enable = true;
boot.loader.grub.efiSupport = true;
boot.loader.grub.efiInstallAsRemovable = true;
boot.loader.grub.device = "nodev";
networking.hostName = "your hostname here";
time.timeZone = "America/New_York";
boot.kernel.sysctl = {
"fs.inotify.max_user_watches" = 1048576;
"net.ipv4.tcp_congestion_control" = "bbr";
"net.ipv4.tcp_ecn" = 1;
"net.core.default_qdisc" = "fq_codel";
};
systemd.network.enable = true;
systemd.network.links = {
"00-ge0" = {
matchConfig = { PermanentMACAddress = "xx"; };
linkConfig = { Name = "ge0"; };
};
"01-ge1" = {
matchConfig = { PermanentMACAddress = "xx"; };
linkConfig = { Name = "ge1"; };
};
"02-ge2" = {
matchConfig = { PermanentMACAddress = "xx"; };
linkConfig = { Name = "ge2"; };
};
"03-ge3" = {
matchConfig = { PermanentMACAddress = "xx"; };
linkConfig = { Name = "ge3"; };
};
"04-ge4" = {
matchConfig = { PermanentMACAddress = "xx"; };
linkConfig = { Name = "ge4"; };
};
"05-ge5" = {
matchConfig = { PermanentMACAddress = "xx"; };
linkConfig = { Name = "ge5"; };
};
"06-ge6" = {
matchConfig = { PermanentMACAddress = "xx"; };
linkConfig = { Name = "ge6"; };
};
"07-ge7" = {
matchConfig = { PermanentMACAddress = "xx"; };
linkConfig = { Name = "ge7"; };
};
"08-xg0" = {
matchConfig = { PermanentMACAddress = "xx"; };
linkConfig = { Name = "xg0"; };
};
"09-xg1" = {
matchConfig = { PermanentMACAddress = "xx"; };
linkConfig = { Name = "xg1"; };
};
};
networking.useDHCP = false;
networking.interfaces.ge0.useDHCP = true;
networking.interfaces.ge1.useDHCP = false;
networking.interfaces.ge2.useDHCP = false;
networking.interfaces.ge3.useDHCP = false;
networking.interfaces.ge4.useDHCP = false;
networking.interfaces.ge5.useDHCP = false;
networking.interfaces.ge6.useDHCP = false;
networking.interfaces.ge7.useDHCP = false;
networking.interfaces.xg0.useDHCP = false;
networking.interfaces.xg1.useDHCP = false;
networking.dhcpcd.extraConfig = "noipv6";
networking.interfaces.br1.ipv4.addresses =
[ { address = "10.0.0.1"; prefixLength = 24; } ];
networking.bridges = {
br1.interfaces = ["ge1" "ge2" "ge3" "ge4" "ge5" "ge6" "ge7" "xg0" "xg1"];
};
networking.nat.enable = true;
networking.nat.externalInterface = "ge0";
networking.nat.internalInterfaces = [ "br1" ];
networking.firewall.checkReversePath = true;
networking.firewall.logReversePathDrops = true;
networking.firewall.allowedTCPPorts = [
22 # ssh
];
networking.firewall.interfaces.br1.allowedTCPPorts = [
22
];
i18n.defaultLocale = "en_US.UTF-8";
users.users.yourusername= {
isNormalUser = true;
extraGroups = [ "wheel" ];
};
#security.sudo.wheelNeedsPassword = false;
environment.systemPackages = with pkgs; [
vim wget htop git ethtool tcpdump
];
services.openssh.enable = true;
services.openssh.openFirewall = false;
services.openssh.listenAddresses = [ { addr = "0.0.0.0"; port = 22; } ];
services.openssh.settings.PermitRootLogin = "no";
services.irqbalance.enable = true;
system.stateVersion = "unstable";
}
Do note that I went out of my way to both pin the MAC addresses and also to specify where I wanted DHCP to come in. I did that simply because I wanted to make sure it didn’t change on me, but also it renames the interfaces when the system comes up, so instead of those annoying procedurally generated names, I get interface names that match the printing on the front panel. Neat! One could totally set this up in other operating systems too, it’s just using a systemd feature.
I’d previously been compiling the kernel on the box with some kernel features suggested in the documentation - here’s what you would add to configuration.nix if you really want that; I was able to query DPAA2 with it but to be honest it was not useful to me in the end so I just boot the normal upstream kernel. It takes several hours to compile on the Ten64, but it can be done with just 8GB of RAM if you’re patient.
boot.kernelPatches = [ {
name = "ten64-patches";
patch = null;
extraConfig = ''
ARM64_VA_BITS_48 y
ARM64_VA_BITS 48
ARCH_LAYERSCAPE y
FSL_MC_BUS y
CRYPTO_HW y
CRYPTO_DEV_FSL_CAAM_COMMON y
CRYPTO_DEV_FSL_CAAM_CRYPTO_API_DESC y
CRYPTO_DEV_FSL_CAAM_AHASH_API_DESC y
CRYPTO_DEV_FSL_CAAM m
CRYPTO_DEV_FSL_CAAM_JR m
CRYPTO_DEV_FSL_CAAM_JR m
CRYPTO_DEV_FSL_CAAM_CRYPTO_API y
CRYPTO_DEV_FSL_CAAM_CRYPTO_API_QI y
CRYPTO_DEV_FSL_CAAM_AHASH_API y
CRYPTO_DEV_FSL_CAAM_PKC_API y
CRYPTO_DEV_FSL_CAAM_RNG_API y
CRYPTO_DEV_FSL_DPAA2_CAAM y
GPIO_MPC8XXX y
NET_VENDOR_FREESCALE y
FSL_PQ_MDIO y
FSL_DPAA_ETH m
FSL_DPAA2_ETH m
FSL_DPAA2_PTP_CLOCk m
PCI_LAYERSCAPE y
PCIE_LAYERSCAPE y
FSL_MC_DPIO y
FSL_MC_UAPI_SUPPORT y
'';
} ];
This isn’t exactly a complete setup guide but I hope that it helps indicate some of the changes you’ll want to make when installing NixOS.