Platform Setup Guide

This guide walks you through the complete installation and setup process for the NixOS infrastructure platform. Follow these steps in order to deploy your three-machine architecture.

This page describes a fresh installation. For current tmux access, remote builds and routine deployments, see CLAUDE.md in the main repo.

Prerequisites

• Hardware: 1 laptop/desktop, 1 VPS, 1 Raspberry Pi
• Network: Internet connection and router access
• Domain: A domain name with DNS management (Cloudflare recommended)
• Basic Skills: Familiarity with Linux terminal and SSH

---

Step 1: Builder Setup (tnix)

The builder machine serves as your orchestrator - the primary machine from which you'll deploy and manage the entire infrastructure.

1.1 Install NixOS on Builder

1. Download NixOS ISO

   # Download latest NixOS ISO from https://nixos.org/download.html
   # Flash to USB drive
   

2. Install NixOS

   • Boot from USB and follow the graphical installer
   • Create user account (recommend username: builder)
   • Enable SSH during installation
   • Complete base installation and reboot

3. Initial Configuration

   # Enable flakes and nix-command
   sudo nano /etc/nixos/configuration.nix
   # Add: nix.settings.experimental-features = [ "nix-command" "flakes" ];
   sudo nixos-rebuild switch
   

1.2 Clone and Prepare Configuration

1. Clone Repository

   git clone https://git.0ff.space/lukas/nixos-config.git
   cd nixos-config
   

2. Generate SSH Keys

   ssh-keygen -t ed25519 -f ~/.ssh/id_ed25519
   # Add public key to your Git service and VPS authorized_keys
   

3. Setup Agenix Secrets

   # Install agenix
   nix shell github:ryantm/agenix
   # Initialize secrets (will be configured per host later)
   

4. Set local infrastructure parameters

   Create the git-ignored variables-local.nix for your domains, public IP, Tailnet IPs, administrator email, and LDAP base DN. Keep credentials in agenix instead. See Configuration and secrets for the complete example and secret inventory.

1.3 Deploy Builder Configuration

# Build and apply tnix configuration
sudo nixos-rebuild switch --flake .#tnix

---

Step 2: Connector Setup (blunix)

The connector is your VPS that handles networking, authentication, and routing.

2.1 Rent and Prepare VPS

1. VPS Requirements

   • Minimum: 2GB RAM, 20GB storage, 1 CPU
   • Ubuntu/Debian base image
   • Public IPv4 address
   • SSH root access

2. Initial VPS Access

   ssh root@YOUR_VPS_IP
   # Ensure SSH key access works
   

2.2 DNS and Cloudflare Setup

1. Domain Configuration

   • Point your domain to VPS IP: example.com A YOUR_VPS_IP
   • Create wildcard subdomain: *.example.com A YOUR_VPS_IP

2. Cloudflare API Setup

   • Create Cloudflare API token with Zone:Read, DNS:Edit permissions
   • Note your Zone ID and API token for later use

2.3 Deploy NixOS with nixos-anywhere

1. Prepare Deployment

   # From your builder machine
   cd nixos-config
   
   # Update hosts/blunix/configuration.nix with your domain
   nano hosts/blunix/configuration.nix
   # Set: domain = "example.com";
   

2. Run nixos-anywhere

   nix run github:nix-community/nixos-anywhere -- \
     --flake .#blunix \
     --build-on-remote \
     root@YOUR_VPS_IP
   

2.4 Initial Connector Configuration

1. Get Host SSH Key

   ssh drone@YOUR_VPS_IP "sudo cat /etc/ssh/ssh_host_ed25519_key.pub"
   # Add this key to secrets/secrets.nix for agenix
   

2. Configure Secrets

   # Add VPS host key to secrets/secrets.nix
   nano secrets/secrets.nix
   # Add: "drone@blunix:/etc/ssh/ssh_host_ed25519_key.pub"
   
   # Rekey secrets
   agenix --rekey
   

2.5 Deploy Traefik (Phase 1)

Deploy only Traefik first to establish SSL certificates:

1. Minimal Deployment

   # Comment out headscale in hosts/blunix/configuration.nix temporarily
   nano hosts/blunix/configuration.nix
   # Comment: # ./localServices/headscale.nix
   
   # Deploy with Traefik only
   nixos-rebuild switch --target-host drone@YOUR_VPS_IP --flake .#blunix
   

2. Verify SSL Certificate

   # Check if wildcard cert is obtained
   curl -I https://test.example.com
   # Should return valid SSL response
   

2.6 Deploy Headscale (Phase 2)

Once Traefik is working with SSL:

1. Enable Headscale

   # Uncomment headscale in configuration
   nano hosts/blunix/configuration.nix
   # Uncomment: ./localServices/headscale.nix
   
   # Deploy complete connector configuration
   nixos-rebuild switch --target-host drone@YOUR_VPS_IP --flake .#blunix
   

2. Create Headscale User

   ssh drone@YOUR_VPS_IP
   sudo headscale users create main
   

2.7 Connect Builder to Mesh Network

1. Install Tailscale on Builder

   # Already included in tnix configuration
   # Get auth key from headscale
   ssh drone@YOUR_VPS_IP "sudo headscale preauthkeys create --user main --expiration 1h"
   

2. Join Network

   sudo tailscale up --login-server https://head.example.com --authkey YOUR_AUTH_KEY
   

---

Step 3: Worker Setup (pinix)

The worker is your Raspberry Pi that runs the core services.

3.1 Prepare Raspberry Pi

1. Flash NixOS Image

   # Download NixOS Raspberry Pi image
   # Use Raspberry Pi Imager (installed on tnix)
   rpi-imager
   # Flash SD card with NixOS ARM64 image
   

2. Initial Boot Setup

   • Connect monitor and keyboard
   • Boot Pi and wait for login prompt
   • Default login: nixos (no password)

3.2 Network Configuration

1. Configure WiFi

   # On the Pi
   sudo loadkeys de  # or your keyboard layout
   sudo systemctl start wpa_supplicant.service
   
   # Configure WiFi
   wpa_cli
   > add_network
   > set_network 0 ssid "YOUR_WIFI_NAME"
   > set_network 0 psk "YOUR_WIFI_PASSWORD"
   > enable_network 0
   > save_config
   > quit
   
   # Set password for SSH access
   passwd
   

2. Find Pi IP Address

   # From builder machine
   nmap -sn 192.168.1.0/24
   # Look for the Pi's IP address
   

3.3 Get Pi Host Key and Deploy

1. Get SSH Host Key

   # From builder
   ssh nixos@PI_IP_ADDRESS "sudo cat /etc/ssh/ssh_host_ed25519_key.pub"
   

2. Add Key to Secrets

   # Add Pi host key to secrets/secrets.nix
   nano secrets/secrets.nix
   # Add: "nixos@pinix:/etc/ssh/ssh_host_ed25519_key.pub"
   
   # Rekey secrets
   agenix --rekey
   

3. Deploy Worker Configuration

   # Deploy to Pi
   nixos-rebuild switch --target-host nixos@PI_IP_ADDRESS --flake .#pinix
   

3.4 Building pinix updates

Use the laptop (tnix) for small pinix changes. The nrsp shell alias builds the configuration locally, copies it to pinix, and activates it:

nrsp

Large updates, kernel builds, or changes with many ARM derivations should be built on nixbuild.net instead of using slow local aarch64-linux emulation:

nix build \
  --json \
  --eval-store auto \
  --store ssh-ng://eu.nixbuild.net \
  ~/platform/nixos-config#nixosConfigurations.pinix.config.system.build.toplevel

The JSON result contains the finished system path in outputs.out:

{
  "outputs": {
    "out": "/nix/store/...-nixos-system-pinix-..."
  }
}

Assign that path and retrieve the complete closure from the remote store:

SYSTEM=/nix/store/...-nixos-system-pinix-...

sudo nix copy \
  --option extra-trusted-public-keys \
  'nixbuild.net/ACCOUNT-1:PUBLIC_SIGNING_KEY=' \
  --from ssh-ng://eu.nixbuild.net \
  "$SYSTEM"

The account-specific public signing key can be shown in the interactive nixbuild.net administration shell:

ssh -T eu.nixbuild.net
settings signing-key-for-builds --show

Copy and activate the already built system:

nixos-rebuild switch \
  --store-path "$SYSTEM" \
  --target-host pinix \
  --sudo

Always verify that the running and boot profiles match after a manual deployment:

ssh -T pinix \
  'readlink -f /run/current-system
   readlink -f /nix/var/nix/profiles/system
   systemctl is-system-running'

The private SSH key remains on tnix. Only the public nixbuild.net signing key is added to Nix trust. Build outputs should be retrieved with signature verification before they are activated as a root-owned NixOS system.

3.5 Join Pi to Mesh Network

1. Get Auth Key

   ssh drone@YOUR_VPS_IP "sudo headscale preauthkeys create --user main --expiration 1h"
   

2. Connect Pi to Network

   ssh drone@PI_IP_ADDRESS
   sudo tailscale up --login-server https://head.example.com --authkey YOUR_AUTH_KEY
   

---

Step 4: Final Configuration

4.1 Verify Network Connectivity

# From builder, test connectivity to all hosts
tailscale ping blunix
tailscale ping pinix

# Check headscale status
ssh drone@blunix "sudo headscale nodes list"

4.2 Complete Service Deployment

1. Deploy All Services

   # Ensure all hosts have latest configuration
   nixos-rebuild switch --flake .#tnix                    # Builder
   nixos-rebuild switch --target-host drone@blunix --flake .#blunix  # Connector
   nixos-rebuild switch --target-host drone@pinix --flake .#pinix    # Worker
   

2. Initialize Services

   # Create initial user accounts in LLDAP
   # Access https://lldap.example.com
   
   # Setup Forgejo admin account
   # Access https://git.example.com
   
   # Configure other services as needed
   

4.3 Backup Setup

1. Initialize Backup Storage

   # Connect external storage to builder (tnix)
   # Backup services are already configured
   
   # Run initial backups
   sudo systemctl start backup-srv
   sudo systemctl start backup-pi
   sudo systemctl start backup-vps
   

---

Troubleshooting

Common Issues

1. SSL Certificate Problems

   • Verify Cloudflare API token permissions
   • Check DNS propagation: dig example.com
   • Review Traefik logs: ssh drone@blunix "sudo journalctl -u traefik"

2. Headscale Connection Issues

   • Ensure Traefik is working first
   • Check headscale logs: ssh drone@blunix "sudo journalctl -u headscale"
   • Verify firewall settings on VPS

3. Pi Deployment Failures

   • Ensure SSH key authentication works
   • Check network connectivity
   • Verify agenix secrets are properly rekeyed

Getting Help

• Check service logs: sudo journalctl -u SERVICE_NAME
• Review configuration: nixos-option services.SERVICE_NAME
• Test network connectivity: tailscale ping HOST

---

Next Steps

Once installation is complete:

1. Read the User Guide for service usage instructions
2. Review Software Overview for detailed service information
3. Configure user accounts and team access
4. Set up regular backups and monitoring
5. Customize services for your team's needs

Your NixOS infrastructure platform is now ready for use!