nix-config

https://git.jan-leila.com/jan-leila/nix-config

nix multi user, multi system, configuration with sops secret management, home-manager, and nixos-anywhere setup via disko with zfs + impermanence

Hosts

Host Map

Hostname	Device Description	Primary User	Role	Provisioned	Using Nix
twilight	Desktop Computer	Leyla	Desktop	✅	✅
horizon	13 inch Framework Laptop	Leyla	Laptop	✅	✅
defiant	NAS Server	Leyla	Server	✅	✅
beacon	Public VPS (blog)	Leyla	Server	✅	✅
hesperium	Mac	?????	Mac	❌	❌
emergent	Desktop Computer	Eve	Desktop	✅	✅
threshold	Laptop	Eve	Laptop	❌	❌
wolfram	Steam Deck	House	Handheld	✅	❌
ceder	A5 Tablet	Leyla	Tablet	✅	❌
skate	A6 Tablet	Leyla	Tablet	❌	❌
shale	A6 Tablet	Eve	Tablet	✅	❌
coven	Pixel 8	Leyla	Android	✅	❌

Tooling

Rebuilding

./rebuild.sh

Updating dependencies

npins update

New host setup

./install.sh --target 192.168.1.130 --config hostname

Updating Secrets

Secrets live in the separate nix-config-secrets repo (pulled in here as the secrets npins input).

Inspecting a configuration

nix-inspect -p .

Running tests

nix-build tests

Building the installer ISO

nix-build -A nixosConfigurations.installer.config.system.build.isoImage

Notes:

Research topics

• Look into this for auto rotating sops keys https://technotim.live/posts/rotate-sops-encryption-keys/
• proton mail now has an smtp server we could use that for our zfs and SMART test emails
• VR https://lvra.gitlab.io/docs/distros/nixos/

Tasks:

Documentation

• project layout
• users file structure
• reverse proxy design
  • public service compatibility
  • vpn based services compatibility
• the choice of impermanence
• storage module design
  • base impermanence compatibility and structure reason
    • what does local vs persist mean in pool names (do we need a second layer? ephemeral, local, and persist? local exist only on this machine and is not backed up, persist is backed up to other machines (I think we need to redo the sops and torrent/media folders?))
  • plans to possibly support btrfs in the future
  • plans for home manager datasets
  • plans for auto systemd service datasets
• plans to migrate to some kind of acl structure for user management

Tech Debt

• monitor configuration in ~/.config/monitors.xml should be sym linked to /run/gdm/.config/monitors.xml (https://www.reddit.com/r/NixOS/comments/u09cz9/home_manager_create_my_own_symlinks_automatically/)
• host.users should be redone so that we just extend the base users.users object. This was previously blocked by circular dependencies between disko/impermanence and home-manager enabling/disabling users per device, but that blocker is resolved now, so the redo is unblocked and hasn't been started yet
• Home manager impermanence is preventing updates to the latest version of the module
• CI credentials (forgejo-ci group secrets, runner token) are provisioned through the deploy config of defiant; look into decoupling CI concerns from system deploy config
• after the network is reworked around a proper internal VPN: build a declarative deployment story (CI builds of host closures, an internal-only binary cache, push/pull deploys) in a dedicated deployment repo that takes this repo as an input. earlier drafts of this lived on the ci-pipeline branch history if reference is ever needed
• break project into smaller modules that are usable by others and keeps library code out of my configs
  • storage module
  • impermanence apps modules (design as 2 modules, 1 that specifies what needs to be saved and another that uses those paths to bind to storage module)
  • declarative home assistant module
  • extra packages module
• set up DDNS of beacon.jan-leila.com and defiant.jan-leila.com

Broken things

• figure out steam vr things?
• auto loading of ssh agent keys that we auto generate per system

Data Integrity

• zfs encryption FIDO2 2fa (look into shavee)
• rotate sops encryption keys periodically (and somehow sync between devices?)
• Secure Boot - https://github.com/nix-community/lanzaboote
• every service needs to have its own data pool
• secondary server with data sync. Maybe a Pi with a usb hdd enclosure and use rtcwake to only turn on once a week to sync data over tailscale with connection initiated from pi's side. We could probably put this at LZ. Hoping for it to draw only like $1 of power a month. Initial sync should probably be done here before we move it over because that will take a while. Data should be encrypted so that devices doesn't have access to it. Project will prob cost like $1800

Data Access

• nfs export should be backed by the same values for server and client (const/network_storage.nix drives both the samba exports and the client mounts)
• samba mounts (replaced the nfs exports; per user auth so clients cant impersonate each other)
• offline access for network mounts (overlay with rsync might be a good option here? https://www.spinics.net/lists/linux-unionfs/msg07105.html note about nfs4 and overlay fs)
• figure out why syncthing and jellyfins permissions don't propagate downwards
• make radarr, sonarr, and bazarr accessible over vpn with fully qualified names via reverse proxy
• move searx, home-assistant, actual, vikunja, jellyfin, paperless, and immich to only be accessible via vpn
• FreeIPA/SSSD/LDAP/Kerberos to manage uid and gid's

Services

• ddns server on device
• ntfy service for unified push
• signal socket server
• vikunja service for project management
• Penpot services (need to make this custom)
• storj server
• XMR miner used to heat home based on smart thermostat
• Create Tor guard/relay server
• Create i2P node
• screeps server
• mastodon instance
• omni tools

DevOps

• wake on LAN for updates
• remote distributed builds - https://nix.dev/tutorials/nixos/distributed-builds-setup.html
• fix panoramax package
• unifi module to configure unifi declaratively

Observability

• graphana for dashboards
• prometheus and loki for metric and log collection
  • zfs storage usage
  • zfs drive health status
  • service version lag
  • network/cpu/ram utilization
  • http latency
  • postgres db load
  • nginx queries
• ntfy.sh for push notifications
• kuma for uptime visualization

Packages

• Custom private fork of MultiMC