nix-config

https://git.jan-leila.com/jan-leila/nix-config

nix multi user, multi system, configuration with sops secret management, home-manager, and nixos-anywhere setup via disko with zfs + impermanence

Hosts

Host Map

Hostname	Device Description	Primary User	Role	Provisioned	Using Nix
twilight	Desktop Computer	Leyla	Desktop	✅	✅
horizon	13 inch Framework Laptop	Leyla	Laptop	✅	✅
defiant	NAS Server	Leyla	Server	✅	✅
hesperium	Mac	?????	Mac	❌	❌
emergent	Desktop Computer	Eve	Desktop	✅	✅
threshold	Laptop	Eve	Laptop	❌	❌
wolfram	Steam Deck	House	Handheld	✅	❌
ceder	A5 Tablet	Leyla	Tablet	✅	❌
skate	A6 Tablet	Leyla	Tablet	❌	❌
shale	A6 Tablet	Eve	Tablet	✅	❌
coven	Pixel 8	Leyla	Android	✅	❌

Tooling

Rebuilding

./rebuild.sh

Updating dependencies

npins update

New host setup

./install.sh --target 192.168.1.130 --config hostname

Updating Secrets

sops secrets/secrets_file_here.yaml

Inspecting a configuration

nix-inspect -p .

Building the installer ISO

nix-build -A nixosConfigurations.installer.config.system.build.isoImage

CI

CI jobs are external services, not workflows committed into the repos they act on — a repo controls nothing about its own CI/CD. Everything is server configuration: the nixos modules that deploy the daemons to defiant also declare, per repo, which commands run (each a [[cmd, arg, ...], ...] list, run in order). That keeps the services language agnostic — onboarding another project is a new repos entry with its own commands, not a change in that project.

• lint: the lint-daemon service (pinned via npins, configured in modules/nixos/programs/lint-daemon). A repo webhook fires on every push to any branch; the service pulls that branch into a local cache, runs the lint commands configured for that repo (here: alejandra), and pushes the fixes back as an ssh-signed commit by the lint-bot account. Results are visible as commit statuses (context lint).
• PR check gate: the build-daemon service (configured in modules/nixos/programs/build-daemon). A webhook fires on pull request events; the service fetches the PR head and runs the check commands configured for that repo (here: build-all-hosts, which builds every nixosConfiguration — deliberately all hosts, so neither fork can merge something that breaks the other's machines). Branch protection requires its build status context. Evaluation reaches the private npins pins by rewriting their ssh urls to authenticated https with the build-bot token.
• peer sync: the sync-daemon service (configured in modules/nixos/programs/sync-daemon). On every push to main it force-pushes a sync/from-<owner> branch to each peer fork and opens a PR there (as sync-bot), with ancestor and tree-equality guards against sync ping-pong. Both forks run one, so changes flow both ways through the receiving fork's own PR checks.
• The old .forgejo/workflows/pr-check.yml and sync-peers.yml actions (and the runner from modules/nixos/programs/forgejo/actions-runner.nix) stay in place only until build-daemon and sync-daemon are deployed and branch protection is switched over; then they get deleted and the runner retired.

Notes:

Research topics

• Look into this for auto rotating sops keys https://technotim.live/posts/rotate-sops-encryption-keys/
• proton mail now has an smtp server we could use that for our zfs and SMART test emails
• VR https://lvra.gitlab.io/docs/distros/nixos/

Tasks:

Documentation

• project layout
• users file structure
• reverse proxy design
  • public service compatibility
  • vpn based services compatibility
• the choice of impermanence
• storage module design
  • base impermanence compatibility and structure reason
    • what does local vs persist mean in pool names (do we need a second layer? ephemeral, local, and persist? local exist only on this machine and is not backed up, persist is backed up to other machines (I think we need to redo the sops and torrent/media folders?))
  • plans to possibly support btrfs in the future
  • plans for home manager datasets
  • plans for auto systemd service datasets
• plans to migrate to some kind of acl structure for user management

Chores:

• test out crab hole service

Tech Debt

• horizon and twilight reuse defiant's services/zfs_smtp_token from defiant-services.yaml for their mail module; each should eventually get its own Proton SMTP token (generated in the Proton web UI) stored in a per-host secrets file (e.g. horizon-services.yaml, twilight-services.yaml) so the tokens can be revoked independently
• monitor configuration in ~/.config/monitors.xml should be sym linked to /run/gdm/.config/monitors.xml (https://www.reddit.com/r/NixOS/comments/u09cz9/home_manager_create_my_own_symlinks_automatically/)
• host.users should be redone so that we just extend the base users.users object. Right now we cant quite do this because we have weird circular dependencies with disko/impermanence (not sure which one) and home manger enabling/disabling users per devices. NOTE: This circular dependency issue should be fixed now
• Home manager impermanence is preventing updates to the latest version of the module
• CI credentials (forgejo-ci group secrets, runner token) are provisioned through the deploy config of defiant; look into decoupling CI concerns from system deploy config
• finish the CI service cutover: push + pin the lint-daemon, build-daemon, and sync-daemon repos, wire their modules into the programs aggregate, enable them on defiant, do the manual forge setup (bots, tokens, webhooks, branch protection), then delete .forgejo/workflows/pr-check.yml/sync-peers.yml and retire the actions runner
• after the network is reworked around a proper internal VPN: build a declarative deployment story (CI builds of host closures, an internal-only binary cache, push/pull deploys) in a dedicated deployment repo that takes this repo as an input. earlier drafts of this lived on the ci-pipeline branch history if reference is ever needed

Broken things

• figure out steam vr things?
• whisper was having issues
• auto loading of ssh agent keys that we auto generate per system

Data Integrity

• zfs email after scrubbing # TODO: test this
• SMART test with email results
• zfs encryption FIDO2 2fa (look into shavee)
• rotate sops encryption keys periodically (and somehow sync between devices?)
• Secure Boot - https://github.com/nix-community/lanzaboote
• auto turn off on power loss - nut
• every service needs to have its own data pool
• secondary server with data sync. Maybe a Pi with a usb hdd enclosure and use rtcwake to only turn on once a week to sync data over tailscale with connection initiated from pi's side. We could probably put this at LZ. Hoping for it to draw only like $1 of power a month. Initial sync should probably be done here before we move it over because that will take a while. Data should be encrypted so that devices doesn't have access to it. Project will prob cost like $1800

Data Access

• nfs export should be backed by the same values for server and client
• samba mounts
• offline access for nfs mounts (overlay with rsync might be a good option here? https://www.spinics.net/lists/linux-unionfs/msg07105.html note about nfs4 and overlay fs)
• figure out why syncthing and jellyfins permissions don't propagate downwards
• make radarr, sonarr, and bazarr accessible over vpn with fully qualified names via reverse proxy
• move searx, home-assistant, actual, vikunja, jellyfin, paperless, and immich to only be accessible via vpn
• FreeIPA/SSSD/LDAP/Kerberos to manage uid and gid's

Services

• ddns server on device
• ntfy service for unified push
• signal socket server
• vikunja service for project management
• Penpot services (need to make this custom)
• storj server
• XMR miner used to heat home based on smart thermostat
• Create Tor guard/relay server
• Create i2P node
• screeps server
• mastodon instance
• omni tools

DevOps

• wake on LAN for updates
• remote distributed builds - https://nix.dev/tutorials/nixos/distributed-builds-setup.html
• ISO target that contains authorized keys for nixos-anywhere github.com/diegofariasm/yggdrasil@4acc43ebc7/hosts/bootstrap/default.nix
• fix panoramax package
• claude code MCP servers should bundle node with them so they work in all environments
• unifi module to configure unifi declaratively

Observability

• graphana for dashboards
• prometheus and loki for metric and log collection
  • zfs storage usage
  • zfs drive health status
  • service version lag
  • network/cpu/ram utilization
  • http latency
  • postgres db load
  • nginx queries
• ntfy.sh for push notifications
• kuma for uptime visualization

Packages

• Custom private fork of MultiMC