sync-daemon

A small webhook-driven fork synchronization service for repos hosted on Forgejo, Gitea, or GitHub. When a change lands on a configured branch, it pushes that branch to a sync/from-<owner> branch on each peer fork and opens a pull request there — or force-updates the branch behind the already-open one. Every sync lands through the receiving fork's own PR checks. Run it on both forks and changes flow in both directions.

It replaces sync CI actions committed into the repos themselves: the job is an external service, and the repos only need a webhook pointed at it.

How it works

push to main ──▶ forge webhook ──▶ POST /webhook
                                        │  verify HMAC signature
                                        ▼
                          per-repo worker (serialized)
                            1. fetch source main into a cache clone
                            2. commit status: pending  (context sync)
                            3. for each peer:
                                 already has the commit,
                                 or identical tree      → skip
                                 else                   → force-push sync branch,
                                                          open PR unless one is open
                            4. status: success (per-peer summary)
                               / failure (which peers failed)

• Ping-pong guard: after a peer merges a sync PR its content matches the source even though the commit graphs differ; the tree-equality check keeps two forks from opening empty sync PRs at each other forever. The ancestor check handles plain fast-forwards.
• Queueing: one worker per source repo; a dirty flag coalesces stacked events, so any number of pushes during a run collapse into one follow-up run against the latest head.
• Startup sweep: each worker runs once at daemon start, covering pushes that happened while the service was down.
• Failure isolation: one failing peer does not stop the others; the run reports failure naming the peers that failed. Transient network failures are retried (3 attempts, backoff); everything else waits for the next push event.
• The daemon is a single-file Python program using only the standard library. Secrets are read from files at runtime and scrubbed from logs.

Config

The daemon takes one argument, a JSON config file:

{
  "listen": {"host": "127.0.0.1", "port": 8425},
  "stateDir": "/var/lib/sync-daemon",
  "statusContext": "sync",
  "repos": [
    {
      "host": "git.example.com",
      "forge": "forgejo",
      "name": "leyla/nix-config",
      "branch": "main",
      "tokenFile": "/run/secrets/sync_bot_token",
      "webhookSecretFile": "/run/secrets/sync_webhook_secret",
      "peers": [
        {"host": "git.example.com", "forge": "forgejo", "name": "eve/nix-config"}
      ]
    }
  ]
}

All top-level fields are required — defaults live in the NixOS module that renders this file, never in the daemon. The only values the daemon fills in itself are derived ones: peers inherit the source repo's tokenFile unless they set their own, and cloneUrlTemplate / apiBaseUrl on repos and peers derive from host/forge unless overridden (same semantics as lint-daemon).

NixOS module

module.nix (exposed as nixosModules.default) runs the daemon as a hardened systemd service under a dedicated sync-daemon user with state in /var/lib/sync-daemon — a re-derivable cache.

services.sync-daemon = {
  enable = true;
  repos = [
    {
      host = "git.example.com";
      name = "leyla/nix-config";
      tokenFile = "/run/secrets/sync_bot_token";
      webhookSecretFile = "/run/secrets/sync_webhook_secret";
      peers = [
        {
          host = "git.example.com";
          name = "eve/nix-config";
        }
      ];
    }
  ];
};

Forge setup

1. Create a bot account; it needs read on the source repo and write (collaborator) on every peer, so it can push sync branches and open PRs there.
2. Create an access token for it → tokenFile (also authorizes the status API on the source repo).
3. Pick a random webhook secret → webhookSecretFile.
4. Add a webhook on the source repo: target http://<daemon host>:<port>/webhook, POST, the secret from step 3, push events, branch filter for the synced branch.

Forgejo note: for a loopback daemon on the Forgejo host itself, set [webhook] ALLOWED_HOST_LIST = loopback.

Development

nix-shell                     # python3, flake8, shellcheck, git, npins
nix-shell --run tests/e2e.sh  # end-to-end suite, no forge needed
nix-build -A packages.default

The e2e suite drives a real daemon against local bare source/peer repos and a stub forge API: the full sync lifecycle (fork in sync → change → PR opened → PR reused → merged → in sync again), the tree-equality ping-pong guard, bad signatures, and ignored events — for both the forgejo and github webhook flavors.