Moves the Prosody server, xmpp-web frontends (chat/plaza), and Forgejo's wiki-update webhook notifier from pinix to blunix, co-located with Traefik/ACME so TLS comes straight from Traefik's cert store instead of the old manual copy process. pinix's prosody/xmpp-web/webhook modules stay in place but deactivated as a fallback, matching the earlier Forgejo migration's pattern. Implements docs/intent/xmpp-umzug-pinix-blunix.md. Deployed and verified live on blunix (nixos-rebuild switch, chat/plaza returning 200, prosody healthy, agent XMPP listeners reconnected). Two first-bootstrap bugs found and fixed during that verification, both specific to bringing Prosody up on a host it never ran on before: - prosody-certs-sync.service used to call a blocking `systemctl reload-or-restart prosody.service` whenever the cert file differed from what's installed -- including on first activation, where prosody.service's own start job is still queued behind this unit (Requires+After), deadlocking forever. Now only reloads an already-running Prosody; ordering alone brings up a fresh one with the cert just installed. - hosts/blunix/localServices/secrets.nix force-owned the global `bindPW` secret to root with the (previously correct) rationale that it belongs to a pinix-only service. Prosody now runs on blunix too and needs to read it directly as its own LDAP bind config include, so that override no longer applies to it. pinix's own deploy (data migration, DNS/webhook cutover) is separate follow-up work, not part of this commit. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ANg2L2xUmNJ45t9mcv6HVT
11 KiB
Prosody
Moved to blunix on 2026-09-06 (docs/intent/xmpp-umzug-pinix-blunix.md);
see hosts/blunix/localServices/prosody/README.md for the active setup.
This module and its data stay in place as a deliberate, disabled fallback
(systemd.services.prosody.enable = false; in setup-prosody.nix) —
everything below describes the old, now-inactive pinix topology, including
the manual TLS copy process this move specifically replaced with an
automated one on blunix.
Prosody ran on pinix. Public XMPP traffic reached port 5222 on
blunix, where Traefik forwarded it through the Headscale/Tailscale network
to 100.64.0.3:5222.
The service uses the upstream NixOS Prosody module. LDAP authentication is
enabled by adding lualdap to the Prosody package's Lua environment in
setup-prosody.nix; no store paths or copied NixOS module are required.
The LDAP password and MOTD are included from agenix files at runtime. Consequently, the module's build-time configuration check is disabled and the live configuration must be checked after activation.
Web clients
xmpp-web.nix builds two frontends from the same pinned xmpp-web release:
chat.0ff.spaceis the registered-account client;plaza.0ff.spaceis the anonymous guest lobby and redirects directly intolobby@muc.0ff.space.
Both builds receive the same generated 0ff-space.css. Its background comes
from the shared hosts/globalServices/branding/0ff-space-background.css,
also used by LLDAP and Phanpy. It reproduces the launcher's 15-pixel dotted
pattern and follows prefers-color-scheme for its light and dark variants.
The stylesheet is copied into each immutable build output and injected into
its index.html; there is no separately managed file below /var/www.
The module also keeps the launcher's five-color palette in one
launcherPalette attribute set and maps xmpp-web/Bulma's semantic colors to
it: primary uses strawberry red, danger neon red, success pink, info violet,
and warning orange. Base, hover, active, and focus colors are replaced during
both frontend builds, so the upstream teal, blue, and green accents do not
reappear during interaction.
nginx terminates TLS on the Tailscale address 100.64.0.3:8443, serves the
two static frontends, and proxies /xmpp-websocket to Prosody on
127.0.0.1:5280.
On blunix, traefik-xmpp-web.nix uses TCP TLS passthrough for the public web
chat hostnames. TLS is therefore terminated on pinix; blunix neither receives
the certificate key nor sees XMPP message contents.
Guests authenticate anonymously against anon.0ff.space and join
lobby@muc.0ff.space. The room is persistent and
lukas@0ff.space is its owner, so the normal LDAP-backed account can join
and administer it from Monal without a second account.
The MUC archive is retained for 30 days and cleaned every four hours.
mod_muc_moderation is installed from the Prosody community-modules revision
pinned by nixpkgs and enabled only on muc.0ff.space. Clients supporting
XEP-0425 can therefore retract abusive messages from the room and its archive;
the moderator must join with an owner or administrator account.
Update notifications
Forgejo sends push and pull-request events to the local webhook receiver on
pinix. The receiver posts short notifications as forgeBot to
update@muc.0ff.space using vikunja@bot.0ff.space.
The repository hook targets
http://127.0.0.1:9000/hooks/wiki-update; Forgejo permits that exact
loopback address in addition to its default external targets.
The update room is members-only. The bot therefore needs a member
affiliation in the room; its password remains in go-sendxmpp.age.
Second channel: agent@bot.0ff.space
Vikunja updates additionally go out as a direct (1:1) message from vikunja
to agent@bot.0ff.space, alongside the existing vikunja@muc.0ff.space post.
Unlike a MUC room, a 1:1 JID needs no room creation or affiliation dance —
Prosody delivers chat messages between local accounts regardless of roster
subscription. The point of the separate account is to have a single-JID
message archive that only carries this one event stream, instead of parsing
it back out of the shared MUC room log.
agent@bot.0ff.space is intentionally never added to a human XMPP client:
go-sendxmpp sends plaintext (no OMEMO), so a client like Monal shows this
conversation unencrypted next to otherwise-encrypted chats.
The listener now runs on blunix, where the persistent Codex checkout and CLI
authentication already live. hosts/blunix/localServices/agent-codex.nix
keeps go-sendxmpp --listen connected as agent, archives its stdout in the
journal, and only queues messages from the explicit sender allowlist:
journalctl -u agent-xmpp-listener
Accepted messages are atomically written below /var/lib/agent-codex/inbox.
Messages from the same sender that arrive within a three-second quiet window
are combined into one queue job. Sending /neu is handled immediately after
any already pending text and resets the sender's active conversation.
agent-codex-worker.path starts a separate oneshot worker, which invokes
codex exec in /home/drone/nixos-config/agents/platform. The nested
agents/platform/AGENTS.md supplies the XMPP role in addition to the root
repository instructions.
The first turn uses JSONL output to capture Codex's thread.started ID.
Successful replies store the sender-to-thread mapping below
/var/lib/agent-codex/sessions; later turns use
codex exec resume <SESSION_ID>. A mapping expires after 30 minutes without a
successful turn. /neu removes it explicitly, so the next content message
starts a fresh thread. Codex's own session rollout files remain in the
isolated /var/lib/agent-codex/codex-home.
Every invocation ignores personal Codex configuration, uses the isolated
CODEX_HOME containing only the copied authentication, explicitly selects a
read-only sandbox, and sends only the final agent message back to the original
XMPP sender. systemd serializes the queue; tmux is not involved. Failed jobs
remain in /var/lib/agent-codex/failed for inspection.
The first successful reply to each allowlisted sender includes a one-time usage note. In normal use:
- send related short fragments within three seconds to combine them;
- wait for the reply before beginning a different request;
- send
/neuto discard the current conversational context; - wait 30 minutes to let the active mapping expire automatically.
The account password lives in agentXmpp.age, decrypted at runtime to
/run/agenix/agentXmpp on blunix, owner drone. It is not decrypted on
pinix after the listener migration. The listener reads it through a
mode-0600 runtime configuration, so it is not exposed in process arguments.
Useful checks on blunix:
systemctl status agent-xmpp-listener agent-codex-worker.path
journalctl -u agent-xmpp-listener -u agent-codex-worker
find /var/lib/agent-codex -maxdepth 2 -type f
Run the unit tests from the repository root:
nix shell --inputs-from . nixpkgs#python3 --command \
python3 -m unittest discover \
-s hosts/blunix/localServices/agent-codex \
-p 'test_*.py' -v
Enable and deploy
Enable the Prosody configuration in ../default.nix:
./prosody/setup-prosody.nix
Build locally on tnix, copy the closure to pinix, and activate it:
nrsp
Refresh the TLS certificate
Traefik renews the wildcard Let's Encrypt certificate on blunix, but
Prosody reads separate certificate files on pinix. Copy the current
certificate manually before the previous one expires. Let's Encrypt
certificates are normally valid for about 90 days.
Run all commands from tnix.
1. Check source and installed certificates
Check the current Traefik certificate without exposing the private key:
ssh -T blunix \
'sudo jq -r '"'"'.cloudflare.Certificates[] |
select(.domain.main == "0ff.space") | .certificate'"'"' \
/var/lib/traefik/acme.json |
base64 -d |
openssl x509 -noout -subject -issuer -dates -fingerprint -sha256'
Check the certificate currently installed on pinix:
ssh -T pinix \
'sudo openssl x509 \
-in /var/lib/prosody/certs/0ff.space.crt.pem \
-noout -subject -issuer -dates -fingerprint -sha256'
2. Transfer certificate and key
Transfer both files directly over SSH. The private key is never written to
the tnix filesystem or printed to the terminal.
set -o pipefail
ssh -T blunix \
'sudo jq -r '"'"'.cloudflare.Certificates[] |
select(.domain.main == "0ff.space") | .certificate'"'"' \
/var/lib/traefik/acme.json | base64 -d' |
ssh -T pinix \
'sudo sh -c '"'"'umask 022
tee /var/lib/prosody/certs/0ff.space.crt.pem.new >/dev/null
chown prosody:prosody /var/lib/prosody/certs/0ff.space.crt.pem.new'"'"''
set -o pipefail
ssh -T blunix \
'sudo jq -r '"'"'.cloudflare.Certificates[] |
select(.domain.main == "0ff.space") | .key'"'"' \
/var/lib/traefik/acme.json | base64 -d' |
ssh -T pinix \
'sudo sh -c '"'"'umask 077
tee /var/lib/prosody/certs/0ff.space.key.pem.new >/dev/null
chown prosody:prosody /var/lib/prosody/certs/0ff.space.key.pem.new'"'"''
3. Validate and install
Validate the certificate, private key, and file permissions on pinix:
ssh -T pinix 'sudo sh -c '"'"'
set -eu
cert=/var/lib/prosody/certs/0ff.space.crt.pem.new
key=/var/lib/prosody/certs/0ff.space.key.pem.new
openssl x509 -in "$cert" -noout -dates -fingerprint -sha256
openssl pkey -in "$key" -check -noout
cert_hash=$(openssl x509 -in "$cert" -pubkey -noout |
openssl pkey -pubin -outform DER 2>/dev/null |
sha256sum | cut -d" " -f1)
key_hash=$(openssl pkey -in "$key" -pubout -outform DER 2>/dev/null |
sha256sum | cut -d" " -f1)
test "$cert_hash" = "$key_hash"
stat -c "%U:%G %a %n" "$cert" "$key"
'"'"''
Expected permissions:
prosody:prosody 644 ...crt.pem.new
prosody:prosody 600 ...key.pem.new
Back up the existing files, install the validated files, and restart Prosody:
ssh -T pinix 'sudo sh -c '"'"'
set -eu
dir=/var/lib/prosody/certs
stamp=$(date -u +%Y%m%dT%H%M%SZ)
cp -a "$dir/0ff.space.crt.pem" \
"$dir/0ff.space.crt.pem.previous-$stamp"
cp -a "$dir/0ff.space.key.pem" \
"$dir/0ff.space.key.pem.previous-$stamp"
mv "$dir/0ff.space.crt.pem.new" "$dir/0ff.space.crt.pem"
mv "$dir/0ff.space.key.pem.new" "$dir/0ff.space.key.pem"
chmod 640 "$dir/0ff.space.key.pem"
systemctl restart prosody
systemctl restart nginx
'"'"''
The installed key is 0640 prosody:prosody: nginx is a supplementary
member of the prosody group and receives read-only access. The temporary
key remains 0600 while it is being transferred and validated.
4. Verify
Check the service and listeners:
ssh -T pinix \
'systemctl is-active prosody
ss -lnt | grep -E ":(5222|5269|5280|5281)[[:space:]]"'
Verify the certificate delivered through the public XMPP endpoint:
set -o pipefail
timeout 20 openssl s_client \
-starttls xmpp \
-connect 23.88.33.137:5222 \
-xmpphost 0ff.space \
-servername 0ff.space \
</dev/null 2>/dev/null |
openssl x509 -noout -subject -issuer -dates -fingerprint -sha256