Root cause chain of recurring Matrix outages (2026-08-16):
1. DNS on m3-hermes resolves exclusively via the Netbird-managed
resolver. When the Netbird nameserver group is unreachable
(observed 14:03-15:24), matrix.m3ta.dev fails to resolve.
2. A gateway restart during that window fails the Matrix login, and
without MATRIX_ACCESS_TOKEN the gateway drops Matrix permanently
from its reconnect queue ('no bot credential on queued config').
Recovery happens only on the next restart - with working DNS.
3. The stable-token setup from 2026-08-14 was wiped because it was
only written to the regenerated .env, not to the agenix secret.
Changes:
- Pin matrix.m3ta.dev to the m3-atlas public IP (TLS termination)
so the Matrix connection survives Netbird DNS outages.
- Declare MATRIX_DEVICE_ID=HERMES01 in the module environment so the
device id survives .env regeneration. The matching
MATRIX_ACCESS_TOKEN must be added to secrets/hermes-env.age
(separate manual step: agenix edit + re-encrypt).
aardvark-dns 2.0.0 has a DNS forwarding bug breaking external name
resolution in podman networks. Pin to 1.17.1 (last 1.x) from
nixpkgs-stable; the nixpkgs podman wrapper picks it up via helpersBin
automatically.
Root cause: The complex concatStringsSep command with power commands
(--power-shutdown, --power-reboot) and multiple --remember flags was
causing tuigreet to display its usage/flags instead of the login UI.
The quoting in the systemd binary paths may have been problematic.
Changes:
- Use lib.getExe instead of manual bin path for tuigreet
- Use simple 'Hyprland' command (found via PATH) instead of
the full start-hyprland path which may have issues
- Remove unverified options: --remember-session, --remember-user-session,
--user-menu, --user-menu-min-uid, --power-shutdown, --power-reboot
- Keep only verified options: --time, --remember, --asterisks, --cmd
- Update tmpfiles comment to reflect actual requirement
This provides a minimal, stable login that works reliably.
User can reboot to test.
m3ta-chiron
m3tam3re