talk/docs/spec.md

Spec: talk 2.0: Cross-Platform Voice Notifications for AI Agents

Status: ready-for-agent Created: 2026-10-18 Source: grilling session 2026-10-18 (shared understanding, all branches visited)

---

Problem Statement

Today, AI agents (coding agents such as opencode/Claude Code, company chats, background jobs) can only reach me visually: terminal output, toast, push notification. When I am away from the screen (coffee, another window, a meeting on the second monitor), I miss that a build has finished, that an agent is waiting for my approval, or that a job has crashed.

The existing talk/crunch stack solves this in principle but is unusable as a foundation: bash + mpv + notify-send + agenix means Linux desktop only, ElevenLabs only (key per host, in practice confirmed working on just one), no native tool interface for agents (shell workarounds only), no queue (audio overlaps), a volatile mute toggle, and no story at all for the company's Windows fleet.

Solution

talk 2.0: a cross-platform tool (Windows/macOS/Linux, amd64+arm64) written in Go as static binaries: talk (CLI with a Bubble Tea TUI) and talkd (daemon with queue). Any AI agent that can run a program can speak with it: talk "Build done". Local MCP clients additionally get a native MCP endpoint on localhost. TTS quality scales with the available keys through a four-stage fallback chain down to free/offline voices, so voice works everywhere from minute one. Speaking is strictly opt-in by the user; the daemon enforces rate limit, mute and quiet hours as a backstop beneath all prompt behaviour.

Distribution: Gitea releases following the agent-lib pattern (asset schema talk_<os>_<arch> + sha256); company: internal mirror + fleet role (Ansible, Program Files + PATH); private: Nix flake. The skill lives in the repo and rolls out in the company via az-agent-defaults/agent-lib.

User Stories

1. As a user I want to call talk "text" so that I immediately know how the interface works without reading docs.
2. As a user I want the call to return non-blocking so that my script/agent does not wait for the audio.
3. As a script author I want talk "text" --wait so that cron jobs only continue or exit after the announcement.
4. As a user I want to use talk wrap -- <command> so that the duration and success/failure of any command are announced to me.
5. As an AI agent I want to feed the program via argument or stdin so that I can call it from any environment.
6. As an AI agent (MCP-capable) I want a local MCP endpoint with a speak tool so that I have voice natively as a tool instead of shell workarounds.
7. As a browser AI chat (later) I want to call POST /speak on localhost with CORS permission so that a web UI (e.g. LibreChat) can address me without talk having to change.
8. As a user with an ElevenLabs key I want the premium voice so that announcements sound the way I am used to.
9. As a user with only an OpenAI key I want to use OpenAI TTS as a provider so that I do not need a second account.
10. As a user without any key I want to get Edge TTS (free) automatically so that voice works out of the box.
11. As an offline user I want to fall back to OS built-in TTS voices (say/SAPI/espeak) so that speech works even without network or key.
12. As a user I want the provider chain to fall through automatically (invalid key, quota exhausted, network gone) so that it never stays silent when it does not have to.
13. As a user I want the language of the text detected automatically (de/en) so that English agent replies are not mangled by a German voice.
14. As a user I want to set a voice per provider and language in config.toml so that my familiar voice sounds the same everywhere.
15. As a user I want talk setup as a guided wizard (preview voices, test key, write config) so that I am set up in 2 minutes without terminal experience.
16. As a user I want talk status as a live dashboard (queue, mute, active provider, version) so that I always know what the tool is doing.
17. As a user I want talk voices as an interactive list with audio preview so that I can compare voices without memorising IDs.
18. As a user I want talk mute / talk unmute as a persistent switch so that all agents and scripts stay silent until I allow it.
19. As a user I want configurable quiet hours (quiet_hours = "18:00-08:00") so that 23:47 cron jobs do not wake me.
20. As a user I want talk cancel so that a running or scheduled announcement stops immediately.
21. As a user I want --priority so that urgent items skip to the front of the queue and crowding under the rate limit may be discarded.
22. As a user I want audio messages to never overlap (the queue serialises) so that everything stays intelligible with three agents at once.
23. As a user I want a rate limit as daemon law (default ~1 announcement/30 s) so that agent spam is impossible, whatever the prompt says.
24. As an agent-using user I want opt-in etiquette documented in the skill: the agent speaks only on my instruction or a standing order in AGENTS.md, session-sticky until revoked, so that no agent chatters on its own.
25. As an agent-using user I want the announcement to be useful (status / next step / what I am needed for), not the long answer read aloud, so that 10 seconds of announcement are enough.
26. As a fleet administrator I want to roll out static binaries from Gitea releases, version-pinned and sha256-verified (agent-lib/officecli pattern), so that the Ansible rollout is deterministic.
27. As a fleet administrator I want to roll out keys per user as environment variables (TALK_*) following our AZ_FLEET pattern so that no secret ends up in config files.
28. As a fleet administrator I want talk doctor as diagnostic output so that "no sound at colleague X" is self-explanatory in 5 seconds.
29. As a fleet user without admin rights I want talk to run without runtime dependencies (no ffmpeg, no player, no Python) so that it simply works on every company machine.
30. As a user on a machine without a running daemon I want a degraded mode (lock file, sequential) so that talk is never broken even without talkd.
31. As a private user with NixOS hosts I want a flake output in the repo so that my hosts pull talk declaratively as usual.
32. As an OS user I want a desktop notification alongside the audio with a cancel button, where the OS allows it, so that I can silence it without opening a terminal.
33. As a non-terminal user I want TUI commands to fall back to plain output without a TTY so that pipes and automation never break.
34. As a project maintainer I want PR-driven tests + a release matrix in Gitea Actions so that every tag automatically produces consistently built artifacts for all platforms.

Implementation Decisions

• Language/runtime: Go; static binaries for win/mac/linux x amd64/arm64. No runtime dependencies.
• Binaries: talk (CLI) + talkd (daemon). Repo: code.m3ta.dev/m3tam3re/talk, MIT licence.
• Daemon: talkd owns the queue (serialisation, priorities, rate limit, discarding under crowding), the persistent mute state, the quiet-hours windows (local time, discard silently or toast-only, configurable), the HTTP API (POST /speak on 127.0.0.1, CORS domains configurable) and an MCP endpoint (Streamable HTTP, localhost).
• Degraded mode: without a running talkd the CLI falls back to lock-file sequential; nothing breaks silently.
• Audio: embedded playback (oto), no mpv/ffmpeg needed. Audio sink selectable, including null (CI, server, doctor).
• Notifications: best effort via beeep (notify-send / macOS Notification Center / Windows toast); cancel button where the OS allows it.
• Provider chain V1 (auto-fallback in this order, overridable via config/flag): ElevenLabs -> OpenAI TTS -> Edge TTS (free, no key) -> OS built-in (macOS say / Windows SAPI / Linux espeak).
• Keys/config: env-first (TALK_ELEVENLABS_KEY, TALK_OPENAI_KEY, ...) plus an optional config.toml in the platform-native config dir (~/.config/talk / ~/Library/Application Support/talk / %APPDATA%\talk). Provider endpoints configurable (testability, company server later). Per-user keys in V1; a central company server/gateway deliberately later.
• Language/voices: de/en heuristic (pure function, no ML); voice.<provider>.<lang> mapping in config.toml; personal ElevenLabs voice (KXxZ...) as private default, neutral defaults in the repo.
• CLI behaviour: non-blocking by default; flags --wait, --now (queue bypass for urgent items), --priority, --voice, --lang, --provider. Commands: say, wrap, mute, unmute, status, cancel, voices, test, doctor, setup.
• TUI (Bubble Tea): interactive = setup (wizard), status (live dashboard), voices (picker with preview); plain and scriptable = say, wrap, mute, cancel, test, doctor; no TTY -> always plain.
• Etiquette (skill policy): strict opt-in. Speaking only on a user trigger of the skill or a standing order in AGENTS.md; session-sticky until revoked (prompt level, no daemon session state); announcement content = status / next step / what the user is needed for, never the long answer. Daemon backstop: rate limit + mute + quiet hours apply regardless of the prompt.
• Distribution: GoReleaser -> Gitea releases, asset schema talk_<os>_<arch> + sha256 (agent-lib contract); the company mirrors internally (pin_mirror pattern) + fleet role modelled on agent-lib/officecli (Program Files + machine PATH, admin-protected ProgramData config); company skill rollout via az-agent-defaults -> agent-lib three-state sync; private via flake.nix.
• CI: Gitea Actions: PR -> tests, tag -> release matrix.
• Legacy teardown: the old voice-notify/crunch-jobs skills (AGENTS repo) become redirect stubs; nixpkgs PRs #24/#7 are closed as superseded; m3ta-nixpkgs talk/crunch deprecated after migration; the ElevenLabs key (exposed in the old prototype script) is rotated, then the script is deleted.
• Naming: the name stays talk (accepted collision with the historical Unix talk; say avoided because of macOS).

Testing Decisions

• A good test tests only externally observable behaviour at the interface, never implementation details.
• Primary seam (one): the process boundary of talkd. Daemon + CLI as a black box, with two fakes injectable via config: (1) provider endpoints point to a fake HTTP server serving audio bytes in tests; (2) audio.sink = "null" disables real playback. Tested at this seam: queue serialisation/priority/rate limit/discarding, mute persistence, quiet hours, fallback chain (fake provider 500 -> next provider), non-blocking + --wait, degraded mode, HTTP /speak contract including CORS, MCP endpoint basic contract, status/cancel/doctor output.
• Two pure side seams (table-driven unit tests, no mocks): language detection (de/en) and config resolution (env + TOML merge).
• Deliberately untested (hardware/OS-bound): real audio rendering and OS toasts, covered by talk doctor as a manual diagnostic tool.
• Predecessors: none, greenfield. CI runs in Gitea Actions from ticket 1 onwards.

Out of Scope

• LibreChat frontend integration (the CORS door is built, the integration itself is not)
• Browser extension as a bridge
• Central company gateway server (one key for everyone)
• Timers/scheduling (crunch at|in|daily stays OS-native: cron/systemd/Task Scheduler)
• Local neural voices (Piper/Kokoro), later expansion
• Calendar-aware quiet hours (static time windows only in V1)
• Homebrew/Scoop/AUR feeds, optional later
• Multi-user machines (one user per host as in the fleet)

Further Notes

• Pending (user): rotate the ElevenLabs key in the dashboard BEFORE deleting the old prototype script.
• Personal default voice: ElevenLabs KXxZd16DiBqt82nbarJx (carried over from the old stack as private default, not the repo default).
• Session stickiness of the etiquette is prompt/skill level; the daemon knows nothing about agent sessions.
• Open decision for later: LibreChat integration via frontend patch vs. browser extension. Blocks nothing; the door is open with the CORS-capable HTTP API.
• Related company ADRs: 0009 (agent-lib), 0012/0013 (secrets env bridge / machine env keys), 0016/0017 (user-scoped keys, MCP audience scoping). talk follows these patterns.

---

Addendum 2026-10-24: decisions after the architecture review

These points replace the corresponding statements above (grilling state of 2026-10-18):

• One binary instead of talk + talkd. The daemon is talk daemon run; talk say/talk wrap start it themselves in the background on the first speak command (daemon.autostart, on by default; talk daemon start|stop explicitly). The degraded mode with a lock file is dropped; without a reachable or startable daemon, talk speaks directly (unserialised, with a notice).
• The CLI is the agent interface. MCP remains as a door but is no longer a focus; browser access is restricted server-side to POST /speak from listed origins (origin guard, not just CORS).
• now:true deliberately breaks out of quiet hours (confirmed).
• Two more laws: queue.max_text (400 characters, truncated at a word boundary) and queue.max_age (2 min, stale backlog announcements expire).
• Target picture for publication: talk should eventually be usable publicly. The top criterion is usability for non-terminal users; the Bubble Tea TUI is not an end in itself. Simple, clear commands (doctor, setup, test, voices) take precedence.
• Release without GoReleaser: scripts/release.sh builds the six targets with go build; .gitea/workflows/release.yml creates the Gitea release on tag v*. Asset contract unchanged (talk_<os>_<arch>[.exe] + checksums.txt). flake.nix provides package, app, overlay and devShell.
• Notifications without beeep: desktop toast via the OS's own tools (notify-send / osascript / PowerShell), exec-based like osbuiltin; initially only for quiet_mode = "toast". Cancel button: later.
• Icon and toast identity (2026-10-25): one SVG icon (speech bubble with level bars), rendered into the binary: PNG for notifications (Linux app_icon, Windows toast), .syso resources for talk.exe. On Windows the daemon registers talk's own AppUserModelID m3ta.talk under HKCU (no shortcut, no admin), so toasts appear as "talk" with the icon instead of Windows PowerShell.