I often have several agents working in different repositories at once. Their updates arrive in terminals and files. Reading every pane is an awkward way to answer a simple question: what changed, and which agent needs me?

I want a workspace I can operate by voice and hearing, while retaining the original text for precision. The first step is not a voice-controlled desktop that can do everything. It is a smaller, testable path from microphone to a confirmed request, and from an agent's written update to speech.

This is an accessibility problem, not just a convenience feature. When reading multiple terminals or tracking small changes across panes is difficult, the interface must tell me where I am, which agent is speaking, how recent its answer is, and whether my instruction was actually accepted. A pretty waveform alone will not solve that. I need a way to keep working when visual scanning is not the easiest channel, without losing the precise text that engineering work requires.

What my desktop actually looks like

These are screenshots from my working desktop, not product mockups. The dark space, compact type, tiling, and dense terminals are the interface I use to work with several agents at once. It is not designed to look like a landing page. It is designed to keep many live threads visible. The accessibility problem begins when visibility alone is not enough to remember which pane holds which agent, which result is new, and who needs a response.

In the agent grid, each pane has its own conversation and context. A model-parity investigation, a TTS bring-up, and a training task can all be active without belonging to one shared thread. A voice interface cannot simply read “the terminal.” It has to identify which agent, which project, which pane, and whether the visible words are a final result, a progress note, or an unanswered question.

The pane list makes a subtle issue visible: several panes can have nearly identical titles, and a listed order is not necessarily their physical position on screen. Selecting a pane is not the same as proving that a request was accepted by the coding agent inside it. The interface therefore needs stable semantic labels and request acknowledgments, not just more window buttons.

The goal is not to replace this desktop with a glossy dashboard. It is to add another reliable way to move through the same work: announce where I am, summarize what changed, let me request exact text when needed, and allow a confirmed instruction to reach the intended agent. The dense visual workspace and the spoken workspace should refer to the same agents and the same evidence.

My Linux workspace already has a map

My setup has three durable locations: ~/Programs for built tools and runtimes, ~/Software for third-party source, and ~/Workspace for the projects I actively change. Shell aliases p, s, and w make those locations quick to reach. LinuxUtilities is in Workspace alongside C-Kernel-Engine and antsand.com. This structure matters because a spoken interface should use names I already recognize rather than invent a second organization system.

AwesomeWM organizes the graphical desktop into screens and tags. Inside a tag, I may have a Terminator window. That window may contain tabs and several split panes. One pane can contain a running coding agent while another contains a plain shell or log. The correct spoken address therefore has multiple parts: project, screen, tag, window, pane, and agent. A raw X11 window ID is only a temporary technical handle.

The hierarchy is practical. If I hear, “CKE runtime agent, working, last update two minutes ago,” I can choose to hear the update, repeat it, or open that agent's terminal. If I hear only “window 0x00a5ac3e,” I still have to visually inspect the screen to know what will happen. Stable spoken names are part of accessibility, not mere polish.

The path we have today

Microphone -> CKE Whisper -> transcript -> human confirmation
                                      |
                                      +-> new read-only Codex conversation
                                      |       -> full text + shortened system speech
                                      |
                                      +-> saved voice note -> optional Terminator draft

Agent-written server_summary.txt -> freshness check -> system speech
                                             |
                                             +-> CKE Kokoro, if vocabulary is supported

LinuxUtilities owns desktop controls and orchestration. C-Kernel-Engine (CKE) supplies speech recognition and an experimental speech-synthesis path. The agent is a separate process: CKE does not make the decision or authorize a command.

Audio has a mundane but important role in this flow. The Sony Bluetooth headset can be in a playback profile or a microphone/call profile. LinuxUtilities' audio-profile script switches modes around capture and restores playback afterward. voice_note.py records a 16 kHz mono WAV through PipeWire, keeps the audio and transcript in a private ~/VoiceNotes/ directory, and can refuse to fall back silently to the laptop microphone. That refusal is useful: otherwise I might think the headset captured my request while a different device actually did.

Those details create a chain of custody for a voice request: selected input device, saved recording, transcript, correction, confirmation, destination, and response. If any step is missing, a successful-looking final answer can hide a wrong-device or wrong-target failure.

Speaking to an agent

The most direct loop starts a read-only conversation for a chosen repository:

python3 scripts/voice_agent.py --repo ~/Workspace/C-Kernel-Engine

The script records speech, asks CKE Whisper to transcribe it, and waits for me to inspect and confirm the transcript before sending. It then uses its own read-only Codex conversation and reads a shortened answer using system speech. The full answer remains printed. This is useful for asking about a repository, but it does not connect to an already-running agent in a Terminator pane.

For a microphone-free check, --typed exercises the conversation path, and --text 'question' --once sends one typed turn. Those modes diagnose routing independently of audio capture. They do not replace tests with my headset, room noise, and confirmation controls.

The spoken answer is deliberately summarized. That is a convenience, not an exact representation of technical text. If a response contains a path, a negative number, a command, or a failing assertion, the original must remain available. The next accessibility step is full reading with paragraph navigation and an exact mode for spelling technical tokens.

Imagine I say, “Ask the CKE agent what happened in the layer-three replay.” Speech recognition might hear “layer free.” I need to review and correct that before sending. After the answer comes back, a short spoken summary can tell me the result, but I also need “read the exact failing assertion” or “spell the path” on demand. Otherwise the voice interface becomes a filter that discards the very evidence I need to debug.

Hearing an existing agent's update

An existing agent can write a short server_summary.txt. I can ask LinuxUtilities to read that file instead of trying to scrape its terminal:

python3 scripts/read_server_summary.py --host p3 --system-voice

The reader checks freshness, with a two-hour maximum age by default. An agent should write its identity, repository, revision, time, completed checks, pending checks, and any question requiring my decision. It should replace the file atomically: write a temporary file, then rename it. That prevents a reader from catching half a summary. The summary is still an agent's report, not independent proof that a test passed.

This file-based route fits how the agents are already working. An agent can keep a readable latest-status file while its terminal contains hundreds of lines of logs. LinuxUtilities can ask, “What changed?” without guessing which lines in a terminal represent a final answer. A future history store should keep the original update, its source, and its revision, then link any spoken summary back to that original. It should not overwrite the only copy of an earlier decision.

The CKE Kokoro path is separate:

python3 scripts/speak_server_summary_cke.py --host p3 --dry-run
python3 scripts/speak_server_summary_cke.py --host p3

The dry run checks whether the text fits the pinned speech frontend before requesting synthesis. Earlier normalization could substitute words and discard punctuation to satisfy a restricted lexicon. In a technical update, changing a month, removing a minus sign, or dropping part of an identifier can change the claim. The current adapter rejects unsupported input instead of silently rewriting it.

Qwen TTS is another CKE backend under development. It may expand the range of text we can speak naturally, especially technical updates, but LinuxUtilities does not currently route to it. We should integrate it behind the same text-preservation and provenance rules, then test real phrases and latency before claiming an improvement. A model's presence in another repository is not an accessibility feature until the end-to-end path works.

For either Kokoro or Qwen TTS, the right acceptance question is not just “did inference produce audio?” It is: Can I understand the spoken output through my headset? Did it preserve -3.2 as a negative value? Can it say a filename, model name, and test result without changing their meaning? Can I interrupt, replay, and resume after a device disconnect? System speech is an honest fallback while those checks are still open.

Why Terminator needs care

Terminator is useful because it shows the real agent conversations. Existing tooling can enumerate windows and type into a selected one:

python3 scripts/voice_note.py --list-windows
python3 scripts/voice_note.py
bash scripts/test_terminator_send.sh --list

voice_note.py saves a private recording and transcript. With an explicit window target, it can type a confirmed draft; --submit also presses Enter. But a window ID is not an agent identity. Terminator can have multiple panes, and the last-focused pane might be a shell. Sending an instruction to the wrong pane is a safety and reliability failure, not a cosmetic bug.

The target-discovery script asks Terminator's D-Bus interface for window and terminal identifiers, titles, and the focused terminal. It can print or speak labels for the discovered panes. That improves orientation, but titles are not strong identities and focus can change. The pane bridge takes a different approach: a listener runs inside a specific Terminator pane and receives bounded text over a private local Unix socket. It displays the text; it does not execute it. That is safer for testing exact delivery, yet it still does not prove that a coding agent accepted the instruction.

The layers should stay separate: AwesomeWM tells me which screen and tag I am on; Terminator tells me which window and pane exist; an agent adapter must tell me which agent session is actually available; an acknowledgment must refer to a specific request. No single window title can answer all four questions.

The pane bridge can test exact text delivery, but its acknowledgment means the text appeared in the pane, not that an agent accepted or completed a task. I treat terminal injection as a secondary path. The primary workflow should eventually address a named agent through a supported adapter, report whether it is available or busy, and give each request a stable ID. If delivery becomes uncertain after a timeout, it must not silently retry a potentially accepted command.

How I want to navigate when reading the screen is hard

I want a short, consistent set of questions rather than an endless spoken terminal dump: “Where am I?”, “Which agents need me?”, “Read the next update,” “Repeat that paragraph,” and “Open this agent's terminal.” The answer should announce project and agent before content. A longer answer should be navigable by paragraph, not one uninterruptible audio file.

The planned panel has three views. A voice area shows recording level, waveform, transcript, and explicit send confirmation. An agent navigator groups stable agent names by project and reports status, age of last update, and pending questions. An activity view keeps original requests, answers, and handoffs. The waveform helps me see recording and playback position when I can use vision, but spoken status and keyboard or gamepad controls must cover the same tasks.

I also want to transfer information between agents deliberately. If the ebook agent finds a reader bug, I should be able to say, “Ask the testing agent to reproduce this finding.” The system should fetch the exact source update, quote it separately from my instruction, announce the destination, let me review the prepared handoff, and only then send it. An agent's suggestion must never silently become my authorized instruction.

The distinction between “delivery unknown” and “failed” is essential. If an agent accepted a request just before the connection timed out, automatically sending it again might launch duplicate work or repeat an action. The interface should tell me what it knows and let me decide whether to inspect, retry, or cancel.

A realistic morning with three agents

I start in the CKE tag and ask for updates. The interface says the runtime agent is working, the Qwen TTS agent has a new report, and the accessibility agent has a question. I hear the TTS report, ask to repeat its exact test command, and decide that the accessibility agent should know about its current pronunciation limitation. The handoff preview says who will receive it and shows the quoted source report separately from my instruction. I confirm; it receives a request ID. Later I can ask, “Did that handoff finish?” without hunting for the right terminal tab.

That scene is a product requirement, not today's completed implementation. Today I can record, transcribe, confirm, use a separate read-only agent loop, listen to a fresh summary with system speech, and test bounded pane delivery. Named-agent routing, persistent request history, handoff acknowledgment, paragraph playback, and Qwen TTS integration still need to be built and tested.

What the tests prove, and what they do not

The CLI foundation is in LinuxUtilities local commit 22fd1f5. Its voice and summary unit suites passed, as did shell syntax checks and the repository's commit-hook fast checks. They cover confirmation, freshness, bounded input, failure handling, and audio-file replacement. Most agent and device interactions in those tests are mocked. The tests do not prove microphone accuracy, human intelligibility, reliable live-agent routing, or daily use after a headset disconnect.

The next end-to-end exercise is concrete: visit three named agents; hear each identity and latest update; record and correct one transcript; send one confirmed request; interrupt and replay the answer; then disconnect a service and recover without losing the request. We need to measure transcription quality on my own phrases and the delay to first audible response, not just count passing unit tests.

The larger goal is not to speak into an arbitrary terminal. It is to know which agent heard me, what it said back, and how to recover when part of the path fails.

This first slice is intentionally modest. It gives us a safer voice loop and a place to attach better CKE speech backends. The larger accessible workspace needs stable project names, persistent original updates, navigable history, deliberate handoffs, and a reliable way to hear exactly what happened without depending on whichever pane happens to have focus.