I have a lot of shells open. On my AwesomeWM desktop, Terminator holds agents working on CKE, Antsand, writing, and whatever else I am investigating that day. That arrangement works for me until reading every pane becomes the work. I want to speak to a specific agent, let it finish, and hear the useful part of its answer without losing the original text.

Today I tried the loop for real with LinuxUtilities, the collection of desktop scripts I am building around this workflow. I spoke into a microphone, watched a live level bar, got a transcript from CKE Whisper, and aimed the message at a named Terminator pane. The agent was also given a unique place to write its reply for a Kokoro-powered reader. This is not a finished voice operating system. It is a small, inspectable path through the desktop I actually use.

I am not claiming to have invented talking to an AI agent. A voice command inside one client is useful. My problem is different: I move among Linux workspaces, many Terminator panes, and more than one agent harness. I want the microphone, destination check, answer file, and reader to belong to my desktop workflow rather than to whichever model provider happens to own a voice button. CKE is the speech backend I am using today; LinuxUtilities should remain useful if I change that backend later.

The route: choose a pane, record and check the transcript, confirm delivery, wait for a matching summary, then open a captioned reader. Each step has its own file or status. If one step fails, I do not have to guess which machine or model to blame.

The desktop is not the agent registry

AwesomeWM gives me predictable workspaces across my screens. Terminator gives me windows, tabs, and split shells. Inside a shell I might run Codex, Qwen, or an SSH session. That nesting is the memory palace: I know roughly where the Antsand work lives, where CKE lives, and which agents are usually in each place. But a place like "the Antsand window, Shell 5" is not an identity. Another window opens and the number can change. A pane title can change while the agent thinks. Two panes can even show the same title.

voice_terminator_targets.py --focused-window --scan-titles reads the pane headers already on screen through accessibility tooling. It also checks local processes attached to each terminal TTY to distinguish a verified Codex or Qwen process from a shell that merely has an impressive title. For delivery, the stronger handle is the Terminator pane UUID. It lasts while that pane exists, even if the window list is renumbered. If the pane closes, the request fails rather than landing in a convenient but wrong shell.

python3 ~/Workspace/LinuxUtilities/scripts/voice_terminator_targets.py --focused-window --scan-titles
python3 ~/Workspace/LinuxUtilities/scripts/voice_note.py --pane-id urn:uuid:YOUR-PANE-UUID --summary-return

The UUID is not a universal agent identity. It is a better local address for this one Terminator pane. A future workspace still needs persistent agent names and a supported adapter for each agent harness.

One Antsand request from one control shell

Suppose one local agent is working on Antsand application changes, another is editing the blog, and several CKE agents are separately investigating vision, audio, serving, TTS, ASR, and training. I am still typing and switching among them today. The practical goal is to stay in one control shell, speak a request, and choose the destination deliberately instead of hunting for the right split pane with my eyes.

1. Find: run voice_note.py --list-panes or scan the focused Terminator window. Read the title and the locally verified agent label. A title alone is not proof.
2. Address: copy the chosen pane UUID. Window and shell numbers are useful for a quick selection, but the UUID survives list renumbering while that pane exists.
3. Speak and correct: record with the selected microphone, review the Whisper transcript, and use --send-file if I need to correct it.
4. Confirm: see the exact target and words. Press Enter to submit or SEND to leave a draft. A keystroke is still not agent acknowledgment.
5. Return: the agent writes the request-matched summary; the inbox recognizes it; the Kokoro reader lets me listen and follow the full text.

Watch the whole desk, not just the voice pane

I recorded the screen for just over two minutes. The edited video is 1:52 after removing the idle opening. I kept the full desktop in view because the surrounding agents are part of the problem I am trying to solve. Play it here, or open the frames below to inspect the text at a larger size. The timestamps refer to this edit.

The screen recorder's own audio track was nearly silent. Fortunately voice_note.py saved my microphone WAV and CKE Whisper transcript separately, so I aligned the actual speech with the video above. The four frames are a readable map of what changes on screen. This is a working capture-to-agent-send demonstration, not yet a completed return-to-speech demonstration.

If I then want to ask a CKE audio agent a different question, I choose that pane's UUID and start a new request. I do not reuse the Antsand destination or assume that changing an AwesomeWM tag changed the script's target. A remote agent behind SSH may appear in a terminal, but local process verification does not establish its remote identity; that needs its own adapter before I can call it dependable routing.

The command I run, step by step

These are the commands for my current laptop, not a magic voice button. First I ask Terminator what panes exist. The numbers are from Terminator's discovered windows, not AwesomeWM workspace numbers. AwesomeWM's tags arrange my two screens; Terminator's D-Bus API lists windows and pane UUIDs inside them. The optional title scan briefly visits pane headers through accessibility tooling and restores focus. A title is a helpful label; the local process check is what lets the list say a Codex or Qwen process is actually attached to that TTY.

cd ~/Workspace/LinuxUtilities
python3 scripts/voice_terminator_targets.py --focused-window --scan-titles
python3 scripts/voice_terminator_targets.py --focused-window --agents-only
python3 scripts/voice_note.py --list-panes

For a quick test on my current default microphone, I can name a pane by its displayed window and shell numbers. A UUID is safer when I might open or close other Terminator windows. I use pactl get-default-source to select the actual PipeWire device, then check the friendly device name printed before recording. If I leave out --mic-target, the Sony headset helper tries its mic-capable call profile; --allow-default-mic permits the system mic when Sony is absent.

pactl get-default-source
python3 scripts/voice_note.py \
  --mic-target "$(pactl get-default-source)" \
  --pane-window 1 --pane-shell 5 --summary-return

# Prefer this after copying a UUID from --list-panes:
python3 scripts/voice_note.py \
  --mic-target "$(pactl get-default-source)" \
  --pane-id 'urn:uuid:YOUR-PANE-UUID' --summary-return

voice_note.py --help is the full option list. The useful branches are --audio-file recording.wav to re-transcribe a saved WAV, --send-file transcript.txt to skip capture and Whisper after editing text, --max-seconds to change the 180-second default limit, and --out-dir to choose where recordings live. --window is an older window-level typing path, not exact-pane delivery and not eligible for --summary-return. The older --submit switch changes what typing SEND means; in the current prompt, plain Enter submits, SEND types a draft without Enter, and CANCEL stops. I prefer leaving --submit off so the confirmation is unambiguous.

The script writes recording.wav, transcript.txt, and whisper.json inside a private timestamped directory under ~/VoiceNotes. The transcript is shown before sending; if a name is wrong, I can edit that saved file and use --send-file rather than repeating the recording. The live bar is computed from audio samples, not a decorative animation.

The control flow in voice_note.py is deliberately small. This is a readable sketch of the real call order, not a replacement for the source or its error handling:

record_audio(audio, selected_mic, max_seconds=180)
transcript = transcribe(audio, whisper_report, cke, model, run_dir)
transcript_path.write_text(transcript + "\n")
target = resolve_pane_id(pane_uuid)  # or resolve_target(window, shell)
record = new_request(target, verified_agent_workspace)
# Show target, words, and return instruction; wait for my confirmation.
send_to_pane(target, marked_one_line_message, submit=True)

There are three separate failure points in that sketch. Whisper may hear the wrong words, pane resolution may reject an uncertain destination, or typing may succeed without the agent accepting the task. Keeping the WAV, transcript, and ledger separate lets me see which one happened.

What happens when I speak

voice_note.py asks me to start recording. It names the selected microphone, shows the PipeWire source ID, and draws a bar from recent samples in the growing WAV file. I can see waiting for mic, silence, sound, or CLIPPING. A final line says whether the meter observed signal at any point; the last frame alone might say silence because I stopped talking before pressing Enter. The bar does not know whether my words were intelligible.

My Sony headphones can play the answer while the laptop microphone captures my request. If I want the Sony microphone, I omit the explicit --mic-target override and let the Bluetooth helper switch the headset into call mode. That has lower playback quality than music mode, so the helper releases it after recording. The label printed before capture matters: I should not have to guess whether I am speaking into the laptop or the headset.

The recording is saved locally, then the script invokes the CKE Whisper Base runner from the laptop's prepared bundle. This specific command is not sending my microphone audio to the P3 speech server. The transcript and Whisper report are retained beside the WAV. Before anything reaches an agent, I can read the transcript and decide whether to submit it, leave a draft, or cancel. When Whisper turns an unfamiliar name into something else, that is a correction problem, not a reason to pretend the transcription was exact.

In one test today, the recorded WAVs were 22.3, 35.4, and 20.5 seconds long. The corresponding Whisper runs generated 31, 20, and 44 tokens, all far below their configured 224-token cap. Some words were wrong, but the audio files themselves were not cut short. Another 20.5-second clip had more than six seconds of silence at the end. I needed to inspect the files rather than call every strange transcript "truncation."

Which computer does the speech work?

There are two CKE roles here, but only one is an HTTP service in this command path. voice_agent.py supplies record_audio() and transcribe() to voice_note.py. The former captures a 16 kHz WAV with the chosen PipeWire source; the latter launches the prepared CKE Whisper Base runner and writes the JSON report. That ASR work runs on this ThinkPad. It does not send the recording over SSH to P3. P3 runs a resident CKE Kokoro HTTP service for the answer after the agent has written its summary.

How the words reach one agent

voice_send_pane.py resolves the target and checks Terminator's D-Bus view, the X11 window, and the focused pane UUID before typing. If the pane disappears or focus shifts, it refuses to choose another destination. Pressing Enter at the confirmation prompt types the request and sends Enter to the verified pane. Typing SEND types a draft without Enter. Canceling keeps the saved transcript. The text is delivered as one line with marked [User request] and [Accessibility response instruction] sections. Literal newlines through terminal keystrokes could submit the first half of the message too early, so the preview is spaced into separate blocks while the injected text remains one safe line.

This is a keystroke delivery contract, not an agent acknowledgment. Seeing text in a pane proves neither that the harness accepted it nor that the work finished. That distinction is why the second half of the loop exists.

Getting a reply I can actually listen to

Each voice request gets an ID and a private return path under the target workspace's .codex_local/agent-replies/ directory. The path includes the pane UUID and the request ID. The instruction asks the agent to write a plain-text summary with matching Request ID: and Status: complete headers, then atomically rename its temporary file into place. LinuxUtilities keeps a separate ledger under my local state directory. agent_summary_inbox.py --list tells me which requests have a matching completed file; --wait polls without treating a missing file as proof of failure.

python3 ~/Workspace/LinuxUtilities/scripts/agent_summary_inbox.py --list
python3 ~/Workspace/LinuxUtilities/scripts/agent_summary_inbox.py \
  --id voice-YOUR-REQUEST-ID --wait 600 --listen --open

The inbox checks the request ID, completion header, file age, size, and a content hash. If the text is unchanged, it reuses the existing reader rather than synthesizing again. If it changes, cke_summary_reader.py asks the P3 CKE Kokoro endpoint over SSH for speech chunks. P3's local service reported ready: true when I checked it for this post. The reader validates the chunk IDs, WAV hashes, audio format, and source-text coverage before making a local WAV and captioned HTML page. The text stays available beside the audio. Phrase spans come from the checked chunks; word highlighting is estimated, so it should not be mistaken for exact forced alignment.

One very ordinary failure taught me something: Kokoro's checked text planner rejected a full http://localhost:8085/... link in the first spoken reply. The written answer was valid; speech was not. A shorter, speech-safe reference to the article is the workaround, while the exact URL stays in text. The right long-term fix is a tested technical-text frontend that can say URLs without silently changing their meaning. The reader should fail visibly rather than play a stale WAV or pretend a system voice is Kokoro.

I can open that reader, see the full script, and hear the answer while following along. A browser page is a practical teleprompter for now, not the final native LinuxUtilities interface. The same pattern could later sit in a small GTK window with a real pause, repeat, and paragraph navigation surface.

The lime highlight follows the current phrase from checked source spans. The smaller word highlight is estimated, not a timestamped alignment produced by the model. Previous and Next move between phrases; the speed selector changes playback speed; the fixed player keeps controls reachable as I scroll. The guide says exactly what is precise and what is approximate. I can also select a phrase in the script instead of scrubbing a waveform blind.

How this reader is built

If you are looking at the screenshot and wondering how the browser knows which words to show, it does not guess from the audio. The agent first writes a specific text file for a specific request ID. My inbox checks that file, hashes the words, and sends those same words to CKE for speech. The returned chunk records carry source-text spans. I put the original words, those spans, and the WAV filename into one local HTML reader. So the page knows the script because it was built from that exact script. Chrome just opens the result; it is not discovering a transcript by listening to the WAV.

The public LinuxUtilities Kokoro reader source and its setup guide are available now. That adapter reads complete text and controls CKE playback. The browser page in my screenshot is a newer, separate local prototype. I have not published its source yet, so I do not want someone to clone the repo expecting the screenshot to appear.

For the browser path, agent_summary_inbox.py first validates the completed request ID and hashes the summary body. cke_summary_reader.py sends that exact text over SSH to P3's loopback Kokoro endpoint. It checks the returned request IDs, WAV hashes, sample format, and phrase coverage, then assembles a local WAV. summary_script_player.py checks that the speech events cover the original text and audio in order before putting the text, cue spans, and WAV filename into an offline HTML page. The browser plays that local WAV; it does not call the model or send the transcript to a web service.

python3 ~/Workspace/LinuxUtilities/scripts/agent_summary_inbox.py --list
python3 ~/Workspace/LinuxUtilities/scripts/agent_summary_inbox.py \
  --id voice-YOUR-REQUEST-ID --listen --open

# Or, with a trusted plain-text summary and the local prototype:
python3 ~/Workspace/LinuxUtilities/scripts/cke_summary_reader.py \
  --text /absolute/path/to/summary.txt

The first command selects the exact request and reuses a reader if the source hash has not changed. The second route bypasses the voice ledger and creates a reader directly from a text file. Both browser routes currently require this local checkout, SSH access to the configured P3 host, and its running CKE Kokoro service. A clone of public LinuxUtilities alone does not yet provide this end-to-end browser flow.

What gets saved, and what does not

The transcript does not get concatenated into one giant forever-file. Each request has a fresh voice-... ID. agent_reply_ledger.py records the target pane, workspace, creation time, delivery state, summary path, and reader hash in a private local JSON ledger, replaced atomically under a file lock. The agent writes the matching reply in that workspace's .codex_local/agent-replies/ tree. agent_summary_inbox.py rejects a missing or stale envelope and only generates speech for a new content hash. The browser opens the resulting local HTML beside its WAV; Chrome is a viewer, not the archive or source of truth.

If I want to revisit this in ten years, today's local files alone are not a retention plan. I need backed-up, portable source text, request and agent identities that survive pane closure, a versioned record format, a retention policy, and a way to reconstruct a reader when CKE or the browser changes. The original text and recording matter more than keeping every generated WAV forever.

The scripts and the PR trail

I want this to become something another Linux user can try, not just a story about my laptop. For now, the LinuxUtilities repository contains the merged Kokoro reader and guide; the CKE speech work is in the CKE repository. The exact-pane voice command and browser teleprompter are still in my local checkout. My next LinuxUtilities change should package those pieces together with setup steps, tests, and replaceable host and microphone settings. Until that is merged, these commands describe my setup rather than a ready-to-install release.

Here is the script map: voice_note.py orchestrates recording and confirmation; voice_agent.py supplies capture and local Whisper transcription; voice_terminator_targets.py discovers panes and verifies local agent processes; voice_send_pane.py enforces exact-pane delivery; agent_reply_ledger.py creates request-specific return paths; agent_summary_inbox.py polls and validates completed files; cke_summary_reader.py asks P3 for speech and builds the reader; summary_script_player.py renders the text-and-audio page. The newer exact-pane and ledger files are still local work in my LinuxUtilities checkout as I write this, not yet a reproducible install from the public repository. Committing them with tests and setup instructions is the next sharing step.

That separation is the point. The desktop layer should choose a target, preserve what I said, and verify which answer came back. An agent adapter should report acceptance and completion. A speech adapter should turn the answer into accessible output without quietly changing it. Today those boundaries are implemented unevenly; the article and the linked PRs show which parts are merged and which are still being hardened.

What is proven, and what still needs a real day of use

The most recent request to write this article is itself a useful acceptance test. I should be able to ask for the article by voice, see it on localhost, get a request-matched summary, and listen to that summary. Publishing the article proves the content path. The summary file and reader are separate gates; I will not call them complete merely because a blog page loads.

I am building this because accessibility is not a layer of polish after the computer works. It changes what "works" means. A correct agent response that I cannot find, read, or safely replay is not useful to me. This first loop is imperfect, but it gives me concrete places to harden: capture, transcription, identity, delivery, response, and speech, one boundary at a time.

Where I started

The earlier visionless-workstation post explains why I wanted this in the first place. Talking to the Agents Without Losing the Thread sketches the broader agent workspace. This post is narrower: the first inspectable route through the hardware and scripts on my actual desk.