A few weeks ago I wrote about running Parakeet and Cohere Transcribe across my complete 42-minute recording. That work was real and useful. CKE converted the weights, ran native kernels, produced complete transcripts, retained token trajectories, and compared results across Intel and AMD CPUs. But Python still knew too much about how those models were assembled and executed.

That was the next architectural problem to solve. It is one thing to have a collection of correct C kernels and call them from a model-specific Python session. It is another thing for the C-Kernel-Engine (CKE) compiler to understand the circuit, resolve its providers, plan its memory, generate the C execution path, and then certify the resulting runtime end to end.

CKE has now crossed much more of that boundary for both NVIDIA Parakeet TDT and Cohere Transcribe. This does not make every audio problem complete, and it does not suddenly make these paths as optimized as Whisper. It does mean they are becoming CKE models rather than Python programs that happen to call CKE kernels.

I also repeated the complete 42:22 workload on two CPU nodes at one pinned commit. The six runs now give this article a measured baseline: Whisper Base completed in 2:01.74 on the Intel node and 1:05.25 on Ryzen; Parakeet completed in 25:15.19 and 14:41.95; Cohere completed in 21:20.99 and 9:43.02. Both machines and their roles are documented in the CKE CPU AI hardware lab. The details below explain what each number includes, what improved, and where CKE is still leaving performance on the table.

Parakeet and Cohere audio waveforms passing through declared CKE circuits into generated native C runtimes on CPUs.
The important change is ownership: model structure is moving into declared circuits, resolved kernel maps, planned memory, and generated C.

Why The Previous Work Still Mattered

The first native Parakeet and Cohere paths were not throwaway prototypes. They answered the hardest numerical question first: do we have the kernels and model arithmetic required to reproduce these architectures?

Parakeet introduced a FastConformer encoder, grouped convolutions, relative attention, an LSTM prediction network, a joint network, and a token-and-duration transducer state machine. Cohere reused part of the Conformer foundation but added its own 48-layer encoder, an eight-layer cross-attention decoder, tokenizer behavior, timestamp attention, and a VAD-bounded long-audio policy.

The earlier Python sessions made those pieces observable. We could compare intermediate tensors against independent references, fix one-thread and long-audio failures, measure word error rate on a manually curated five-minute excerpt, and process the complete recording. That evidence showed that the kernel foundation was coherent before we asked the compiler to own the complete path.

It also exposed the limitation clearly. Model-specific Python code was still deciding execution order, allocating some intermediate arrays, carrying state, and connecting outputs to the next operation. The native kernels were reusable, but the complete model composition was not yet represented by the same CKE circuit and code-generation machinery used elsewhere.

Comparison between the earlier Python-orchestrated native kernel path and the current declared circuit and generated C path for CKE audio models.
The kernels did not become less important. The compiler learned how to compose them while leaving long-audio policy at an explicit host boundary.

Parakeet: From Frontend To Generated TDT

The Parakeet migration happened in layers rather than one large rewrite.

  • PR #526 moved the Parakeet subsampling component into generated execution.
  • PR #528 generated the FastConformer blocks from the resolved circuit.
  • PR #529 completed the generated encoder schedule, including relative positions, all 24 blocks, buffer reuse, and the final 1024-to-640 projection.
  • PR #530 moved the TDT decode graph and state loop into compiler-emitted C and added a complete single-window WAV-to-text entry point.

PR #530 is the main ownership change. The Parakeet circuit now declares the ordered prediction-LSTM, decoder projection, encoder projection, joint network, token logits, duration logits, and selection operations. Code generation refuses an incomplete or invalid graph. The generated entry point initializes the decoder state, advances through encoder frames, applies the blank and duration rules, and calls only resolved provider functions.

The certification then exercises two related boundaries. An isolated generated TDT trajectory must match the retained tokens, durations, and first joint logits. Complete single-window transcription must produce the same token and duration sequence twice, reject an undersized workspace before producing output, and decode a non-empty transcript through the native tokenizer.

Long audio remains a separate policy. The small C host loads the generated model, divides a recording into bounded overlapping windows, asks the generated runtime to transcribe each window, and gives overlap words deterministic ownership using their timestamps. That split is intentional. The model mathematics belongs in the generated circuit. Window size, overlap, reconciliation, resume, and streaming policy belong at the host boundary.

Cohere: Generated Encoder-Decoder Audio

Cohere required a second path because it is not a TDT model. It uses an encoder-decoder architecture, so CKE needed to prove that the audio frontend and Conformer work could be reused without turning the compiler into a list of model names.

  • PR #533 generated the Cohere audio frontend.
  • PR #534 generated the complete Cohere Transcribe encoder.
  • PR #536 added dynamic audio-encoder memory support so real audio geometry did not depend on one fixed fixture.
  • PR #537 generated the eight-layer cross-attention decoder.
  • PR #538 certified standalone short WAV-to-text through generated frontend, encoder, decoder, and tokenizer components.
  • PR #540 extended that boundary to a persistent native long-audio session.

The standalone certifier deliberately removes Python environment variables before launching the native executable. It also inspects dynamic dependencies to reject a hidden libpython dependency. The generated token trajectory and final transcript must match the retained reference exactly.

For long audio, the native process loads the generated model libraries once, consumes every declared speech segment, resets decoder key-value and position state at the segment boundary, and retains per-segment token evidence. The segment plan is versioned and checked against the source audio. Today that plan still comes from an external pinned VAD export. CKE owns the generated transcription path, but it does not yet claim to own speech detection.

What Generated-C Compatibility Means

I use this phrase carefully because there are several layers that can otherwise be collapsed into one claim.

LayerParakeetCohere Transcribe
Native reusable kernelsCertifiedCertified
Declared model circuit and resolved providersFrontend, encoder, and TDT graphFrontend, encoder, and decoder graphs
Compiler-generated model executionSingle-window WAV-to-textShort WAV-to-text through generated components
Native long-audio hostBounded overlap windows and word ownershipPersistent session over an explicit speech-segment plan
Still outside the claimMultilingual quality, diarization, tuned performanceCKE-owned VAD, resampling, timestamps, multilingual quality, diarization, tuned performance

The DSL does not need to generate command-line parsing, WAV file management, VAD policy, or every user-interface decision for the architecture to be sound. It needs to own the model program: operations, dependencies, tensor geometry, provider resolution, state, planned memory, and calls into exact C kernels. The host should remain thin and explicit about the policy it adds around that program.

How We Know It Is More Than A Build

Compiling libmodel.so is only the first gate. The useful evidence comes from climbing a longer ladder.

CKE audio certification ladder from kernel tests and component parity through generated circuit execution, standalone native transcription, long audio, quality and performance.
Each gate answers a different question. A passing kernel test cannot substitute for complete generated-runtime evidence.

CKE now checks the circuit shape, resolved operation inventory, generated symbols, workspace geometry, repeated token trajectories, complete transcript, native dependency closure, stale runtime bundles, exact segment plans, complete source consumption, and long-audio replay. PR #541 also added coverage for the audio FP32 projection shapes that these encoders exercise.

PR #542 begins the next phase by recording generated-encoder GEMM shapes, active threads, call counts, and elapsed time. That is measurement infrastructure, not an optimization claim. It led to two measured provider changes. PR #550 stopped dividing relative attention only by eight heads and scheduled independent query rows across the worker pool. PR #551 reused each AVX-512 activation load across four independent FP32 GEMM outputs while preserving each output's reduction order.

The Complete 42-Minute Two-CPU Result

I reran all three models after those changes rather than extending a five-minute result into a full-recording claim. Every run used clean CKE commit 87b49720d and the same 42:22 mono 16 kHz PCM source: 40,672,000 frames with SHA-256 5690063a6575...10116fe. The source is my published presentation, How AI Runs on CPUs: A Beginner's Guide to Kernels, so readers can listen to the recording that produced these transcripts. Generated runtimes were compiled natively on each machine before timing. Model conversion and C compilation are therefore excluded; model loading, complete transcription, and host orchestration are included.

The machines are my Intel P3 node, an i7-14700T with 20 physical cores, 28 logical CPUs, AVX2, and 61.4 GiB of visible memory, and the Ryzen node, a Ryzen 9 9950X3D with 16 physical cores, 32 logical CPUs, AVX-512, and 150.7 GiB of visible memory. The wider lab inventory and each node's research role are recorded on the CKE CPU AI hardware page.

Model and hostWall timeSource speedAverage CPU coresPeak RSS
Whisper Base - P32:01.7420.88x real-time11.04525.4 MiB
Whisper Base - Ryzen1:05.2538.96x real-time8.68526.0 MiB
Parakeet TDT 0.6B v3 - P325:15.191.68x real-time13.596.11 GiB
Parakeet TDT 0.6B v3 - Ryzen14:41.952.88x real-time14.176.10 GiB
Cohere Transcribe 03-2026 - P321:20.991.98x source real-time6.718.91 GiB
Cohere Transcribe 03-2026 - Ryzen9:43.024.36x source real-time12.378.91 GiB
Measured wall times for Whisper Base, Parakeet TDT 0.6B v3, and Cohere Transcribe on Intel i7-14700T and AMD Ryzen 9 9950X3D CPU nodes.
Complete wall time, not a selected kernel. Lower is faster. All six runs used the same source and CKE commit; model policies remain different and are described below.

The model sizes explain part of the gap. The retained FP32 weight bundles contain 291,707,364 bytes for Whisper Base, 2,508,517,559 bytes for Parakeet, and 8,602,286,583 bytes for Cohere's encoder and decoder together. Whisper is also the most optimized CKE audio path today. It would be misleading to call this a universal model ranking or to imply that NVIDIA's published GPU throughput should automatically appear in a new CPU runtime.

The Workload Boundary Is Not Identical

Whisper consumed all 40,672,000 source frames through 92 timestamp-seek windows and emitted 9,870 tokens. Parakeet also covered the complete source through seventeen 180-second windows with 30 seconds of overlap and selected 7,225 words. Cohere consumed a pinned 123-segment external VAD plan covering 2,288.04 seconds of speech inside the 2,542-second recording and emitted 10,212 tokens. Its source-speed column uses the full recording duration; relative to admitted speech, Cohere reached 1.79x real-time on P3 and 3.92x on Ryzen.

The complete recording does not have a fully aligned human transcript, so I am not claiming full-track word error rate. The earlier manually curated five-minute fixture remains the quality comparison: 5.65% WER for Whisper Base and 7.25% for Parakeet's overlapping-window policy. WER means word error rate: substitutions, deletions, and insertions divided by the words in a human reference. This run measures execution, source coverage, and cross-host reproducibility.

That reproducibility result is strong. Parakeet produced the same 7,225-word transcript on AVX2 and AVX-512, SHA-256 2909c88f...f5ffe2c. Cohere produced the same 6,883-word transcript on both, SHA-256 b57297d5...1db370. Whisper produced the same 6,967-word transcript and 9,870-token command output on both, transcript SHA-256 910addb4...cae16.

What Actually Improved

The older retained Ryzen Parakeet run used the same source, seventeen-window policy, 16-thread setting, and 7,225-word output. It took 2,154.18 seconds. The current generated path took 881.95 seconds: 59.1% less wall time, or 2.44 times the throughput. That is the accumulated result of circuit generation, row-parallel relative attention, shared GEMM work, and the other provider changes between the two commits. It is not a claim that PR #551 alone delivered 59.1%.

For Cohere, the earlier Ryzen native runner reported 646.48 seconds and the current generated session took 583.02 seconds, a directional 9.8% reduction. I do not treat that as a controlled kernel A/B because orchestration and generated-runtime ownership changed, and the current generated trajectory contains 10,212 tokens rather than the older runner's 10,207. The controlled PR #551 five-minute A/B is narrower and cleaner: 64.383 seconds before versus a 59.205-second candidate median, with byte-identical output.

Whisper also improved on P3. The retained PR #482 persistent-worker result was 138.73 seconds; the current clean run completed in 121.74 seconds with the same 92-window, 9,870-token trajectory. That is 12.2% lower wall time across the intervening CKE work, not an isolated attribution to the two FastConformer changes.

What The Measurements Say To Optimize Next

Before PR #551, generated-runtime profiles attributed approximately 59% of Parakeet samples and 71% of Cohere samples to the shared FP32 GEMM. Before PR #550, relative attention could activate at most eight workers because work ownership followed eight heads. Those were real measured bottlenecks, and both changes retained exact arithmetic checks.

The full run now changes the question. P3 Cohere requested eight threads and averaged 6.71 CPU-core equivalents. Ryzen Cohere requested sixteen and averaged 12.37. Parakeet averaged 13.59 and 14.17 cores respectively with sixteen requested. There is still unused machine capacity, but another optimization should begin with a fresh post-#551 profile. Optimizing yesterday's largest stack frame after it has already moved is how performance work becomes folklore.

Cohere's retained phase totals make the host difference more specific. On P3, the frontend used 29.31 seconds, the encoder 1,123.02 seconds, and the decoder 124.76 seconds. On Ryzen, those phases used 20.66, 336.14, and 225.23 seconds. The Ryzen encoder is 3.34 times faster, while its decoder is slower in this run. That is a reason to profile encoder and decoder separately and sweep thread count and affinity separately, especially on P3's hybrid P-core/E-core topology.

One deployment failure was useful too. Copying the Ryzen-generated libmodel.so to P3 failed immediately with SIGILL because it had been compiled with native AVX-512 instructions. The generated C source and weight maps were portable; the compiled binary was not. Recompiling the same generated source on P3 produced the successful AVX2 run. CKE should treat generated source as portable and generated native libraries as ISA-bound artifacts with explicit provenance.

The canonical high-level Whisper command exposed a separate regression on both hosts after compilation: Python could not pickle _persistent_worker_main because the dynamically loaded ck_v8_audio_runtime module was not importable by a spawned worker. The lower-level generated runtime completed normally. This is a front-door integration bug, not a numerical failure, but it needs a non-skippable spawn-path test because a user should not have to know the internal runner.

What This Says About CKE

This is the type of progression I want from CKE. A new model first reveals missing math. We add or reuse kernels, verify them independently, declare the circuit, map weights and providers, lower it through the DSL, inspect the generated C, and certify the complete path. The first implementation may be slow and may still have a reference session around it. That is acceptable as long as the boundary is honest and the next architectural step is clear.

Parakeet and Cohere are useful because they are different enough from Whisper to test whether the architecture actually composes. The resulting changes were not two giant hard-coded model branches. They added reusable audio frontend, encoder, decoder, dynamic-memory, state, and native-session capabilities that another model family can exercise.

There is still cleanup to do. Some public documentation describes the older Python-owned boundary and needs to be updated. Performance is not where I want it. VAD remains external for Cohere. Diarization is a separate model and timeline problem. Quantized audio paths, broader multilingual evidence, resampling, cancellation, and stronger concurrent-session tests remain future work.

But the ownership direction is now much better. The earlier article proved that CKE kernels could run these models on real recordings. This work moves the model itself into the compiler.

Implementation Trail

Continue Reading