CKE made meaningful progress today. Not because another model name appeared in a compatibility table, but because the evidence became harder to misread and several measured CPU bottlenecks became smaller.
The work covered three connected problems. First, a nightly report needed to prove which capabilities actually ran on the current commit. Second, a Qwen3-VL failure turned out to be a validation-harness error rather than a broken vision model. Third, the 42-minute audio recording from my own kernel-engineering video became a practical Whisper workload for finding and removing serial CPU work.
The result is a better version of the same C-Kernel-Engine idea: explicit circuits, generated C, independently checked numerical boundaries, retained real workloads, and optimization only after measurement.
A Green Nightly Must Say What Actually Ran
PR #474 changed the nightly dashboard from a list of suite outcomes into current-run capability evidence. Every report can now bind its result to the commit, event, capability-manifest hash, scheduled case inventory, and summary. Capabilities render as PASS, FAIL, ERROR, TIMEOUT, or NOT_TESTED.
That final state is important. A skipped test is not a pass. Historical evidence from another commit is not a current pass. A component-level kernel test is not automatically full-model certification. The verifier now rejects stale commits, changed manifests, missing required cases, malformed summaries, and incomplete current-run evidence.
PR #472 applied the same discipline to the public README. Qwen3.8 Flash Next and Gemma3 now have explicit tested artifact and context boundaries, while open gates remain visible. Documentation should help someone reproduce a result, not turn partial evidence into a larger claim.
The Qwen3-VL Model Had Not Regressed
The scheduled Q8 vision lane reported a failure. The easy response would have been to loosen its tolerance or start changing kernels. PR #475 instead traced the evidence path.
The parity harness addressed an activation buffer without applying the generated arena's base alignment. A four-byte offset was enough to compare the wrong values. The harness could also load stale llama.cpp libraries, creating another way for a current CKE runtime to be measured against an unintended oracle bundle.
After correcting arena addressing and fingerprinting the active oracle libraries, cosine similarity improved from 0.990852 to 0.999291, RMSE fell from 0.035114 to 0.009780, and the gate passed. More importantly, the corrected frontend capture matched all 3,096,576 im2patch values exactly.
This was a harness repair, not a model-kernel repair. CKE now preserves nested parity failures, records the scheduled phase and thread boundary, and uploads the matching artifacts. The lesson is broader than vision: an oracle result is only useful when buffer extent, offset, library identity, model identity, and execution phase are part of the evidence.
The Video Audio Became A Real CPU Workload
I recently used CKE's Whisper path to transcribe the video How AI Runs on CPUs: A Beginner's Guide to Kernels. That first practical use found compiler and audio-tail bugs. The repaired five-minute excerpt then became a nightly regression fixture. Today the complete 42:22 recording became the profiling workload.
The P3 is an Intel Core i7-14700T system with 20 physical P/E cores and 28 logical CPUs. The recording contains 2,542 seconds of mono 16 kHz audio. The retained PR #480 comparison used 92 windows; the fresh current-main run below used 95 timestamp-seek windows under its current generation behavior. This is long enough to expose repeated frontend work, serial decoder regions, thread-pool overhead, cache behavior, and resource-policy failures that a ten-second sample can hide.
Four Bottlenecks, Four Focused Repairs
1. Compute The Long-Audio Frontend Once
The old runner recomputed the complete-recording STFT and globally normalized log-Mel frontend for every timestamp-selected window. PR #476 generated a circuit-derived full-feature entry point, computed the feature tensor once per request, and sliced exact hop-aligned windows for the unchanged encoder.
On the five-minute fixture, frontend time fell from 9.16 seconds to a 0.86-second median. Wall time fell from 37.26 seconds to 29.04 seconds, a 22.1% reduction. All 974 generated token IDs, 11 source-window boundaries, feature hashes, and encoder hashes matched the retained baseline, while the five-minute certification remained at 5.65% WER.
PR #477 then hardened the resource boundary. The runner probes cache capacity before materializing the full feature tensor, falls back only for unsupported runtimes or storage exhaustion, keeps unexpected frontend failures fatal, exposes --temp-dir, and reports whether and why reuse was disabled. A fast path that silently fails on an older runtime is not a robust optimization.
2. Parallelize Exact ERF GELU
Profiling showed that Whisper-base spent roughly half of its encoder time evaluating independent FP32 ERF GELU elements serially. PR #478 added a thread-pool provider that partitions disjoint output tiles without changing the arithmetic within an element.
Real-size isolated kernels improved by 12.17x to 13.60x at 20 threads and remained byte-exact. On the five-minute fixture, encoder time fell from 10.45 seconds to a 5.56-second median and wall time fell from 29.04 to 24.33 seconds. The retained full recording's encoder time fell from 87.56 to 46.55 seconds in that PR's measurement.
3. Stop Sending Large Single-Row Projections To A Serial Leaf
Whisper decode has many M=1 projections. Treating every single-row matrix operation as too small for threading left large MLP and vocabulary projections serial even though output rows were independently owned. PR #479 routes sufficiently large FP32 work through the persistent CKE thread pool while retaining small operations on the serial path through a measured 512 KiFMA threshold.
Five-minute decoder generation fell from 12.673 to a 9.871-second median. The complete-track measurement fell from 128.635 to 99.556 seconds for decoder generation. Eight production oracle shapes remained bit-exact across 1, 16, 20, and 24 threads.
4. Partition Decode Cross-Attention By Head
The next bottleneck exposed a shape problem. Whisper cross-attention decode has one query row, so a provider partitioned by query rows could activate only one worker. It still had eight independent attention heads. PR #480 introduced a decode-only provider that partitions those heads while preserving the serial arithmetic within each head.
Median five-minute decoder generation fell from 9.760 to 6.458 seconds, a 33.8% reduction against that stage's retained baseline. On the complete recording it fell from 99.556 to 65.673 seconds, or 34.0%. That PR path completed all 42:22 of source audio in 3:22.40 wall time, averaged 739% process CPU, used 525 MiB maximum RSS, and incurred no major page faults or swaps.
The Combined Current-Main Run Took 3:10
After the changes merged, I ran a clean benchmark from CKE commit 4a433162. The generated Whisper-base encoder and decoder were warmed and provenance-verified first, so the measured full-track command includes model loading and audio orchestration but excludes one-time model conversion and C compilation.
The 42:22 recording completed in 3:10.40 wall time, equivalent to 13.35x real-time throughput. The process averaged 953% CPU, reached 523 MiB maximum resident memory, performed no swaps, and exited successfully. CKE generated 10,102 tokens across 95 segments and consumed all 40,672,000 source frames, ending exactly at 2,542.0 seconds.
This is about 12 seconds faster than the earlier PR #480 path, but it is not a strict isolated 6% kernel comparison: current main contains the combined merged changes and its run produced a different window and token count. The defensible claim is that the complete current runtime now performs this specific warm transcription in approximately three minutes, not that one additional kernel alone caused the entire difference.
PR #482 Removed The Repeated Worker Lifecycle
The 191 initialization messages in that clean run identified a concrete orchestration problem. The long-audio runner was loading encoder and decoder weights and creating native thread pools for every timestamp-selected window. PR #482 now keeps one isolated encoder worker and one isolated decoder worker alive for the complete transcription request.
The important engineering detail is what persists and what does not. Immutable weights, prepared constants, tokenizer state, the full-feature mapping, and native thread pools remain resident. Before every decoder window, CKE calls ck_model_kv_cache_reset(). It then binds the new encoder memory, which invalidates persistent cross-attention K/V state. The old --worker-lifecycle per-window mode remains available as a same-runtime diagnostic control, and workers have bounded response deadlines plus terminate/kill cleanup if graceful shutdown fails.
In the controlled P3 A/B, both paths used the same Whisper-base generated runtimes, 16 kHz mono source, timestamp decoding, greedy generation, and a 256-token limit per window. Persistent workers matched all 9,870 token IDs, 92 feature hashes, 92 encoder-output hashes, window boundaries, timestamp events, stop reasons, and transcript text. Wall time fell from 171.60 to 138.73 seconds, a 19.2% improvement, while time outside the named frontend, encoder, prefill, and decode phases fell from 32.75 to 2.14 seconds.
That is approximately 18.3 times real time for this retained 42:22 workload. It should not be compared directly with the earlier 3:10.40 run as though only worker lifecycle changed: that run used the current command's 128-token per-window setting and produced 10,102 tokens over 95 windows, while PR #482's controlled A/B used 256 tokens and produced 9,870 tokens over 92 windows. The valid performance claim comes from the PR's same-runtime A/B, where every retained output boundary matched.
The lifecycle improvement also reproduced on the Ryzen 9 9950X3D five-minute fixture: 11.708 seconds per-window versus 9.490 seconds persistent, with all 1,126 tokens and intermediate evidence matching. That test deliberately reused the same AVX2 binary to isolate lifecycle behavior; it is not a native AVX-512 Ryzen result. Full-track results for other Whisper sizes and native Ryzen provider performance remain open.
Try CKE Whisper On Your Own Recording
This is now a practical CKE workflow rather than only a benchmark. I recorded the presentation through my ATEM Mini setup, selected the microphone track I actually used, normalized the recording, and passed it through CKE's generated Whisper-base runtime. An ATEM recording may arrive as an MP4 or MOV with embedded audio; another recorder may give you MP3, M4A, FLAC, or WAV. CKE deliberately accepts one narrow input contract, so the first step is to convert any of those sources into mono 16 kHz signed 16-bit PCM WAV.
sudo apt install ffmpeg jq
mkdir -p "$HOME/cke-audio/results"
# Use this form when the ATEM recording is an MP4 or MOV.
ffmpeg -y -i /path/to/atem-recording.mp4 \
-vn -ac 1 -ar 16000 -c:a pcm_s16le \
"$HOME/cke-audio/input-16k-mono.wav"
# Or normalize an MP3 with the same audio contract.
ffmpeg -y -i /path/to/recording.mp3 \
-ac 1 -ar 16000 -c:a pcm_s16le \
"$HOME/cke-audio/input-16k-mono.wav"
ffprobe -v error \
-show_entries stream=codec_name,sample_rate,channels,duration \
-of default=noprint_wrappers=1 \
"$HOME/cke-audio/input-16k-mono.wav"This preprocessing is not speech recognition. FFmpeg decodes the recording, -vn discards the video stream, -ac 1 selects one output channel, -ar 16000 resamples it to 16 kHz, and pcm_s16le stores uncompressed little-endian PCM samples. Normalizing the input once gives the model and the regression harness the same deterministic audio representation.
From the root of a current CKE checkout, this command selects OpenAI's Whisper Base checkpoint, creates a reusable run directory, generates and compiles the CKE encoder and decoder, and transcribes the WAV:
RUN_DIR="$HOME/.cache/ck-engine-v8/models/whisper-base-local"
CK_NUM_THREADS="$(nproc)" OMP_NUM_THREADS="$(nproc)" \
version/v8/scripts/cks-v8-run audio hf://openai/whisper-base \
--run "$RUN_DIR" \
--wav "$HOME/cke-audio/input-16k-mono.wav" \
--language en \
--task transcribe \
--timestamps \
--max-tokens 448 \
--output "$HOME/cke-audio/results/transcript.json" \
| tee "$HOME/cke-audio/results/transcript.stdout.txt"
jq -r '.decoder.transcript_text' \
"$HOME/cke-audio/results/transcript.json" \
> "$HOME/cke-audio/results/transcript.txt"
jq '{status, windowing, worker_lifecycle, timing}' \
"$HOME/cke-audio/results/transcript.json"The first invocation may download the safetensors checkpoint, convert it, generate C, and compile the encoder and decoder. Later runs reuse provenance-verified generated artifacts unless you request a forced rebuild. I used Whisper Base because it was small enough to iterate on quickly while still producing useful English transcription on my CPU. CKE currently documents generated FP32 Whisper Tiny, Base, and Small paths; model quality, memory use, and runtime will differ by artifact and machine.
In Whisper, the frontend is the signal-processing path before transformer inference: waveform framing, short-time Fourier transform, power spectrum, Mel filter-bank projection, logarithm, normalization, and feature layout. CKE lowers those steps as seven circuit operations into ck_model_run_audio_wav; Python coordinates the request and worker lifecycles rather than secretly replacing the generated frontend math. The CKE v8 runbook documents the command and artifact layout, the Whisper implementation page exposes the model stages and numerical evidence, the model/kernel matrix shows current provider coverage, and CKE concepts explains how kernels, circuits, lowering, code generation, and runtimes fit together.
Did The Transcript Match What I Meant?
Mostly, yes, but not word for word. I tend to elaborate, repeat myself, and move away from the prepared speaker notes while recording. The prepared introduction and slide notes contained 1,725 normalized words, while the CKE transcript contained 7,139 words. An ordered comparison found 81.5% of the speaker-note words in the transcript, and every major planned section appeared in the intended sequence: activations, GEMM and GEMV, SIMD, transformer structure, RMSNorm and SwiGLU, attention, KV and recurrent state, quantization, prefill and decode, roofline reasoning, CKE architecture, and the contributor invitation.
That 81.5% is a speaker-note coverage measurement, not WER. The recording is not a verbatim reading, so treating every improvised phrase as a recognition error would be misleading. The separate five-minute fixture has a manually curated reference and remains the proper accuracy gate at 5.65% WER.
The raw transcript was useful for reviewing content, locating sections, finding pauses, preparing captions, and supporting the video-editing workflow. It was not publication-ready without correction. Technical names were the predictable weak point: examples included "colonels" for kernels, "GEMB" for GEMV, "swigloo" for SwiGLU, "RMSNOM" for RMSNorm, and "SQL engine" for CKE. A project glossary, constrained terminology repair, and final human review are therefore part of the practical pipeline. They should correct spelling and names without inventing speech that is absent from the recording.
Why The Exactness Evidence Matters
The optimization target was not merely a plausible transcript. Across the complete-track comparison, all 9,870 token IDs, the transcript, all 92 source-window boundaries, every feature hash, and every encoder-output hash matched the retained output-parallel baseline. The head-parallel attention provider also survived an 8,192-dispatch replay with poisoned output buffers and matched the serial provider bit exactly.
That protects against several tempting mistakes: changing reduction order to gain speed, accepting stale generated libraries, comparing different token limits, skipping the final partial window, or letting an old buffer value survive because a threaded worker did not write its complete output range.
There is still an important quality boundary. The five-minute fixture has a reference transcript and a retained WER gate. The complete 42-minute recording does not yet have a fully curated reference transcript, so it proves complete source consumption and identity against the retained baseline, not an independent full-track WER claim.
There Is Still Considerable Optimization Room
The current-main run averaged 953% process CPU, or roughly 9.53 logical-CPU equivalents on a 28-logical-CPU hybrid processor. That does not mean every phase should sustain 2,800% process CPU: autoregressive token dependencies, small kernels, synchronization, I/O, and serial orchestration impose real limits. It does show that the machine is not uniformly saturated.
The earlier log recorded 191 thread-pool initializations across 95 audio windows. PR #482 has now removed that repeated lifecycle from the default persistent path and added schema-v5 lifecycle and orchestration evidence. The next profiler pass should therefore measure what remains inside encoder transfer, per-window file transport, decoder kernels, barriers, and hybrid-core scheduling rather than continuing to attribute the old setup cost to model arithmetic.
The next work should be measured rather than guessed:
- Repeat the clean current-
mainbenchmark to establish run-to-run variance and retain it as a reproducible performance lane. - Attribute time and worker occupancy by frontend, encoder operation, decoder prefill, decoder self-attention, decoder cross-attention, MLP, vocabulary projection, and window orchestration.
- Measure thread-pool wakeup and barrier costs instead of assuming more workers always help.
- Compare P-core, E-core, and mixed-core pinning on the i7-14700T.
- Inspect memory bandwidth, cache misses, NUMA policy where relevant, and per-kernel arithmetic intensity.
- Preserve token IDs, timestamp monotonicity, feature hashes, encoder hashes, and the five-minute WER gate after every optimization.
The objective is not a screenshot showing every CPU bar at 100%. The objective is lower wall time and better throughput per watt without weakening the numerical or provenance contract.
The Server Gate Is Now Retained
PR #473 is now merged. It adds a bounded localhost E2E test that starts the generated Qwen3-0.6B server, waits for health, streams a Responses API request, validates the event sequence and stored response, then proves process shutdown and port cleanup.
I corrected that PR's metadata before merge as well. The commit message was already valid; the PR description lacked the required evidence, validation, regression, and documentation sections. Keeping that distinction matters because metadata gates should identify the actual missing evidence rather than encourage unnecessary history rewriting.
What Today Says About CKE
CKE is slowly moving from a compiler research project that can run models into a tool I can use in my own workflow and then improve from the resulting evidence. The YouTube recording found the Whisper regression. The retained audio became the nightly fixture. The complete recording became the performance workload. Profiling found serial regions. Each repair added an explicit provider, contract, oracle, report, and regression gate.
The vision fix tells the other half of the story: hardening does not always mean changing model math. Sometimes it means proving that the evidence harness is reading the correct byte from the correct library on the correct commit.
This is the kind of progress I want from CKE. More models still matter because they expose new kernels and compositions. But practical workloads, trustworthy reports, and retained numerical boundaries are what turn those compatibility claims into a runtime that can eventually support more ambitious CPU inference, distributed execution, and training research.
Implementation And Evidence
- PR #473: generated server localhost E2E gate
- PR #474: current-run capability evidence
- PR #475: Qwen3-VL parity-harness repair
- PR #476: request-scoped long-audio frontend reuse
- PR #477: frontend cache and temporary-storage fallback
- PR #478: exact parallel ERF GELU
- PR #479: large FP32 decode output parallelism
- PR #480: decode cross-attention head parallelism
- PR #482: persistent Whisper request workers
- Long-audio frontend reuse report
- Parallel ERF GELU report
- Decode output-parallel report
- Decode cross-attention report
- Persistent-worker validation report