For anyone new to the project, C-Kernel-Engine (CKE) is an open-source CPU runtime and kernel compiler for transformer-based language, vision, and audio models. A model is described as mathematical circuits, resolved against explicit kernel contracts, and lowered into generated C that is compared with reference implementations such as PyTorch, llama.cpp, and whisper.cpp. The code lives at github.com/C-Kernel-Engine.

This article is the sequel to two earlier posts. Teaching C-Kernel-Engine To Listen walked the Whisper Tiny circuit kernel by kernel — the bring-up. The July engineering recap then told the first performance chapter: a 49.15x gap to whisper.cpp on the first controlled benchmark, narrowed to roughly 2.7x by the end of July. If you want the conceptual introduction to audio transformers — encoder paths, timestamps, why audio is not a VLM patch — that is How Audio Transformers Work. This post covers exactly what landed after the recap closed: August 1–2, pull requests #280–#282 on Whisper speed, #286–#290 on evidence, #283/#284 on the native session boundary, and #291 on moving Whisper execution policy out of converter branches and into declarative model maps.

What We Measured

Everything performance-related in this article is one controlled lane: Whisper Base on the 11-second JFK fixture, transcribed on the commodity lab node — a 14th Gen Intel Core i7-14700T (AVX2/FMA/AVX-VNNI, no native BF16 or AMX) running minimal Ubuntu Server, profiled with VTune. The optimized lane stores encoder linear weights as real IEEE FP16 through the opt-in --encoder-linear-weight-dtype fp16 contract; the FP32 provider path is unchanged and remains the default.

Where July Ended

The first controlled Whisper benchmark, recorded in the July recap, was deliberately unflattering: CKE at 26.751 seconds of compute against 0.544 seconds for whisper.cpp on the same fixture — a 49.15x gap, with all three backends producing the same 25 transcript tokens. That baseline isolated the generated decoder and attention execution as the real problem, and it stayed on the books as the yardstick. Through the end of July, profiling, packed-K attention, persistent-thread scheduling, Conv1D parallelism, oneDNN integration, and validated FP16 encoder storage reduced the exact CKE hybrid path to 1.53–1.58 seconds, against a recorded whisper.cpp FP16 result of 0.567 seconds.

Two late-July changes set up everything in August and are worth one paragraph each. PR #278 pinned OpenBLAS and MKL workers to one thread inside Whisper execution — the generated runtime was competing with its own BLAS pool for cores — and parallelized the exact encoder kernels. PR #279 added validated FP16 encoder storage, and its real contribution was a contract, not a dtype: the converter can now label a tensor FP16 only when it actually serializes IEEE FP16 payload bytes. A manifest that disagrees with its payload fails closed. Metadata alone is not evidence.

Rung 1: FP16 Attention And Scheduling (PR #280)

With FP16 storage in place, VTune named the next constraint precisely: Whisper Base FP16 was using about 3.3 effective physical cores, and the encoder was materializing a complete attention score matrix per layer. Two changes landed together:

A tiled FP16 K/V online-softmax attention provider, selected by the circuit — not by a compiler branch — replaced the full-score implementation. The provider measured 15.58 ms per encoder layer against 26.87 ms for the prior full-score path. Topology-aware scheduling derived performance-core affinity from Linux SMT sibling data (thread_siblings_list) instead of hardcoded CPU IDs, while still honoring an explicit CK_NUM_THREADS override and recording the resolved worker CPUs in the end-to-end report.

The same-host result: the JFK fixture improved from repeated 1.53–1.58 second runs to 1.25–1.29 seconds, the 25-token transcript remained exact, retired instructions fell from 48.71 billion to 42.79 billion, and effective physical-core utilization rose from 3.31 to 3.89 cores. And the engineering note immediately recorded the new queue: "FP16 GEMM is now the largest remaining hotspot, followed by Conv1D and tiled attention."

Rung 2: FP16 GEMM Weight Reuse (PR #281)

The FP16 encoder GEMM was streaming the same weight rows independently for every prompt row — each worker re-read weights that its neighbor had just touched. The fix is an AVX2 tile that computes four prompt rows by two output rows together, reusing each loaded FP16 weight pair across four activations. The arithmetic contract is stated explicitly: the eight-lane FMA chain and the horizontal-reduction order are preserved independently for every output, and the tile is compiled out on AVX-512 so it cannot alter that ISA's 16-lane reduction contract.

The evidence is the part worth reading twice. Full production-shape baseline comparisons are byte-exact for Tiny, Base, and Small projections and MLPs, and the independent llama.cpp production-graph oracle remains bit-exact at 1, 16, 20, and 24 threads — thread count is part of the tested contract, not an afterthought. Isolated provider benchmarks measured 1.28–1.70x speedups; alternating same-library end-to-end runs improved the encoder by about 4 percent for Tiny, 5–6 percent for Base, and 7 percent for Small, with unchanged transcripts. VTune attributes 97.6 percent of the provider-only CPU time to ck_gemm_f16_input_fp16_work, which is what makes the claim attributable rather than atmospheric.

Rung 3: Killing The Stride-Two Gather (PR #282)

Whisper's second encoder stem convolution reads every other input frame — stride two. The AVX2 implementation was issuing an indexed gather for every input channel, kernel tap, and frame tile: the most expensive way to ask the CPU for strided data. The optimized provider loads two contiguous eight-float vectors and compacts their even lanes deterministically, then executes the unchanged per-lane FMA sequence. Boundary, non-AVX2, and generic-stride paths keep the gather/scalar handling.

On the full Base stem shape — 512 input channels, 512 output channels, 3000 frames in, 1500 out — the new provider is byte-identical to the gather implementation, and the corrected alternating benchmark (provider selection resolved once before dispatch, not inside the inner loop — a benchmark-hygiene fix in itself) improved the median from 47.40 ms to 23.54 ms, roughly 2.0x. End to end, alternating FP16 encoder runs improved from 0.899/0.907 seconds to 0.883/0.871 seconds — about 2.9 percent — with the same transcript. The note is careful not to claim more: a disposable test runtime had replaced the original oneDNN-linked engine, making decoder-prefill timing non-comparable, so no whole-run speedup is claimed from that experiment.

The Evidence Discipline: The Other Half Of The Two Days

Speed claims are cheap. The same two days also landed five pull requests that made CKE's evidence system stricter about what is allowed to count as a result — and this half matters more, because it is what separates an engineering measurement from a benchmark screenshot. Three refuse bad evidence, one makes scheduling comparisons explicit, and the fifth moves neutrality into the nightly gate. Taken together they say: the system must be able to tell you no.

The next gates use Qwen3-VL, Qwen3.6, and Gemma rather than Whisper. They belong here because CKE does not maintain a separate standard of evidence for each model family. The same native-session, scheduler, corpus, tokenizer, and X-Ray machinery decides whether an audio, language, or vision result is admissible. A gate that catches an empty Gemma response or observer-induced Qwen drift is the same kind of gate that prevents a fast Whisper transcript from being accepted for the wrong reason. These are cross-runtime controls, not additional Whisper speed claims.

The roofline we refused to publish

The engineering note records a quiet, important sentence: Intel Advisor 2026.0 could not ingest the Python-driven Whisper trace on this host — it reported _advi_dynamic_regions_table followed by no data — so no Advisor roofline claim is made. That is the whole story, and it is the point. The Python runner orchestrates artifacts, backend rotation, and reports; it must not be the sampled process for a roofline claim, because Python startup, NumPy, and subprocess plumbing obscure the generated C runtime you are actually trying to measure.

Instead of a shiny wrong diagram, the note defines the native profiler boundary as future work: a small audio-specific executable that loads the generated encoder and decoder libraries once, reads the WAV in C, calls ck_model_run_audio_wav_window directly, runs warmups before measurement, exposes separate encoder / decoder-prefill / decode / provider-only regions, and emits the runtime hashes, compiler flags, thread affinity, transcript tokens, and phase timings. VTune and Advisor attach to that process. Until it exists, the roofline section stays empty. Claims made must be traceable; claims that cannot be traced are refused.

Capture Neutrality: Is This Tensor Dump Actually Evidence? (PR #289)

CKE X-Ray explains numerical differences by dumping tensors at circuit checkpoints. But a checkpoint export is work inside the observed process — environment checks, memory copies, path formatting, file opens and writes between model operations. That work changes timing and cache state. Correct deterministic kernels should still return the same values; a race, an unsafe scratch-buffer lifetime, an uninitialized read, or a scheduling-dependent reduction can become visible only when instrumentation changes timing. Which raises an uncomfortable question: is a captured tensor evidence of the model's behavior, or of the observer's presence?

PR #289 turns the answer into a contract. A captured tensor is evidence only if the captured execution reproduces an uncaptured execution on the same causal token history. The harness now enforces an acceptance sequence automatically whenever capture is requested:

Two design details deserve emphasis. The comparison is bit-exact — every float32 logit, including signed zero and NaN payloads — because a numerical tolerance could hide exactly the small perturbation a deep recurrent model later amplifies into a flipped token. And the fallback is deliberately unavailable for KV-cache and binary-parity captures, whose coupled state cannot be reconstructed by blindly splitting an operation list. The contract knows its own limits.

PR #290 closes an operational gap in that contract: capture-neutrality tests now run in the nightly X-Ray lane rather than depending on someone remembering to invoke a focused test. It does not add a new numerical claim. It makes the existing rule — rejected observations must never be promoted as evidence — part of the recurring gate.

Corpus Parity, BOS Ownership, And Scheduling Gates (PRs #286–#288)

Certify the native binary, not just the Python harness (#286). The private Qwen3-VL corpus compared Python CKE with llama.cpp but never independently certified the native CLI — and completed numerical divergences were being misfiled as infrastructure errors. The fix adds atomic native token traces, requires a three-way native CLI / Python CKE / llama.cpp comparison, and — the cultural change — preserves completed divergence evidence instead of discarding it. A fresh production image matched 8/8 pre-EOS token IDs across all three runtimes; a separate image-25 run kept its real step-6 CKE-versus-llama.cpp divergence on the record as a divergence, where it can be investigated, rather than swept into "the test flaked."

Honor tokenizer BOS ownership (#288). The generated native chat formatter was prepending the template's textual BOS even when the generated tokenizer had already inserted one. Gemma received two BOS markers and terminated at immediate EOS — and the scheduler matrix then recorded that zero-token execution as passing, because it had prefill timing and exit code zero. The fix resolves formatting against declarative add_bos_token metadata, and the gate now fails closed on zero decoded tokens with an actionable failure_reason. A fresh Gemma 3 270M runtime went from immediate EOS to a coherent native reply. A benchmark that cannot detect an empty answer is not a benchmark; now it can.

Balance GEMM tiles dynamically — and prove the text did not change (#287). Static tile ownership leaves fast workers waiting on slow ones on hybrid P/E-core CPUs. CKE now has a typed auto/static/dynamic scheduling ABI, exposed as --gemm-schedule across the native CLI, Python chat, parity, certification, and benchmark tools — no scheduler environment variables. The alternating 40-image matrix measured 48.95 versus 53.51 seconds per image (8.4% faster by mean, 9.0% by paired median), with generated text matching on all 40 images, and a typed-auto smoke matching 12/12 pre-EOS tokens across native CLI, Python CKE, and pinned llama.cpp. Ordered reductions and split attention keep their existing schedules — dynamism is applied only where the contract allows it.

Where The Gap Actually Stands

Honest bookkeeping, because this is where performance writing usually goes dishonest. whisper.cpp is still faster. The recorded whisper.cpp FP16 result on this fixture is 0.567 seconds; the best CKE full-fixture number after these rungs is 1.25–1.29 seconds, with separately measured FP16 encoder-phase runs now around 0.87 seconds. Those two numbers are not the same measurement boundary — the 0.567 is a complete whisper.cpp run, the 0.87 is CKE's encoder phase — and comparing them directly would be exactly the kind of claim this article just spent two sections refusing to make. The durable statement is narrower and stronger: on the controlled alternating lane, each named provider got measurably faster, the transcripts stayed exact, and the remaining gap has a named home (decoder and scheduling, with an AVX-512 FP16 GEMM certification still open).

The scope limits are unchanged and worth restating: this is one Base fixture, not a broad audio corpus. Whisper Medium and Large are not certified. FP16 is opt-in, not default. The AVX2 tile does not touch AVX-512 behavior. Corpus coverage — clean, noisy, multilingual, timestamped, and long-form audio — is the certification work that would let FP16 become a default policy, and it is explicitly listed as remaining work, not implied as done.

What Comes Next

The infrastructure for the next chapter already landed in the same two days. PRs #283 and #284 added a generated multimodal session ABI and exposed a native session FFI — the beginnings of CKE as an embeddable runtime rather than a Python-orchestrated experiment, and the natural home for the native profiling executable the Advisor episode demanded. The experimental Responses schema (#272) sits on the same path: wire contract first, native execution design second.

PR #291 also removes a quieter form of technical debt. Whisper precision routing and encoder/decoder artifact ownership were still selected through architecture-name branches in the safetensors converter. They now resolve from model-map fields such as runtime_config_by_linear_weight_dtype and ignored_source_tensors. The arithmetic is unchanged, but a future audio encoder can select the same FP16 attention and topology policy through data instead of another hardcoded if whisper branch. That is exactly how CKE's DSL becomes more mechanical as model support grows.

On the numerical side, the capture-neutrality finding reframes August's biggest open item. The Qwen3.6 1K-context drift that keeps optimized batched prefill behind CK_V8_FORCE_BATCHED_PREFILL=1 — documented in How CKE Made Qwen3.6 Prefill 2.6x Faster — is now known to include run-to-run nondeterminism on the production path itself, first visible at step 76 under 24 threads. Closing it means isolating a parallel reduction, a recurrent-state update, or a scratch-buffer race — with a harness that can now prove whether the observer is innocent before anyone blames the math.

That is the real takeaway of these two days. The ladder — attention, then GEMM, then Conv1D — is the theory of constraints doing what it always does. The discipline — fail-closed storage, a refused roofline, neutrality gates, preserved divergences, empty-output detection — is what makes the ladder's numbers worth believing. A runtime that can make Whisper fast is useful. A runtime that can tell you no is the one you can build on.

Related Notes And Further Reading

• What CKE Built In July 2026 — the recap this post continues from, including the 49.15x → 2.7x Whisper chapter.
• Teaching C-Kernel-Engine To Listen — the Whisper Tiny circuit, kernel by kernel.
• How Audio Transformers Work — the conceptual prequel.
• How CKE Made Qwen3.6 Prefill 2.6x Faster — the sister performance story on the language side.
• How CKE X-Ray Found Qwen3.6's First Bad Circuit — X-Ray as the investigation surface.
• What Numerical Parity Actually Requires — the parity method behind every contract cited here.
• What Is A CPU PMU? — how perf, VTune, and Advisor measure what a processor actually did.

The code is at github.com/C-Kernel-Engine — the engineering note behind the Whisper numbers is docs/notes/WHISPER_FP16_TOPOLOGY_ATTENTION_2026-08-01.md, and the capture-neutrality contract is docs/notes/xray_capture_neutrality.md. The documentation lives at c-kernel-engine.github.io/C-Kernel-Engine. The engineering Discord is open at discord.gg/J7wNx8xS — bring questions, skepticism, or a fixture you want to see run on CPUs.