I opened our nightly test report this morning and caught myself doing the thing I always do: glancing at the green badge, nodding, moving on. And then it hit me that the badge had been telling me almost nothing. One hundred and fourteen checks were green — on whose machine, with what instruction set, proving what, exactly? I spent the day making the report answer those questions itself, and this post is the why and the how.
Two lessons I want to write down before they blur into the week. First: the report is part of the proof. A runtime that refuses to publish numbers it cannot defend has to name the machine its numbers came from — evidence that cannot name its origin is a claim, not evidence. Second: "not broken" and "not measured" must never look the same on a dashboard. The moment they share a color, your status page is lying to someone who trusts it.
This one is for anyone running CI for numerically sensitive software, for operators who have ever stared at a wall of amber asking "wait… is this broken?", and for people following C-Kernel-Engine who want to see what changed and why it matters.
A CI badge that says ALL PASS is making a claim, and a claim is only as good as what it can name. On what machine did these tests run? Which instruction-set surface did the kernel dispatch actually follow? And does each status on the page mean breakage, debt, or simply "not measured here"? This week C-Kernel-Engine's public nightly test report crossed the line from decoration to evidence: the report now captures and publishes the hardware identity of the runner that produced it, renders that identity as the first thing on the page, and teaches the operator what PASS, WARN, SKIP, and FAIL each prove.
For anyone new to the project, C-Kernel-Engine (CKE) is an open-source CPU runtime and kernel compiler for transformer-based language, vision, and audio models — start with What Is the C-Kernel-Engine if you want the full framing. A model is described as mathematical circuits, resolved against explicit kernel contracts, and lowered into generated C that is compared with reference implementations such as PyTorch, llama.cpp, and whisper.cpp. The code lives at github.com/C-Kernel-Engine, and the artifact this article is about is the public nightly test report — the page where the project's 114 nightly checks announce themselves to the world.
This post continues a thread that has run through the last several weeks of writing here. What Numerical Parity Actually Requires laid out why a number is only meaningful with its method attached. X-Ray's first divergence showed the investigation surface, and the Whisper two-day ladder showed a runtime that refuses to publish numbers it cannot defend — a refused roofline, fail-closed FP16 storage, capture-neutrality gates. This week the same discipline reached the page that aggregates everything: pull requests #296, #300, #303, and #304 turned the nightly report into something that names its own machine.
The Badge Is An Argument
Every project with a CI pipeline eventually puts a green checkmark somewhere public. Most of those checkmarks are unfalsifiable. ALL PASS — on what? A badge that cannot answer "on what machine, with which ISA surface, and what does each status actually prove?" is not evidence; it is marketing with good hygiene. For a project like CKE, whose entire selling point is that kernel dispatch follows the detected instruction-set surface of the machine in front of it, this gap was especially pointed: the runtime cares deeply about whether the silicon has AVX2 or AVX-512, but the public report said nothing about the silicon the tests ran on. The results were real, and the page was honest about pass counts — yet a skeptic landing on it could not reconstruct the stage the results were produced on. The argument had a conclusion and no premises.
The Runner Names Itself (PRs #296, #300)
The first two PRs changed what the nightly runner is willing to say about itself. scripts/nightly_runner.py now captures its own hardware evidence at run time and publishes it inside the report JSON as a schema-1 runner_hardware object. The capture is deliberately boring and therefore trustworthy: structured lscpu output with a redacted /proc/cpuinfo fallback if lscpu is unavailable, free -h and /proc/meminfo for memory, the runner image and version, the kernel version, the workflow run id and attempt, and a structured table of detected ISA flags — avx, avx2, fma, the AVX-512 family, AMX — each recorded as present or absent rather than implied.
Two design choices deserve emphasis. First, the raw captures ship with the report: the parsed fields are a convenience, but the underlying lscpu and meminfo text is right there in the JSON for anyone who distrusts the parser. Second, the feature is honest about history. Reports generated before this schema have no runner_hardware object, and the page renders NOT PUBLISHED for those runs instead of backfilling an invented machine. Backward compatibility here does not mean pretending the past was measured; it means labeling it as unmeasured. That is the same instinct as the refused Advisor roofline from the Whisper ladder: when the evidence does not exist, the correct output is a visible gap, not a plausible fill-in.
The Identity Plate (PR #303)
Data in a JSON file is provenance; data rendered at the top of the page is a commitment. PR #303 renders the capture as what the docs call the identity plate: an amber, CKE-brand card that sits above every test row. Its kicker reads EXECUTED ON · NIGHTLY CI RUNNER. The headline is the CPU model — today, AMD EPYC 9V74 80-Core Processor. A stat strip carries the coordinates an operator actually asks for: Architecture x86_64, Logical CPUs 4 with the topology sub-label (1 socket × 2 cores × 2 threads), Memory 15.6 GiB with 14.5 GiB available, and Kernel 6.17.0-1020-azure. Under the strip, a linkage line with an amber down-arrow says the quiet part out loud: "Every result below was produced on this machine — kernel dispatch follows its detected ISA surface."
The plate is disciplined about what is glanceable and what is on demand. Provenance details — vendor AuthenticAMD, runner image ubuntu24 20260720.247.2, the workflow run number and attempt, the capture source (lscpu) — live in a slim accordion together with the ISA chips and the raw captures, one click away but not competing with the headline. The card answers the five-second questions; the accordion answers the five-minute ones.
Dashboards That Teach (PR #304)
Naming the machine is half the story; the other half is naming what the statuses mean. The report carries two contract dashboards whose statuses were, until this week, expert-only vocabulary. PR #304 adds explainer cards and status legends to both, so the page now teaches its own semantics.
The Architecture Contract Dashboard checks the promoted circuit templates: each template declares its critical tensor edges, and the suite verifies whether those edges are explicit in the template or still compiler-inferred. Its legend now states the reading plainly. PASS means every critical edge is explicit. WARN means correct wiring with implicit edges remaining — hardening debt, not brokenness. FAIL means a declared contract was violated. Today's reading: glm4 and kimi_vl PASS with all edges explicit, and 10 of 12 templates WARN — 35 edges explicit, 78 still inferred, zero failed. A wall of amber that used to look like a sick project now reads as what it is: a measured backlog of contract hardening, with the wiring verified correct underneath.
The v8 DSL & Compiler Contracts panel reports three suites, each with its own card and its own guarantee. Zero Hardcoding — generated code may not bake in model-specific constants, so a new model family never needs a compiler branch. Numerical Kernel Contracts — dtype, reduction-order, and oracle-parity declarations are checked against reality. Circuit/Dataflow Audit — tensors may flow only through declared edges. And the legend names the fourth status the other dashboards rarely explain: SKIP means not applicable or not run on this runner — not a failure. Today the panel reads PARTIAL overall precisely because the Numerical suite skipped on the hosted runner. Before the legend, PARTIAL looked like trouble; now it reads as an absence, correctly located.
The Free-Diversity Angle
There is a quiet strategic benefit hiding in the capture. CKE does not own AMD silicon; the lab is Intel. But GitHub-hosted runners happen to be AMD EPYC 9V74, which means every nightly exercises AVX2 lanes on hardware the project does not own, selected by someone else's procurement. Cross-vendor coverage that would otherwise require buying machines arrives free with the CI bill — and now the report proves which silicon produced the evidence instead of assuming it. When GitHub changes its fleet, the identity plate changes with it, visibly, and a dispatch regression that only appears on new silicon will show up with the new machine's name already attached to every row.
The Operator Test
The design bar for all of this was stated as a thought experiment: someone landing on the page in six months — a new contributor, a skeptical evaluator, a future maintainer who has forgotten this week entirely — should know within five seconds whether a WARN is breakage or debt. The old page failed that test; amber rows and a PARTIAL panel looked like symptoms. The new page passes it: the machine is named at the top, the linkage line binds every row to that machine, and each status carries its own definition at the point of use. The page answers "wait… is this broken?" before the question finishes forming. A report that can field its own hardest first question is a report you can hand to a stranger.
The Report Is Part Of The Proof
This is the thesis worth carrying out of the week. CKE has been building a runtime that can say no — X-Ray capture neutrality refuses tensor dumps the observer may have contaminated; BOS handling fails closed on empty generations rather than recording them as passes; the Whisper ladder refused a roofline its tooling could not trace (all in the two-day ladder post and the July recap). A runtime with that character has one more obligation: the page that publishes its numbers has to name the machine the numbers came from. Evidence discipline that stops at the repository boundary is incomplete. The nightly report is not a summary of the proof — it is part of the proof, and this week it started acting like it.
Related Notes And Further Reading
- How CKE Made Whisper Faster In Two Days — the evidence discipline this post extends: capture neutrality, fail-closed gates, refused claims.
- What CKE Built In July 2026 — the recap whose evidence chapter this week's report work continues.
- How CKE X-Ray Found Qwen3.6's First Bad Circuit — the investigation surface behind the numerical contracts.
- What Numerical Parity Actually Requires — why a number without its method is not a result.
- The CKE nightly test report — the live artifact, identity plate included.
The code is at github.com/C-Kernel-Engine — the runner capture lives in scripts/nightly_runner.py, and the report work this article covers landed in PRs #296, #300, #303, and #304. The engineering Discord is open at discord.gg/J7wNx8xS — bring questions, skepticism, or a runner whose hardware you would like to see named.