If you are new to the project, start with What Is the C-Kernel-Engine?, then read Templates Are Circuit Maps and Prefill vs Decode. This post sits one layer lower: after CKE knows the model circuit, how does it decide where tensors live?

The actual v8 mechanism

The important file is version/v8/scripts/memory_planner_v8.py. Its own header says the goal plainly: assign physical buffers from the IR1 dataflow graph and replace older brittle ping-pong behavior with explicit assignments.

That means the current planner is not doing magical global optimization. It is doing something more practical and auditable:

1. IR1 describes dataflow. Each lowered op carries inputs, outputs, dtype hints, slots, and producer/consumer intent.
2. The memory planner maps logical slots to physical buffers. For example, the main activation stream can live in A_EMBEDDED_INPUT or A_LAYER_INPUT, attention work goes through A_ATTN_SCRATCH, MLP work goes through A_MLP_SCRATCH, residuals use A_RESIDUAL, and logits use A_LOGITS.
3. Codegen consumes those assignments. The generated C receives exact buffer names and offsets instead of guessing where an op should read or write.

So the honest sentence is this: CKE v8 has explicit dataflow buffer assignment and deterministic named buffers; it is still being hardened toward fewer copies and better reuse. That is stronger than a marketing claim because the mechanism is visible in scripts, generated C, and IR reports.

What memory planning is trying to prevent

A transformer layer produces many temporary values: normalized hidden states, Q/K/V projections, attention output, residuals, MLP gate/up activations, down projection output, logits, and cache state. Some of those values die quickly. Some must survive. Some are layout handoffs between two kernels that do not yet agree on the same memory order.

The planner has to keep three categories separate:

This is why a wrong memory planner can be worse than a crash. A crash is loud. A stale scratch buffer or misplaced cache write can still produce finite logits and a readable answer, just with the wrong numerical path underneath.

Prefill and decode do not want the same plan

Prefill reads a prompt and builds the cache. Decode uses that cache one token at a time. The weights are shared, but the memory pressure is not.

Layer sketch:
1. x_i enters as the current hidden stream.
2. RMSNorm reads the current stream and writes normalized hidden state.
3. Q/K/V projection writes attention scratch or cache-facing buffers.
4. Attention reads Q plus K/V cache and writes attention output.
5. MLP gate/up/down uses MLP scratch.
6. Residual add produces the next layer stream.

Scratch can be reused after its consumer.
KV cache cannot be overwritten just because the current op is done.

The invariant, without pretending it solves everything

The basic memory-planning goal is simple:

\[ \text{peak memory} = \max_t \sum_{b \in \text{live}(t)} \text{size}(b) \]

But CKE is not only trying to reduce peak bytes. On CPU, memory movement itself is expensive. A plan that saves peak memory by inserting extra copies can still lose wall-clock time. The better target is: fewer live bytes, fewer unnecessary layout handoffs, and buffer assignments that match the actual kernels.

Why the Qwen3VL memcpy evidence matters

The Qwen3VL IR visualizer screenshot matters because it shows the next bottleneck in a way humans can inspect. In one loaded Qwen3VL-8B-Instruct-Q4_K_M view, each decoder layer appears to expose roughly two memcpy-like handoff nodes. If each handoff is on the order of tens of megabytes, then across 27 layers the pressure can approach about a gigabyte of extra movement per generated token in the worst interpretation of that trace.

That number should be read as an investigation estimate, not a published benchmark. The point is still serious: even if the math kernels are correct, layout handoffs can quietly tax every token. CKE's advantage is that the IR visualizer can show those copies instead of leaving them buried inside a framework graph.

What changed in the engineering direction

The memory direction in CKE has been moving from implicit assumptions toward inspectable contracts:

• Older behavior: ping-pong logic and generated paths could disagree about where a value should live.
• Current v8 direction: IR1 dataflow feeds memory_planner_v8.py, which assigns named physical buffers before lowering/codegen.
• Current hardening gap: some codegen paths still use memcpy for layout transforms, KV movement, bridge outputs, dumps, and API-facing output copies.
• Next target: make more producers write directly into the layout their consumer expects, especially around attention and multimodal prefill.

This ties directly to recent CKE work on Qwen3VL, X-ray, and numerical parity. Parity proves the result is correct. X-ray shows where drift or layout transitions occur. Memory planning asks whether the correct result can be produced with less unnecessary movement.

What to check in PRs

How to use the IR visualizer

Do not link a local IR report from the public article. Generate it as an artifact and attach screenshots or exported reports to the PR instead. The point of the visualizer is to give humans an investigative surface: when AI writes a large codegen change, the reviewer can inspect the circuit, buffer flow, and suspicious handoffs before trusting the speed claim.

This matters even more because a lot of CKE is now built with AI assistance. AI can produce broad codegen changes and optimization attempts quickly, but humans still need a surface that explains what happened: which buffers were used, which kernels were selected, which layout conversions appeared, and where extra movement entered the generated path. The IR visualizer is that surface.

1. Build or load the model runtime you want to inspect.
2. Generate the IR report from the CKE v8 tooling.
3. Open the memory/codegen sections and count explicit copies, layout transforms, and cache handoffs.
4. Use parity gates before and after removing a copy.

Related reading

• C-Kernel-Engine source
• CKE documentation
• v8 memory planner script
• Kernel maps and buffer roles
• Kernel registry and numerical contracts
• Numerical parity and CKE X-ray