ANTSHIV ROBOTICS / BUILDING REAL RUNTIMES
Four bits. From packed weights to onboard inference — with C-Kernel-Engine.
SLIDE 01 / ON ARRIVAL
Welcome to Antshiv Robotics. I have been working through these formats while building C-Kernel-Engine, and I want to start with a question that confused me at first. If they all say four bits, why do they need different decoding code? Let's put the three versions beside each other.
[CLICK] 1/3 — One ruler / 32 values
02 / THE THREE CONTRACTS
Same values. Different rulers. 1 × E8M0 E4M3 / 16 E4M3 / 16 FP32 global 32 four-bit values · Next transforms the diagram.
MXFP4 One E8M0 scale
NVFP4 Two groups of 16.
Tensor scale An FP32 global scale
Conceptual grouping, not physical byte layout. Metadata objects are enlarged for visibility.
SLIDE 02 / ON ARRIVAL
Let's keep the number of values fixed. There are thirty-two little blocks here. The objects are a picture of who shares a scale, rather than a drawing of the exact byte layout.
[CLICK] 1/3 — MXFP4 One E8M0 scale for all 32 values.
03 / THE THREE CONTRACTS
On a robot, memory is shared. Four-bit payload 4bits / weight
7 billion × 2 bytes = 14 GB
7 billion × ½ byte = 3.5 GB
Onboard memory also serves sensor buffers, KV cache and runtime workspace.
SLIDE 03 / ON ARRIVAL
Why spend time on this? For the kind of systems I want to build, the model shares memory with everything else: sensor buffers, the runtime, and the state for active requests. Take an illustrative seven-billion-weight model.
[CLICK] 1/2 — 7 billion × 2 bytes = 14 GB
04 / THE THREE CONTRACTS
One row. One output. A tiny row of W multiplied by x; bias is zero here.
w · x = −2 + .30 − .90 + .85
SLIDE 04 / ON ARRIVAL
Before opening the formats, let me make the computation small. This is one row of weights and one input vector. Each pair contributes to one output. I have left the bias at zero.
[CLICK] 1/2 — w · x = −2 + .30 − .90 + .85
05 / READ THE FORMAT NAME
E2M1: four bits, three jobs. E = exponent bits · M = stored fraction bits · sign is additional.
Sign: 0 means positive. Exponent: binary 10 = 2; subtract bias 1.
Fraction: binary .1 = ½. A normal value has an implicit leading 1.
(+1) × 22−1 × (1 + ½) = 3
A normal E2M1 example used by MXFP4 and NVFP4. Zero and subnormal encodings use a different rule.
SLIDE 05 / ON ARRIVAL
E2M1 looks like a complicated label until we read it literally. Two exponent bits, one stored fraction bit, and one sign bit. Here are the four bits: zero, one, zero, one.
[CLICK] 1/3 — Sign: 0 means positive. Exponent: binary 10 = 2; subtract bias 1.
06 / THE THREE CONTRACTS
Does 0101 mean five? E2M1: one sign bit, two exponent bits, one fraction bit.
E2M1: 0101 means 3. Q4_K: 0101 means integer 5.
E2M1 magnitudes: 0, ½, 1, 1½, 2, 3, 4, 6.
This codebook is shared by the MXFP4 and NVFP4 examples. It is not signed INT4.
SLIDE 06 / ON ARRIVAL
So, does zero-one-zero-one mean five? [PAUSE — give the viewer a moment.] If I read it as an unsigned integer, yes. But we have just read it another way.
[CLICK] 1/2 — E2M1: 0101 means 3. Q4_K: 0101 means integer 5.
07 / PREDICT / THEN MOVE THE RULER
Does a smaller scale always help? Original Rebuilt
① Snap at scale 0.5
② Shrink to 0.125
③ Inspect the errors
Positive half of E2M1; nearest value, midpoint ties toward smaller magnitude. Original values: 0.35 and 1.7.
SLIDE 07 / ON ARRIVAL
Let's try two original values: point three five and one point seven. Before I move anything, would a smaller scale help both of them? [PAUSE.]
[CLICK] 1/3 — ① Snap at scale 0.5
08 / THE THREE CONTRACTS
Move the ruler. Watch the error. Illustrative FP4 reconstruction scale: 0.5 Scales: ⅛, ¼, ½, 1, 2 · deterministic nearest value; midpoint ties use smaller magnitude.
Original w
FP4 values
Reconstructed ŵ
Dot product
Signed output error
Teaching quantizer with selectable scales, not a production scale-selection algorithm or a format-quality benchmark.
SLIDE 08 / ON ARRIVAL
Here is our original row again. At scale one half, the reconstructed weights give an output of minus two. We started at minus one point seven five, so the signed output error is minus point two five. This is a small teaching quantizer; the purpose is to make the error visible. [DEMO — optional, not a Reveal click: move the slider from 0.5 to 0.125, then back to 0.5. At 0.125 the output is −1.625, error +0.125.] Watch the output as well as the individual weights. The input values decide how each weight error contributes to the sum. [FOCUS — click blank slide space before using your presenter key again. If you skip the demo, leave the slider at 0.5.] Now let's put the scale inside a real format.
[CLICK] Next slide — E8M0: just a power of two.
09 / MXFP4 / THE SHARED SCALE
E8M0: just a power of two. 8 exponent bits · 0 fraction bits · no sign bit.
0 1 1 1 1 1 1 0
Stored exponent byte: 0x7E = 126
decoded exponent = 126 − 127 = −1
One byte stretches the ruler for all 32 FP4 values.1.5 .
Finite E8M0 scales are positive powers of two. 0xFF denotes NaN; zero is not representable. Bias:127.
SLIDE 09 / ON ARRIVAL
For MXFP4, that shared scale is E8M0. Eight exponent bits, no stored fraction, and no sign bit. The example byte is hexadecimal seven-E, which is one hundred and twenty-six.
[CLICK] 1/3 — decoded exponent = 126 − 127 = −1
10 / THE THREE CONTRACTS
One ruler for all 32 codes.
Rotate 30° Top view Reset view Drag to orbit · scroll to zoom
32 FP4 codes Mint cubes: E2M1 codes.
One E8M0 scale Gold: one byte for all 32 codes.
Decode the values Read the signed E2M1 codebook. Height shows each value.
Apply the shared scale ŵ = FP4(code) × 2ᵉ1.5
17 bytes / 32 weights = 4.25 bits per weight. Schematic geometry. Blender source · GLB model
SLIDE 10 / ON ARRIVAL
Let's see that grouping in three dimensions. The mint objects are thirty-two four-bit codes. Their raw payload takes sixteen bytes. I will keep the camera still so we can follow what changes.
[CLICK] 1/3 — One E8M0 scale Gold: one byte for all 32 codes. Example: exponent −1 → 0.5.
11 / NVFP4 / THE LOCAL SCALE
E4M3: a finer scale in one byte. 1 sign + 4 exponent + 3 fraction bits = 8 bits.
Subtract exponent bias 7 . The stored fraction adds to an implicit 1.
(+1) × 25−7 × (1 + 2/8) = 0.3125
NVFP4 uses this FP8 scale for 16 E2M1 codes.0.9375 .
E4M3 normal example: byte 0x2A. The weights remain four-bit codes; their shared scale uses eight bits. Special encodings have separate rules.
SLIDE 11 / ON ARRIVAL
NVFP4 keeps the E2M1 weight code, but uses a different local scale. E4M3 has a sign bit, four exponent bits and three stored fraction bits. Let's decode the byte shown here.
[CLICK] 1/3 — Subtract exponent bias 7 . The stored fraction adds to an implicit 1.
12 / THE THREE CONTRACTS
NVFP4: two levels of scale.
Rotate 30° Top view Reset view Drag to orbit · scroll to zoom
32 FP4 codes Each mint cube represents one four-bit code.
Two local scales Gold E4M3 scales: one per 16 codes. Lines show ownership.
One tensor scale Blue FP32 scale: shared across the tensor.
ŵ = FP4(code) × block scale × tensor scale
A grouping diagram, not a chip or byte-proportional model. Blender source · GLB model
SLIDE 12 / ON ARRIVAL
Here are thirty-two NVFP4 codes, so we can compare the picture with MXFP4. Think about ownership of the scales as the objects separate.
[CLICK] 1/2 — Two local scales Gold E4M3 scales: one per 16 codes. Lines show ownership.
13 / THE RECONSTRUCTION
A different decoding rule. 256 weights → eight subblocks of 32.
132 weights
232 weights
332 weights
432 weights
532 weights
632 weights
732 weights
832 weights
ŵᵢ = d × sⱼ × qᵢ − dmin × mⱼ
qᵢ is an unsigned nibble, 0…15.
d and dmin are FP16 superblock factors. This is affine reconstruction, not the E2M1 codebook.
SLIDE 13 / ON ARRIVAL
Q4_K is a different route. Its four-bit payload holds unsigned integers. We are going to reconstruct two hundred and fifty-six weights in one superblock.
[CLICK] 1/2 — 1 32 weights sᵢ · mᵢ 2 32 weights sᵢ · mᵢ 3 32 weights sᵢ · mᵢ 4 32 weights sᵢ · mᵢ 5 32 weights sᵢ · mᵢ 6 32 weights sᵢ · mᵢ 7 32 weights sᵢ · mᵢ 8 32 weights sᵢ · mᵢ
14 / Q4_K / KEEP ONE WEIGHT IN VIEW
Follow w₀. Same identity, new encoding. ORIGINAL 1.1 CODE 5 BYTE 0xA5 DECODED 1.0 × ACTIVATION 2.0 w₀
① Encode
② Pack
③ Reconstruct
④ Multiply
Chosen gain 0.25 and offset 0.25. Illustrative nearest-code quantization, not CKE’s production parameter search.
SLIDE 14 / ON ARRIVAL
Let me keep one weight in view. It starts at one point one. For this illustration, I chose a gain of one quarter and an offset of one quarter. We are following the white outline.
[CLICK] 1/4 — ① Encode
15 / THE RECONSTRUCTION
What is inside 144 bytes? qs[0…31] qs[32…63] qs[64…95] qs[96…127] 12 metadata bytes d dmin 128 mint cubes
12 gold cubes
Two blue blocks
4 chunks × 64 weights256 weights
Memory offsets: d 0–1 · dmin 2–3 · metadata 4–15 · payload 16–143. Exploded view; sizes are schematic.
SLIDE 15 / ON ARRIVAL
Now zoom out to the stored block. Each mint cube is a byte, and each byte contains two four-bit codes. There are one hundred and twenty-eight payload bytes.
[CLICK] 1/3 — 12 gold cubes Eight scales + eight minima, six bits each.
16 / THE RECONSTRUCTION
Split. Scale. Subtract. 0xA5 1010 0101 5 10 × (.125 × 2) × (.125 × 3) − (.0625 × 4) − (.0625 × 8) ŵ₀ 1 ŵ₃₂ 3.25 UNPACK LOCAL GAIN OFFSET WEIGHT ① Low: b & 15
② Multiply each code
③ Subtract dmin × mⱼ.
④ Store at lanes 0 / 32.
Predict: do the two nibbles become neighbouring weights? Advance to reveal their destinations.
SLIDE 16 / ON ARRIVAL
Here is A-five. Before I split it, a question: do its two nibbles become neighbouring weights? [PAUSE.] Let's follow their routes.
[CLICK] 1/4 — ① Low: b & 15 High: b >> 4
17 / THE RECONSTRUCTION
Open a real Q4_K block.
Rotate 30° Top view Reset view Drag to orbit · scroll to zoom
144 bytes → 256 weights 128 payload bytes + 12 metadata bytes + two 2-byte FP16 factors.
① Unpack 8 scale/minimum pairs.
② Split 0xA5 → low 5 / high 10.
③ Gain: d × s₀ = .125 × 2 = .25
④ Offset: .0625 × 4 = .251
White outline: w₀ from the previous walkthrough. Sizes are schematic; height encodes values after splitting. Blender source · CKE-verified fixture
SLIDE 17 / ON ARRIVAL
This is the same contract across the full block. We begin with one hundred and forty-four stored bytes. The white outline keeps our first weight in view.
[CLICK] 1/4 — ① Unpack 8 scale/minimum pairs.
18 / CKE / THE DIAGRAM MEETS THE C
Watch the real decoder execute. 347 const float d1 = d * (float)sc[2 * iter];
348 const float m1 = dmin * (float)m[2 * iter];
352 const uint8_t *qs = &block->qs[iter * 32];
353 float *out = &output[iter * 64];
357 const int q = (qs[l] & 0x0F);
358 out[l] = d1 * (float)q - m1;
363 const int q = (qs[l] >> 4);
364 out[32 + l] = d2 * (float)q - m2;
① Metadata
② Pointers
③ Mask
④ Reconstruct
⑤ Shift + route
Verbatim selected lines from CKE dequant_kernels.c; loops and other setup omitted. First iteration, l = 0.
SLIDE 18 / ON ARRIVAL
Here is the C behind the picture. These are selected lines from the scalar decoder; I have left out loops and setup so we can follow the first byte. We start with iteration zero and lane zero.
[CLICK] 1/5 — ① Metadata
19 / THE RECONSTRUCTION
A model is a collection of tensors. Inspect the tensor inventory.
Q4_K_M selects a mixture of tensor types.
Conceptual inventory, not a claim about any particular checkpoint.
SLIDE 19 / ON ARRIVAL
So far we have opened one block format. A model contains many tensors, and its filename is a broader description.
[CLICK] 1/3 — Tensor A Q4_K
21 / THE RUNTIME
The runtime must keep its promise. Checkpoint tensor types
Converter packed layout
Dispatch matching kernel
Compute wider accumulator
Verify reference parity
Four times smaller. Four times faster?
No guaranteed speed ratio: bandwidth, decoding and compute all matter. Measure on the target hardware.
SLIDE 21 / ON ARRIVAL
Now imagine putting this into a runtime. A much smaller weight file is appealing. But is it automatically four times faster? [PAUSE.] Let me follow what has to happen.
[CLICK] 1/5 — Checkpoint tensor types
22 / ANTSHIV ROBOTICS / PUT THE KERNEL IN CONTEXT
Measure the whole input-to-decision path. INPUT camera / voice PREPROCESS resize / features MODEL quantized weights DECISION downstream logic
① Prepare input
② Run inference
③ Use the result
Conceptual application, not a demonstrated control deployment. Weight storage is one part of memory; kernel time is one part of latency.
SLIDE 22 / ON ARRIVAL
This is where I bring it back to Antshiv Robotics. Suppose we have camera or voice input. This diagram is a proposed application path, not a demonstrated robot controller.
[CLICK] 1/3 — ① Prepare input
23 / THE RUNTIME
Three gates for a useful quant.
Correct bytes Decode known blocks.
Useful behavior Same model and tokenizer.
Target hardware Measure latency and memory.Hold KV precision, workload, hardware and runtime revision fixed when comparing weight formats.
SLIDE 23 / ON ARRIVAL
Before calling a quantized path useful, I would keep these three questions separate.
[CLICK] 1/3 — Correct bytes Decode known blocks. Check scales and lanes.
24 / THE RUNTIME
The code is only half the story. MXFP4 E2M1 values
NVFP4 E2M1 values
Q4_K Unsigned integer codes
The format defines the meaning.
Q4_K_M describes a tensor recipe. None of these labels alone guarantees model quality or inference speed.
SLIDE 24 / ON ARRIVAL
Let's come back to the three pictures we started with. The same number of element bits can lead to different execution rules.
[CLICK] 1/4 — MXFP4 E2M1 values One E8M0 scale / 32
25 / THE RUNTIME
Keep building with Antshiv Robotics. Antshiv Robotics on YouTube · antshiv.com
SLIDE 25 / ON ARRIVAL
If you want to work through the arithmetic slowly, I have linked the companion article on ShivasNotes, along with the foundations and runtime testing posts. I will put those in the video description. Here on Antshiv Robotics, I will keep following this into the implementations and the machines running them. Thanks for watching. [PAUSE — hold the closing slide, then stop the main recording. Do not advance into the appendix.]
[STOP] End of main recording.
MAIN EPISODE COMPLETE
Optional deep dives. Bit packing · numerical examples · alternate animation clips · sources
Use these for a longer cut, a follow-up video, or questions.Return to opening · Open sources
SLIDE 26 / ON ARRIVAL
[OPTIONAL MATERIAL — main recording has ended.] These slides are available for a longer explanation or a separate insert. Choose the one you need from Reveal's overview; they are not required for the main episode.
[CLICK] Next slide — Ask three questions.
APPENDIX 01 / OPTIONAL DEEP DIVE
Ask three questions.
01 · Value What does a four-bit code mean?Integer or FP4?
02 · Scale Which values share a ruler?16, 32, or subblocks?
03 · Recipe Which tensor gets which format?Uniform or mixed? Q4_K_M belongs at the recipe level. Q4_K is an underlying block type.
SLIDE 27 / ON ARRIVAL
When the names start to blur together, I come back to three questions.
[CLICK] 1/3 — 01 · Value What does a four-bit code mean? Integer or FP4?
APPENDIX 02 / OPTIONAL DEEP DIVE
One scale for 32 values. E8M0 scale → power of two
code value 3 × scale ½ = 1.5
One 8-bit scale per 32 four-bit elements. This diagram shows grouping, not physical byte order.
SLIDE 28 / ON ARRIVAL
This is the compact MXFP4 reference. One E8M0 scale belongs to thirty-two values.
[CLICK] 1/2 — ŵᵢ = FP4(codeᵢ) × 2ᵉ
APPENDIX 03 / OPTIONAL DEEP DIVE
A finer local ruler. ŵᵢ = FP4(codeᵢ) × sᵦ × sɢ
sᵦ: E4M3 block scale sɢ: FP32 tensor scale
NVFP4 reconstruction convention. Some APIs store reciprocal scales; inspect the contract.
SLIDE 29 / ON ARRIVAL
Here is the equivalent NVFP4 reference, with sixteen values per local group.
[CLICK] 1/3 — ŵᵢ = FP4(codeᵢ) × sᵦ × sɢ
APPENDIX 04 / OPTIONAL DEEP DIVE
Same row. Different reconstruction. Illustration Reconstructed weights Output Original −1, .30, .90, 1.70 −1.75 FP4 · scale .5 −1, .25, 1, 1.5 −2.00 FP4 · .3125 × 1 −.9375, .3125, .9375, 1.875 −1.5625
Chosen valid reconstruction scales; not matched production quantizers. This example cannot rank model quality.
SLIDE 30 / ON ARRIVAL
This is our original row and its output, minus one point seven five. The input vector is unchanged.
[CLICK] 1/3 — FP4 · scale .5 −1, .25, 1, 1.5 −2.00
APPENDIX 05 / OPTIONAL DEEP DIVE
Decode the number, step by step. d = .125 s = 2 q = 5 dmin = .0625 m = 4
local scale = .125 × 2 = .25
Valid illustrative metadata. It demonstrates decoding, not how the quantizer fits the parameters.
SLIDE 31 / ON ARRIVAL
Here are the chosen Q4_K parameters for one weight. We are decoding a valid illustrative block, not fitting parameters from training data.
[CLICK] 1/3 — local scale = .125 × 2 = .25
APPENDIX 06 / OPTIONAL DEEP DIVE
Right arithmetic. Wrong lane? 0xA3
Low nibble · 3 qs[l] & 0x0Foutput[l]
High nibble · 10 qs[l] >> 4output[l + 32] In each 64-weight chunk of the Q4_K decoder,
CKE dequant_q4_k_block, first iteration. These are not adjacent output weights.
SLIDE 32 / ON ARRIVAL
This alternate byte is A-three. The point here is the destination, so watch both halves.
[CLICK] 1/2 — Low nibble · 3 qs[l] & 0x0F output[l]
APPENDIX 07 / OPTIONAL DEEP DIVE
Watch the layers separate. SLIDE 33 / ON ARRIVAL
[PLAY VIDEO — use its native play control; this is not a Reveal click. The clip is silent. Keep focus off the controls before advancing.] Here are the thirty-two codes. Watch the local scales lift away from the two groups. Then the shared tensor scale appears above them. The camera movement is just to make that ownership easier to see. [WAIT — pause or finish the clip before advancing.]
[CLICK] Next slide — A weight moves. An error remains.
APPENDIX 08 / OPTIONAL DEEP DIVE
A weight moves. An error remains. SLIDE 34 / ON ARRIVAL
[PLAY VIDEO — use its native play control; this is not a Reveal click.] Here the original weight is point nine. At the chosen scale of one half it lands on one. The remaining error is positive point one. This clip illustrates that one rounding decision; it does not compare complete model quality. [WAIT — pause or finish the clip; click blank slide space before advancing.]
[CLICK] Next slide — Twelve bytes. Sixteen parameters.
APPENDIX 10 / OPTIONAL DEEP DIVE
One byte. Two different rulers. Low nibble → weight 0 0xA5 & 15 = 5
.125 × 2 × 5 − .0625 × 4
ŵ₀ = 1
High nibble → weight 32 0xA5 >> 4 = 10
.125 × 3 × 10 − .0625 × 8
ŵ₃₂ = 3.25
Repeat for 32 bytes → 64 weights.
Scalar reference: 144 packed bytes → 1,024 FP32 bytes.
Reconstruction approximates the original weights. Q4_K_M is a tensor-mixing recipe; this demonstrates one Q4_K block.
SLIDE 36 / ON ARRIVAL
Here are both results for A-five together. The low code five reconstructs to one at weight zero. The high code ten reconstructs to three point two five at weight thirty-two. Their gains and offsets differ.
[CLICK] 1/2 — Repeat for 32 bytes → 64 weights. Four chunks → all 256 weights.
APPENDIX 11 / OPTIONAL DEEP DIVE
Specifications, code, and provenance. Local Reveal.js + pinned GSAP 3.13.0. Numerical demonstrations are illustrative; no performance results are claimed.
SLIDE 37 / ON ARRIVAL
These are the specifications, implementations and provenance behind the examples. The companion article links the details as well. [END OF APPENDIX — stop here or use overview to select an insert; no further advance is needed.]
[STOP] End of appendix.