projects

Two LLMs on a single Raspberry Pi 5, a draft and a target on separate CPU cores, with hand-tuned INT8 matmul kernels written in ARM NEON intrinsics and benchmarked against llama.cpp. Grouped quantization cuts the weights’ memory footprint by 75%.

A tiled forward pass that streams KV blocks through an online softmax, so the N×N score matrix is never materialized while still matching standard attention exactly. A custom autograd backward pass recomputes scores in tiles, keeping training memory linear in sequence length.