Skip to main content
Tobias Rapp

Blog

Technical notes and articles.

Published notes
3
Archive
2025–2026
22 min read

Decoding in Tiles: A Static Megakernel for Llama 3.2 1B on an RTX 3050

How close can one statically scheduled persistent kernel get to the memory-bandwidth floor of batch-one decode on a small GPU? This post explains where that floor comes from for Llama 3.2 1B on an 8 GB RTX 3050, how a token is cut into tiles and waves, where the remaining time goes, and how the result compares with PyTorch and llama.cpp.

  • CUDA
  • GPU programming
  • megakernels
Read note

Archive

2 entries

  1. 15 min read

    Persistent Threads and Megakernels

    Reading about recent megakernels for LLM inference brought me back to persistent threads and megakernels I used in my own GPU work. This post traces both ideas from their origins in ray tracing to modern attention kernels and full-model LLM inference

    • CUDA
    • GPU programming
    Read note
  2. 5 min read

    Void-and-Cluster Sampling Beyond Space and Time

    Extending void-and-cluster sampling from scattered spatial data to higher-dimensional point sets, using a neighborhood graph to sample an embedded manifold.

    • scientific visualization
    • sampling
    Read note