Senior Software Engineer at Robert Bosch
Performance engineering for GPU and AI systems
I build and optimize compute-intensive software, from CUDA algorithms to real-time AI inference. The same focus on efficient execution shaped my PhD research in scientific visualization and earlier work in graphics.
Research and software I've worked on
A research runtime for low-latency LLM inference based on scheduling a graph of tiles. An offline planner assigns every tile to a fixed worker, and one persistent CUDA kernel executes that schedule for every token of Llama 3.2 1B on an 8 GB RTX 3050.
A CUDA software rasterizer that runs the graphics pipeline entirely on the GPU. The framework manages rasterization and adaptive sampling in software, generating additional work when needed while achieving interactive frame rates.
Technical articles and notes
An approach to low-latency LLM inference by scheduling a graph of tiles in a megakernel. How close can it get to the memory-bandwidth floor of batch-one decode on a small GPU? This post explains where that floor comes from for Llama 3.2 1B on an RTX 3050, how a token is scheduled into tiles and waves, where the remaining time goes, and how the result compares with PyTorch and llama.cpp.
Reading about recent megakernels for LLM inference brought me back to persistent threads and megakernels I used in my own GPU work. This post traces both ideas from their origins in ray tracing to modern attention kernels and full-model LLM inference
Extending void-and-cluster sampling from scattered spatial data to higher-dimensional point sets, using a neighborhood graph to sample an embedded manifold.
Peer-reviewed research
01
T. Rapp, C. Peters, C. Dachsbacher
02
03
T. Rapp, C. Peters, C. Dachsbacher