ALEX MORGAN / SENIOR AI SYSTEMS ENGINEER LAB STATUS: LIVE [DEMO ENVIRONMENT]
BUILD. MEASURE. UNDERSTAND.
Engineering Autonomous
Intelligence at Scale_
Deep dives on GPU kernel optimization, multi-agent orchestration, low-latency LLM inference engines, and the craft of building systems that work.
NEURAL MESH / 180 NODESDRAG TO ROTATE ↔
GLOBAL INFERENCE TPS4.8k+24.3%
Distributed cluster aggregate
P99 INFERENCE LATENCY42ms−8ms (opt)
Time to first chunk (TTFT)
ACTIVE MODELS IN PROD14multi-tenant
70B, LoRA, MoE, Embeddings
OPEN SOURCE STARS12.4k1.6k forks
Illustrative portfolio metrics
⌁ Ambient Architecture Stream / Video Telemetry
LIVE SIMULATION▰ FEED: CLUSTER-KERNEL-PIPELINE AV-014
REAL-TIME SYSTEM OBSERVABILITY
Intelligence, in motion.
Every connection. Every token. Every millisecond.
LIVE BITRATE: 34.0 MbpsSHADERS: COMPILED [WEB CANVAS]
▪ Lead Research Story
KERNEL ARCHITECTURE / SYSTEMSZero-Overhead Speculative Decoding with CUDA Graphs and Triton
What happens when you stop optimizing individual tokens and rethink the whole pipeline? An illustrated look at draft models, batched verification, and captured execution graphs.
SPEEDUP BENCHMARK (DEMO DATA)LLAMA-70B · BATCH 1
Baseline Autoregressive (vLLM)24.2 tok/s
Standard Speculative (EAGLE Draft)58.5 tok/s
CUDA Graph Captured Verification82.6 tok/s (3.4×)
RESEARCH NOTE / NO. 001
verify_kernel.triton.py
# Capturing verification in a static CUDA Graph
@triton.jit
def speculative_verify_kernel(
draft_tokens_ptr, target_logits_ptr,
accepted_mask_ptr, stride_b, stride_s,
BLOCK_SIZE: tl.constexpr
):
pid = tl.program_id(0)
offsets = tl.arange(0, BLOCK_SIZE)
# Vectorized sampling & greedy comparison
draft = tl.load(draft_tokens_ptr + offsets)
logits = tl.load(target_logits_ptr + offsets)
target = tl.argmax(logits, axis=0)
mask = (draft == target)
tl.store(accepted_mask_ptr + pid, mask)
# Replay the graph — skip Python dispatch
cuda_graph.replay()System Dispatches &
Articles.
Field notes from the intersection of code and curiosity.
※ Interactive Architecture Lab Spotlight
LIVE SIMULATION ENGINEUltra-Low Latency Inference Pipeline Architecture
Click a computational stage to inspect latency budgets and explore the flow.
READY / SELECT A STAGE TO INSPECT
‹› Open Source Toolkit
TOOLS I EXPLORE ↗triton-lang/triton
GPU programming for neural networks.Python / C++ OPEN SOURCE
vllm-project/vllm
A library for LLM inference and serving.Python OPEN SOURCE
ray-project/ray
A unified framework for distributed computing.Python / C++ OPEN SOURCE