Sitemap
What to index.
Investigations, the projects I built, and the rest of https://abiaryan.com. Also at llms.txt and llms-full.txt.
Pages
Maven courses
- maven.com/goabiaryan
amongst the top-most rated instructors on Maven
- AI Inference Engineering & Systems Design
A highly rated masterclass for mid-to-senior engineers. Build the stack by hand. Next live cohort 24 Oct - 19 Dec 2026.
- AI Inference for FDE and PM Roles
A masterclass on what is happening inside the model, for FDE, PM, and adjacent technical roles.
- The Future of LLM Serving
Lightning lesson · 45 min
- How to do Observability right?
Lightning lesson · 60 min
- How much will my model cost? Napkin Maths 101
Lightning lesson · 60 min
- What happens when you make an LLM call?
Lightning lesson · 45 min
- Whats next for AI Engineers?
Lightning lesson · 60 min
- AI Careers in 2025+: What No One's Telling You
Lightning lesson · 30 min
Investigations
- Why GPU Utilization Is a Misleading Metric for LLM Inference: Queueing, KV Cache, and SLOs
GPU utilization is not goodput. Queueing, KV pressure, and why $97 vs $0.38 / MTok.
GPU utilization, goodput, queueing, KV cache, SLOs, TTFT, TPOT, $/MTok
- Why KV Cache Needs a Directory: Locality, Leases, and Cache-Aware Routing
On one GPU, KV is an allocator. Across a cluster it is a phone book.
KV cache, directory, locality, leases, cache-aware routing, allocator
- Why the Same GPU Serves One Request in 3.3 Seconds and Another in 39: Request Shape, Prefill, and Decode
Same model, same card, same code. The request shape did it.
request shape, prefill, decode, latency, same GPU
- One Inference Roadmap, Not Fifty Tools: Request Path Before CUDA
Path, phases, memory, queue, then kernels. Not a bookmark pile.
inference roadmap, request path, CUDA, phases, memory, queue, kernels
- Why Agent Workloads OOM at 62% GPU Utilization: KV Cache, Multi-Turn State, and a $27k Weekend
The demo had headroom. Saturday agents leased HBM. Seven of twelve replicas died.
agent workloads, OOM, GPU utilization, KV cache, multi-turn, HBM, $27k
- The Inference Engineer Owns SLOs, Not vLLM Flags: Why This Job Didn't Exist in 2024
The role owns TTFT, TPOT, KV pressure, and $/token when the demo becomes traffic.
inference engineer, SLOs, vLLM, TTFT, TPOT, KV pressure, $/token
- Junior Throws GPUs. Principal Deletes Them: Why Over-Engineering Hides the Queue
The funny ladder is a critique of architecture theater.
over-engineering, queue, GPU, architecture theater
- The Full Story of LLM Inference: Six Layers from HTTP Request to Multi-GPU Execution
The map: gateway, engine, orchestration, compiler, CUDA, silicon.
LLM inference, gateway, engine, orchestration, compiler, CUDA, silicon, multi-GPU
- Abi's PMPP Notes: Execution Model, Memory Hierarchy, and Why More Work Can Be Faster
Kirk & Hwu: warps, coalescing, occupancy, latency hiding.
PMPP, warps, coalescing, occupancy, latency hiding, memory hierarchy
Archive
- Evaluation and Interpretation for LLMs
How evaluation changes when the model is generative.
- Introduction to Large Language Models
What LLMs are, how they are trained, and the cost of building them.
- Problems & Challenges in MLOps
Open problems in taking models to production.
- Problems & Challenges in AutoML
A 2019 survey of CASH, NAS, and autonomous model design.
- Developing A.I. using Games
Games as a testbed for machine intelligence.
- Why TensorLayer?
TensorLayer between TensorFlow and a usable deep learning API.
- Dense Associative Memory for Pattern Recognition
Notes on Krotov and Hopfield, NIPS 2016.
Projects I built
- Joule
Founder. Joule is an inference power economics engine that ties physical GPU energy to token throughput and SLO goodput.
Joule, joule.lat, inference power economics engine, physical GPU energy, token throughput, SLO goodput, phase-aware inference, power economics, digital twin
- gpuengineering.com
The public list behind the domain: CUDA, kernels, serving engines, multi-GPU systems. ~600 stars as awesome-gpu-engineering.
gpuengineering.com, awesome-gpu-engineering, CUDA, kernels, serving engines, multi-GPU systems
- RelayServe
Minimal LLM inference gateway for heterogeneous devices. OpenAI-compatible chat, batching, streaming, request IDs, /metrics. On PyPI as relayserve.
RelayServe, relayserve, LLM inference gateway, heterogeneous devices, OpenAI-compatible, chat, batching, streaming, request IDs, /metrics, PyPI
- smol-vllm
Educational paged-attention engine: KV cache, continuous batching, preemption, prefill vs decode. Built to be read, not shipped as production vLLM.
smol-vllm, smol_vllm, paged-attention, KV cache, continuous batching, preemption, prefill, decode, vLLM
- fullstack-inferencing
vLLM on Lambda Cloud, FastAPI gateway, nginx, Prometheus, Grafana, optional Jaeger. Documents engine-flag limits, cost metrics, and Lambda image caveats.
fullstack-inferencing, vLLM, Lambda Cloud, FastAPI, nginx, Prometheus, Grafana, Jaeger, engine-flag limits, cost metrics
- AbideX
Zero-code OpenTelemetry monitoring for agent workflows (CrewAI, LangGraph, Pydantic AI). The Abide-era observability product.
AbideX, OpenTelemetry, agent workflows, CrewAI, LangGraph, Pydantic AI, observability
- awesome-observability
Curated tools and frameworks for LLM observability.
awesome-observability, LLM observability, tracing, metrics
- llm-cluster-simulator
Browser simulator for distributed LLM training and inference: memory, throughput, cost, parallelism.
llm-cluster-simulator, distributed LLM, training, inference, memory, throughput, cost, parallelism
- ai-infra-fragility
Code for Cascading Fragility in National AI Infrastructure, market structure, exposure, network models. Submitting to AI & Society (Springer).
ai-infra-fragility, Cascading Fragility, national AI infrastructure, market structure, exposure, network models, AI & Society
Course labs
- nexus-workshop-code
Packt Nexus: build a first agent / multi-agent system with CrewAI, roles, tasks, orchestration.
nexus-workshop-code, Packt Nexus, CrewAI, multi-agent, roles, tasks, orchestration
- inferencing_maven
Labs for the Maven inference engineering cohort.
inferencing_maven, Maven, inference engineering, labs
- class-code
Class exercises that sit next to RelayServe and the serving labs.
class-code, RelayServe, serving labs
- ray_project
Student exercise on concurrency and profiling with Ray.
ray_project, Ray, concurrency, profiling