# Abi Aryan

> Hi, I am Abi. I build inference systems for data centers, neoclouds, and hyperscalers. Founder of Joule. I teach on Maven.

Hi, I am Abi. I am an AI Infrastructure Engineer building inference systems for data centers, neoclouds, and hyperscalers. I write and teach inference engineering: serving path, KV cache, SLOs, GPUs, and the job around them.

## Joule
- [Joule](https://joule.lat/): Joule is an inference power economics engine that ties physical GPU energy to token throughput and SLO goodput.

## Teaching / Maven
- School: [https://maven.com/goabiaryan](https://maven.com/goabiaryan). 1.7K subscribers. amongst the top-most rated instructors on Maven.
- Invited Expert and Faculty Member on AI Inferencing for [Andreessen Horowitz Academy (The Academy SF)](https://theacademysf.com/faculty-hiring-partners).
- [AI Inference Engineering & Systems Design](https://maven.com/goabiaryan/inferencing) (8 weeks): A highly rated masterclass for mid-to-senior engineers. Build the stack by hand. Next live cohort 24 Oct - 19 Dec 2026.
- [AI Inference for FDE and PM Roles](https://maven.com/goabiaryan/inferencing-intro) (Inferencing 101): A masterclass on what is happening inside the model, for FDE, PM, and adjacent technical roles.
- Lightning lesson: [The Future of LLM Serving](https://maven.com/p/1b67c0/) (45 min)
- Lightning lesson: [How to do Observability right?](https://maven.com/p/d74587/) (60 min)
- Lightning lesson: [How much will my model cost? Napkin Maths 101](https://maven.com/p/f8237b/) (60 min)
- Lightning lesson: [What happens when you make an LLM call?](https://maven.com/p/52b7f0/) (45 min)
- Lightning lesson: [Whats next for AI Engineers?](https://maven.com/p/e10955/) (60 min)
- Lightning lesson: [AI Careers in 2025+: What No One's Telling You](https://maven.com/p/4ee6de/) (30 min)
- 1:1 mentorship: [https://topmate.io/goabiaryan/294674](https://topmate.io/goabiaryan/294674)

## The stack I teach
- 01 Model & Framework: Transformers, GGUF, Hugging Face, PyTorch, Accelerate
- 02 Compiler & Kernel: LLVM, Triton, TorchDynamo, TorchInductor, CUDA Graphs, PTX, SASS, FlashAttention, CUTLASS
- 03 Memory & Storage: HBM, DDR, PagedAttention, Mooncake, LMCache, KV cache, Lustre, NVMe
- 04 GPU & Communication: CUDA, MIG, DRA, HAMi, DCGM, DGCM, NCCL, MPI, NIXL, NVLink/NVSwitch, UCX, GPUDirect RDMA
- 05 Networking: InfiniBand, UFM, RDMA/RoCE, Ethernet, CXL, PCIe
- 06 Serving: vLLM, SGLang, TensorRT-LLM, KServe, llm-d, llama.cpp, OpenJoule, RelayServe, LiteLLM, Portkey, OpenRouter
- 07 Scheduling & Orchestration: Kubernetes, EKS, GKE, Ray, KubeRay, Volcano, Slurm, Flux, Kueue, dstack, Karpenter, KEDA, HPA, Gateway API, Enroot, xCAT
- 08 Automation: Terraform, Ansible
- 09 Observability & Performance: Prometheus, Grafana, Jaeger, Locust, Nsight Systems, Nsight Compute, Perf, HPCG
- 10 Facility & Hardware Management: DCIM platforms, BMS/BAS, Modbus, BACnet, SNMP, OPC UA, Redfish, IPMI, BMC
- 11 $ / Token: Lambda Cloud, Modal

## Doctoral research
- AI HPC distributed systems.
Doctoral research in AI HPC distributed systems: SLO-aware inference optimization, hardware-aware compiler design, and a digital twin for energy optimization for bursty workloads.
- SLO-aware inference optimization: Latency contracts first. Energy and cost moves are not allowed to blow TTFT, TPOT, or the tail.
- Hardware-aware compiler design: The compiler has to see the GPU in front of it: memory hierarchy, occupancy, and the serving loop, not a generic IR.
- Phase-aware observability: Prefill and decode are different workloads. The twin has to see which phase is burning watts and which is blowing the tail.

## Education and theses
- Now Doctoral research, AI HPC distributed systems: SLO-aware inference optimization, hardware-aware compiler design, and a digital twin for energy optimization for bursty workloads.
- 2024–2026 Master's in Information Management Systems, NOVA IMS (Lisbon, Portugal): Thesis: Cascading Fragility in National AI Infrastructure: A Seven-Layer Framework for Resilience Assessment and Policy Design.
Master's thesis, NOVA IMS. Preparing for journal submission.
- 2018–2019 Visiting Research Scholar, UCLA Cognitive Systems Lab (Los Angeles, USA): Supervisor: Dr. Judea Pearl, ACM 2012 Turing Award Winner. Research on intelligence in agents, spanning causal inference, AutoML, emotion recognition, and multi-agent learning.
- 2013–2014 MSc Applicable Mathematics, The London School of Economics and Political Science (London, UK): Youngest postgraduate candidate at age 18.
Thesis: Image Pattern Recognition using Hopfield Neural Networks.

Courses included Algorithms and Computation, Game Theory, Cryptography, Control Optimization, Non-Linear Dynamics, Financial Risk Analysis, Behavioral Finance, and Quantitative Methods in Financial Modelling.
- 2010–2013 BSc (Honors) Mathematics, MDU (India): Distinction (Top 5).
Thesis: Are Twin Primes Infinite? (Literature Review)

Majors in pure and applied mathematics, with minors in statistics, computer science, and operations research. University record holder in Statistics.
- Thesis 2026: Cascading Fragility in National AI Infrastructure: A Seven-Layer Framework for Resilience Assessment and Policy Design. Master's thesis, NOVA IMS.

By evaluating 22 global suppliers across a seven-layer dependency model, this work quantifies how critical single points of failure in semiconductor supply chains and hyperscalers trigger cascading systemic collapse. It provides a quantitative framework to measure concentration risk, guide sovereign capacity planning, and ensure continuity for mission-critical AI services. Preparing for journal submission.
- Thesis 2014: Image Pattern Recognition using Hopfield Neural Networks. MSc Applicable Mathematics, The London School of Economics and Political Science, 2013–2014.

My early research in 2014 at LSE focused on energy-based associative memory (Hopfield networks), a class of architectures now recognized as mathematically isomorphic to modern Transformer self-attention. This thread of viewing compute through physical energy landscapes informs my current work on inference economics at Joule.
- Thesis 2013: Are Twin Primes Infinite? (Literature Review). BSc (Honors) Mathematics, MDU.

## Books
- [GPU Engineering: AI Inference and System Design](https://abiaryan.com/books/): Packt. High-throughput inference serving from the gateway to bare metal: concurrency, multi-tenant isolation, control versus data plane, then fabric, silicon, compilers, and kernels. Almost finished. Launch is end of 2026 or early 2027.
- [LLMOps](https://www.oreilly.com/library/view/llmops/9781098154196/): O'Reilly book. Written 2023-2024, finished December 2024, published 2025. In English, now translated into Korean, Japanese, Russian, and Simplified Chinese.
- [What is LLMOps](https://www.oreilly.com/library/view/what-is-llmops/9781098154301/): O'Reilly report. Written 2023, published 2024. A separate brief from the LLMOps book.

## Pages
- [Home](https://abiaryan.com/)
- [About](https://abiaryan.com/about/)
- [Writing](https://abiaryan.com/writing/)
- [Books](https://abiaryan.com/books/)
- [Teaching](https://abiaryan.com/teaching/)
- [Advisory](https://abiaryan.com/advisory/)
- [Code](https://abiaryan.com/code/)
- [Speaking](https://abiaryan.com/speaking/)
- [Conferences](https://abiaryan.com/conferences/)
- [llms.txt](https://abiaryan.com/llms.txt)
- [llms-full.txt](https://abiaryan.com/llms-full.txt)

- [Sitemap](https://abiaryan.com/sitemap/)

## Investigations
- [Why Your AI Factory Is Secretly a Dynamical System (And 5 Other Counter-Intuitive Truths About Modern AI Inference)](https://abiaryan.com/writing/ai-factory-dynamical-system/): Power, cooling, KV, queues, and retries are one coupled system, not a dashboard of independent numbers. Keywords: AID, AI Infrastructure Dynamics, OpenJoule, AI factory, dynamical system, KV cache, OOM, workload burstiness, retry storm, PagedAttention, RadixAttention, intervention.
- [Why GPU Utilization Is a Misleading Metric for LLM Inference: Queueing, KV Cache, and SLOs](https://abiaryan.com/writing/eighty-busy-still-slow/): GPU utilization is not goodput. Queueing, KV pressure, and why $97 vs $0.38 / MTok. Keywords: GPU utilization, goodput, queueing, KV cache, SLOs, TTFT, TPOT, $/MTok.
- [Why KV Cache Needs a Directory: Locality, Leases, and Cache-Aware Routing](https://abiaryan.com/writing/kv-needs-a-phone-book/): On one GPU, KV is an allocator. Across a cluster it is a phone book. Keywords: KV cache, directory, locality, leases, cache-aware routing, allocator.
- [Why the Same GPU Serves One Request in 3.3 Seconds and Another in 39: Request Shape, Prefill, and Decode](https://abiaryan.com/writing/same-gpu-different-century/): Same model, same card, same code. The request shape did it. Keywords: request shape, prefill, decode, latency, same GPU.
- [One Inference Roadmap, Not Fifty Tools: Request Path Before CUDA](https://abiaryan.com/writing/one-roadmap/): Path, phases, memory, queue, then kernels. Not a bookmark pile. Keywords: inference roadmap, request path, CUDA, phases, memory, queue, kernels.
- [Why Agent Workloads OOM at 62% GPU Utilization: KV Cache, Multi-Turn State, and a $27k Weekend](https://abiaryan.com/writing/sunday-slack-27k/): The demo had headroom. Saturday agents leased HBM. Seven of twelve replicas died. Keywords: agent workloads, OOM, GPU utilization, KV cache, multi-turn, HBM, $27k.
- [The Inference Engineer Owns SLOs, Not vLLM Flags: Why This Job Didn't Exist in 2024](https://abiaryan.com/writing/job-that-didnt-exist/): The role owns TTFT, TPOT, KV pressure, and $/token when the demo becomes traffic. Keywords: inference engineer, SLOs, vLLM, TTFT, TPOT, KV pressure, $/token.
- [Junior Throws GPUs. Principal Deletes Them: Why Over-Engineering Hides the Queue](https://abiaryan.com/writing/junior-throws-gpus/): The funny ladder is a critique of architecture theater. Keywords: over-engineering, queue, GPU, architecture theater.
- [CASCADE: Why Fault Tolerance Without Adaptive Consistency Still Breaks GPU Clusters](https://abiaryan.com/writing/cascade/): Horovod, Ray, and DeepSpeed recover the job. They do not change how the cluster is allowed to disagree. Keywords: CASCADE, adaptive consistency, causal consistency, fault tolerance, GPU clusters, Horovod, Ray, DeepSpeed.
- [The Full Story of LLM Inference: Six Layers from HTTP Request to Multi-GPU Execution](https://abiaryan.com/writing/llm-inference/): The map: gateway, engine, orchestration, compiler, CUDA, silicon. Keywords: LLM inference, gateway, engine, orchestration, compiler, CUDA, silicon, multi-GPU.
- [Abi's PMPP Notes: Execution Model, Memory Hierarchy, and Why More Work Can Be Faster](https://abiaryan.com/writing/pmpp-notes/): Kirk & Hwu: warps, coalescing, occupancy, latency hiding. Keywords: PMPP, warps, coalescing, occupancy, latency hiding, memory hierarchy.

## Archive writing
- [Evaluation and Interpretation for LLMs](https://abiaryan.com/writing/llm-evals/): How evaluation changes when the model is generative.
- [Introduction to Large Language Models](https://abiaryan.com/writing/intro-llms/): What LLMs are, how they are trained, and the cost of building them.
- [Problems & Challenges in MLOps](https://abiaryan.com/writing/mlops-open-problems/): Open problems in taking models to production.
- [Problems & Challenges in AutoML](https://abiaryan.com/writing/automl-problems/): A 2019 survey of CASH, NAS, and autonomous model design.
- [Developing A.I. using Games](https://abiaryan.com/writing/game-ai/): Games as a testbed for machine intelligence.
- [Why TensorLayer?](https://abiaryan.com/writing/why-tensorlayer/): TensorLayer between TensorFlow and a usable deep learning API.
- [Dense Associative Memory for Pattern Recognition](https://abiaryan.com/writing/associative-memory/): Notes on Krotov and Hopfield, NIPS 2016.

## Projects I built
- [Joule](https://joule.lat/) (Company): Founder. Joule is an inference power economics engine that ties physical GPU energy to token throughput and SLO goodput. Keywords: Joule, joule.lat, inference power economics engine, physical GPU energy, token throughput, SLO goodput, phase-aware inference, power economics, digital twin.
- [OpenJoule](https://github.com/joule-lat/OpenJoule) (Engine): Open-source plant-level inference and control engine. Instantiates AID in a real system. Keywords: OpenJoule, openjoule, AID, AI Infrastructure Dynamics, plant, inference control, vLLM.
- [gpuengineering.com](https://gpuengineering.com/) (Curriculum): The public list behind the domain: CUDA, kernels, serving engines, multi-GPU systems. ~600 stars as awesome-gpu-engineering. Keywords: gpuengineering.com, awesome-gpu-engineering, CUDA, kernels, serving engines, multi-GPU systems.
- [RelayServe](https://github.com/goabiaryan/RelayServe) (Serving): Minimal LLM inference gateway for heterogeneous devices. OpenAI-compatible chat, batching, streaming, request IDs, /metrics. On PyPI as relayserve. Keywords: RelayServe, relayserve, LLM inference gateway, heterogeneous devices, OpenAI-compatible, chat, batching, streaming, request IDs, /metrics, PyPI.
- [smol-vllm](https://github.com/goabiaryan/smol_vllm) (Engine): Educational paged-attention engine: KV cache, continuous batching, preemption, prefill vs decode. Built to be read, not shipped as production vLLM. Keywords: smol-vllm, smol_vllm, paged-attention, KV cache, continuous batching, preemption, prefill, decode, vLLM.
- [fullstack-inferencing](https://github.com/goabiaryan/fullstack-inferencing) (Lab / Lambda): vLLM on Lambda Cloud, FastAPI gateway, nginx, Prometheus, Grafana, optional Jaeger. Documents engine-flag limits, cost metrics, and Lambda image caveats. Keywords: fullstack-inferencing, vLLM, Lambda Cloud, FastAPI, nginx, Prometheus, Grafana, Jaeger, engine-flag limits, cost metrics.
- [AbideX](https://github.com/abide-ai/abidex) (Observability): Zero-code OpenTelemetry monitoring for agent workflows (CrewAI, LangGraph, Pydantic AI). The Abide-era observability product. Keywords: AbideX, OpenTelemetry, agent workflows, CrewAI, LangGraph, Pydantic AI, observability.
- [awesome-observability](https://github.com/goabiaryan/awesome-observability) (Observability): Curated tools and frameworks for LLM observability. Keywords: awesome-observability, LLM observability, tracing, metrics.
- [llm-cluster-simulator](https://github.com/goabiaryan/llm-cluster-simulator) (Systems): Browser simulator for distributed LLM training and inference: memory, throughput, cost, parallelism. Keywords: llm-cluster-simulator, distributed LLM, training, inference, memory, throughput, cost, parallelism.
- [ai-infra-fragility](https://github.com/goabiaryan/ai-infra-fragility) (Thesis): Code for Cascading Fragility in National AI Infrastructure, market structure, exposure, network models. Submitting to AI & Society (Springer). Keywords: ai-infra-fragility, Cascading Fragility, national AI infrastructure, market structure, exposure, network models, AI & Society.

## Course labs
- [nexus-workshop-code](https://github.com/goabiaryan/nexus-workshop-code) (Workshop): Packt Nexus: build a first agent / multi-agent system with CrewAI, roles, tasks, orchestration. Keywords: nexus-workshop-code, Packt Nexus, CrewAI, multi-agent, roles, tasks, orchestration.
- [concurrency](https://github.com/goabiaryan/concurrency) (Course): Concurrency exercises for inference workloads. Keywords: concurrency, inference workloads.
- [ray_project](https://github.com/goabiaryan/ray_project) (Course): Student exercise on concurrency and profiling with Ray. Keywords: ray_project, Ray, concurrency, profiling.

## Elsewhere
- [ModelCraft](https://modelcraft.substack.com/)
- [GitHub](https://github.com/goabiaryan)
- [LinkedIn](https://www.linkedin.com/in/goabiaryan/)
- [X](https://twitter.com/GoAbiAryan)
- [YouTube](https://www.youtube.com/@goabiaryan)
- [Podcast](https://modelcraft.substack.com/podcast)
- [gpuengineering.com](https://gpuengineering.com/)

## Papers
- 2026 AID: A Framework for AI Infrastructure Dynamics (https://arxiv.org/abs/2610.04801). arXiv:2610.04801. October 2026.
- 2026 Adaptive Causal Consistency for Fault-Resilient GPU Clusters in Large-Scale AI Workloads. Working paper for DAIS 2026.
- 2026 Reflective Causal Agents: A NeuroSymbolic Approach to explainable LLMs (https://openreview.net/forum?id=lE2UP2pZ1g). Working paper for KR 2026. OpenReview lE2UP2pZ1g.
- 2026 AbideGym: Turning Static RL Worlds into Adaptive Challenges (https://arxiv.org/abs/2509.21234). arXiv:2509.21234. In review at GECCO 2026.
- 2025 Causal Reflection with Language Models (https://arxiv.org/abs/2508.04495). NeurIPS 2025 Efficient Reasoning Workshop.
- 2024 The Costly Dilemma: Generalization, Evaluation and Cost Optimal Deployment of Large Language Models (https://arxiv.org/abs/2308.08061). arXiv:2308.08061.
- 2020 Arena: A General Evaluation Platform and Building Toolkit for Multi-Agent Intelligence (https://ojs.aaai.org/index.php/AAAI/article/view/6216). AAAI 2020.
- 2019 Problems & Challenges in AutoML. With Shubhi Sareen.

## Stack tools, one per line
- Transformers (Model & Framework layer, taught by Abi Aryan)
- GGUF (Model & Framework layer, taught by Abi Aryan)
- Hugging Face (Model & Framework layer, taught by Abi Aryan)
- PyTorch (Model & Framework layer, taught by Abi Aryan)
- Accelerate (Model & Framework layer, taught by Abi Aryan)
- LLVM (Compiler & Kernel layer, taught by Abi Aryan)
- Triton (Compiler & Kernel layer, taught by Abi Aryan)
- TorchDynamo (Compiler & Kernel layer, taught by Abi Aryan)
- TorchInductor (Compiler & Kernel layer, taught by Abi Aryan)
- CUDA Graphs (Compiler & Kernel layer, taught by Abi Aryan)
- PTX (Compiler & Kernel layer, taught by Abi Aryan)
- SASS (Compiler & Kernel layer, taught by Abi Aryan)
- FlashAttention (Compiler & Kernel layer, taught by Abi Aryan)
- CUTLASS (Compiler & Kernel layer, taught by Abi Aryan)
- HBM (Memory & Storage layer, taught by Abi Aryan)
- DDR (Memory & Storage layer, taught by Abi Aryan)
- PagedAttention (Memory & Storage layer, taught by Abi Aryan)
- Mooncake (Memory & Storage layer, taught by Abi Aryan)
- LMCache (Memory & Storage layer, taught by Abi Aryan)
- KV cache (Memory & Storage layer, taught by Abi Aryan)
- Lustre (Memory & Storage layer, taught by Abi Aryan)
- NVMe (Memory & Storage layer, taught by Abi Aryan)
- CUDA (GPU & Communication layer, taught by Abi Aryan)
- MIG (GPU & Communication layer, taught by Abi Aryan)
- DRA (GPU & Communication layer, taught by Abi Aryan)
- HAMi (GPU & Communication layer, taught by Abi Aryan)
- DCGM (GPU & Communication layer, taught by Abi Aryan)
- DGCM (GPU & Communication layer, taught by Abi Aryan)
- NCCL (GPU & Communication layer, taught by Abi Aryan)
- MPI (GPU & Communication layer, taught by Abi Aryan)
- NIXL (GPU & Communication layer, taught by Abi Aryan)
- NVLink/NVSwitch (GPU & Communication layer, taught by Abi Aryan)
- UCX (GPU & Communication layer, taught by Abi Aryan)
- GPUDirect RDMA (GPU & Communication layer, taught by Abi Aryan)
- InfiniBand (Networking layer, taught by Abi Aryan)
- UFM (Networking layer, taught by Abi Aryan)
- RDMA/RoCE (Networking layer, taught by Abi Aryan)
- Ethernet (Networking layer, taught by Abi Aryan)
- CXL (Networking layer, taught by Abi Aryan)
- PCIe (Networking layer, taught by Abi Aryan)
- vLLM (Serving layer, taught by Abi Aryan)
- SGLang (Serving layer, taught by Abi Aryan)
- TensorRT-LLM (Serving layer, taught by Abi Aryan)
- KServe (Serving layer, taught by Abi Aryan)
- llm-d (Serving layer, taught by Abi Aryan)
- llama.cpp (Serving layer, taught by Abi Aryan)
- OpenJoule (Serving layer, taught by Abi Aryan)
- RelayServe (Serving layer, taught by Abi Aryan)
- LiteLLM (Serving layer, taught by Abi Aryan)
- Portkey (Serving layer, taught by Abi Aryan)
- OpenRouter (Serving layer, taught by Abi Aryan)
- Kubernetes (Scheduling & Orchestration layer, taught by Abi Aryan)
- EKS (Scheduling & Orchestration layer, taught by Abi Aryan)
- GKE (Scheduling & Orchestration layer, taught by Abi Aryan)
- Ray (Scheduling & Orchestration layer, taught by Abi Aryan)
- KubeRay (Scheduling & Orchestration layer, taught by Abi Aryan)
- Volcano (Scheduling & Orchestration layer, taught by Abi Aryan)
- Slurm (Scheduling & Orchestration layer, taught by Abi Aryan)
- Flux (Scheduling & Orchestration layer, taught by Abi Aryan)
- Kueue (Scheduling & Orchestration layer, taught by Abi Aryan)
- dstack (Scheduling & Orchestration layer, taught by Abi Aryan)
- Karpenter (Scheduling & Orchestration layer, taught by Abi Aryan)
- KEDA (Scheduling & Orchestration layer, taught by Abi Aryan)
- HPA (Scheduling & Orchestration layer, taught by Abi Aryan)
- Gateway API (Scheduling & Orchestration layer, taught by Abi Aryan)
- Enroot (Scheduling & Orchestration layer, taught by Abi Aryan)
- xCAT (Scheduling & Orchestration layer, taught by Abi Aryan)
- Terraform (Automation layer, taught by Abi Aryan)
- Ansible (Automation layer, taught by Abi Aryan)
- Prometheus (Observability & Performance layer, taught by Abi Aryan)
- Grafana (Observability & Performance layer, taught by Abi Aryan)
- Jaeger (Observability & Performance layer, taught by Abi Aryan)
- Locust (Observability & Performance layer, taught by Abi Aryan)
- Nsight Systems (Observability & Performance layer, taught by Abi Aryan)
- Nsight Compute (Observability & Performance layer, taught by Abi Aryan)
- Perf (Observability & Performance layer, taught by Abi Aryan)
- HPCG (Observability & Performance layer, taught by Abi Aryan)
- DCIM platforms (Facility & Hardware Management layer, taught by Abi Aryan)
- BMS/BAS (Facility & Hardware Management layer, taught by Abi Aryan)
- Modbus (Facility & Hardware Management layer, taught by Abi Aryan)
- BACnet (Facility & Hardware Management layer, taught by Abi Aryan)
- SNMP (Facility & Hardware Management layer, taught by Abi Aryan)
- OPC UA (Facility & Hardware Management layer, taught by Abi Aryan)
- Redfish (Facility & Hardware Management layer, taught by Abi Aryan)
- IPMI (Facility & Hardware Management layer, taught by Abi Aryan)
- BMC (Facility & Hardware Management layer, taught by Abi Aryan)
- Lambda Cloud ($ / Token layer, taught by Abi Aryan)
- Modal ($ / Token layer, taught by Abi Aryan)

## Project keywords, one per line
- Joule (Joule, Company)
- joule.lat (Joule, Company)
- inference power economics engine (Joule, Company)
- physical GPU energy (Joule, Company)
- token throughput (Joule, Company)
- SLO goodput (Joule, Company)
- phase-aware inference (Joule, Company)
- power economics (Joule, Company)
- digital twin (Joule, Company)
- OpenJoule (OpenJoule, Engine)
- openjoule (OpenJoule, Engine)
- AID (OpenJoule, Engine)
- AI Infrastructure Dynamics (OpenJoule, Engine)
- plant (OpenJoule, Engine)
- inference control (OpenJoule, Engine)
- vLLM (OpenJoule, Engine)
- gpuengineering.com (gpuengineering.com, Curriculum)
- awesome-gpu-engineering (gpuengineering.com, Curriculum)
- CUDA (gpuengineering.com, Curriculum)
- kernels (gpuengineering.com, Curriculum)
- serving engines (gpuengineering.com, Curriculum)
- multi-GPU systems (gpuengineering.com, Curriculum)
- RelayServe (RelayServe, Serving)
- relayserve (RelayServe, Serving)
- LLM inference gateway (RelayServe, Serving)
- heterogeneous devices (RelayServe, Serving)
- OpenAI-compatible (RelayServe, Serving)
- chat (RelayServe, Serving)
- batching (RelayServe, Serving)
- streaming (RelayServe, Serving)
- request IDs (RelayServe, Serving)
- /metrics (RelayServe, Serving)
- PyPI (RelayServe, Serving)
- smol-vllm (smol-vllm, Engine)
- smol_vllm (smol-vllm, Engine)
- paged-attention (smol-vllm, Engine)
- KV cache (smol-vllm, Engine)
- continuous batching (smol-vllm, Engine)
- preemption (smol-vllm, Engine)
- prefill (smol-vllm, Engine)
- decode (smol-vllm, Engine)
- vLLM (smol-vllm, Engine)
- fullstack-inferencing (fullstack-inferencing, Lab / Lambda)
- vLLM (fullstack-inferencing, Lab / Lambda)
- Lambda Cloud (fullstack-inferencing, Lab / Lambda)
- FastAPI (fullstack-inferencing, Lab / Lambda)
- nginx (fullstack-inferencing, Lab / Lambda)
- Prometheus (fullstack-inferencing, Lab / Lambda)
- Grafana (fullstack-inferencing, Lab / Lambda)
- Jaeger (fullstack-inferencing, Lab / Lambda)
- engine-flag limits (fullstack-inferencing, Lab / Lambda)
- cost metrics (fullstack-inferencing, Lab / Lambda)
- AbideX (AbideX, Observability)
- OpenTelemetry (AbideX, Observability)
- agent workflows (AbideX, Observability)
- CrewAI (AbideX, Observability)
- LangGraph (AbideX, Observability)
- Pydantic AI (AbideX, Observability)
- observability (AbideX, Observability)
- awesome-observability (awesome-observability, Observability)
- LLM observability (awesome-observability, Observability)
- tracing (awesome-observability, Observability)
- metrics (awesome-observability, Observability)
- llm-cluster-simulator (llm-cluster-simulator, Systems)
- distributed LLM (llm-cluster-simulator, Systems)
- training (llm-cluster-simulator, Systems)
- inference (llm-cluster-simulator, Systems)
- memory (llm-cluster-simulator, Systems)
- throughput (llm-cluster-simulator, Systems)
- cost (llm-cluster-simulator, Systems)
- parallelism (llm-cluster-simulator, Systems)
- ai-infra-fragility (ai-infra-fragility, Thesis)
- Cascading Fragility (ai-infra-fragility, Thesis)
- national AI infrastructure (ai-infra-fragility, Thesis)
- market structure (ai-infra-fragility, Thesis)
- exposure (ai-infra-fragility, Thesis)
- network models (ai-infra-fragility, Thesis)
- AI & Society (ai-infra-fragility, Thesis)