SLO-aware inference optimization
Latency contracts first. Energy and cost moves are not allowed to blow TTFT, TPOT, or the tail.
About
Abi Aryan
I am an AI Infrastructure Engineer building inference systems for data centers, neoclouds, and hyperscalers. I founded Joule, an inference power economics engine connecting physical GPU energy to token throughput and SLO goodput.
I have spent over a decade building production ML systems and work across the inference stack: runtimes, distributed systems, GPUs, compilers, networking, and cluster efficiency. I architect infrastructure for serving systems built on TensorRT-LLM, TorchDynamo, SGLang, and vLLM, with a focus on high-concurrency inference at scale, from requests and tokens to GPUs, network fabric, power, cooling, and data-center capacity.
I am a doctoral researcher in HPC distributed systems, studying SLO-aware inference optimization and hardware-aware compiler design for supercomputers, and developing a digital twin for LLM inference that lets operators manage energy while protecting latency SLOs.
I build, measure, research, write about, and teach these systems. I am an Invited Expert and Faculty Member on AI Inferencing for Andreessen Horowitz Academy (The Academy SF by a16z). I teach one of the highest-rated Maven courses on AI Inference Engineering & Systems Design, an 8-week live cohort for professionals looking for an engineering-heavy, practical infrastructure deep dive, and a shorter introductory course on AI Inference for FDE and PM Roles. Ex-students have been AI Infra Founders, Head of AI, AI engineers, Solutions Architects, Principal Engineers, SWEs from FAANG, National banks and Energy companies, from across 5 continents.
I am the author of LLMOps (2025), translated into Korean, Japanese, Russian, and Simplified Chinese, and What is LLMOps (O'Reilly, 2024). My next book, GPU Engineering: AI Inference and System Design (Packt), is nearing completion, with publication planned for late 2026 or early 2027. I write ModelCraft and review for NeurIPS, ACL, EMNLP, ICML, ICLR, TMLR, and AABI.
Previously, I co-founded Abide, a research lab focused on reflective intelligence and neuro-symbolic models for agents, where I contributed platform and observability solutions for agentic reasoning in LLMs. This work was presented at NeurIPS 2025. Abide is prior work; Joule is the company I run today.
A mathematician by training, I was a visiting research scholar at UCLA's Cognitive Systems Lab from 2018-19 under Dr. Judea Pearl, ACM 2012 Turing Award Winner, where my research spanned intelligence in agents, causal inference, AutoML, and multi-agent learning.
For talks, see Speaking. For audit and advisory work, see Advisory.
Doctoral work
Doctoral research in AI HPC distributed systems: SLO-aware inference optimization, hardware-aware compiler design, and a digital twin for energy optimization for bursty workloads.
Hardware-aware compiler design
The compiler has to see the GPU in front of it: memory hierarchy, occupancy, and the serving loop, not a generic IR.
Phase-aware observability
Prefill and decode are different workloads. The twin has to see which phase is burning watts and which is blowing the tail.
Education
- Doctoral research, AI HPC distributed systems
SLO-aware inference optimization, hardware-aware compiler design, and a digital twin for energy optimization for bursty workloads.
- Master's in Information Management Systems, NOVA IMS (Lisbon, Portugal)
Thesis: Cascading Fragility in National AI Infrastructure: A Seven-Layer Framework for Resilience Assessment and Policy Design. Master's thesis, NOVA IMS. Preparing for journal submission.
- Visiting Research Scholar, UCLA Cognitive Systems Lab (Los Angeles, USA)
Supervisor: Dr. Judea Pearl, ACM 2012 Turing Award Winner. Research on intelligence in agents, spanning causal inference, AutoML, emotion recognition, and multi-agent learning.
- MSc Applicable Mathematics, The London School of Economics and Political Science (London, UK)
Youngest postgraduate candidate at age 18. Thesis: Image Pattern Recognition using Hopfield Neural Networks. Courses included Algorithms and Computation, Game Theory, Cryptography, Control Optimization, Non-Linear Dynamics, Financial Risk Analysis, Behavioral Finance, and Quantitative Methods in Financial Modelling.
- BSc (Honors) Mathematics, MDU (India)
Distinction (Top 5). Thesis: Are Twin Primes Infinite? (Literature Review) Majors in pure and applied mathematics, with minors in statistics, computer science, and operations research. University record holder in Statistics.
Theses
- Cascading Fragility in National AI Infrastructure: A Seven-Layer Framework for Resilience Assessment and Policy Design
Master's thesis, NOVA IMS. By evaluating 22 global suppliers across a seven-layer dependency model, this work quantifies how critical single points of failure in semiconductor supply chains and hyperscalers trigger cascading systemic collapse. It provides a quantitative framework to measure concentration risk, guide sovereign capacity planning, and ensure continuity for mission-critical AI services. Preparing for journal submission.
- Image Pattern Recognition using Hopfield Neural Networks
MSc Applicable Mathematics, The London School of Economics and Political Science, 2013–2014. My early research in 2014 at LSE focused on energy-based associative memory (Hopfield networks), a class of architectures now recognized as mathematically isomorphic to modern Transformer self-attention. This thread of viewing compute through physical energy landscapes informs my current work on inference economics at Joule.
- Are Twin Primes Infinite? (Literature Review)
BSc (Honors) Mathematics, MDU.
Papers
- Adaptive Causal Consistency for Fault-Resilient GPU Clusters in Large-Scale AI Workloads
Working paper for DAIS 2026.
- Reflective Causal Agents: A NeuroSymbolic Approach to explainable LLMs
Working paper for KR 2026. OpenReview lE2UP2pZ1g.
- AbideGym: Turning Static RL Worlds into Adaptive Challenges
arXiv:2509.21234. In review at GECCO 2026.
- Causal Reflection with Language Models
NeurIPS 2025 Efficient Reasoning Workshop.
- Problems & Challenges in AutoML
With Shubhi Sareen.
Reviewing and service
Ongoing 2025
Jun 2024
Feb 2024
Mar 2024
Reviewer, AABI 2024
Oct 2023
Reviewer, NeurIPS Workshop: I (Still) Can't Believe It's Not Better
Sep 2023
Reviewer, DGM4H NeurIPS 2023
Mar 2023
Reviewer, AABI 2023
Sep 2022
Proposal reviewer, PyData NYC
Sep 2021
Reviewer, NeurIPS Workshop: I (Still) Can't Believe It's Not Better
Aug 2021
Research mentorship, Association for Computational Linguistics
Nov 2018
Area chair, AutoML, NeurIPS 2018
May–Sep 2018
Organising committee co-chair, PyData Los Angeles
2016–2021
Director, Women Who Code Los Angeles
Media
October 2024I was featured on the Times Square Billboard by Topmate.