SLO-aware inference optimization
Latency contracts first. Energy and cost moves are not allowed to blow TTFT, TPOT, or the tail.
About
Abi Aryan
I am an AI Infrastructure Engineer building inference systems for data centers, neoclouds, and hyperscalers. I founded Joule, an inference power economics engine that ties physical GPU energy to token throughput and SLO goodput.
I have spent over a decade building production ML systems. Today, I work at the intersection of inference, GPUs, distributed systems, and AI infrastructure: from the runtime down to the hardware.
I build the systems, measure them, research them, write about them, and teach them. I am a doctoral researcher in HPC distributed systems, studying SLO-aware inference optimization and hardware-aware compiler design. I am the author of LLMOps (written 2023-2024, finished December 2024, published 2025, in English, now translated into Korean, Japanese, Russian, and Simplified Chinese) and the report What is LLMOps (written 2023, published 2024, O'Reilly). GPU Engineering: AI Inference and System Design (Packt) is almost finished, launch is end of 2026 or early 2027.
Invited Faculty for AI Inference Engineering at Andreessen Horowitz Academy (The Academy SF).
I am developing a digital twin for LLM inference so operators can manage energy while latency SLOs stay intact. That work sits in Joule.
Previously, I founded Abide, a lab for reflective intelligence and neuro-symbolic models for agents. That research was presented at NeurIPS 2025. Abide is prior work. Joule is the company I run now.
I am a mathematician by training. I was a visiting research scholar at UCLA's Cognitive Systems Lab under the 2012 Turing Award winner, Dr. Judea Pearl, working on causal inference, AutoML, and multi-agent learning.
I work on performance-critical inference runtimes (TensorRT-LLM, TorchDynamo, SGLang, vLLM) and GPU cluster efficiency. I am not trying to be the person who maintains those engines. I am trying to make the serving path intelligible, and to do original work in the same territory.
I teach popular, highly rated masterclasses on Maven: AI Inference Engineering & Systems Design for mid-to-senior engineers, and AI Inference for FDE and PM Roles. I write ModelCraft. I review for NeurIPS, ACL, EMNLP, ICML, ICLR, TMLR, and AABI.
For talks, see speaking. For the audit and the seat, see advisory.
Doctoral work
I am doing doctoral research in HPC distributed systems: SLO-aware inference optimization and hardware-aware compiler design. I am developing a digital twin that lets operators manage energy while protecting latency SLOs, via phase-aware observability, predictive forecasting, and human-centred decision support.
Hardware-aware compiler design
The compiler has to see the GPU in front of it: memory hierarchy, occupancy, and the serving loop, not a generic IR.
Phase-aware observability
Prefill and decode are different workloads. The twin has to see which phase is burning watts and which is blowing the tail.
Education
- Doctoral research, HPC distributed systems
SLO-aware inference optimization, hardware-aware compiler design, and a digital twin for energy against latency SLOs.
- Master's in Information Management Systems, NOVA IMS
Thesis: Cascading Fragility in National AI Infrastructure.
- MSc Applicable Mathematics, The London School of Economics and Political Science (LSE)
Thesis: Pattern Recognition using Hopfield Neural Networks. Courses included algorithms and computation, game theory, cryptography, control optimization, non-linear dynamics, corporate finance, behavioral finance, and quantitative methods in finance.
- BSc (Honors) Mathematics, Maharshi Dayanand University
Distinction (top 5). University record holder in Statistics. Pure and applied mathematics, with statistics, computer science, and operations research. Thesis: literature review on whether twin primes are infinite.
Theses
- Cascading Fragility in National AI Infrastructure
Master's thesis for the Master's in Information Management Systems at NOVA IMS. Submitting to AI & Society (Springer). Code on GitHub.
- Pattern Recognition using Hopfield Neural Networks
MSc Applicable Mathematics, The London School of Economics and Political Science (LSE), 2013-2014. Image pattern recognition using Hopfield models, energy-based recurrent neural networks.
- Literature Review on the Open Question in Mathematics: Are Twin Primes Infinite
BSc (Honors) Mathematics, Maharshi Dayanand University. Distinction (top 5). University record holder in Statistics.
Papers
- Adaptive Causal Consistency for Fault-Resilient GPU Clusters in Large-Scale AI Workloads
Working paper for DAIS 2026.
- Reflective Causal Agents: A NeuroSymbolic Approach to explainable LLMs
Working paper for KR 2026. OpenReview lE2UP2pZ1g.
- AbideGym: Turning Static RL Worlds into Adaptive Challenges
arXiv:2509.21234. In review at GECCO 2026.
- Causal Reflection with Language Models
NeurIPS 2025 Efficient Reasoning Workshop.
- Problems & Challenges in AutoML
With Shubhi Sareen.
Reviewing and service
Ongoing 2025
Jun 2024
Feb 2024
Mar 2024
Reviewer, AABI 2024
Oct 2023
Reviewer, NeurIPS Workshop: I (Still) Can't Believe It's Not Better
Sep 2023
Reviewer, DGM4H NeurIPS 2023
Mar 2023
Reviewer, AABI 2023
Sep 2022
Proposal reviewer, PyData NYC
Sep 2021
Reviewer, NeurIPS Workshop: I (Still) Can't Believe It's Not Better
Aug 2021
Research mentorship, Association for Computational Linguistics
Nov 2018
Area chair, AutoML, NeurIPS 2018
May–Sep 2018
Organising committee co-chair, PyData Los Angeles
2016–2021
Director, Women Who Code Los Angeles
Media
October 2024I was featured on the Times Square Billboard by Topmate.