Emerging Technologies and Intelligent Systems: Theory, Applications and Innovations
HELIOS: Failure-Aware Distributed Scheduling for Heterogeneous AI Services
Authors: Mohammed Kamran Syed, Anudeep Rentala, Somesh Rahul, Raghuram Malpe Pai and Krishna Suman Dara
Publishing Date: 15-11-2026
ISBN: 978-81-975670-4-9
Abstract
Distributed AI services now run across mixed GPU generations, CPU pools, memory tiers, network domains, caches, replicas, and geographically separated clusters. Most cluster schedulers still optimize for instantaneous load, locality, or fairness, so they cope poorly when a workload burst, a degrading network link, and a component failure all hit at once. HELIOS is a failure-aware distributed scheduler that ties placement, locality, replication, migration, and recovery into a single decision loop instead of running them as separate reactive mechanisms. It keeps a resource and failure-risk graph up to date, forecasts short-horizon saturation, scores candidate placements on latency, locality, cost, and resilience, and moves state, through replication or live migration, before an overload becomes visible to users. On heterogeneous AI workloads, HELIOS cuts P95 latency by 35.8% and P99 by 38.3%, lifts throughput 1.34×, pushes GPU utilization to 84.6%, and cuts SLA violations by 62.8% against round-robin scheduling. Under injected failures it recovers successfully 93.7% of the time on average and cuts weighted MTTR by 53.6%. It does this for 5.4 ms of mean scheduling overhead and 18.9% lower operating cost overall. The takeaway is that performance, placement, and recovery work better as one coordinated control problem than as three independent ones.
Keywords
Distributed systems, cluster scheduling, heterogeneous computing, fault tolerance, GPU scheduling, predictive placement, live migration, replication, AI infrastructure.
Cite as
Mohammed Kamran Syed, Anudeep Rentala, Somesh Rahul, Raghuram Malpe Pai and Krishna Suman Dara, "HELIOS: Failure-Aware Distributed Scheduling for Heterogeneous AI Services", In: Harish Sharma and Mukesh Saraswat (eds), Emerging Technologies and Intelligent Systems: Theory, Applications and Innovations, SCRS, India, 2026, pp. 120-141. https://doi.org/10.56155/978-81-975670-4-9-7