Emerging Technologies and Intelligent Systems: Theory, Applications and Innovations
POLARIS: Policy-Orchestrated Adaptive LLM Inference Across Heterogeneous
Authors: Somesh Rahul, Krishna Suman Dara and Raghuram Malpe Pai
Publishing Date: 15-11-2026
ISBN: 978-81-975670-4-9
Abstract
Production language-model serving now composes retrieval, multimodal encoders, speculative decoding, quantization, edge–cloud partitioning, privacy isolation and fault recovery, and most serving frameworks tune those mechanisms one at a time inside a fixed pipeline. POLARIS treats serving as a constrained online composition problem instead: it derives a policy vector per request, prunes a capability graph down to plans that satisfy the request’s hard constraints, and scores what survives against a published objective. We evaluate it in a discrete-event simulation of 12,000 requests across six workload classes, against six independently implemented baseline policies and an offline oracle, with 24 seeds and bootstrap intervals throughout. The headline result isn’t a speedup. Against a latency-greedy adaptive controller, the strongest deployable comparator here, POLARIS is 11.3% slower at P95 (95% CI [10.0, 12.7]), and that penalty holds in all 72 one-at-a-time parameter sweeps. What it buys is compliance and cost: violations of the per-request output-confidence floor drop from 18.68% to 6.60% (64.7% lower, CI [62.9, 66.4]), serving cost drops 14.7% and energy 10.2%, putting realized utility within 2.7% of the offline oracle. Planning costs 0.841 ms of measured CPU per decision. Two mechanisms the design argues for don’t pay under this workload: the carbon term in the objective produces no measurable utility change, though the workload carries no binding carbon budget and a single grid-intensity pair, so that result bounds the mechanism rather than refuting it; and recovery re-planning cuts mean time to recovery 19.3% without improving utility at all. The ablation also shows most of the end-to-end gain comes from speculative decoding being available rather than from orchestration. Policy-aware composition is a compliance-and-cost mechanism, and it should be argued for on those terms. The architectural claim behind it is separable from the simulation: hard constraints are enforced structurally, by pruning the plan space before anything is scored, because a constraint expressed as a weighted term is tradeable by construction.
Keywords
Large language model serving, adaptive inference, edge–cloud computing, constrained optimization, runtime verification, simulation study.
Cite as
Somesh Rahul, Krishna Suman Dara and Raghuram Malpe Pai, "POLARIS: Policy-Orchestrated Adaptive LLM Inference Across Heterogeneous", In: Harish Sharma and Mukesh Saraswat (eds), Emerging Technologies and Intelligent Systems: Theory, Applications and Innovations, SCRS, India, 2026, pp. 88-108. https://doi.org/10.56155/978-81-975670-4-9-5