Departmental Papers (ESE)


We consider a queueing system with n parallel queues, which receives a reward for the service it provides. Our aim is to maximize the expected reward obtained per unit time (utility) while ensuring that the mean queue length in each of the queues is bounded (stability). We show that the optimal policy has counter intuitive properties because of the general reward states and stability constraint. For example, the greedy policy of serving a customer that fetches maximum reward need not be optimal. In addition, the optimal policy may belong to a class of non work-conserving policies. We obtain two different policies that attain the above optimality goal. The first policy arbitrates service randomly based on the current reward states and probabilities that depend on system statistics. The second policy arbitrates service deterministically based only on the queue lengths and the current reward states, and does not require any knowledge of the system statistics. The proposed policies are optimal in a large class of policies that includes off-line policies, which use knowledge of past, present and even future arrival and reward states in their decision processes.

Document Type

Conference Paper

Date of this Version



Suggested Citation:
Chaporkar, P. and S. Sarkar. (2005). "Utility Optimal Scheduling for General Reward States and Stability Constraint." Proceedings of the 44th IEEE Conference on Decision and Control and the European Control Conference 2005. Seville, Spain. December 12-15, 2005.

©2005 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.


Queueing theory, utility maximization, stability, randomized algorithms



Date Posted: 12 October 2010

This document has been peer reviewed.