Modeling, Analysis, Design, and Control of Stochastic Systems

This is an introductory level text on stochastic modeling. It is suited for undergraduate or graduate students in actuarial science, business management, computer science, engineering, operations research, public policy, statistics, and mathematics. It employs a large number of examples to teach how to build stochastic models of physical systems, analyze these models to predict their performance, and use the analysis to design and control them. The book provides a self-contained review of the relevant topics in probability theory. The rest of the book is devoted to important classes of stochastic models. In discrete and continuous time Markov models it covers the transient and long term behavior, cost models, and first passage times. Under generalized Markov models, it covers renewal processes, cumulative processes and semi-Markov processes. All the material is illustrated with many examples. There is a separate chapter on queueing models. In the chapter on design the author shows how the techniques developed in the text can be used to optimize the performance of a system. Finally, in the last chapter, linear programming is used to compute optimal control policies for stochastic systems. The book emphasizes numerical answers to the problems. A software package called MAXIM, which runs on MATLAB, is made available for downloading. Vidyadhar G. Kulkarni is Professor of Operations Research at the University of North Craolina at Chapel Hill. He has authored a graduate level text 'Modeling and Analysis of Stochastic Systems' and research articles on stochastic models of queues, computer systems and telecommunication systems. He holds a patent on traffic management in telecommunication networks, and he has served as an editor and associate editor of Stochastic Models and Operations Research Letters.