This book examines optimization problems that in practice involve random model parameters. It outlines the computation of robust optimal solutions, i.e., optimal solutions that are insensitive to random parameter variations, where appropriate deterministic substitute problems are needed. Based on the probability distribution of the random data and using decision theoretical concepts, optimization problems under stochastic uncertainty are converted into corresponding deterministic problems. Due to the probabilities and expectations involved, the book also shows how to apply approximative solution techniques. Several deterministic and stochastic approximation methods are provided: Taylor expansion methods, regression and response surface methods (RSM), probability inequalities, multiple linearization of survival/failure domains, discretization methods, convex approximation/deterministic descent directions/efficient points, stochastic approximation and gradient procedures, and differentiation formulas for probabilities and expectations. The fourth edition of this classic text has been carefully and thoroughly revised. It includes new chapters on the solution of stochastic linear programs by discretization of the underlying probability distribution, and on solving deterministic optimization problems by means of controlled random search methods and multiple random search procedures. It also presents a new application of stochastic optimization methods to machine learning problems with different loss functions. For the computation of optimal feedback controls under stochastic uncertainty, besides the open-loop feedback procedures, a new method based on Taylor expansions with respect to the gain parameters is presented. The book is intended for researchers and graduate students who are interested in stochastics, stochastic optimization, and control. It will also benefit professionals and practitioners whose work involves technical, economicand/or operations research problems under stochastic uncertainty.