A class of optimization methods and problem formulations in which the objective function, constraints, or both depend on random variables or noisy observations, and solutions are sought that account for that randomness through expectations, samples, or probabilistic guarantees.

Principle

Replace or augment deterministic objectives and constraints with statistical descriptions (expectations, sample averages, chance constraints) and use randomized sampling, stochastic gradients, or simulation optimization to guide search toward solutions that perform well on average or with specified probability.

Demonstration

Training a machine learning model by minimizing expected loss using stochastic gradient descent: each iteration uses a minibatch sampled from the data distribution to approximate the gradient of the expected loss and update parameters toward lower expected error.

Misapplication

Treating a single noisy sample as a surrogate for the entire distribution without variance estimation, which can lead to overfitting to sample noise or failure to satisfy probabilistic constraints when applied out-of-sample.

Consequence

When used appropriately, yields solutions that generalize across random realizations, provides trade-offs between computational cost and statistical accuracy, and allows incorporation of simulation-based or data-driven uncertainty; it typically requires more iterations or sampling than deterministic counterparts.

Reversal

Deterministic optimization assumes fixed model parameters and exact evaluations of objectives and constraints; reversing stochastic assumptions yields solutions optimized for a single realization but potentially fragile under variability.

Boundary

Applies when model evaluations are noisy, depend on random inputs, or are obtained via Monte Carlo simulation; it excludes purely deterministic problems without uncertainty and methods that do not account for statistical variability (unless used as approximations).

Semantic Tension

Competes with robust optimization: stochastic optimization focuses on performance averaged over a distribution or probabilistic guarantees, whereas robust optimization targets worst-case guarantees over uncertainty sets; they can be complementary but lead to different solution conservatisms.

Synthesis

Stochastic optimization integrates sampling-based approximations and probabilistic constraint handling into optimization to produce solutions that explicitly account for randomness, balancing statistical fidelity and computational effort to achieve reliable average-case performance.