Statistical Global Optimization Algorithms

Overview

Statistical global optimization algorithms employ a statistical model of the objective function to bias the selection of new sample points. These methods are justified with Bayesian arguments that suppose that the particular objective function that is being optimized comes from a class of functions that is modeled by a particular stochastic function. Information from previous samples of the objective function can be used to estimate parameters of the stochastic function, and this refined model can subsequently be used to bias the selection of points in the search domain.

This framework is designed to cover average conditions of optimization. One of the challenges of using statistical methods is the verification that the statistical model is appropriate for the class of problems to which they are applied. Additionally, it has proved difficult to devise computationally interesting versions of these algorithms for high dimensional optimization problems.

Application Domains

Virtually all statistical methods have been developed for objective functions defined over the reals. Statistical methods generally assume that the objective function is sufficiently expensive that it is reasonable for the optimization method to perform some nontrivial analysis of the points that have been previously sampled. Many statistical methods rely on dividing the search region into partitions. In practice, this limits these methods to problems with a moderate number of dimensions.

Statistical global optimization algorithms have been applied to some challenging problems. However, their application has been limited due to the complexity of the mathematical software needed to implement them.

Software

The Bayesian methods developed by A. Mockus are available here.

References

J. Mockus, Application of Bayesian Approach to Numerical Methods of Global and Stochastic Optimization, J. Global Optimization. pp. 347-356. Vol. 4 No. 4 (1994).

J.Mockus "Bayesian Approach to Global Optimization", Kluwer, 1989.

J. Mockus and L. Mockus L., Bayesian approach to global optimization and applications to multiobjective and constrained optimization, of Optimization Theory and Applications 70, July, No. 1, 1991.

Miscellaneous Links

None.

Global Optimization Survey - Main Page
Sandia National Laboratories

This page maintained by wehart@cs.sandia.gov

Last modified: March 10, 1997