![\[a_l \leq Ax \leq a_u\]](form_0.png)
Several examples follow. The first example shows a minimal specification for an optimization method.
method, \ dot_sqp
A more sophisticated example would be
method, \ id_method = 'NLP1' \ model_type single \ variables_pointer = 'V1' \ interface_pointer = 'I1' \ responses_pointer = 'R1' \ dot_sqp \ max_iterations = 50 \ convergence_tolerance = 1e-4 \ output verbose \ optimization_type minimize
max_iterations, convergence_tolerance, and output settings are method independent controls, in that they are defined for a variety of methods (see DOT method independent controls for DOT usage of these controls). The optimization_type control is a method dependent control, in that it is only meaningful for DOT methods (see DOT method dependent controls).The next example shows a specification for a least squares method.
method, \ optpp_g_newton \ max_iterations = 10 \ convergence_tolerance = 1.e-8 \ search_method trust_region \ gradient_tolerance = 1.e-6
search_method and gradient_tolerance) which are only meaningful for OPT++ methods (see OPT++ method dependent controls).The next example shows a specification for a nondeterministic iterator with several method dependent controls (refer to Nondeterministic sampling method).
method, \ nond_sampling \ samples = 100 seed = 12345 \ sample_type lhs \ response_levels = 1000. 500.
The last example shows a specification for a parameter study iterator where, again, each of the controls are method dependent (refer to Vector parameter study).
method, \ vector_parameter_study \ step_vector = 1. 1. 1. \ num_steps = 10
method, \ <method independent controls> \ <method selection> \ <method dependent controls>
where <method selection> is one of the following: dot_frcg, dot_mmfd, dot_bfgs, dot_slp, dot_sqp, conmin_frcg, conmin_mfd, npsol_sqp, nlssol_sqp, nl2sol, reduced_sqp, optpp_cg, optpp_q_newton, optpp_fd_newton, optpp_g_newton, optpp_newton, optpp_pds, coliny_apps, coliny_cobyla, coliny_direct, coliny_pattern_search, coliny_solis_wets, coliny_misc_solver, sgopt_pga_real, sgopt_pga_int, sgopt_epsa, sgopt_pattern_search, sgopt_solis_wets, sgopt_strat_mc, nond_polynomial_chaos, nond_sampling, nond_reliability, dace, vector_parameter_study, list_parameter_study, centered_parameter_study, or multidim_parameter_study.
The <method independent controls> are those controls which are valid for a variety of methods. In some cases, these controls are abstractions which may have slightly different implementations from one method to the next. The <method dependent controls> are those controls which are only meaningful for a specific method or library. Referring to dakota.input.spec, the method independent controls are those controls defined externally from and prior to the method selection blocks. They are all optional. The method selection blocks are all required group specifications separated by logical OR's. The method dependent controls are those controls defined within the method selection blocks. Defaults for method independent and method dependent controls are defined in DataMethod. The following sections provide additional detail on the method independent controls followed by the method selections and their corresponding method dependent controls.
The method identifier string is supplied with id_method and is used to provide a unique identifier string for use with strategy specifications (refer to Strategy Description). It is appropriate to omit a method identifier string if only one method is included in the input file and single_method is the selected strategy (all other strategies require one or more method pointers), since the single method to use is unambiguous in this case.
The type of model to be used by the method is supplied with model_type and can be single, nested, or layered (refer to Model for the class hierarchy involved). In the single model case, the optional variables_pointer, interface_pointer, and responses_pointer specifications provide strings for cross-referencing with id_variables, id_interface, and id_responses string inputs from particular variables, interface, and responses keyword specifications. These pointers identify which specifications will be used in building the single model, which is to be iterated by the method to map the variables into responses through the interface. In the layered model case, the specification is similar, except that the interface_pointer specification is required in order to identify a global, multipoint, local, or hierarchical approximation interface (see Approximation Interface) to use in the layered model. In the nested model case, a sub_method_pointer must be provided in order to specify the nested iterator, and interface_pointer and interface_responses_pointer provide an optional group specification for the optional interface portion of nested models (where interface_pointer points to the interface specification and interface_responses_pointer points to a responses specification describing the data to be returned by this interface). This interface is used to provide non-nested data, which is then combined with data from the nested iterator using the primary_response_mapping and secondary_response_mapping inputs (see mapping discussion below).
For each of these cases, if a pointer string is specified and no corresponding id is available, DAKOTA will exit with an error message. If the pointer is optional and no pointer string is specified, then the last specification parsed will be used. It is appropriate to omit optional cross-referencing whenever the relationships are unambiguous due to the presence of only one specification. Since the method specification is responsible for cross-referencing with the interface, variables, and responses specifications, identification of methods at the strategy layer is often sufficient to completely specify all of the object interrelationships. Table 5.1 provides the specification detail for the method independent controls involving identifiers, pointers, and model type controls.
| Description | Keyword | Associated Data | Status | Default |
| Method set identifier | id_method | string | Optional | strategy use of last method parsed |
| Model type | model_type | single | nested | layered | Optional group | single |
| Variables set pointer | variables_pointer | string | Optional | method use of last variables parsed |
| Interface set pointer | interface_pointer | string | single: Optional, nested: Optional group, layered: Required | single: method use of last interface parsed, nested: no optional interface, layered: N/A |
| Responses set pointer | responses_pointer | string | Optional | method use of last responses parsed |
| Responses pointer for nested model optional interfaces | interface_responses_pointer | string | Required within optional group | N/A |
| Sub-method pointer for nested models | sub_method_pointer | string | Required | N/A |
Nested models may employ mappings for both the variable inputs to the sub-model and the response outputs from the sub-model. In the former case, the primary_variable_mapping and secondary_variable_mapping specifications are used to map from the top-level variables into the sub-model variables, and in the latter case, the primary_response_mapping and secondary_response_mapping specifications are used to map from the sub-model responses back to the top-level responses. For the variable mappings, the primary and secondary specifications provide lists of strings which are used to target active sub-model variables and their distribution parameters, respectively. The primary strings are matched to variable labels such as 'cdv_1' (either user-supplied or default labels), and the secondary strings are matched to distribution parameters such as 'mean' or 'std_deviation' (the singular form of the uncertain distribution parameter keywords, lacking the prepended distribution type identifier). Both specifications are optional, which is designed to support three possibilities:
These different variable mapping possibilities may be used in any combination by employing empty strings ('') for particular omitted mappings (the number of strings in user-supplied primary and secondary variable mapping specifications must equal the number of active top-level variables).
For the response mappings, the primary and secondary specifications provide real-valued multipliers to be applied to sub-iterator response results. The sub-iterator response results are defined as follows for different sub-iterator types:
response_levels, probability_levels, and/or reliability_levels, in that order.The primary values map sub-iterator response results into top-level objective functions, least squares terms, or generic response functions, depending on the declared top-level response set. The secondary values map sub-iterator response results into top-level nonlinear inequality and equality constraints. Refer to NestedModel::response_mapping() for additional details.
An example of variable and response mappings is provided below:
primary_variable_mapping = '' '' 'X' 'Y' \ secondary_variable_mapping = '' '' 'mean' 'mean' \ primary_response_mapping = 1. 0. 0. 0. 0. 0. 0. 0. 0. \ secondary_response_mapping = 0. 0. 0. 1. 3. 0. 0. 0. 0. \ 0. 0. 0. 0. 0. 0. 1. 3. 0. \
'X' and 'Y' (option 1 above). The response mappings correspond to 9 sub-iterator response functions (e.g., a set of UQ final statistics for 3 response functions, each with a mean, a standard deviation, and a reliability level). The primary response mapping maps the first sub-iterator response function (mean) into a single objective function, least squares term, or generic response function (as dictated by the top-level response specification), and the secondary response mapping maps the fourth sub-iterator response function plus 3 times the fifth sub-iterator response function (mean plus 3 standard deviations) into one top-level nonlinear constraint and the seventh sub-iterator response function plus 3 times the eighth sub-iterator response function (mean plus 3 standard deviations) into another top-level nonlinear constraint (these top-level nonlinear constraints may be inequality or equality, as dictated by the top-level response specification).Table 5.2 provides the specification detail for the method independent controls involving nested model mappings.
| Description | Keyword | Associated Data | Status | Default |
| Primary variable mappings for nested models | primary_variable_mapping | list of strings | Optional | augmentation of sub-model variables (no insertion) |
| Secondary variable mappings for nested models | secondary_variable_mapping | list of strings | Optional | primary mappings into sub-model variables are value-based |
| Primary response mappings for nested models | primary_response_mapping | list of reals | Optional | no sub-iterator contribution to primary functions |
| Secondary response mappings for nested models | secondary_response_mapping | list of reals | Optional | no sub-iterator contribution to secondary functions |
When performing gradient-based optimization in parallel, speculative gradients can be selected to address the load imbalance that can occur between gradient evaluation and line search phases. In a typical gradient-based optimization, the line search phase consists primarily of evaluating the objective function and any constraints at a trial point, and then testing the trial point for a sufficient decrease in the objective function value and/or constraint violation. If a sufficient decrease is not observed, then one or more additional trial points may be attempted sequentially. However, if the trial point is accepted then the line search phase is complete and the gradient evaluation phase begins. By speculating that the gradient information associated with a given line search trial point will be used later, additional coarse grained parallelism can be introduced by computing the gradient information (either by finite difference or analytically) in parallel, at the same time as the line search phase trial-point function values. This balances the total amount of computation to be performed at each design point and allows for efficient utilization of multiple processors. While the total amount of work performed will generally increase (since some speculative gradients will not be used when a trial point is rejected in the line search phase), the run time will usually decrease (since gradient evaluations needed at the start of each new optimization cycle were already performed in parallel during the line search phase). Refer to [Byrd et al., 1998] for additional details. The speculative specification is implemented for the gradient-based optimizers in the DOT, CONMIN, and OPT++ libraries, and it can be used with dakota numerical or analytic gradient selections in the responses specification (refer to Gradient Specification for information on these specifications). It should not be selected with vendor numerical gradients since vendor internal finite difference algorithms have not been modified for this purpose. In full-Newton approaches, the Hessian is also computed speculatively. NPSOL and NLSSOL do not support speculative gradients, as their gradient-based line search in user-supplied gradient mode (dakota numerical or analytic gradients) is a superior approach for load-balanced parallel execution.
Output verbosity control is specified with output followed by silent, quiet, verbose or debug. If there is no user specification for output verbosity, then the default setting is normal. This gives a total of five output levels to manage the volume of data that is returned to the user during the course of a study, ranging from full run annotation plus internal debug diagnostics (debug) to the bare minimum of output containing little more than the total number of simulations performed and the final solution (silent). Output verbosity is observed within the Iterator (algorithm verbosity), Model (synchronize/fd_gradients verbosity), Interface (map/synch verbosity), Approximation (global data fit coefficient reporting),and AnalysisCode (file operation reporting) class hierarchies; however, not all of these software components observe the full granularity of verbosity settings. Specific mappings are as follows:
output silent (i.e., really quiet): silent iterators, silent model, silent interface, quiet approximation, quiet file operations output quiet: quiet iterators, quiet model, quiet interface, quiet approximation, quiet file operations output normal: normal iterators, normal model, normal interface, quiet approximation, quiet file operations output verbose: verbose iterators, normal model, verbose interface, verbose approximation, verbose file operations output debug (i.e., really verbose): debug iterators, normal model, debug interface, verbose approximation, verbose file operationsquiet suppresses parameter and response set reporting and silent further suppresses function evaluation headers and scheduling output. Similarly, verbose adds file management, approximation evaluation, and global approximation coefficient details, and debug further adds diagnostics from nonblocking schedulers.
The constraint_tolerance specification determines the maximum allowable value of infeasibility that any constraint in an optimization problem may possess and still be considered to be satisfied. It is specified as a positive real value. If a constraint function is greater than this value then it is considered to be violated by the optimization algorithm. This specification gives some control over how tightly the constraints will be satisfied at convergence of the algorithm. However, if the value is set too small the algorithm may terminate with one or more constraints being violated. This specification is currently meaningful for the NPSOL, NLSSOL, DOT and CONMIN constrained optimizers (refer to DOT method independent controls and NPSOL method independent controls).
The convergence_tolerance specification provides a real value for controlling the termination of iteration. In most cases, it is a relative convergence tolerance for the objective function; i.e., if the change in the objective function between successive iterations divided by the previous objective function is less than the amount specified by convergence_tolerance, then this convergence criterion is satisfied on the current iteration. Since no progress may be made on one iteration followed by significant progress on a subsequent iteration, some libraries require that the convergence tolerance be satisfied on two or more consecutive iterations prior to termination of iteration. This control is used with optimization and least squares iterators (DOT, CONMIN, NPSOL, NLSSOL, OPT++, and Coliny) and is not used within the uncertainty quantification, design of experiments, or parameter study iterator branches. Refer to DOT method independent controls, NPSOL method independent controls, OPT++ method independent controls, and Coliny method independent controls for specific interpretations of the convergence_tolerance specification.
The max_iterations and max_function_evaluations controls provide integer limits for the maximum number of iterations and maximum number of function evaluations, respectively. The difference between an iteration and a function evaluation is that a function evaluation involves a single parameter to response mapping through an interface, whereas an iteration involves a complete cycle of computation within the iterator. Thus, an iteration generally involves multiple function evaluations (e.g., an iteration contains descent direction and line search computations in gradient-based optimization, population and multiple offset evaluations in nongradient-based optimization, etc.). This control is not currently used within the uncertainty quantification, design of experiments, and parameter study iterator branches, and in the case of optimization and least squares, does not currently capture function evaluations that occur as part of the method_source dakota finite difference routine (since these additional evaluations are intentionally isolated from the iterators).
Table 5.3 provides the specification detail for the method independent controls involving tolerances, limits, output verbosity, and speculative gradients.
| Description | Keyword | Associated Data | Status | Default |
| Speculative gradients and Hessians | speculative | none | Optional | no speculation |
| Output verbosity | output | silent | quiet | verbose | debug | Optional | normal |
| Maximum iterations | max_iterations | integer | Optional | 100 |
| Maximum function evaluations | max_function_evaluations | integer | Optional | 1000 |
| Constraint tolerance | constraint_tolerance | real | Optional | Library default |
| Convergence tolerance | convergence_tolerance | real | Optional | 1.e-4 |
Linear inequality constraints can be supplied with the linear_inequality_constraint_matrix, linear_inequality_lower_bounds, and linear_inequality_upper_bounds specifications, and linear equality constraints can be supplied with the linear_equality_constraint_matrix and linear_equality_targets specifications. In the inequality case, the constraint matrix provides coefficients for the variables and the lower and upper bounds provide constraint limits for the following two-sided formulation:
As with nonlinear inequality constraints (see Objective and constraint functions (optimization data set)), the default linear inequality constraint bounds are selected so that one-sided inequalities of the form
![\[Ax \leq 0.0\]](form_1.png)
result when there are no user bounds specifications (this provides backwards compatibility with previous DAKOTA versions). In a user bounds specification, any upper bound values greater than +bigRealBoundSize (1.e+30, as defined in Minimizer) are treated as +infinity and any lower bound values less than -bigRealBoundSize are treated as -infinity. This feature is commonly used to drop one of the bounds in order to specify a 1-sided constraint (just as the default lower bounds drop out since -DBL_MAX < -bigRealBoundSize). In the equality case, the constraint matrix again provides coefficients for the variables and the targets provide the equality constraint right hand sides:
![\[Ax = a_t\]](form_2.png)
and the defaults for the equality constraint targets enforce a value of 0. for each constraint
![\[Ax = 0.0\]](form_3.png)
Currently, DOT, CONMIN, NPSOL, NLSSOL, and OPT++ all support specialized handling of linear constraints (either directly through the algorithm itself or indirectly through the DAKOTA wrapper). Coliny optimizers will support linear constraints in future releases. Linear constraints need not be computed by the user's interface on every function evaluation; rather the coefficients, bounds, and targets of the linear constraints can be provided at start up, allowing the optimizers to track the linear constraints internally. It is important to recognize that linear constraints are those constraints that are linear in the design variables, e.g.:
![\[0.0 \leq 3x_1 - 4x_2 + 2x_3 \leq 15.0\]](form_4.png)
![\[x_1 + x_2 + x_3 \geq 2.0\]](form_5.png)
![\[x_1 + x_2 - x_3 = 1.0\]](form_6.png)
which is not to be confused with something like
![\[s(X) - s_{fail} \leq 0.0\]](form_7.png)
where the constraint is linear in a response quantity, but may be a nonlinear implicit function of the design variables. For the three linear constraints above, the specification would appear as:
linear_inequality_constraint_matrix = 3.0 -4.0 2.0 \
1.0 1.0 1.0 \
linear_inequality_lower_bounds = 0.0 2.0 \
linear_inequality_upper_bounds = 15.0 1.e+50 \
linear_equality_constraint_matrix = 1.0 1.0 -1.0 \
linear_equality_targets = 1.0 \
1.e+50 is a dummy upper bound value which defines a 1-sided inequality since it is greater than bigRealBoundSize. The constraint matrix specifications list the coefficients of the first constraint followed by the coefficients of the second constraint, and so on. They are divided into individual constraints based on the number of design variables, and can be broken onto multiple lines for readability as shown above.Table 5.4 provides the specification detail for the method independent controls involving linear constraints.
| Description | Keyword | Associated Data | Status | Default |
| Linear inequality coefficient matrix | linear_inequality_constraint_matrix | list of reals | Optional | no linear inequality constraints |
| Linear inequality lower bounds | linear_inequality_lower_bounds | list of reals | Optional | vector values = -DBL_MAX |
| Linear inequality upper bounds | linear_inequality_upper_bounds | list of reals | Optional | vector values = 0. |
| Linear equality coefficient matrix | linear_equality_constraint_matrix | list of reals | Optional | no linear equality constraints |
| Linear equality targets | linear_equality_targets | list of reals | Optional | vector values = 0. |
dot_bfgs method) and Fletcher-Reeves conjugate gradient (DAKOTA's dot_frcg method) methods for unconstrained optimization, and the modified method of feasible directions (DAKOTA's dot_mmfd method), sequential linear programming (DAKOTA's dot_slp method), and sequential quadratic programming (DAKOTA's dot_sqp method) methods for constrained optimization. DAKOTA provides access to the DOT library through the DOTOptimizer class.max_iterations and max_function_evaluations limit the number of major iterations and the number of function evaluations that can be performed during a DOT optimization. The convergence_tolerance control defines the threshold value on relative change in the objective function that indicates convergence. This convergence criterion must be satisfied for two consecutive iterations before DOT will terminate. The constraint_tolerance specification defines how tightly constraint functions are to be satisfied at convergence. The default value for DOT constrained optimizers is 0.003. Extremely small values for constraint_tolerance may not be attainable. The output verbosity specification controls the amount of information generated by DOT: the silent and quiet settings result in header information, final results, and objective function, constraint, and parameter information on each iteration; whereas the verbose and debug settings add additional information on gradients, search direction, one-dimensional search results, and parameter scaling factors. DOT contains no parallel algorithms which can directly take advantage of concurrent evaluations. However, if numerical_gradients with method_source dakota is specified, then the finite difference function evaluations can be performed concurrently (using any of the parallel modes described in the Users Manual). In addition, if speculative is specified, then gradients (dakota numerical or analytic gradients) will be computed on each line search evaluation in order to balance the load and lower the total run time in parallel optimization studies. Lastly, specialized handling of linear constraints is supported with DOT; linear constraint coefficients, bounds, and targets can be provided to DOT at start-up and tracked internally. Specification detail for these method independent controls is provided in Tables 5.1 through 5.4.optimization_type which may be either minimize or maximize. DOT provides the only set of methods within DAKOTA which support this control; to convert a maximization problem into the minimization formulation assumed by other methods, simply change the sign on the objective function (i.e., multiply by -1). Table 5.5 provides the specification detail for the DOT methods and their method dependent controls.| Description | Keyword | Associated Data | Status | Default |
| Optimization type | optimization_type | minimize | maximize | Optional group | minimize |
npsol_sqp method). SQP is a nonlinear programming optimizer for constrained minimization. DAKOTA provides access to the NPSOL library through the NPSOLOptimizer class.max_iterations and max_function_evaluations limit the number of major SQP iterations and the number of function evaluations that can be performed during an NPSOL optimization. The convergence_tolerance control defines NPSOL's internal optimality tolerance which is used in evaluating if an iterate satisfies the first-order Kuhn-Tucker conditions for a minimum. The magnitude of convergence_tolerance approximately specifies the number of significant digits of accuracy desired in the final objective function (e.g., convergence_tolerance = 1.e-6 will result in approximately six digits of accuracy in the final objective function). The constraint_tolerance control defines how tightly the constraint functions are satisfied at convergence. The default value is dependent upon the machine precision of the platform in use, but is typically on the order of 1.e-8 for double precision computations. Extremely small values for constraint_tolerance may not be attainable. The output verbosity setting controls the amount of information generated at each major SQP iteration: the silent and quiet settings result in only one line of diagnostic output for each major iteration and print the final optimization solution, whereas the verbose and debug settings add additional information on the objective function, constraints, and variables at each major iteration.
NPSOL is not a parallel algorithm and cannot directly take advantage of concurrent evaluations. However, if numerical_gradients with method_source dakota is specified, then the finite difference function evaluations can be performed concurrently (using any of the parallel modes described in the Users Manual). An important related observation is the fact that NPSOL uses two different line searches depending on how gradients are computed. For either analytic_gradients or numerical_gradients with method_source dakota, NPSOL is placed in user-supplied gradient mode (NPSOL's "Derivative Level" is set to 3) and it uses a gradient-based line search (the assumption is that user-supplied gradients are inexpensive). On the other hand, if numerical_gradients are selected with method_source vendor, then NPSOL is computing finite differences internally and it will use a value-based line search (the assumption is that finite differencing on each line search evaluation is too expensive). The ramifications of this are: (1) performance will vary between method_source dakota and method_source vendor for numerical_gradients, and (2) gradient speculation is unnecessary when performing optimization in parallel since the gradient-based line search in user-supplied gradient mode is already load balanced for parallel execution. Therefore, a speculative specification will be ignored by NPSOL, and optimization with numerical gradients should select method_source dakota for load balanced parallel operation and method_source vendor for efficient serial operation.
Lastly, NPSOL supports specialized handling of linear inequality and equality constraints. By specifying the coefficients and bounds of the linear inequality constraints and the coefficients and targets of the linear equality constraints, this information can be provided to NPSOL at initialization and tracked internally, removing the need for the user to provide the values of the linear constraints on every function evaluation. Refer to Method Independent Controls for additional information and to Tables 5.1 through 5.4 for method independent control specification detail.
verify_level, function_precision, and linesearch_tolerance. The verify_level control instructs NPSOL to perform finite difference verifications on user-supplied gradient components. The function_precision control provides NPSOL an estimate of the accuracy to which the problem functions can be computed. This is used to prevent NPSOL from trying to distinguish between function values that differ by less than the inherent error in the calculation. And the linesearch_tolerance setting controls the accuracy of the line search. The smaller the value (between 0 and 1), the more accurately NPSOL will attempt to compute a precise minimum along the search direction. Table 5.6 provides the specification detail for the NPSOL SQP method and its method dependent controls.| Description | Keyword | Associated Data | Status | Default |
| Gradient verification level | verify_level | integer | Optional | -1 (no gradient verification) |
| Function precision | function_precision | real | Optional | 1.e-10 |
| Line search tolerance | linesearch_tolerance | real | Optional | 0.9 (inaccurate line search) |
conmin_frcg method) method for unconstrained optimization, and the method of feasible directions (DAKOTA's conmin_mfd method) for constrained optimization. As CONMIN was a predecessor to the DOT commercial library, the algorithm controls are very similar. DAKOTA provides access to the CONMIN library through the CONMINOptimizer class.optpp_cg method), quasi-Newton (DAKOTA's optpp_q_newton method), finite difference Newton (DAKOTA's optpp_fd_newton method), and full Newton (DAKOTA's optpp_newton method). The conjugate gradient method is strictly unconstrained, and each of the Newton-based methods are automatically bound to the appropriate OPT++ algorithm based on the user constraint specification (unconstrained, bound-constrained, or generally-constrained). In the generally-constrained case, the Newton methods use a nonlinear interior-point approach to manage the constraints. The library also contains a direct search algorithm, PDS (parallel direct search, DAKOTA's optpp_pds method), which supports bound constraints. DAKOTA provides access to the OPT++ library through the SNLLOptimizer class, where "SNLL" denotes Sandia National Laboratories - Livermore.max_iterations and max_function_evaluations limit the number of major iterations and the number of function evaluations that can be performed during an OPT++ optimization. The convergence_tolerance control defines the threshold value on relative change in the objective function that indicates convergence. The output verbosity specification controls the amount of information generated from OPT++ executions: the debug setting turns on OPT++'s internal debug mode and also generates additional debugging information from DAKOTA's SNLLOptimizer wrapper class. OPT++'s gradient-based methods are not parallel algorithms and cannot directly take advantage of concurrent function evaluations. However, if numerical_gradients with method_source dakota is specified, a parallel DAKOTA configuration can utilize concurrent evaluations for the finite difference gradient computations. OPT++'s nongradient-based PDS method can directly exploit asynchronous evaluations; however, this capability has not yet been implemented in the SNLLOptimizer class.
The speculative specification enables speculative computation of gradient and/or Hessian information, where applicable, for parallel optimization studies. By speculating that the derivative information at the current point will be used later, the complete data set (all available gradient/Hessian information) can be computed on every function evaluation. While some of these computations will be wasted, the positive effects are a consistent parallel load balance and usually shorter wall clock time. The speculative specification is applicable only when parallelism in the gradient calculations can be exploited by DAKOTA (it will be ignored for vendor numerical gradients).
Lastly, linear constraint specifications are supported by each of the Newton methods (optpp_newton, optpp_q_newton, optpp_fd_newton, and optpp_g_newton); whereas optpp_cg must be unconstrained and optpp_pds can be, at most, bound-constrained. Specification detail for the method independent controls is provided in Tables 5.1 through 5.4.
max_step, gradient_tolerance, search_method, merit_function, central_path, steplength_to_boundary, centering_parameter, and search_scheme_size. The max_step control specifies the maximum step that can be taken when computing a change in the current design point (e.g., limiting the Newton step computed from current gradient and Hessian information). It is equivalent to a move limit or a maximum trust region size. The gradient_tolerance control defines the threshold value on the L2 norm of the objective function gradient that indicates convergence to an unconstrained minimum (no active constraints). The gradient_tolerance control is defined for all gradient-based optimizers.
max_step and gradient_tolerance are the only method dependent controls for the OPT++ conjugate gradient method. Table 5.7 covers this specification.
| Description | Keyword | Associated Data | Status | Default |
| OPT++ conjugate gradient method | optpp_cg | none | Required | N/A |
| Maximum step size | max_step | real | Optional | 1000. |
| Gradient tolerance | gradient_tolerance | real | Optional | 1.e-4 |
The search_method control is defined for all Newton-based optimizers and is used to select between trust_region, gradient_based_line_search, and value_based_line_search methods. The gradient_based_line_search option uses the line search method proposed by [More and Thuente, 1994]. This option satisfies sufficient decrease and curvature conditions; whereas, value_base_line_search only satisfies the sufficient decrease condition. At each line search iteration, the gradient_based_line_search method computes the function and gradient at the trial point. Consequently, given expensive function evaluations, the value_based_line_search method is preferred to the gradient_based_line_search method. Each of these Newton methods additionally supports the tr_pds selection for unconstrained problems. This option performs a robust trust region search using pattern search techniques. Use of a line search is the default for bound-constrained and generally-constrained problems, and use of a trust_region search method is the default for unconstrained problems.
The merit_function, central_path, steplength_to_boundary, and centering_parameter selections are additional specifications that are defined for the solution of generally-constrained problems with nonlinear interior-point algorithms. A merit_function is a function in constrained optimization that attempts to provide joint progress toward reducing the objective function and satisfying the constraints. Valid string inputs are "el_bakry", "argaez_tapia", or "van_shanno", where user input is not case sensitive in this case. Details for these selections are as follows:
merit_function to "argaez_tapia" or "van_shanno".
The central_path specification represents a measure of proximity to the central path and specifies an update strategy for the perturbation parameter mu. Refer to [Argaez et al., 2002] for a detailed discussion on proximity measures to the central region. Valid options are, again, "el_bakry", "argaez_tapia", or "van_shanno", where user input is not case sensitive. The default value for central_path is the value of merit_function (either user-selected or default). The steplength_to_boundary specification is a parameter (between 0 and 1) that controls how close to the boundary of the feasible region the algorithm is allowed to move. A value of 1 means that the algorithm is allowed to take steps that may reach the boundary of the feasible region. If the user wishes to maintain strict feasibility of the design parameters this value should be less than 1. Default values are .8, .99995, and .95 for the "el_bakry", "argaez_tapia", and "van_shanno" merit functions, respectively. The centering_parameter specification is a parameter (between 0 and 1) that controls how closely the algorithm should follow the "central path". See [Wright] for the definition of central path. The larger the value, the more closely the algorithm follows the central path, which results in small steps. A value of 0 indicates that the algorithm will take a pure Newton step. Default values are .2, .2, and .1 for the "el_bakry", "argaez_tapia", and "van_shanno" merit functions, respectively.
Table 5.8 provides the details for the Newton-based methods.
| Description | Keyword | Associated Data | Status | Default |
| OPT++ Newton-based methods | optpp_q_newton | optpp_fd_newton | optpp_newton | none | Required group | N/A |
| Search method | value_based_line_search | gradient_based_line_search | trust_region | tr_pds | none | Optional group | trust_region (unconstrained), value_based_line_search (bound/general constraints) |
| Maximum step size | max_step | real | Optional | 1000. |
| Gradient tolerance | gradient_tolerance | real | Optional | 1.e-4 |
| Merit function | merit_function | string | Optional | "argaez_tapia" |
| Central path | central_path | string | Optional | value of merit_function |
| Steplength to boundary | steplength_to_boundary | real | Optional | Merit function dependent: 0.8 (el_bakry), 0.99995 (argaez_tapia), 0.95 (van_shanno) |
| Centering parameter | centering_parameter | real | Optional | Merit function dependent: 0.2 (el_bakry), 0.2 (argaez_tapia), 0.1 (van_shanno) |
The search_scheme_size is defined for the PDS method to specify the number of points to be used in the direct search template. PDS does not support parallelism at this time due to current limitations in the OPT++ interface. Table 5.9 provides the detail for the parallel direct search method.
| Description | Keyword | Associated Data | Status | Default |
| OPT++ parallel direct search method | optpp_pds | none | Required group | N/A |
| Search scheme size | search_scheme_size | integer | Optional | 32 |
sgopt_pga_real, sgopt_pga_int, and sgopt_epsa) and stratified Monte Carlo (sgopt_strat_mc). Additionally, SGOPT includes nongradient-based local search algorithms such as Solis-Wets (sgopt_solis_wets) and pattern search (sgopt_pattern_search). With the exception of the unconstrained sgopt_solis_wets method, each of the SGOPT methods support bound constraints. DAKOTA provides access to the SGOPT library through the SGOPTOptimizer class.max_iterations and max_function_evaluations limit the number of major iterations and the number of function evaluations that can be performed during an SGOPT optimization. The convergence_tolerance control defines the threshold value on relative change in the objective function that indicates convergence. The output verbosity specification controls the amount of information generated by SGOPT: the silent, quiet, and normal settings correspond to minimal reporting from SGOPT, whereas the verbose setting corresponds to a higher level of information, and debug outputs method initialization and a variety of internal SGOPT diagnostics. The majority of SGOPT's methods have independent function evaluations that can directly take advantage of DAKOTA's parallel capabilities. Only sgopt_solis_wets and certain exploratory_moves options in sgopt_pattern_search (multi_step, best_first, biased_best_first, and adaptive_pattern; see Pattern search) are inherently serial. The parallel methods automatically utilize parallel logic when the DAKOTA configuration supports parallelism. Lastly, neither speculative gradients nor specialized handling of linear constraints are currently supported with SGOPT since SGOPT methods are nongradient-based and support, at most, bound constraints. Specification detail for method independent controls is provided in Tables 5.1 through 5.4.solution_accuracy is a method dependent control which is defined for all SGOPT methods. Solution accuracy defines a convergence criterion in which the optimizer will terminate if it finds an objective function value lower than the specified accuracy. Table 5.10 provides the specification detail for recurring method dependent controls.| Description | Keyword | Associated Data | Status | Default |
| Desired solution accuracy | solution_accuracy | real | Optional | -DBL_MAX |
Each SGOPT method supplements the settings of Table 5.10 with controls which are specific to its particular class of method.
sgopt_pga_real), an integer-valued genetic algorithm (sgopt_pga_int), and an evolutionary pattern search technique (sgopt_epsa), where "real-valued" and "integer-valued" refer to the use of continuous or discrete variable domains, respectively (the response data are real-valued in all cases).The basic steps of an evolutionary algorithm are as follows:
new_solutions_generated new individuals from the selected parents
Controls for seed, population size, selection, and replacement are identical for the three EA methods, whereas the crossover and mutation controls contain slight differences and the sgopt_epsa specification contains an additional num_partitions input. Table 5.11 provides the specification detail for the controls which are common between the three EA methods.
| Description | Keyword | Associated Data | Status | Default |
| EA selection | sgopt_pga_real | sgopt_pga_int | sgopt_epsa | none | Required group | N/A |
| Random seed | seed | integer | Optional | randomly generated seed |
| Number of population members | population_size | integer | Optional | 100 |
| Selection pressure | selection_pressure | rank | proportional | Optional | proportional |
| Replacement type | replacement_type | random | chc | elitist | Optional group | elitist = 1 |
| Random replacement | random | integer | Required | N/A |
| CHC replacement type | chc | integer | Required | N/A |
| Elitist replacement type | elitist | integer | Required | N/A |
| New solutions generated | new_solutions_generated | integer | Optional | population_size - replacement_size |
The random seed control provides a mechanism for making a stochastic optimization repeatable. That is, the use of the same random seed in identical studies will generate identical results. The population_size control specifies how many individuals will comprise the EA's population. The selection_pressure controls how strongly differences in "fitness" (i.e., the objective function) are weighted in the process of selecting "parents" for crossover:
rank setting uses a linear scaling of probability of selection based on the rank order of each individual's objective function within the populationproportional setting uses a proportional scaling of probability of selection based on the relative value of each individual's objective function within the populationreplacement_type controls how current populations and newly generated individuals are combined to create a new population. Each of the replacement_type selections accepts an integer value, which will is referred to below and in Table 5.11 as the replacement_size:
random setting (the default) creates a new population using (a) replacement_size randomly selected individuals from the current population, and (b) population_size - replacement_size individuals randomly selected from among the newly generated individuals (the number of which is optionally specified using new_solutions_generated) that are created for each generation (using the selection, crossover, and mutation procedures).CHC setting creates a new population using (a) the replacement_size best individuals from the combination of the current population and the newly generated individuals, and (b) population_size - replacement_size individuals randomly selected from among the remaining individuals in this combined pool. CHC is the preferred selection for many engineering problems.elitist setting creates a new population using (a) the replacement_size best individuals from the current population, (b) and population_size - replacement_size individuals randomly selected from the newly generated individuals. It is possible in this case to lose a good solution from the newly generated individuals if it is not randomly selected for replacement; however, the default new_solutions_generated value is set such that the entire set of newly generated individuals will be selected for replacement.sgopt_pga_real, sgopt_pga_int, and sgopt_epsa, respectively.| Description | Keyword | Associated Data | Status | Default |
| Crossover type | crossover_type | two_point | blend | uniform | Optional group | two_point |
| Crossover rate | crossover_rate | real | Optional | 0.8 |
| Mutation type | mutation_type | replace_uniform | offset_normal | offset_cauchy | offset_uniform | offset_triangular | Optional group | offset_normal |
| Mutation scale | mutation_scale | real | Optional | 0.1 |
| Mutation dimension rate | dimension_rate | real | Optional |  |
| Mutation population rate | population_rate | real | Optional | 1.0 |
| Non-adaptive mutation flag | non_adaptive | none | Optional | Adaptive mutation |
| Description | Keyword | Associated Data | Status | Default |
| Crossover type | crossover_type | two_point | uniform | Optional group | two_point |
| Crossover rate | crossover_rate | real | Optional | 0.8 |
| Mutation type | mutation_type | replace_uniform | offset_uniform | Optional group | replace_uniform |
| Mutation range | mutation_range | integer | Optional | 1 |
| Mutation dimension rate | dimension_rate | real | Optional | |
| Mutation population rate | population_rate | real | Optional | 1.0 |
| Description | Keyword | Associated Data | Status | Default |
| Crossover type | crossover_type | two_point | uniform | Optional group | two_point |
| Crossover rate | crossover_rate | real | Optional | 0.8 |
| Mutation type | mutation_type | unary_coord | unary_simplex | multi_coord | multi_simplex | Optional group | unary_coord |
| Mutation dimension rate | dimension_rate | real | Optional | |
| Mutation scale | mutation_scale | real | Optional | 0.1 |
| Minimum mutation scale | min_scale | real | Optional | 0.001 |
| Mutation population rate | population_rate | real | Optional | 1.0 |
| Number of partitions | num_partitions | integer | Optional | 100 |
The crossover_type controls what approach is employed for combining parent genetic information to create offspring, and the crossover_rate specifies the probability of a crossover operation being performed to generate a new offspring. SGOPT supports two generic forms of crossover, two_point and uniform, which generate a new individual through coordinate-wise combinations of two parent individuals. Two-point crossover divides each parent into three regions, where offspring are created from the combination of the middle region from one parent and the end regions from the other parent. Since SGOPT does not utilize bit representations of variable values, the crossover points only occur on coordinate boundaries, never within the bits of a particular coordinate. Uniform crossover creates offspring through random combination of coordinates from the two parents. The sgopt_pga_real optimizer supports a third option, the blend crossover method, which generates a new individual randomly along the multidimensional vector connecting the two parents.
The mutation_type controls what approach is employed in randomly modifying design variables within the EA population. Each of the mutation methods generates coordinate-wise changes to individuals, usually by adding a random variable to a given coordinate value (an "offset" mutation), but also by replacing a given coordinate value with a random variable (a "replace" mutation). The population_rate controls the probability of mutation being performed on an individual, both for new individuals generated by crossover (if crossover occurs) and for individuals from the existing population (if crossover does not occur; see algorithm description in Evolutionary Algorithms). The dimension_rate specifies the probabilities that a given dimension is changed given that the individual is having mutation applied to it. The default dimension_rate uses the special formula shown in the preceding tables, where n is the number of design variables and e is the natural logarithm constant. The mutation_scale specifies a scale factor which scales mutation offsets for sgopt_pga_real and sgopt_epsa; this is a fraction of the total range of each dimension, so mutation_scale is a relative value between 0 and 1. The mutation_range provides an analogous control for sgopt_pga_int, but is not a relative value in that it specifies the total integer range of the mutation. The offset_normal, offset_cauchy, offset_uniform, and offset_triangular mutation types are "offset" mutations in that they add a 0-mean random variable with a normal, cauchy, uniform, or triangular distribution, respectively, to the existing coordinate value. These offsets are limited in magnitude by mutation_scale. The replace_uniform mutation type is not limited by mutation_scale; rather it generates a replacement value for a coordinate using a uniformly distributed value over the total range for that coordinate. The real-valued genetic algorithm supports each of these 5 mutation types, and integer-valued genetic algorithm supports the replace_uniform and offset_uniform types. The mutation types for evolutionary pattern search are more specialized:
multi_coord: Mutate each coordinate dimension with probability dimension_rate using an "offset" approach with initial scale mutation_scale * variable range. Multiple coordinates may or may not be mutated.unary_coord: Mutate a single randomly selected coordinate dimension using an "offset" approach with initial scale mutation_scale * variable range. One and only one coordinate is mutated.multi_simplex: Apply each of the vector offsets from a regular simplex (n+1 vectors for n dimensions) with probability dimension_rate and initial scale mutation_scale * variable range. A single vector offset may alter multiple coordinate dimensions. Multiple simplex vectors may or may not be applied.unary_simplex: Add a single randomly selected vector offset from a regular simplex with an initial scale of mutation_scale * variable range. One and only one simplex vector is applied, but this simplex vector may alter multiple coordinate dimensions.non_adaptive flag can be used to deactivate the self-adaptation in real-valued genetic algorithms, which may facilitate a more global search. The adaptive mutation in evolutionary pattern search is an inherent component that cannot be deactivated. The min_scale input specifies the minimum mutation scale for evolutionary pattern search; sgopt_epsa terminates if the adapted mutation scale falls below this threshold.
The num_partitions specification is not part of the crossover or mutation group specifications; it specifies the number of possible values for each dimension (fractions of the variable ranges) used in the initial evolutionary pattern search population. It is needed for theoretical reasons.
For additional information on these options, see the user and reference manuals for SGOPT [Hart, 2001a; Hart, 2001b].
sgopt_pattern_search, coliny_pattern_search, and coliny_apps. The SGOPT pattern search technique is invoked using a sgopt_pattern_search group specification. Components within this specification group include initial_delta, threshold_delta, pattern_basis, total_pattern_size, expand_after_success, no_expansion, contraction_factor, stochastic, seed, and exploratory_moves specifications. The initial_delta and threshold_delta specifications are required in order to provide the initial offset size and the threshold size at which to terminate the algorithm, respectively. These values are relative percentages of the bounded region. The pattern_basis specification is used to select between a coordinate basis or a simplex basis. The former uses a plus and minus offset in each coordinate direction, for a total of 2n function evaluations in the pattern, whereas the latter uses a minimal positive basis simplex for the parameter space, for a total of n+1 function evaluations in the pattern. The total_pattern_size specification can be used to augment the basic coordinate and simplex patterns with additional function evaluations, and is particularly useful for parallel load balancing. For example, if some function evaluations in the pattern are dropped due to duplication or bound constraint interaction, then the total_pattern_size specification instructs the algorithm to generate new offsets to bring the total number of evaluations up to this consistent total. The expand_after_success control specifies how many successful objective function improvements must occur with a specific delta prior to expansion of the delta, whereas the no_expansion flag instructs the algorithm to forgo pattern expansion altogether. The contraction_factor specification selects the scaling factor used in computing a reduced offset for a new pattern search cycle after the previous cycle has been unsuccessful in finding an improved point. The SGOPT pattern search provides the capability for stochastic shuffling of offset evaluation order, for which the random seed can be used to make the optimizations repeatable. Finally, the exploratory_moves setting controls how the offset evaluations are ordered as well as the logic for acceptance of an improved point. The following exploratory moves selections are supported by SGOPT:
multi_step case examines each trial step in the pattern in turn. If a successful step is found, the pattern search continues examining trial steps about this new point. In this manner, the effects of multiple successful steps are cumulative within a single iteration. This option does not support any parallelism and will result in a serial pattern search.best_all case waits for completion of all offset evaluations in the pattern before selecting a new iterate. This method is most appropriate for parallel execution of the pattern search.best_first case immediately selects the first improving point found as the new iterate, without waiting for completion of all offset evaluations in the cycle. This option does not support any parallelism and will result in a serial pattern search.biased_best_first case immediately selects the first improved point as the new iterate, but also introduces a bias toward directions in which improving points have been found previously by reordering the offset evaluations. This option does not support any parallelism and will result in a serial pattern search.adaptive_pattern case invokes a pattern search technique that adaptively rescales the different search directions to maximize the number of redundant function evaluations. See [Hart et al., 2001] for details of this method. In preliminary experiments, this method had more robust performance than the standard best_first case. This option does not support any parallelism and will result in a serial pattern search.test case is used for development purposes. This currently utilizes a nonblocking scheduler (i.e., DakotaModel::synchronize_nowait()) for performing the function evaluations.| Description | Keyword | Associated Data | Status | Default |
| SGOPT pattern search method | sgopt_pattern_search | none | Required group | N/A |
| Stochastic pattern search | stochastic | none | Optional group | N/A |
| Random seed for stochastic pattern search | seed | integer | Optional | randomly generated seed |
| Initial offset value | initial_delta | real | Required | N/A |
| Threshold for offset values | threshold_delta | real | Required | N/A |
| Pattern basis selection | pattern_basis | coordinate | simplex | Optional | simplex |
| Total number of points in pattern | total_pattern_size | integer | Optional | no augmentation of basic pattern |
| No expansion flag | no_expansion | none | Optional | algorithm may expand pattern size |
| Number of consecutive improvements before expansion | expand_after_success | integer | Optional | 1 |
| Pattern contraction factor | contraction_factor | real | Optional | 0.5 |
| Exploratory moves selection | exploratory_moves | multi_step | best_all | best_first | biased_best_first | adaptive_pattern | test | Optional group | best_first for serial, best_all for parallel |
| Description | Keyword | Associated Data | Status | Default |
| SGOPT Solis-Wets method | sgopt_solis_wets | none | Required group | N/A |
| Random seed for stochastic pattern search | seed | integer | Optional | randomly generated seed |
| Initial offset value | initial_delta | real | Required | N/A |
| Threshold for offset values | threshold_delta | real | Required | N/A |
| No expansion flag | no_expansion | none | Optional | algorithm may expand pattern size |
| Number of consecutive improvements before expansion | expand_after_success | integer | Optional | 5 |
| Number of consecutive failures before contraction | contract_after_failure | integer | Optional | 3 |
| Pattern contraction factor | contraction_factor | real | Optional | 0.5 |
The seed, initial_delta, threshold_delta, no_expansion, expand_after_success, and contraction_factor specifications have identical meaning to the corresponding specifications for sgopt_pattern_search (see Pattern search). The only new specification is contract_after_failure, which specifies the number of unsuccessful cycles which must occur with a specific delta prior to contraction of the delta.
solution_accuracy (see SGOPT method dependent controls), the sMC algorithm can terminate adaptively when a design point with a desired performance has been located. Table 5.17 provides the specification detail for this method and its method dependent controls.| Description | Keyword | Associated Data | Status | Default |
| SGOPT stratified Monte Carlo method | sgopt_strat_mc | none | Required group | N/A |
| Random seed for stochastic pattern search | seed | integer | Optional | randomly generated seed |
| Number of samples per stratification | batch_size | integer | Optional | 1 |
| Partitions per variable | partitions | list of integers | Optional | No partitioning |
As for other SGOPT methods, the random seed is used to make stochastic optimizations repeatable. The batch_size input specifies the number samples to be evaluated in each multidimensional partition. And the partitions list is used to specify the number of partitions for each design variable. For example, partitions = 2, 4, 3 specifies 2 partitions in the first design variable, 4 partitions in the second design variable, and 3 partitions in the third design variable. This creates a total of 24 multidimensional partitions, and a batch_size of 2 would select 2 random samples in each partition, for a total of 48 samples on each iteration of the sMC algorithm. Iterations containing 48 samples will continue until the maximum number of iterations or function evaluations is exceeded, or the desired solution accuracy is obtained.
coliny_apps, coliny_cobyla, coliny_direct, coliny_pattern_search, coliny_solis_wets, and coliny_misc_solver. Of these, coliny_apps, coliny_cobyla, and coliny_direct are new methods which will be discussed below. The coliny_pattern_search and coliny_solis_wets methods are updated versions of sgopt_pattern_search and sgopt_solis_wets and have new features focused primarily on more general support of constraints. The Coliny method dependent controls are very similar to those described in Pattern search and Solis-Wets, respectively. Finally, the coliny_misc_solver method is a convenient hook for new algorithm testing. Additional Coliny information is available from http://software.sandia.gov/Acro/Coliny/.
Coliny solvers now support bound constraints and general nonlinear constraints. Supported nonlinear constraints include both equality and two-sided inequality constraints. Coliny solvers do not yet support linear constraints. Most Coliny optimizers treat constraints with a simple penalty scheme that adds constraint_penalty times the sum of squares of the constraint violations. Specific exceptions to this method for handling constraint violations are noted below. (The default value of constraint_penalty is 1000.0, except for methods that dynamically adapt their constraint penalty, for which the default value is 1.0.)
max_iterations and max_function_evaluations limit the number of major iterations and the number of function evaluations that can be performed during a Coliny optimization. The convergence_tolerance control defines the threshold value on relative change in the objective function that indicates convergence. The output verbosity specification controls the amount of information generated by Coliny: the silent, quiet, and normal settings correspond to minimal reporting from Coliny, whereas the verbose setting corresponds to a higher level of information, and debug outputs method initialization and a variety of internal Coliny diagnostics. The majority of Coliny's methods have independent function evaluations that can directly take advantage of DAKOTA's parallel capabilities. Only coliny_solis_wets and certain configurations of coliny_pattern_search are inherenty serial (see Pattern search). The parallel methods automatically utilize parallel logic when the DAKOTA configuration supports parallelism. Lastly, neither speculative gradients nor linear constraints are currently supported with Coliny. Specification detail for method independent controls is provided in Tables 5.1 through 5.4.show_misc_options optional specification which results in a dump of all the allowable method inputs. Note that the information provided by this command refers to optimizer parameters that are internal to Coliny, and which may differ from corresponding parameters used by the DAKOTA interface. The misc_options optional specification provides a means for inputing additional settings supported by the Coliny methods but which are not currently mapped through the DAKOTA input specification.
Each of the Coliny methods supports the solution_accuracy control, which defines a convergence criterion in which the optimizer will terminate if it finds an objective function value lower than the specified accuracy. Specification detail for method dependent controls for all Coliny methods is provided in Table 5.18.
| Description | Keyword | Associated Data | Status | Default |
| Show miscellaneous options | show_misc_options | none | Optional | no dump of specification options |
| Specify miscellaneous options | misc_options | list of strings | Optional | no miscellaneous options specified |
| Desired solution accuracy | solution_accuracy | real | Optional | -DBL_MAX |
Each Coliny method supplements the settings of Table 5.18 with controls which are specific to its particular class of method.
coliny_pattern_search group specification, which includes a variety of specification components.
Traditional pattern search methods search with a fixed pattern of search directions to try to find improvements to the current iterate. The Coliny pattern search methods generalize this simple algorithmic strategy to enable control of how the search pattern is adapted, as well as how each search pattern is evaluated. The stochastic and synchronization specifications denote how the the trial points are evaluated. The stochastic specification indicates that the trial points are considered in a random order. For parallel pattern search, synchronization dictates whether the evaluations are scheduled using a blocking scheduler or a nonblocking scheduler (i.e., Model::synchronize() or Model::synchronize_nowait(), respectively). In the blocking case, all points in the pattern are evaluated (in parallel), and if the best of these trial points is an improving point, then it becomes the next iterate. These runs are reproducible, assuming use of the same seed in the stochastic case. In the nonblocking case, then all points in the pattern may not be evaluated, since the first improving point found becomes the next iterate. Since the algorithm steps will be subject to parallel timing variabilities, these runs will not generally be repeatable. The synchronization specification has similar connotations for sequential pattern search. If blocking is specified, then each sequential iteration terminates after all trial points have been considered, and if nonblocking is specified, then each sequential iteration terminates after the first improving trial point is evaluated.
The particular form of the search pattern is controlled by the pattern_basis specification. If pattern_basis is coordinate basis, then the pattern search uses a plus and minus offset in each coordinate direction, for a total of 2n function evaluations in the pattern. If pattern_basis is simplex_basis, then pattern search uses a minimal positive basis simplex for the parameter space, for a total of n+1 function evaluations in the pattern. The total_pattern_size specification can be used to augment the basic coordinate and simplex patterns with additional function evaluations, and is particularly useful for parallel load balancing. For example, if some function evaluations in the pattern are dropped due to duplication or bound constraint interaction, then the total_pattern_size specification instructs the algorithm to generate new offsets to bring the total number of evaluations up to this consistent total.
The exploratory_moves specification controls how the search pattern is adapted. (The search pattern can be adapted after an improving trial point is found, or after all trial points in a search pattern have been found to be unimproving points.) The following exploratory moves selections are supported by Coliny:
basic_pattern case is the simple pattern search approach, which uses the same pattern in each iteration.multi_step case examines each trial step in the pattern in turn. If a successful step is found, the pattern search continues examining trial steps about this new point. In this manner, the effects of multiple successful steps are cumulative within a single iteration. This option does not support any parallelism and will result in a serial pattern search.adaptive_pattern case invokes a pattern search technique that adaptively rescales the different search directions to maximize the number of redundant function evaluations. See [Hart et al., 2001] for details of this method. In preliminary experiments, this method had more robust performance than the standard basic_pattern case in serial tests. This option does not support any parallelism and will result in a serial pattern search.initial_delta and threshold_delta specifications provide the initial offset size and the threshold size at which to terminate the algorithm. For any dimension that has both upper and lower bounds, this step length will be internally rescaled to provide search steps of length initial_delta * range. This rescaling does not occur for other dimensions, so search steps in those directions have length initial_delta.
In general, pattern search methods can expand and contract their step lengths. Coliny pattern search methods contract the step length by the value contraction_factor, and they expand the step length by the value (1/contraction_factor). The expand_after_success control specifies how many successful objective function improvements must occur with a specific step length prior to expansion of the step length, whereas the no_expansion flag instructs the algorithm to forgo pattern expansion altogether.
Finally, constraint infeasibility can be managed in a somewhat more sophisticated manner than the simple weighted penalty function. If the constant_penalty specification is used, then the simple weighted penalty scheme described above is used. Otherwise, the constraint penalty is adapted to the value constraint_penalty/L, where L is the the smallest step length used so far.
Table 5.19 and Table 5.20 provide the specification detail for the Coliny pattern search method and its method dependent controls.
| Description | Keyword | Associated Data | Status | Default |
| Coliny pattern search method | coliny_pattern_search | none | Required group | N/A |
| Stochastic pattern search | stochastic | none | Optional group | N/A |
| Random seed for stochastic pattern search | seed | integer | Optional | randomly generated seed |
| Initial offset value | initial_delta | real | Required | N/A |
| Threshold for offset values | threshold_delta | real | Required | N/A |
| Constraint penalty | constraint_penalty | real | Optional | 1.0 |
| Control of dynamic penalty | constant_penalty | none | Optional | algorithm dynamically adapts the constraint penalty |
| Description | Keyword | Associated Data | Status | Default |
| Pattern basis selection | pattern_basis | coordinate | simplex | Optional | simplex |
| Total number of points in pattern | total_pattern_size | integer | Optional | no augmentation of basic pattern |
| No expansion flag | no_expansion | none | Optional | algorithm may expand pattern size |
| Number of consecutive improvements before expansion | expand_after_success | integer | Optional | 1 |
| Pattern contraction factor | contraction_factor | real | Optional | 0.5 |
| Evaluation synchronization | synchronization | blocking | nonblocking | Optional | nonblocking |
| Exploratory moves selection | exploratory_moves | basic_pattern | multi_step | adaptive_pattern | Optional | basic_pattern |
| Description | Keyword | Associated Data | Status | Default |
| Coliny Solis-Wets method | coliny_solis_wets | none | Required group | N/A |
| Random seed for stochastic pattern search | seed | integer | Optional | randomly generated seed |
| Initial offset value | initial_delta | real | Required | N/A |
| Threshold for offset values | threshold_delta | real | Required | N/A |
| No expansion flag | no_expansion | none | Optional | algorithm may expand pattern size |
| Number of consecutive improvements before expansion | expand_after_success | integer | Optional | 5 |
| Number of consecutive failures before contraction | contract_after_failure | integer | Optional | 3 |
| Pattern contraction factor | contraction_factor | real | Optional | 0.5 |
| Constraint penalty | constraint_penalty | real | Optional | 1.0 |
| Control of dynamic penalty | constant_penalty | none | Optional | algorithm dynamically adapts the constraint penalty |
These specifications have the same meaning as corresponding specifications for coliny_pattern_search. In particular, coliny_solis_wets supports dynamic rescaling of the step length, and dynamic rescaling of the constraint penalty. The only new specification is contract_after_failure, which specifies the number of unsuccessful cycles which must occur with a specific delta prior to contraction of the delta.
synchronization nonblocking). APPS is currently interfaced to DAKOTA as part of Coliny (method coliny_apps). APPS-specific software documentation is available from http://software.sandia.gov/appspack/.
The only method independent control currently mapped to APPS is the output verbosity control. The APPS internal "debug" level is mapped to the DAKOTA debug, verbose, normal, quiet, and silent settings as follows:
coliny_apps group specification. The method dependent controls are a subset of the Coliny controls for coliny_pattern_search described in Pattern search. In particular, APPS supports initial_delta, threshold_delta, and contraction_factor, and the APPS step lengths are dynamically rescaled like the steps in coliny_pattern_search. Coliny specifications such as pattern_basis, total_pattern_size, and no_expansion are not supported since APPS only supports coordinate bases with a total of 2n function evaluations in the pattern, and these patterns may only contract. The synchronization specification can be used to specify the use of either blocking or nonblocking schedulers for APPS. Table 5.22 summarizes the APPS specification.| Description | Keyword | Associated Data | Status | Default |
| APPS method | coliny_apps | none | Required group | N/A |
| Initial offset value | initial_delta | real | Required | N/A |
| Threshold for offset values | threshold_delta | real | Required | N/A |
| Pattern contraction factor | contraction_factor | real | Optional | 0.5 |
| Evaluation synchronization | synchronization | blocking | nonblocking | Optional | nonblocking |
| Constraint penalty | constraint_penalty | real | Optional | 1000.0 |
initial_delta to threshold_delta. COBYLA has an advantage over many of its competitors, however, which is that it treats each constraint individually when calculating a change to the variables, instead of lumping the constraints together into a single penalty function.
COBYLA currently only supports termination based on the max_function_evaluations and solution_accuracy specifications. The search performed by COBYLA is currently not parallelized.
Table 5.23 summarizes the COBYLA specification.
| Description | Keyword | Associated Data | Status | Default |
| COBYLA method | coliny_cobyla | none | Required group | N/A |
| Initial offset value | initial_delta | real | Required | N/A |
| Threshold for offset values | threshold_delta | real | Required | N/A |
Figure 5.1 Design space partitioning with DIRECT
In practice, DIRECT has proven an effective heuristic for engineering design applications, for which it is able to quickly identify candidate solutions that can be further refined with fast local optimizers. This capability is currently operational for serial executions.
DIRECT uses the solution_accuracy, constraint_penalty and show_misc_options specifications that are described in Coliny method dependent controls. Note, however, that DIRECT uses a fixed penalty value for constraint violations (i.e. it is not dynamically adapted as is done in coliny_pattern_search).
The division specification determines how DIRECT subdivides each subregion of the search space. If division is set to major_dimension, then the dimension representing the longest edge of the subregion is subdivided (this is the default). If division is set to all_dimensions, then all dimensions are simultaneously subdivided.
Each subregion considered by DIRECT has a size, which corresponds to the longest diagonal of the subregion. The global_balance_parameter controls how much global search is performed by only allowing a subregion to be subdivided if the size of the subregion divided by the size of the largest subregion is at least global_balance_parameter. Intuitively, this forces large subregions to be subdivided before the smallest subregions are refined. The local_balance_parameter provides a tolerance for estimating whether the smallest subregion can provide a sufficient decrease to be worth subdividing; the default value is a small value that is suitable for most applications.
DIRECT can be terminated with the standard max_function_evaluations and solution_accuracy specifications. Additionally, the max_boxsize_limit specification will terminated DIRECT if the size of the largest subregion falls below this threshold. The min_boxsize_limit specification terminates DIRECT if the size of the smallest subregion falls below this threshold. In practice, this later specification is likely to be more effective at limited DIRECT's search.
Table 5.24 summarizes the DIRECT specification.
| Description | Keyword | Associated Data | Status | Default |
| DIRECT method | coliny_direct | none | Required group | N/A |
| Box subdivision approach | division | major_dimension | all_dimensions | Optional group | major_dimension |
| Global search balancing parameter | global_balance_parameter | real | Optional | 0.0 |
| Local search balancing parameter | local_balance_parameter | real | Optional | 1.e-8 |
| Maximum boxsize limit | max_boxsize_limit | real | Optional | 0.0 |
| Minimum boxsize limit | min_boxsize_limit | real | Optional | 0.0001 |
| Constraint penalty | constraint_penalty | real | Optional | 1000.0 |
moga and soga within DAKOTA. DAKOTA provides access to the JEGA library through the JEGAOptimizer class.The JEGA/DAKOTA interface is still undergoing refinements and is in a beta release state for DAKOTA v3.2.
max_iterations and max_function_evaluations method independent controls to provide integer limits for the maximum number of generations and function evaluations, respectively. Note that currently, the DAKOTA default for max_iterations is 100 and for max_function_evaluations is 1000. These are the default settings that will be used to "stop" the JEGA algorithms, unless the user resets them or unless some specific convergence criteria are set (this is only in the case of SOGA, see Table 5.28 below).moga), and a single- objective genetic algorithm (soga). Both of these GAs can take real-valued inputs, integer-valued inputs, or a mixture of real and integer-valued inputs. "Real-valued" and "integer-valued" refer to the use of continuous or discrete variable domains, respectively (the response data are real-valued in all cases).The basic steps of the genetic algorithm are as follows:
replacement_type) called domination_count. If this replacement mechanism is specified, the user defines a domination_cutoff. If a particular solution is dominated by more than domination_cutoff population members, then it is discarded. Otherwise, it is kept. Thus, this particular replacement type does not need a fitness assessor. There are fitness assessors available that go with some other replacement types, however. For example, in the case of a SOGA, one may apply an exterior penalty multiplier to the constraint violations and sum this penalty term with the objective function. Then, for example, this fitness may be used in a roulette wheel replacement scheme. replacement_type of roulette_wheel or unique_roulette_wheel may be used either with MOGA or SOGA problems. If a roulette wheel replacement is used with a MOGA, the fitness used is a "Layered fitness." In a layered scheme, the solutions are assigned "layers" based on their rank in domination and feasibility, and the layers are translated to fitness values. If roulette wheel replacement is used with a SOGA, the objective is calculated by applying the exterior penalty multiplier to the sum of constraint violations. The replacement_type of domination_count is specific to a MOGA. The replacement_type of favor_feasible is specific to a SOGA. This replacement operator will always take a feasible design over an infeasible one. Beyond that, if favors solutions based on an assigned fitness value which must have been installed by some fitness assessor. max_function_evaluations or max_iterations. This convergence stops the optimization after max_function_evaluations or max_iterations or both. In addition, there are two convergence types for SOGA problems which stop the GA after the average fitness or best fitness in the population has remained basically unchanged for a certain number of generations. There are many controls which can be used for both MOGA and SOGA methods. These include random seed, initialization types, crossover and mutation types, main loop controls, and some replacement types. These are described in Tables 5.25 and 5.26 below.
The seed control defines the starting seed for the random number generator. initialization_type defines the type of initialization for the GA. There are three types: random, unique_random, and flat_file. random creates initial solutions with random variable values according to a uniform random number distribution. It gives no consideration to any previously generated designs. The number of designs is specified by the population_size. unique_random is the same as random, except that when a new solution is generated, it is checked against the rest of the solutions. If it duplicates any of them, it is rejected. flat_file allows the initial population to be read from a flat file. If flat_file is specified, a file name must be given. Variables must be delimited with a tab in the input file. The input file will continue to read until the end of the file. The algorithm will discard any configurations for which it was unable to retrieve at least the number of design variables. The objective and constraint entries are not required but if all are present, they will be recorded and the Design will be tagged as evaluated so that evaluators may choose not to re-evaluate them. Setting the size for this initializer has the effect of requiring a minimum number of Designs to create. If this minimum number has not been created once the files are all read, the rest are created using the unique_random initializer.
Note that the population_size only sets the size of the initial population. The population size varies in the JEGA methods according to the type of operators chosen for a particular optimization run.
There are many crossover types available. multi_point_binary crossover requires an integer number, N, of crossover points. This crossover type performs a bit switching crossover at N crossover points in the binary encoded genome of two designs. Thus, crossover may occur at any point along a solution chromosome (in the middle of a gene representing a design variable, for example). multi_point_parameterized_binary crossover is similar, in that it performs a bit switching crossover routine at N crossover points. However, the crossover points are only between design variables. Thus, this crossover type performs crossover on a design variable or sets of design variables. multi_point_real crossover performs a variable switching crossover routing at N crossover points in the real encoded genome of two designs. In this scheme as in multi_point_parameterized_binary, crossover only occurs between design variables. Note that the standard solution chromosome representation in the JEGA algorithm is real encoded and can handle integer or real decision variables. For the first two crossover types that use a binary representation, real variables are converted to long integers by multiplying the real number by 10^6 and then truncating. Note that this assumes a precision of only six decimal places. Discrete variables are treated as integers.
The final crossover type is shuffle_random. This crossover type performs crossover by choosing design variables at random from a specified number of parents enough times that the requested number of children are produced. For example, consider the case of 3 parents producing 2 children. This operator would go through and for each design variable, select one of the parents as the donor for the child. So it creates a random shuffle of the parent design variable values. The relative numbers of children and parents are controllable to allow for as much mixing as desired. The more parents involved, the less likely that the children will wind up exact duplicates of the parents.
All crossover types take a crossover_rate. The crossover rate is used to calculate the number of crossover operations that take place. The number of crossovers is equal to the rate * population_size.
There are five mutation types allowed. replace_uniform introduces random variation by first randomly choosing a design variable of a randomly selected design and reassigning it to a random valid value for that variable. No consideration of the current value is given when determining the new value. All mutation types have a population_rate. The number of mutations for the replace_uniform mutator is the product of the population_rate and the population_size.
The bit_random mutator introduces random variation by first converting a randomly chosen variable of a randomly chosen Design into a binary string. It then flips a randomly chosen bit in the string from a 1 to a 0 or visa versa. This mutator is similar to the replace_uniform, only it is mutating on a binary representation and not a real representation. Also, the resulting value from a bit_random mutator has a high probability that it will be similar to the original value, but the resulting value from a replace_uniform mutator is more likely to be significantly different than the original value. The number of mutations performed is the product of the population_rate, the number of design variables, and the population size.
The offset mutators all act by adding an "offset" random amount to the variable. The random amount has a mean of zero in all cases. The offset_normal mutator introduces random variation by adding a Gaussian random amount to a variable value. The random amount has a mean of 0 and a standard deviation dependent on the offset range. For the offset_normal mutator, the offset range is interpreted as a fraction of the total range of the variable. The standard deviation is computed as the product of the offset range and the total range of the variable. mutation_scale is a fraction in the range [0, 1] and is meant to help control the amount of variation that takes place when a variable is mutated. mutation_scale is multiplied by the range of the variable being mutated to serve as standard deviation. offset_cauchy is similar to offset_normal, except that a Cauchy random variable is added to the variable being mutated. The mutation_scale also defines the standard deviation for this mutator. Finally, offset_uniform adds a uniform random amount to the variable value. For the offset_uniform mutator, the offset range is interpreted as a fraction of the total range of the variable. The magnitude of the deviation is +/- 1/2 * (offset range * variable range). The offset range is defined by mutation_scale. The number of mutations for all offset mutators is defined as the product of population_rate and population_size.
The replacement types that are common to both MOGA and SOGA are roulette_wheel and unique_roulette_wheel. In roulette_wheel replacement, each Design is allotted a portion of a wheel proportional to its fitness relative to the fitnesses of the other Designs. Then portions of the wheel are chosen at random and the Design occupying those portions are duplicated into the next population. Those Designs allotted larger portions of the wheel are more likely to be selected (potentially many times). unique_roulette_wheel replacement is the same as roulette_wheel replacement, with the exception that a Design may only be selected once.
| Description | Keyword | Associated Data | Status | Default |
| GA Method | moga| soga | none | Required group | N/A |
| Random Seed | seed | integer | Optional | Randomly generated seed |
| Initialization type | initialization_type | flat_file | random | unique_random | Required | unique_random |
| Mutation type | mutation_type | replace_uniform | bit_random | offset_cauchy | offset_uniform | offset_normal | Optional group | None |
| Mutation scale | mutation_scale | real | Optional | 0.15 |
| Mutation population rate | population_rate | real | Optional | 0.08 |
| Replacement type | replacement_type | roulette_wheel | unique_roulette_wheel | Optional group | None |
| Description | Keyword | Associated Data | Status | Default |
| Crossover type | crossover_type | multi_point_binary | multi_point_parameterized_binary | multi_point_real | shuffle_random | Optional group | none |
| Multi point binary crossover | multi_point_binary | integer | Required | N/A |
| Multi point parameterized binary crossover | multi_point_parameterized_binary | integer | Required | N/A |
| Multi point real crossover | multi_point_real | integer | Required | N/A |
| Random shuffle crossover | shuffle_random | num_parents, num_offspring | Required | N/A |
| Number of parents in random shuffle crossover | num_parents | integer | optional | 2 |
| Number of offspring in random shuffle crossover | num_offspring | integer | optional | 2 |
| Crossover rate | crossover_rate | real | optional (applies to all crossover types) | 0.8 |
moga as the method.
The initialization, crossover, and mutation controls were all described in the preceding section. There are no MOGA specific aspects to these controls. The replacement_type for a MOGA may be roulette_wheel, unique_roulette_wheel, or domination_count. The domination_count replacement is the default and is recommended. It works especially well on multi-objective problems because it has been specifically designed to avoid problems with aggregating and scaling objective function values and transforming them into a single objective. Instead, domination_count works by ordering population members by number of dominated designs. If a design is dominated by more than a number of designs (domination_cutoff), then it is discarded. Otherwise it is kept and selected to go to the next generation. The one catch is that this replacement will require that a minimum number of selections take place. shrinkage_percentage defines the minimum amount of selections that will take place if enough designs are available. It is interpreted as a percentage of the population size that must go on to the subsequent generation. To enforce this, domination_count makes all the replacements it would make anyway and if that is not enough, it re-ranks what is left and makes selections from those. It continues until it has made enough selections. The shrinkage_percentage is designed to prevent extreme decreases in the population size at any given generation, and thus prevent a big loss of genetic diversity in a very short time. Without a shrinkage limit, a small group of "super" designs may appear and quickly cull the population down to a size on the order of domination_count. In this case, all the diversity of the population is lost and it is expensive to re-diversify and spread the population. Another instance when it is beneficial to keep a population of reasonable size is when the algorithm has settled into one area of the response space and then happens on a new, better area through exploration. The shrinkage_percentage prevents a fast move to the new area.
The MOGA specific controls are described in Table 5.27 below. Note that MOGA and SOGA create additional output files during execution. "Finaldata.dat" is a file that holds the Pareto members of the population in the final generation. "Discards.dat" holds solutions that were discarded from the population during the course of evolution. It can often be useful to plot objective function values from these files to visually see the Pareto front and ensure that finaldata.dat solutions dominate discards.dat solutions. The solutions are written to these output files in the format "Input1...InputN..Output1...OutputM". If MOGA is used in a multi-level optimization strategy (which requires one optimal solution from each individual optimization method to be passed to the subsequent optimization method as its starting point), the solution in the Pareto set closest to the "utopia" point is given as the best solution. This solution is also reported in the DAKOTA output. This "best" solution in the Pareto set has minimum distance from the utopia point. The utopia point is defined as the poitn of extreme (best) values for each objective function. For example, if the Pareto front is bounded by (1,100) and (90,2), then (1,2) is the utopia point. There will be a point in the Pareto set that has minimum L2-norm distance to this point, for example (10,10) may be such a point. In SOGA, the solution that minimizes the single objective function is returned as the best solution.
| Description | Keyword | Associated Data | Status | Default |
| Replacement type | replacement_type | domination_count | roulette_wheel | unique_roulette_wheel | Optional group | domination_count |
| Domination cutoff | domination_cutoff | integer | Optional | 6 |
| Shrinkage percentage | shrinkage_percentage | real | Optional | 0.9 |
soga as the method.
The initialization, crossover, and mutation controls were all described above. There are no SOGA specific aspects to these controls. The replacement_type for a SOGA may be roulette_wheel, unique_roulette_wheel, or favor_feasible. The favor_feasible replacement type always takes a feasible design over an infeasible one. Beyond that, it selects designs based on a fitness value. For SOGA problems, the user is allowed to specify an exterior_penalty_multiplier with roulette_wheel or unique_roulette_wheel replacement. The penalty multiplier is a parameter that multiplies the constraint violation penalty prior to summation with a weighted sum of objectives to obtain a fitness value.
The SOGA controls allow two additional convergence types. The convergence_type called average_fitness_tracker keeps track of the average fitness in a population. If this average fitness does not change more than percent_change over some number of generations, num_generations, then the solution is reported as converged and the algorithm terminates. The best_fitness_tracker works in a similar manner, only it tracks the best fitness in the population. Convergence occurs after num_generations has passed and there has been less than percent_change in the best fitness value.
The SOGA specific controls are described in Table 5.28 below.
| Description | Keyword | Associated Data | Status | Default |
| Replacement type | replacement_type | favor_feasible | unique_roulette_wheel | roulette_wheel | Optional group | |
| Exterior penalty multiplier | exterior_penalty_multiplier | real | Optional | |
| Convergence type | convergence_type | best_fitness_tracker | average_fitness_tracker | Optional | |
| Number of generations (for convergence test) | num_generations | integer | Optional | 15 |
| Percent change in fitness | percent_change | real | Optional | 0.1 |
The important difference of these algorithms from general-purpose optimization methods is that the response set is defined by least squares terms, rather than an objective function. Thus, a finer granularity of data is used by least squares solvers as compared to that used by optimizers. This allows the exploitation of the special structure provided by a sum of squares objective function. Refer to Least squares terms and constraint functions (least squares data set) for additional information on the least squares response data set.
nl2sol and addresses unconstrained and bound-constrained problems. It uses a trust-region method (and thus can be viewed as a generalization of the Levenberg-Marquardt algorithm) and adaptively chooses between two Hessian approximations, the Gauss-Newton approximation alone and the Gauss-Newton approximation plus a quasi-Newton approximation to the rest of the Hessian. Even on small-residual problems, the latter Hessian approximation can be useful when the starting guess is far from the solution. On problems that are not over-parameterized (i.e., that do not involve more optimization variables than the data support), NL2SOL usually exhibits fast convergence.
NL2SOL has a variety of internal controls as described in AT&T Bell Labs CS TR 153 (http://cm.bell-labs.com/cm/cs/cstr/153.ps.gz). A number of existing DAKOTA controls (method independent controls and responses controls) are mapped into these NL2SOL internal controls. In particular, DAKOTA's convergence_tolerance, max_iterations, max_function_evaluations, and fd_gradient_step_size are mapped directly into NL2SOL's rfctol, mxiter, mxfcal, and dltfdj controls, respectively. In addition, DAKOTA's fd_hessian_step_size is mapped into both delta0 and dltfdc, and DAKOTA's output verbosity is mapped into NL2SOL's auxprt and outlev (for normal/ prints initial guess, final solution, solution statistics, nondefault values, and changes to the active bound constraint set on every iteration; for verbose/ NL2SOLdebug output,quiet output, NL2SOL prints only the initial guess and final solution; and for silent output, NL2SOL output is suppressed).
Several NL2SOL convergence tolerances are adjusted in response to function_precision, which gives the relative precision to which responses are computed. These tolerances may also be specified explicitly: convergence_tolerance (NL2SOL's rfctol, as mentioned previously) is the relative-function convergence tolerance (on the accuracy desired in the sum-of-squares function); x_conv_tol (NL2SOL's xctol) is the X-convergence tolerance (scaled relative accuracy of the solution variables); absolute_conv_tol (NL2SOL's afctol) is the absolute function convergence tolerance (stop when half the sum of squares is less than absolute_conv_tol, which is mainly of interest on zero-residual test problems); singular_conv_tol (NL2SOL's sctol) is the singular convergence tolerance, which works in conjunction with singular_radius (NL2SOL's lmaxs) to test for underdetermined least-squares problems (stop when the relative reduction yet possible in the sum of squares appears less then singular_conv_tol for steps of scaled length at most singular_radius); false_conv_tol (NL2SOL's xftol) is the false-convergence tolerance (stop with a suspicion of discontinuity when a more favorable stopping test is not satisfied and a step of scaled length at most false_conv_tol is not accepted). Finally, the initial_trust_radius specification (NL2SOL's lmax0) specifies the initial trust region radius for the algorithm.
The internal NL2SOL defaults can be obtained for many of these controls by specifying the value -1. For both the singular_radius and the initial_trust_radius, this results in the internal use of steps of length 1. For other controls, the internal defaults are often functions of machine epsilon (as limited by function_precision). Refer to CS TR 153 for additional details on these formulations.
Whether and how NL2SOL computes and prints a final covariance matrix and regression diagnostics is affected by several keywords. covariance (NL2SOL's covreq) specifies the desired covariance approximation:
fd_hessian_step_size) fd_hessian_step_size)regression_diagnostics (NL2SOL's rdreq) is specified and a positive-definite final Hessian approximation H is computed, NL2SOL computes and prints a regression diagnostic vector RD such that if omitting the i-th observation would cause alpha times the change in the solution that omitting the j-th observation would cause, then RD[i] = |alpha| RD[j]. The finite-difference step-size tolerance affecting H is fd_hessian_step_size (NL2SOL's delta0 and dltfdc, as mentioned previously).Table 5.29 provides the specification detail for the NL2SOL method dependent controls.
| Description | Keyword | Associated Data | Status | Default |
| Relative precision in least squares terms | function_precision | real | Optional | 1e-10 |
| Absolute function convergence tolerance | absolute_conv_tol | real | Optional | -1. (use NL2SOL internal default) |
| Convergence tolerance for change in parameter vector | x_conv_tol | real | Optional | -1. (use NL2SOL internal default) |
| Singular convergence tolerance | singular_conv_tol | real | Optional | -1. (use NL2SOL internal default) |
Step limit for sctol | singular_radius | real | Optional | -1. (use NL2SOL internal default of 1) |
| False convergence tolerance | false_conv_tol | real | Optional | -1. (use NL2SOL internal default) |
| Initial trust region radius | initial_trust_radius | real | Optional | -1. (use NL2SOL internal default of 1) |
| Covariance post-processing | covariance | integer | Optional | 0 (no covariance) |
| Regression diagnostics post-processing | regression_diagnostics | none | Optional | no regression diagnostics |
nlssol_sqp and supports unconstrained, bound-constrained, and generally-constrained problems. It exploits the structure of a least squares objective function through the periodic use of Gauss-Newton Hessian approximations to accelerate the SQP algorithm. DAKOTA provides access to the NLSSOL library through the NLSSOLLeastSq class. The method independent and method dependent controls are identical to those of NPSOL as described in NPSOL method independent controls and NPSOL method dependent controls.optpp_g_newton and supports unconstrained, bound-constrained, and generally-constrained problems. The code for the Gauss-Newton approximation (objective function value, gradient, and approximate Hessian defined from residual function values and gradients) is provided outside of OPT++ within SNLLLeastSq::nlf2_evaluator_gn(). When interfaced with the unconstrained, bound-constrained, and nonlinear interior point full-Newton optimizers from the OPT++ library, it provides a Gauss-Newton least squares capability which -- on zero-residual test problems -- can exhibit quadratic convergence rates near the solution. (Real problems almost never have zero residuals, i.e., perfect fits.)
Mappings for the method independent and dependent controls are the same as for the OPT++ optimization methods and are as described in OPT++ method independent controls and OPT++ method dependent controls. In particular, since OPT++ full-Newton optimizers provide the foundation for Gauss-Newton, the specifications from Table 5.8 are also applicable for optpp_g_newton.
Each of these techniques supports response_levels, probability_levels, and reliability_levels specifications along with optional num_response_levels, num_probability_levels, and num_reliability_levels keys. The keys define the distribution of the levels among the different response functions. For example, the following specification
num_response_levels = 2 4 3 \ response_levels = 1. 2. .1 .2 .3 .4 10. 20. 30. \
num_response_levels key were omitted from this example, then the response levels would be evenly distributed among the response functions (three levels each in this case).
The response_levels specification provides the target response values for generating probabilities and/or reliabilities (forward mapping). The selection of probability or reliability results for the forward mapping can be performed with the compute keyword followed by either probabilities or reliabilities. Conversely, the probability_levels and reliability_levels specifications provide target levels for which response values will be computed (inverse mapping). The mapping results (probabilities or reliabilities for the forward mapping and response values for the inverse mapping) define the final statistics of the nondeterministic analysis that can be accessed via the primary and secondary mapping matrices for nested models (see Method Independent Controls). Sets of response-probability pairs computed with the forward/inverse mappings define either a cumulative distribution function (CDF) or a complementary cumulative distribution function (CCDF) for each response function. The selection of a CDF or CCDF can be performed with the distribution keyword followed by either cumulative for the CDF option or complementary for the CCDF option. Table 5.30 provides the specification detail for the forward/inverse mappings used by each of the nondeterministic analysis methods.
| Description | Keyword | Associated Data | Status | Default |
| Distribution type | distribution | cumulative | complementary | Optional group | cumulative (CDF) |
| Response levels | response_levels | list of reals | Optional group | No CDF/CCDF probabilities/reliabilities to compute |
| Number of response levels | num_response_levels | list of integers | Optional | response_levels evenly distributed among response functions |
| Target statistics for response levels | compute | probabilities | reliabilities | Optional | probabilities |
| Probability levels | probability_levels | list of reals | Optional group | No CDF/CCDF response levels to compute |
| Number of probability levels | num_probability_levels | list of integers | Optional | probability_levels evenly distributed among response functions |
| Reliability levels | reliability_levels | list of reals | Optional group | No CDF/CCDF response levels to compute |
| Number of reliability levels | num_reliability_levels | list of integers | Optional | reliability_levels evenly distributed among response functions |
nond_sampling specification. This iterator performs sampling within specified uncertain variable probability distributions in order to determine distribution statistics for response functions. DAKOTA currently provides access to nondeterministic sampling methods through the combination of the NonDSampling base class and the NonDLHSSampling derived class.CDF/CCDF probabilities are calculated for specified response levels using a simple binning approach. Response levels are calculated for specified CDF/CCDF probabilities by indexing into a sorted samples array (the response levels computed are not interpolated and will correspond to one of the sampled values). CDF/CCDF reliabilities are calculated for specified response levels by computing the number of sample standard deviations separating the sample mean from the response level. Response levels are calculated for specified CDF/CCDF reliabilities by projecting out the prescribed number of sample standard deviations from the sample mean.
The seed integer specification specifies the seed for the random number generator which is used to make sampling studies repeatable. The fixed_seed flag is relevant if multiple sampling sets will be generated during the course of a strategy (e.g., surrogate-based optimization, optimization under uncertainty). Specifying this flag results in the reuse of the same seed value for each of these multiple sampling sets, which can be important for reducing variability in the sampling results. However, this behavior is not the default as the repetition of the same sampling pattern can result in a modeling weakness that an optimizer could potentially exploit (resulting in actual reliabilities that are lower than the estimated reliabilities). In either case (fixed_seed or not), the study is repeatable if the user specifies a seed and the study is random is the user omits a seed specification.
The number of samples to be evaluated is selected with the samples integer specification. The algorithm used to generate the samples can be specified using sample_type followed by either random, for pure random Monte Carlo sampling, or lhs, for Latin Hypercube sampling.
The nondeterministic sampling iterator also supports a design of experiments mode through the all_variables flag. Normally, nond_sampling generates samples only for the uncertain variables, and treats any design or state variables as constants. The all_variables flag alters this behavior by instructing the sampling algorithm to treat any continuous design or continuous state variables as parameters with uniform probability distributions between their upper and lower bounds. Samples are then generated over all of the continuous variables (design, uncertain, and state) in the variables specification. This is similar to the behavior of the design of experiments methods described in Design of Computer Experiments Methods, since they will also generate samples over all continuous design, uncertain, and state variables in the variables specification. However, the design of experiments methods will treat all variables as being uniformly distributed between their upper and lower bounds, whereas the nond_sampling iterator will sample the uncertain variables within their specified probability distributions.
Finally, the nondeterministic sampling iterator supports two types of sensitivity analysis. In this context of sampling, we take sensitivity analysis to be global, not local as when calculating derivatives of output variables with respect to input variables. Our definition is similar to that of [Saltelli et al., 2004]: "The study of how uncertainty in the output of a model can be apportioned to different sources of uncertainty in the model input." As a default, DAKOTA provides correlation analyses when running LHS. Correlation tables are printed with the simple, partial, and rank correlations between inputs and outputs. These can be useful to get a quick sense of how correlated the inputs are to each other, and how correlated various outputs are to inputs. In addition, we have the capability to calculate sensitivity indices through Variance Based Decomposition, variance_based_decomp. Variance based decomposition is a way of using sets of samples to understand how the variance of the output behaves, with respect to each input variable. A larger value of the sensitivity index, Si, means that the uncertainty in the input variable i has a larger effect on the variance of the output. More details on the calculations and interpretation of the sensitivity indices can be found in [Saltelli et al., 2004]. Note that variance_based_decomp is extremely computationally intensive since replicated sets of sample values are evaluated. If the user specified a number of samples, N, and a number of nondeterministic variables, M, variance-based decomposition requires the evaluation of N*(M+2) samples. To obtain sensitivity indices that are reasonably accurate, we recommend that N, the number of samples, be at least one hundred and preferably several hundred or thousands. Because of the computational cost, variance_based_decomp is turned off as a default. Table 5.31 provides details of the nondeterministic sampling specifications beyond those of Table 5.30.
| Description | Keyword | Associated Data | Status | Default |
| Nondeterministic sampling iterator | nond_sampling | none | Required group | N/A |
| Random seed | seed | integer | Optional | randomly generated seed |
| Fixed seed flag | fixed_seed | none | Optional | seed not fixed: sampling patterns are variable |
| Number of samples | samples | integer | Optional | minimum required |
| Sampling type | sample_type | random | lhs | Optional group | lhs |
| All variables flag | all_variables | none | Optional | sampling only over uncertain variables |
| Variance based decomposition | variance_based_decomp | none | Optional | No variance_based_decomp |
nond_reliability specification and are implemented within the NonDReliability class. These methods compute approximate response function distribution statistics based on specified uncertain variable probability distributions. Each of the reliability methods can compute the probabilities/reliabilities corresponding to specified response levels and the response levels corresponding to specified probability/reliability levels. Moreover, specifications of response_levels, probability_levels, and reliability_levels may be combined within the calculations for each response function.The Mean Value method (MV, also known as MVFOSM in [Haldar and Mahadevan, 2000]) is the simplest, least-expensive method in that it estimates the response means, response standard deviations, and all CDF/CCDF response-probability-reliability mappings from a single evaluation of response functions and gradients at the uncertain variable means. This approximation can have acceptable accuracy when the response functions are nearly linear and their distributions are approximately Gaussian, but can have very poor accuracy in other situations.
All other reliability methods perform an internal nonlinear optimization to compute a most probable point (MPP). The distance of the MPP from the origin in the transformed standard normal space ("u-space") defines the reliability index. The reliability can then be converted to a probability using either first- or second-order integration. The forward reliability analysis algorithm of computing probabilities for specified response levels is called the Reliability Index Approach (RIA), and the inverse reliability analysis algorithm of computing response levels for specified probability levels is called the Performance Measure Approach (PMA). The different RIA/PMA algorithm options are specified using the mpp_search specification which selects among different linearization approaches that can be used to reduce computational expense during the MPP searches. The x_linearize_mean MPP search option performs a single linearization in the space of the original uncertain variables ("x-space") centered at the uncertain variable means, searches for the MPP for each response/probability level using this linearization, and performs a validation response evaluation at each predicted MPP. This option is commonly known as the Advanced Mean Value (AMV) method. The u_linearize_mean option is identical to the x_linearize_mean option, except that the linearization is performed in u-space. The x_linearize_mpp approach starts with an x-space linearization at the uncertain variable means, but iteratively relinearizes at each MPP prediction until the MPP converges. This option is commonly known as the AMV+ method. The u_linearize_mpp option is identical to the x_linearize_mpp option, except that all linearizations are performed in u-space. And, finally, the no_linearize option performs the MPP search on the original response functions without the use of any linearizations. The optimization algorithm used to perform these MPP searches can be selected to be either sequential quadratic programming (uses the npsol_sqp optimizer) or nonlinear interior point (uses the optpp_q_newton optimizer) algorithms using the sqp or nip keywords.
In addition to the MPP search specifications, one may select among different integration approaches for computing probabilities at the MPP by using the integration keyword followed by either first_order or second_order. Combining the no_linearize option of the MPP search with first- and second-order integrations results in the traditional first- and second-order reliability methods (FORM and SORM). Additional details on these methods are available in [Eldred et al., 2004c].
Table 5.32 provides details of the reliability method specifications beyond those of Table 5.30.
| Description | Keyword | Associated Data | Status | Default |
| Reliability method | nond_reliability | none | Required group | N/A |
| MPP search type | mpp_search | x_linearize_mean | u_linearize_mean | x_linearize_mpp | u_linearize_mpp | no_linearize | Optional group | No MPP search (MV method) |
| MPP search algorithm | sqp, nip | none | Optional | NPSOL's SQP algorithm |
| Integration method | integration | first_order | second_order | Optional group | First-order integration |
The method requires either the expansion_terms or the expansion_order specification in order to specify the number of terms in the expansion or the highest order of Gaussian variable appearing in the expansion. The number of terms, P, in a complete polynomial chaos expansion of arbitrary order, p, for a response function involving n uncertain input variables is given by
![\[P = 1 + \sum_{s=1}^{p} {\frac{1}{s!}} \prod_{r=0}^{s-1} (n + r )\!.\]](form_12.png)
One must be careful when using the expansion_terms specification, as the satisfaction of the above equation for some order p is not rigidly enforced. As a result, in some cases, only a subset of terms of a certain order will be included in the series while others of the same order will be omitted. This omission of terms can increase the efficacy of the methodology for some problems but have extremely deleterious effects for others. The method outputs either the first expansion_terms coefficients of the series or the coefficients of all terms up to order expansion_order in the series depending on the specification. Additional specifications include the level mappings described in Nondeterministic Methods and the seed, fixed_seed, samples, and sample_type specifications described in Nondeterministic sampling method. Table 5.33 provides details of the polynomial chaos expansion specifications beyond those of Table 5.30.
| Description | Keyword | Associated Data | Status | Default |
| Polynomial chaos expansion iterator | nond_polynomial_chaos | none | Required group | N/A |
| Expansion terms | expansion_terms | integer | Required | N/A |
| Expansion order | expansion_order | integer | Required | N/A |
| Random seed | seed | integer | Optional | randomly generated seed |
| Fixed seed flag | fixed_seed | none | Optional | seed not fixed: sampling patterns are variable |
| Number of samples | samples | integer | Optional | minimum required |
| Sampling type | sample_type | random | lhs | Optional group | lhs |
grid), pure random sampling (random), orthogonal array sampling (oas), latin hypercube sampling (lhs), orthogonal array latin hypercube sampling (oa_lhs), Box-Behnken (box_behnken), and central composite design (central_composite). It is worth noting that there is some overlap in sampling techniques with those available from the nondeterministic branch. The current distinction is that the nondeterministic branch methods are designed to sample within a variety of probability distributions for uncertain variables, whereas the design of experiments methods treat all variables as having uniform distributions. As such, the design of experiments methods are well-suited for performing parametric studies and for generating data sets used in building global approximations (see Global approximation interface), but are not currently suited for assessing the effect of uncertainties. If a design of experiments over both design/state variables (treated as uniform) and uncertain variables (with probability distributions) is desired, then nond_sampling can support this with its all_variables option (see Nondeterministic sampling method). DAKOTA provides access to the DDACE library through the DDACEDesignCompExp class.
In terms of method dependent controls, the specification structure is straightforward. First, there is a set of design of experiments algorithm selections separated by logical OR's (grid or random or oas or lhs or oa_lhs or box_behnken or central_composite). Second, there are optional specifications for the random seed to use in generating the sample set (seed), for fixing the seed (fixed_seed) among multiple sample sets (see Nondeterministic sampling method for discussion), for the number of samples to perform (samples), and for the number of symbols to use (symbols). The seed control is used to make sample sets repeatable, and the symbols control is related to the number of replications in the sample set (a larger number of symbols equates to more stratification and fewer replications). The quality_metrics control is available for the DDACE library. This control turns on calculation of volumetric quality measures which measure the uniformity of the point samples. More details on the quality measures are given under the description of the FSU sampling methods. The variance_based_decomp control is also available. This control enables the calculation of sensitivity indices which indicate how important the uncertainty in each input variable is in contributing to the output variance. More details on variance based decomposition are given in Nondeterministic sampling method. Design of experiments specification detail is given in Table 5.34.
| Description | Keyword | Associated Data | Status | Default |
| Design of experiments iterator | dace | none | Required group | N/A |
| dace algorithm selection | grid | random | oas | lhs | oa_lhs | box_behnken | central_composite | none | Required | N/A |
| Random seed | seed | integer | Optional | randomly generated seed |
| Fixed seed flag | fixed_seed | none | Optional | seed not fixed: sampling patterns are variable |
| Number of samples | samples | integer | Optional | minimum required |
| Number of symbols | symbols | integer | Optional | default for sampling algorithm |
| Quality metrics | quality_metrics | none | Optional | No quality_metrics |
| Variance based decomposition | variance_based_decomp | none | Optional | No variance_based_decomp |
fsu_quasi_mc) based on the Halton sequence (halton) or the Hammersley sequence (hammersley), and Centroidal Voronoi Tessellation (fsu_cvt). All three methods generate sets of uniform random variables on the interval [0,1]. If the user specifies lower and upper bounds for a variable, the [0,1] samples are mapped to the [lower, upper] interval. The quasi-Monte Carlo and CVT methods are designed with the goal of low discrepancy. Discrepancy refers to the nonuniformity of the sample points within the hypercube. Discrepancy is defined as the difference between the actual number and the expected number of points one would expect in a particular set B (such as a hyper-rectangle within the unit hypercube), maximized over all such sets. Low discrepancy sequences tend to cover the unit hypercube reasonably uniformly. Quasi-Monte Carlo methods produce low discrepancy sequences, especially if one is interested in the uniformity of projections of the point sets onto lower dimensional faces of the hypercube (usually 1-D: how well do the marginal distributions approximate a uniform?) CVT does very well volumetrically: it spaces the points fairly equally throughout the space, so that the points cover the region and are isotropically distributed with no directional bias in the point placement. There are various measures of volumetric uniformity which take into account the distances between pairs of points, regularity measures, etc. Note that CVT does not produce low-discrepancy sequences in lower dimensions, however: the lower-dimension (such as 1-D) projections of CVT can have high discrepancy.The quasi-Monte Carlo sequences of Halton and Hammersley are deterministic sequences determined by a set of prime bases. Generally, we recommend that the user leave the default setting for the bases, which are the lowest primes. Thus, if one wants to generate a sample set for 3 random variables, the default bases used are 2, 3, and 5 in the Halton sequence. To give an example of how these sequences look, the Halton sequence in base 2 starts with points 0.5, 0.25, 0.75, 0.125, 0.625, etc. The first few points in a Halton base 3 sequence are 0.33333, 0.66667, 0.11111, 0.44444, 0.77777, etc. Notice that the Halton sequence tends to alternate back and forth, generating a point closer to zero then a point closer to one. An individual sequence is based on a radix inverse function defined on a prime base. The prime base determines how quickly the [0,1] interval is filled in. Generally, the lowest primes are recommended.
The Hammersley sequence is the same as the Halton sequence, except the values for the first random variable are equal to 1/N, where N is the number of samples. Thus, if one wants to generate a sample set of 100 samples for 3 random variables, the first random variable has values 1/100, 2/100, 3/100, etc. and the second and third variables are generated according to a Halton sequence with bases 2 and 3, respectively. For more information about these sequences, see [Halton, 1960, Halton and Smith, 1964, and Kocis and Whiten, 1997].
The specification for specifying quasi-Monte Carlo (fsu_quasi_mc) is given below in Table 5.35. The user must specify if the sequence is (halton) or (hammersley). The user must also specify the number of samples to generate for each variable (samples). Then, there are three optional lists the user may specify. The first list determines where in the sequence the user wants to start. For example, for the Halton sequence in base 2, if the user specifies sequence_start = 2, the sequence would not include 0.5 and 0.25, but instead would start at 0.75. The default sequence_start is a vector with 0 for each variable, specifying that each sequence start with the first term. The sequence_leap control is similar but controls the "leaping" of terms in the sequence. The default is 1 for each variable, meaning that each term in the sequence be returned. If the user specifies a sequence_leap of 2 for a variable, the points returned would be every other term from the QMC sequence. The advantage to using a leap value greater than one is mainly for high-dimensional sets of random deviates. In this case, setting a leap value to the next prime number larger than the largest prime base can help maintain uniformity when generating sample sets for high dimensions. For more information about the efficacy of leaped Halton sequences, see [Robinson and Atcitty, 1999]. The final specification for the QMC sequences is the prime base. It is recommended that the user not specify this and use the default values. For the Halton sequence, the default bases are primes in increasing order, starting with 2, 3, 5, etc. For the Hammersley sequence, the user specifies (s-1) primes if one is generating an s-dimensional set of random variables.
The fixed_sequence control is similar to fixed_seed for other sampling methods. If fixed_sequence is specified, the user will get the same sequence (meaning the same set of samples) for subsequent calls of the QMC sampling method (for example, this might be used in a surrogate based optimization method or a parameter study where one wants to fix the uncertain variables). The latinize command takes the QMC sequence and "latinizes" it, meaning that each original sample is moved so that it falls into one strata or bin in each dimension as in Latin Hypercube sampling. The default setting is NOT to latinize a QMC sample. However, one may be interested in doing this in situations where one wants better discrepancy of the 1-dimensional projections (the marginal distributions). The variance_based_decomp control is also available. This control enables the calculation of sensitivity indices which indicate how important the uncertainty in each input variable is in contributing to the output variance. More details on variance based decomposition are given in Nondeterministic sampling method.
Finally, quality_metrics calculates four quality metrics relating to the volumetric spacing of the samples. The four quality metrics measure different aspects relating to the uniformity of point samples in hypercubes. Desirable properties of such point samples are: are the points equally spaced, do the points cover the region, and are they isotropically distributed, with no directional bias in the spacing. The four quality metrics we report are h, chi, tau, and d. h is the point distribution norm, which is a measure of uniformity of the point distribution. Chi is a regularity measure, and provides a measure of local uniformity of a set of points. Tau is the second moment trace measure, and d is the second moment determinant measure. All of these values are scaled so that smaller is better (the smaller the metric, the better the uniformity of the point distribution). Complete explanation of these measures can be found in [Gunzburger and Burkhardt, 2004.].
| Description | Keyword | Associated Data | Status | Default |
| FSU Quasi-Monte Carlo | fsu_quasi_mc | none | Required group | N/A |
| Sequence type | halton | hammersley | none | Required group | N/A |
| Number of samples | samples | integer | Optional | (0) for standalone sampling, (minimum required) for surrogates |
| Sequence starting indices | sequence_start | integer list (one integer per variable) | Optional | Vector of zeroes |
| Sequence leaping indices | sequence_leap | integer list (one integer per variable) | Optional | Vector of ones |
| Prime bases for sequences | prime_base | integer list (one integer per variable) | Optional | Vector of the first s primes for s-dimensions in Halton, First (s-1) primes for Hammersley |
| Fixed sequence flag | fixed_sequence | none | Optional | sequence not fixed: sampling patterns are variable |
| Latinization of samples | latinize | none | Optional | No latinization |
| Variance based decomposition | variance_based_decomp | none | Optional | No variance_based_decomp |
| Quality metrics | quality_metrics | none | Optional | No quality_metrics |
The FSU CVT method (fsu_cvt) produces a set of sample points that are (approximately) a Centroidal Voronoi Tessellation. The primary feature of such a set of points is that they have good volumetric spacing; the points tend to arrange themselves in a pattern of cells that are roughly the same shape. To produce this set of points, an almost arbitrary set of initial points is chosen, and then an internal set of iterations is carried out. These iterations repeatedly replace the current set of sample points by an estimate of the centroids of the corresponding Voronoi subregions. [Du, Faber, and Gunzburger, 1999].
The user may generally ignore the details of this internal iteration. If control is desired, however, there are a few variables with which the user can influence the iteration. The user may specify max_iterations, the number of iterations carried out; num_trials, the number of secondary sample points generated to adjust the location of the primary sample points; and trial_type, which controls how these secondary sample points are generated. In general, the variable with the most influence on the quality of the final sample set is num_trials, which determines how well the Voronoi subregions are sampled. Generally, num_trials should be "large", certainly much bigger than the number of sample points being requested; a reasonable value might be 10,000, but values of 100,000 or 1 million are not unusual.
CVT has a seed specification similar to that in DDACE: there are optional specifications for the random seed to use in generating the sample set (seed), for fixing the seed (fixed_seed) among multiple sample sets (see Nondeterministic sampling method for discussion), and for the number of samples to perform (samples). The seed control is used to make sample sets repeatable. Finally, the user has the option to specify the method by which the trials are created to adjust the centroids. The trial_type can be one of three types: random, where points are generated randomly; halton, where points are generated according to the Halton sequence; and grid, where points are placed on a regular grid over the hyperspace.
Finally, latinization is available for CVT as with QMC. The latinize control takes the CVT sequence and "latinizes" it, meaning that each original sample is moved so that it falls into one strata or bin in each dimension as in Latin Hypercube sampling. The default setting is NOT to latinize a CVT sample. However, one may be interested in doing this in situations where one wants better discrepancy of the 1-dimensional projections (the marginal distributions). The variance_based_decomp control is also available. This control enables the calculation of sensitivity indices which indicate how important the uncertainty in each input variable is in contributing to the output variance. More details on variance based decomposition are given in Nondeterministic sampling method. The quality_metrics control is available for CVT as with QMC. This command turns on calculation of volumetric quality measures which measure the "goodness" of the uniformity of the point samples. More details on the quality measures are given under the description of the QMC methods.
The specification detail for the FSU CVT method is given in Table 5.36.
| Description | Keyword | Associated Data | Status | Default |
| FSU CVT sampling | fsu_cvt | none | Required group | N/A |
| Random seed | seed | integer | Optional | randomly generated seed |
| Fixed seed flag | fixed_seed | none | Optional | seed not fixed: sampling patterns are variable |
| Number of samples | samples | integer | Required | (0) for standalone sampling, (minimum required) for surrogates |
| Number of trials | num_trials | integer | Optional | 10000 |
| Trial type | trial_type | random | grid | halton | Optional | random |
| Latinization of samples | latinize | none | Optional | No latinization |
| Variance based decomposition | variance_based_decomp | none | Optional | No variance_based_decomp |
| Quality metrics | quality_metrics | none | Optional | No quality_metrics |
With the exception of output verbosity (a setting of silent will suppress some parameter study diagnostic output), DAKOTA's parameter study methods do not make use of the method independent controls. Therefore, the parameter study documentation which follows is limited to the method dependent controls for the vector, list, centered, and multidimensional parameter study methods.
vector_parameter_study specification followed by either a final_point or a step_vector specification.
The vector for the study can be defined in several ways (refer to dakota.input.spec). First, a final_point specification, when combined with the initial values from the variables specification (see cdv_initial_point, ddv_initial_point, csv_initial_state, and dsv_initial_state in Variables Commands), uniquely defines an n-dimensional vector's direction and magnitude through its start and end points. The intervals along this vector may either be specified with a step_length or a num_steps specification. In the former case, steps of equal length (Cartesian distance) are taken from the initial values up to (but not past) the final_point. The study will terminate at the last full step which does not go beyond the final_point. In the latter num_steps case, the distance between the initial values and the final_point is broken into num_steps intervals of equal length. This study performs function evaluations at both ends, making the total number of evaluations equal to num_steps+1. The final_point specification detail is given in Table 5.37.
| Description | Keyword | Associated Data | Status | Default |
| Vector parameter study | vector_parameter_study | none | Required group | N/A |
| Termination point of vector | final_point | list of reals | Required group | N/A |
| Step length along vector | step_length | real | Required | N/A |
| Number of steps along vector | num_steps | integer | Required | N/A |
The other technique for defining a vector in the study is the step_vector specification. This parameter study begins at the initial values and adds the increments specified in step_vector to obtain new simulation points. This process is performed num_steps times, and since the initial values are included, the total number of simulations is again equal to num_steps+1. The step_vector specification detail is given in Table 5.38.
| Description | Keyword | Associated Data | Status | Default |
| Vector parameter study | vector_parameter_study | none | Required group | N/A |
| Step vector | step_vector | list of reals | Required group | N/A |
| Number of steps along vector | num_steps | integer | Required | N/A |
list_parameter_study method specification followed by a list_of_points specification.
The number of real values in the list_of_points specification must be a multiple of the total number of continuous variables contained in the variables specification. This parameter study simply performs simulations for the first parameter set (the first n entries in the list), followed by the next parameter set (the next n entries), and so on, until the list of points has been exhausted. Since the initial values from the variables specification will not be used, they need not be specified. The list parameter study specification detail is given in Table 5.39.
| Description | Keyword | Associated Data | Status | Default |
| List parameter study | list_parameter_study | none | Required group | N/A |
| List of points to evaluate | list_of_points | list of reals | Required | N/A |
centered_parameter_study method specification followed by percent_delta and deltas_per_variable specifications, where percent_delta specifies the size of the increments in percent and deltas_per_variable specifies the number of increments per variable in each of the plus and minus directions. The centered parameter study specification detail is given in Table 5.40.| Description | Keyword | Associated Data | Status | Default |
| Centered parameter study | centered_parameter_study | none | Required group | N/A |
| Interval size in percent | percent_delta | real | Required | N/A |
| Number of +/- deltas per variable | deltas_per_variable | integer | Required | N/A |
multidim_parameter_study method specification followed by a partitions specification, where the partitions list specifies the number of partitions for each continuous variable. Therefore, the number of entries in the partitions list must be equal to the total number of continuous variables contained in the variables specification. Since the initial values from the variables specification will not be used, they need not be specified. The multidimensional parameter study specification detail is given in Table 5.41.| Description | Keyword | Associated Data | Status | Default |
| Multidimensional parameter study | multidim_parameter_study | none | Required group | N/A |
| Partitions per variable | partitions | list of integers | Required | N/A |
1.3.8