TODO List of Tasks for further development o Cleanup the GApoint class. o Consoldate GPSOpt, CoordsPSOpt and CoordSPSOpt into a PatternSearch class. o Add reflection calculations into PatternSearch to eliminate redundant fevals. o Evaluate whether multiple shrinks often occur in succession, and if so implement method for enabling asymptotic 1-feval per shrink. o Implement constraint-handling techniques for Pattern Search. o Parallelize the sMC heirarchy to enable parallel random sampling as well as parallel MS Local Search. o Move opt_name initialization into BaseOptimizer. o Reimplement template stuff in perl/python scripts. o Make doinstemp use a temp file indexed by the users name o Implement sophisticated step management in pattern search methods. o Implement PBIL-like GAs o Reconsider global optimizers that do an initial function evaluation o Revise implementation in MSAppInterface to make the terminate_id method work as expected: when requesting termination, you can only terminate tasks that have not been launched yet. May be able to add more complete termination later. o Add 'can_terminate_asynch_evals' flag to BaseOptProblem, which is used by algorithms to select the type of synchronization used. o SparseMatrix Class o Add constraint handling to pattern search code. o Verify asynch_eval vs nonasynch before running (maybe in the set_func call?) o Verify codes GA CoordPS IntOpt o Setup ASV-style evaluation with gradients, etc o Figure out test problem format for simple gradient usage, and update the testfunction suite o Add JGO test problem to suite, and figure out randomization o Add option for 'synchronous' master-slave. Always wait for a feval before sending off another. Good when load balancing isn't assured. o Figure out semantics for MixedApp. How manage synch vs. asynch? NOTE: MixedApp is not supported right now o Cleanup management of responsesFNameArray in AnalysisCode.C, so it isn't resized for every function evaluation. Greetings All: I wanted to send you a note to summarize my reactions to my recent visit to Sandia CA, and to propose a framework for optimization problems. While at Sandia CA, I spoke with Juan, Todd and Patty about a general framework for optimization problems, using my initial prototype to highlight the basic issues that we face. As I see it, we have several broad goals: a. Provide a syntactic definition of optimization problems on a variety of search domains, in an extensible fashion. b. Provide syntactic differentiation between continuous problems where different levels of derivatives are available. I think this differentiation should also be included in mixed-integer problems as well, but that is less critical right now. c. Provide a general capability for expressing constraints (bound, linear, nonlinear). d. Provide a general capability for flexibly denoting how function/constraints are computed, including: - via a system call - through direct calls to subroutines - through an MPI call - spawning a thread (that uses a subroutine call) Similarly, we want to support various types of parallelization: - master-slave (single-level and bi-level parallelism) - peer-master (everyone executes the optimizer code, but some expensive routines (like distributed parallel matrix multiplies) are parallelized) - parallel optimization algorithms (e.g. parallel GAs) To satisfy these needs, I propose the following class structure, which I worked on with Todd and Patty (this is a simple modification of the prototype that I proposed earlier). The basic idea is to have a class hierarchy for optimization problems that includes syntactic information for domains and gradients. For example, we could have the following class hierarchy (The level of indent indicates subclass relations): BaseOptProblem BinaryOptProblem (or BooleanOptProblem) IntOptProblem RealOptProblem0 RealOptProblem1 RealOptProblem2 MixedOptProblem0 MixedOptProblem1 MixedOptProblem2 This class hierarchy is basically a hybrid of the SGOPT and OPT++ class hierarchies (I think the RealOptProblem2 class is roughly equivalent to the DakotaModel class). This class hierarchy distinguishes between different types of search domains. Within the RealOptProblem and MixedOptProblem classes, there is further distinction based on the level of true derivatives provided by the Problem. This design assumes that additional levels of derivatives could be approximated through explicit method calls (e.g. the method EvalG-FD would evaluate the gradient with finite-differences). To control the manner in which function and constrain evaluations are computed, we use a seperate class hierarchy. We use a handle-body idiom to allow the OptProblem class to interrogate this new class to execute the actual evaluations. The class hierarchy is: ApplicationInterface SyscallInterface DirectInterface (or SubroutineInterface) MPIInterface This hierarchy does not include syntactic differentiation between either search-domain or derivative information. The basic idea is that the user will be exposed only to the OptProblem class hierarchy, so we can utilize less object-oriented design at this level, thereby avoiding a combinatorial explosion of class objects (which even us 'expert' developers would find confusing). At the ApplicationInterface level, there would be methods that the OptProblem could query to determine whether a particular ApplicationInterface object satisfies the syntactic requirements of a particular OptProblem class (presumably these would be setup upon construction of the OptProblem). I think this design balances the need to minimize the number of classes and the need to provide an expressive set of primitives. Our initial thoughts were that different search domains could be supported by over-riding the Eval methods for the different domains. For example, to support RealVectors and IntVectors, we'd have two methods EvalF(RealVector& ...) EvalF(IntVector& ...) This is not unreasonable, but also not terribly extensible. Thus, I wonder if we don't want to include at least a syntactic distinction at the ApplicationInterface. Thus we'd have IntApplicationInterfaces and RealApplicationInterfaces. But this would necessitate using multiple-inheritance to avoid having a lot of duplicated code (e.g. the SystemCall routines are basically the same regardless of the search domain). There are several basic issues that need to be addressed to evaluate whether this new design will work well. CONSTRUCTION/INITIALIZATION One way to manage handle-body idioms, is to provide different constructors for the handle (OptProblem class) that then construct the body (ApplicationInterface class) directly. My only concern in this context is that this imposes a dependence from the handle class on the body class. If you want to add a new ApplicationInterface, you need to add a new constructor to the OptProblem class(es). This necessitates recompilation of your entire library, which is bad news. To avoid this, I propose creating 'contructor functions' that serve the same purpose. For example, we could use the following code: double fn(double* x, int n) { // Function code } int main() { RealOptProblem0 *prob = ConstructRealOptProblem(&fn); } The 'ConstructRealOptProblem' function could be overloaded to provide for the construction of a wide range of ApplicationInterface options. PARALLELISM The present design doesn't address all of the issues needed to utilize parallelism. Each of the application interface classes can be used synchronously and asynchronously, thereby affording some basic level of parallel design (e.g. the asynch DirectInterface would spawn threads of execution). However, the semantics of the MPIInterface class is unclear just now. I am imagining that this could be used to support master-slave parallelism in a transparent manner. However, there has to be some initial setup of the optimizers that at some level would _not_ be transparent to the user. For example, how does the setup for master-slave parallelism differ from perr-master? Do these different modes of parallelism require different compiler flags to be setup in the code? APPROXIMATE DERIVATIVES The management of approximate derivatives is a sticky issue that Dakota and OPT++ have resolved differently. OPT++ has a seperate NLP class that uses derivatives, which can be used with seperate copies of the optimizers. I say 'copies' here, because Todd indicated that there is a lot of code duplication. Dakota buries the derivative estimation inside the DakotaModel class. This offers nice modularity, but could lead to problems where the optimization software doesn't realize that approximate derivatives are being provided. I propose a third alternative in the context of this class design. Suppose that the optimizer classes for 'reals' have a 'set_problem()' method. Then, for example, the an optimization class like NewtonOpt could have the method set_problem(RealOptProblem0& prob). This would imply that the NewtonOpt is responsible for calling the finite-difference routine for the prob object (which is encapsulated at the ApplicationInterface level). Thus we can avoid code duplication while providing syntactic indication of the type of gradient information that the NewtonOpt class can utilize. A simple union structure can be used within the NewtonOpt class to determine what type of optimization problem has been provided.