Forward, Reverse, and Taylor Polynomial Mode Automatic Differentiation Tools for Sandia C++ Applications

Description

Automatic differentiation (AD) is a software tool for efficiently producing analytic derivatives, such as gradients, Jacobians, Hessians, and Taylor polynomials. Forward mode AD efficiently produces the action of the Jacobian, while reverse mode AD produces the adjoint (i.e., transpose). In certain classes of problems, reverse mode AD can deliver function and gradient evaluations in a small multiple of the time needed for evaluating the function alone. This is essential in design optimization and inversion problems with large numbers of design parameters and in adjoint methods for error estimation, e.g., for mesh adaptation. The Taylor polynomial mode produces coefficients for Taylor series expansions of arbitrarily high degree in one or more variables, allowing the development of highly accurate time integration, uncertainty quantification, and stability analysis algorithms. In fact, Taylor polynomial-based time integration algorithms have been shown to be up to 10 times faster than traditional methods on small problems. Similar techniques have never been attempted in a large-scale context, even though scalability is evident. In addition, Taylor polynomials can be used to propagate uncertainties in a manner similar to stochastic finite-elements, but no manual rewriting of application codes is required. All three AD modes share common implementations and once an application code has been modified for any one mode, all three modes become available. While various AD tools are currently under development elsewhere, none are sufficient for Sandia C++ applications. We seek AD tools that work for ANSI-C++ compliant code, provide good performance, are easy to use and easy to integrate with existing build systems. We propose the development of a suite of operator overloading and/or hybrid AD tools geared to Sandia C++ applications to address the following:

1. How can reverse mode AD be implemented to effectively and efficiently (from both computational and software engineering viewpoints) produce adjoint derivatives for large-scale Sandia C++ codes?
2. How efficient can reverse mode AD be while still providing a general tool applicable to all Sandia codes?
3. How can Taylor polynomial information be used to produce more accurate, efficient, and robust time integration, uncertainty quantification, and stability analysis algorithms applicable to Sandia codes?
4. To what extent can Taylor polynomial based time integration algorithms more effectively deal with multiple time scale problems?

The goals of this project are to demonstrate the viability of AD as a technology to increase the accuracy, efficiency, and/or robustness at which problems are currently solved at Sandia and to motivate new algorithms for solving them. Ultimately, we wish to provide the tools, techniques, and experience to make AD a common-place technology used regularly throughout the Sandia complex.

Staff: David M. Gay, Eric T. Phipps.

Consultants and Collaborators: Roscoe Bartlett, Andy Salinger, Bart van Bloemen Waanders.

Privacy and Security. Last modified: 20 Jan. 2005.