Multi-Resolution Methods for Modeling and Control of Dynamical Systems

by

John L. Junkins and Puneet Singla

Department of Aerospace Engineering

Texas A&M University

College Station, TX-77843-3141

Overview

In the proposed book titled“Multi-Resolution Methods for Modeling and Control of Dynamical Systems”, novel modeling and control methodologies will be presented to address various problems associated with the design of large scale dynamical systems. This book would cover issues from classical finite element methods to adaptive control to neural network approximation. These diverse topics will be presented in an integrated fashion, for the first time using a framework derived from dynamical systems, estimation, optimization, and approximation theory. It will also introduce some powerful new methods and demonstrate their broad utility. The main contribution of this book will be the development of adaptable, robust and computationally efficient, multiresolution approximation algorithms to solve challenging modeling and control problems, motivated by advanced aerospace systems.

In recent years, we have developed adaptable, robust and computationally efficient, multiresolution approximation algorithms based on Radial Basis Function (RBF) networks and Global-Local Orthogonal MAPping (GLO-MAP) approaches. The main feature of our RBF network approach is the unique direction dependent scaling and rotation of RBF via a novel Directed Connectivity Graph approach. Our approach is also built upon estimation and approximation theory, which inherently depends on metrics that indicate the “health” of a given input/output approximation. Our contributions have led to a broadly useful approximation approach that establishes global approximations capable of good local approximation for many moderate dimensioned applications. However, some applications with many high frequency local input/output variations and a high dimensional input space remain a challenge and motivate us to investigate entirely new approach. The GLO-MAP method is introduced – this novel development is based upon an averaging process to blend independent local approximations into a piecewise continuous global family of local least-squares approximations while retaining the freedom to vary in a general way the resolution (e.g., degrees of freedom) of the local approximations. Both the adaptive RBF and the GLO-MAP approaches are compatible with a wide variety of disciplines such as continuous function approximation, dynamic system modeling and system identification, nonlinear signal processing and time series prediction.

Another contribution of this book will be the development of related GLO-MAP based methods for the modeling of dynamical systems nominally described by nonlinear differential equations and to solve for static and dynamic response of Distributed Parameter Systems (DPS) in an efficient manner. The main focus the dynamical systems presentation will be on illuminating the process of establishing nonlinear dynamical models from input/output experimental data. We will discuss various non-traditional modeling approaches for direct approximation of experimental data. The new nonlinear system approximation algorithms not only have the approximation ability of ANN but also have model reduction ability of algorithms analogous to existing linear system methods (POD/PCA). The main advantage of the GLO-MAP based mesh-less FEM algorithms is that for dynamic calculations, the GLO-MAP approximations can in principle be added or subtracted individually without disturbing the underlying basis functions. The main advantage of the GLO-MAP approximation method is that any kind of prior knowledge (qualitative or geometrical) about the system can be incorporated in the approximation. This is illustrated clearly with several significant applications, with comparisons to existing methods.

The generalized GLO-MAP approach can, in principle, handle high-dimensioned systems. GLO-MAP is the first piecewise continuous approximation to allow the full utilization of locally supported orthogonal function approximation. In the proposed book, the GLO-MAP will be discussed for multi-scale modeling in higher dimensions and to solve several classical problems, including the Fokker Plank Equation to generate response PDF, and the Hamilton-Jacobi-Bellman equation to compute optimal cost-to-go optimal trajectories for a given dynamical system.

In addition, a hierarchical control allocation algorithm will be presented which makes use of the concept of multi-resolution distribution functions to keep in check the “curse of dimensionality” while solving the control allocation problem for highly over-actuated systems that might arise with the development of embedded systems. The main advantage of the proposed hierarchical approach is the systematic means for de-coupling of many small-scale problems from each other, yet blending them appropriately to obtain the global representation. As a consequence, the algorithm can be highly parallelized to reduce the computation burden involved. Comparisons of several approaches to problems drawn from typical applications make the relative merits clear.

Whereas the theoretical framework of this book will lie in fundamental research in approximation theory, the motivation and applications of the methodology will be demonstrated for a diverse set of problems, drawn from autonomous and intelligent systems, flow control, spacecraft maneuvers, active materials/structures. The formulation and case studies in this book will focus on demonstrating, through analysis, simulation, and design, the applicability and feasibility of a substantial set of recent developed approximation ideas to a rich set of examples. The reliability and limitations of the approximation methods discussed will be assessed by considering various academic and engineering problems.

The book will contain total 10 chapters with approximately 400-500 pages, 100 figures and 20-30 tables. The tentative Table of Content for the book is as follows:

Table of Contents

Chapter / Pages
  1. Multi-Resolution Approximation1-25
  2. Analytical Model vs Mathematical Models
  3. Multi-Resolution Analysis
  4. Choice of Basis Functions
  5. Curse of Dimensionality
  6. Model Complexity
  7. Exact Interpolation
  8. Local vs Global Approximation
  9. Finite Element Methods, Wavelets, Artificial Neural Networks and Polynomial Approximation.
  10. Summary
  1. Artificial Neural Networks26-56
  2. Introduction
  3. Single and Multi Layer Neural Network
  4. Sigmoid Units
  5. Radial Basis Functions
  6. Network Training
  7. Regularization Theory
  8. Comparison with Multi-Layer Neural Network
  9. Adaptive Radial Basis Function Networks
  10. Review of Existing Radial Basis Function Learning Algorithms
  11. Summary
  1. Direction Dependent Learning Approach For Radial Basis Function57-112 Networks
  2. Introduction
  3. Rotation of Generalized Gaussian Radial Basis Function
  4. Directed Connectivity Graph: A Direction Dependent Approach
  5. Parameter Estimation Algorithm
  6. Cayley Transformation
  7. Additive Decomposition of the Covariance Matrix, R
  8. Cholesky Decomposition of the Covariance Matrix, R
  9. Modified Minimal Resource Allocating Algorithm
  10. Numerical Simulations and Results
  11. Test Example 1: Function Approximation
  12. Test Example 2: 3 Input- 1 Output Continuous Function Approximation
  13. Test Example 3: Dynamical System Identification
  14. Test Example 4: Chaotic Time Series Prediction Problem
  15. Test Example 5: Benchmark Against On-line Structural Adaptive Hybrid Learning (ONSAHL) Algorithm 55
  16. Summary
  1. Global Orthogonal Mapping113-163
  2. Least Square Approximation
  3. Polynomial Basis Functions
  4. Classical Continuous Orthogonal Polynomials
  5. Gram-Schmidt Procedure of Orthogonalization
  6. Three Term Recurrence Relation to Generate Orthogonal Polynomials
  7. Hypergeometric Function Approach to Generate Orthogonal Polynomials
  8. Discrete Variable Orthogonal Polynomials.
  9. Interpolation Properties of Orthogonal Polynomials
  10. Non-Polynomial Orthogonal Basis Functions.
  11. Summary
  1. Global – Local Approximation Algorithms164-200
  2. Global vs Local Approximation.
  3. Partition of Unity
  4. Finite Element Methods
  5. Wavelets
  6. Spline Approximation
  7. Bezier spline curves
  8. Smooth Irregular Curves
  9. Moving Least Squares
  10. Issue of Independent Local Approximations
  11. Curse of Dimensionality
  12. Summary
  1. Global Local Orthogonal Polynomial Mapping (GLO-MAP) In N-Dimensions 201-265
  2. Introduction
  3. Novel Averaging Approach to Merge Independent Local Approximations:
  4. Basic Ideas
  5. Boundary Value Problem For The Weighting Function
  6. Approximation in 1-, 2- and N- Dimensions Using Weighting Functions
  7. Orthogonal Approximation in 1-, 2- and N-Dimensional Spaces
  8. One Dimensional Case
  9. Two Dimensional Case
  10. N- Dimensional Case
  11. Algorithm Implementation
  12. Sequential Version of the GLO-MAP Algorithm
  13. Illustrative Engineering Applications
  14. Function Approximation
  15. Synthetic Jet Actuator Modeling
  16. Space Based Radar (SBR) Antenna Simulation
  17. Porkchop Plots Approximation for Mission to Near-Earth Objects (NEOs)
  18. Summary
  1. Error Analysis for the GLO-MAP Algorithm266-300
  2. GLO-MAP: A Multi-Resolution Approximation Algorithm
  3. Bias vs Variance
  4. GLO-MAP Approximation Error
  5. Discretization Error
  6. Covariance Analysis for the GLO-MAP Algorithm
  7. GLO-MAP: An Unbiased Approximation Algorithm
  8. Summary
  1. Robust Nonlinear System Identification Algorithms Using Orthogonal Polynomial Network 301-336
  2. Introduction
  3. Problem Statement and Background
  4. Novel System Identification Algorithms
  5. Linear System Identification
  6. State Variable Estimation
  7. Nonlinear System Identification Algorithm
  8. Learning Algorithm for SysID 1
  9. Adaptation Law Derivation using the GLO-MAP Network
  10. Learning Algorithm for SysID 2
  11. Numerical Simulation
  12. Dynamic System Identification of Large Space Antenna
  13. Summary
  1. Meshless Finite Element Methods337-383
  2. Meshless Finite Element Methods
  3. MLPG-Moving Least Square Based Approach
  4. Modification of The MLPG Algorithm using The GLO-MAP Algorithm
  5. Partition of Unity Finite Element Method
  6. Numerical Simulations:
  7. Poisson’s Equation
  8. Solution to Higher Dimension Fokker-Planck- Kolmogorov (FPK) Equations.
  9. Summary
  1. Control Distribution For Over Actuated Systems 384-422
  2. Introduction
  3. Problem Statement and Background
  4. Control Distribution Functions
  5. Radial Basis Functions
  6. Global/Local Orthogonal Basis Functions
  7. Hierarchical Control Distribution Algorithm
  8. Numerical Results
  9. Control Allocation For a Morphing Wing
  10. Summary

Typeset (LaTex) drafts exist for about 75% of the text, and informal notes have been developed to draft the entire manuscript. Based upon our experience with previous book-writing projects of this scale, we anticipate that we can complete the book in about a calendar year following execution of an agreement. At this point, we feel that this book will better serve the purpose of a tutorial monograph for engineers/scientists, researchers and graduate students/faculty, rather than a usual textbook. However, this book will offer sufficient material for one-semester graduate course on approximation theory and estimation methods. The fact that the material crosses many disciplinary boundaries should help in marketing the book. In this first edition, no end of chapter problems will be provided, however each chapter will contain illustrative examples and prototype applications that could be used to develop student exercises.. We will include plenty of results from applications that demonstrate “what it all means” in various settings. Please find 4 sample chapters (Chapters III, VI, VIII and IX) attached with this proposal.

Outstanding Features of the Project

This book would be the first to rigorously integrate a family of artificial neural network methods into an approximation, estimation, and control theory setting and provide insights into when such methods can either work well or fail. This book will discuss in detail the issues related to modeling of large-scale dynamical systems like dimensionality, quality of the measurement data, offline or online learning, approximation accuracy, the computation time associated with the model, complexity of the mathematical model and efficiency of the learning algorithm. This book will describe new methods for modeling dynamical systems, developed largely in the last two decades, with roots in statistics, signal processing and traditional dynamical systems theory. The focus will be on understanding the process of producing models from experimental data. As this book, addressing topics from neural approximation to finite element analysis, is being written from an approximation analysis perspective, therefore, this book will differ from more narrowly focused treatments and research articles in the field of approximation theory and related topics. The one prominent feature of this book is the nontrivial unified perspective, and generalizations that led to new modeling algorithms applicable to high-dimensioned nonlinear systems. The main objectives of this proposal are as follows:

  1. The first and the most important objective of this project is to present adaptable, robust and computationally efficient, multi-resolution approximation algorithms which takes care of local and as well as global complexity of the problem.
  2. The second objective is to set down a theoretical framework including all assumptions to help understand the advantages, the drawbacks and the areas of applications of existing and new algorithms for input/output approximation.
  3. The third objective is establish a rigorous means to merge different local and global approximations that guarantees a prescribed degree of piecewise continuity, while keeping the “curse of dimensionality” in check.
  4. The fourth objective is to present new adaptive learning algorithms to adjust in real time the various parameters of the unknown mathematical models while keeping the number of unknowns to be minimum.
  5. The fifth objective is to compare new approximation algorithms with some existing approximation algorithms while considering various benchmark problems in open literature.
  6. The last but not the least objective is to assess the reliability and limitations of the new and historical approximation methods by focusing attention on various academic and engineering problems where traditional methods either fail or perform very poorly.

Benefits for the Potential Reader

The main goal of this book is to provide engineers in industry and academicians with a thorough understanding of the underlying principles of input-output data approximation and various issues associated with the modeling of large scale dynamical systems. The book will be self-contained in the sense that it will require only rudimentary knowledge of matrix algebra, statistics and dynamical system theory. The proposed ideas will be illustrated by numerous figures, examples and real-world applications for easy understanding of the various concepts. The emphasis of the book will be on an intuitive understanding of the subject and practical applications of the discussed methods so that reader should be able to apply the discussed methods to real engineering problems with the awareness of potential difficulties that might arise in practice. This book will also serve as an introduction to various classical and modern modeling methods and will provide an overview of various modeling and control approaches used in engineering.

Marketing

We can help establish electronic mailing lists for domestic and international colleagues that would be interested in this text. We can identify several professional publications where the book should be sent for review. We can help optimize the advertising materials for electronic and snail mail advertising.

Production

We will provide a camera-ready manuscript: a fully formatted and indexed manuscript with all figures and tables included. We will typeset the document using a LaTex style and the overall quality of the camera-ready manuscript will be analogous to our recent text on Optimal Estimation of Dynamic Systems.

About the Authors

Name:John L. Junkins

Years of Experience:38

Position:Distinguished Professor of Aerospace Engineering, Texas A&M University

Education:B.A.E., Aerospace Engineering, Auburn University, 1965; M.S. (1967) and Ph.D. (1969), Aerospace Engineering, University of California, Los Angeles

Experience: He holds the George Eppright Endowed Chair Professorship in Aerospace Engineering at Texas A&M University, and is the Director of the Center for Mechanics and Control. Prior to joining Texas A&M in 1985, he held previous academic appointments at the University of Virginia (1970-1977) and at Virginia Tech (1978-1985). He has held positions at McDonnell Douglas Astronautics Company (1965-1970) and NASA Marshall Space Flight Center (1962-1965). Dr. Junkins’ technical interests include navigation, guidance, and control of spacecraft and aircraft. He is a member of the National Academy of Engineering and the International Academy of Astronautics; he is a Fellow of the American Institute of Aeronautics and Astronautics (AIAA) and the American Astronautical Society (AAS). He has received a dozen national and international honors, including the AIAA Mechanics and Control of Flight Award, the AIAA Pendray Aerospace Literature Award, the AIAA Theodore von Karman Lectureship, the International Astronautical Federation Frank Malina Medal and the Institute of Navigation Tyco Brahe Award. He has recently invented patented laser sensing technology for applications in navigation, machine vision and multimedia. He is a prolific academic mentor, having directed the graduate study of over 70 students, leading to 37 completed PhDs and four generations of professors, graduate students and engineers in industry. His ideas have been implemented successfully in many industrial and aerospace systems. He has authored or edited 9 books and over 350 technical publications.

Name:Dr. Puneet Singla

Position:Post-Doctorate Research Associate, Aerospace Engineering, Texas A&M University

Education:B.Tech., Aerospace Engineering, Indian Institute of Technology, Kanpur, India, 2000; M.S. (2002) and Ph.D. (2006), Aerospace Engineering, Texas A&M University, College Station, TX.

Experience: Dr. Singla’s six years of experience since completing his BS have been mainly focused on basic and applied research time-shared with pursuit of his PhD at Texas A&M University. He has enjoyed the opportunity to work on a variety of areas that include autonomous and intelligent systems, adaptive control, approximation theory, distributed and redundant actuation and nonlinear dynamical systems. In his short professional career, Dr. Singla has demonstrated engineering ability ranging from basic theory to the design of the algorithms, and notably, to applications/design/experiments that have flown successfully on space missions. His work in attitude estimation included algorithms supporting a successful experiment StarNav that flew on the STS-107. His 17 conference papers to date and 10 journal papers in various stages of editing, and his recent dissertation spans a wide array of problems, including: attitude estimation, dynamics and control, celestial mechanics, adaptive control, distributed parameter systems modeling and control, approximation theory, including a novel method for solving n-dimensional PDEs such as Foker-Plank equations. He has made a number of contributions in these diverse areas and has been especially successful in crossing boundaries of some heretofore disjoint areas to develop unified approaches for broad classes of problems. In some cases, very significant new results emerged from this unified approach to approximation in engineering and applied science. These contributions, along with the collaborations with Dr. Junkins and his historical work, established the foundation upon which the proposed book rests. Dr. Singla is a member of the American Institute of Aeronautics and Astronautics (AIAA), the American Astronautical Society (AAS) and the Institute of Electrical and Electronics Engineers (IEEE).