From Complexity to Creativity-- Copyright Plenum Press, © 1997 / Back to Ben Goertzel's Books
From Complexity to Creativity
Computational Models of Evolutionary, Autopoietic and Cognitive Dynamics
Ben Goertzel
Chairman and CTO Intelligenesis Corp.
Paper Version to be published by Plenum Press, 1996
Contents
- INTRODUCTION
Part I. The Complex Mind/Brain
- CHAPTER 1. DYNAMICS, EVOLUTION, AUTOPOIESIS
- CHAPTER 2. THE PSYNET MODEL
- CHAPTER 3. A MODEL OF CORTICAL DYNAMICS
- CHAPTER 4. PERCEPTION AND MINDSPACE CURVATURE
Part II. Formal Tools for Exploring Complexity
- CHAPTER 5. DYNAMICS AND PATTERN
- CHAPTER 6. EVOLUTION AND DYNAMICS
- CHAPTER 7. MAGICIAN SYSTEMS
Part III. Mathematical Structures in the Mind
- CHAPTER 8. THE STRUCTURE OF CONSCIOUSNESS
- CHAPTER 9. FRACTALS AND SENTENCE PRODUCTION
- CHAPTER 10. DREAM DYNAMICS
- CHAPTER 11. ARTIFICIAL SELFHOOD
Part IV. The Dynamics of Self and Creativity
- CHAPTER 12. SUBSELF DYNAMICS
- CHAPTER 13. ASPECTS OF HUMAN PERSONALITY DYNAMICS
- CHAPTER 14. ON THE DYNAMICS OF CREATIVITY
- References
Ben Goertzel ()
Part I. The Complex Mind/Brain
CHAPTER 1. DYNAMICS, EVOLUTION, AUTOPOIESIS
- 1. Introduction
- 2. Attractors
- 3. Genetic Algorithms
- 4. Magician Systems
- 5. Dynamics and Pattern
PART I
THE COMPLEX MIND/BRAIN
CHAPTER ONE
DYNAMICS, EVOLUTION, AUTOPOIESIS
1.1 INTRODUCTION
In this brief chapter, I will introduce a few preliminary ideas required for understanding the rest of the book: dynamics, attractors, chaos, genetic algorithms, magician systems, and algorithmic patterns in dynamics.
These concepts are diverse, but every one of them is a variation on the theme of the first one: dynamics. For "dynamics," most broadly conceived, is just change, or the study of how things change. Thus, for example, evolution by natural selection is a kind of dynamic. Comparing natural selection with physical dynamics, as portrayed e.g. by the Schrodinger equation, gives one a rough idea of just how broad the concept of dynamics really is.
Physics has focused on dynamics at least since the time of Newton. Psychology and the other human sciences, on the other hand, have tended to focus on statics, ignoring questions of change and process -- a fact which is probably due to the incredible dynamical complexity of human systems. Only now are we beginning to find mathematical and computational tools which are up to dealing with the truly complex dynamical systems that we confront in our everyday mental and social lives.
There is a vast and intricate body of mathematics called "dynamical systems theory." However, the bulk of the concrete results in this theory have to do with dynamical systems that are governed by very special kinds of equations. In studying very complex dynamical systems, as we will be doing here, one is generally involved with conceptual models and computer simulations rather than analytical mathematics. Whether this state of affairs will change in the future, as new kinds of mathematics evolve, is extremely unclear. But at present, some of the most interesting dynamical systems models are so complex as to almost completely elude mathematical analysis.
An example is the genetic algorithm, a discrete and stochastic mathematical model of evolution by natural selection. Another example is the autopoietic system, a model of the self-producing nature of complex systems. Both these dynamical systems models will be discussed in detail in later chapters, and briefly introduced in this chapter. It is these models, I believe, which are most relevant to modeling extremely complex systems like the mind/brain. In order to model psychological complexity, we must leave traditional dynamical systems theory behind, and fully confront the messiness, intricacy and emergent pattern of the real world.
In the last section of this chapter, the relation between dynamics and structure will be discussed, and dealt with in a formal way. The abstract theory of pattern, developed in my previous publications, will be introduced and used to give formal definitions of such concepts as "dynamical pattern" and "system complexity." The concept of pattern unifies all the various dynamics discussed in the previous sections. For these dynamics, like all others, are in the end just ways of getting to appropriate emergent patterns.
Finally, before launching into details, it may be worthwile to briefly reflect on these topics from a more philosophical perspective. Change and structure -- becoming and being -- are elementary philosophical concepts. What we are studying here are the manifestations of these concepts within the conceptual framework of conceptual science. Any productive belief system, any framework for understanding the world, must come to terms with being and becoming in its own way. Science has, until recently, had a very hard time dealing with the "becoming" aspect of very complex systems, e.g. living and thinking systems. But this too is changing!
1.2 ATTRACTORS
Mathematical dynamical systems theory tends to deal with very special cases. For instance, although most real-world dynamical systems are most naturally viewed in terms of stochastic dynamics, most of the hard mathematical results of dynamical systems theory have to do with deterministic dynamics. And even within the realm of deterministic dynamics, most of the important theorems involve quantities that are only possible to compute for systems with a handful of variables, instead of the hundreds, thousands or trillions of variables that characterize many real systems. A course in dynamical systems theory tends to concentrate on what I call "toy iterations" -- very simple deterministic dynamical systems, often in one or two or three variables, which do not accurately model any real situation of interest, but which are easily amenable to mathematical analysis.
The great grand-daddy of the toy iterations is the logistic map, xn+1 = rxn(1-xn); a large portion of Devaney's excellent book Chaotic Dynamical Systems is devoted to this seemingly simple iteration. In the realm of differential equations, the Lorenz equation is a classic "toy model" (its discretization, by which its trajectories are studied on the computer, is thus a toy iteration). Implicit in the research programme of dynamical systems theory is the assumption that the methods used to study these toy iterations will someday be generalizable to more interesting dynamical systems. But this remains a promissory note; and, for the moment, if one wishes to model real-world systems in terms of dynamical systems theory, one must eschew mathematical theorizing and content oneself with qualitative heuristics and numerical simulations.
But even if the deeper results of dynamical systems theory are never generalized to deal with truly complex systems, there is no doubt that the conceptual vocabulary of dynamical systems theory is useful in all areas of study. In the following pages we will repeatedly use the language of dynamical systems and attractors to talk about psychological systems, but it is worth remembering that these concepts did not (and probably could not) have come out of psychology. They were arrived at through years of painstaking experimentation with simple, physics-inspired, few-variable dynamical systems.
So, let us define some terms. A dynamical system, first of all, is just a mapping from some abstract space into itself. The abstract space is the set of "states" of the system (the set of states of the real-world system modelled by the mapping; or the abstract dynamical system implicit in the mapping). The mapping may be repeated over and over again, in discrete time; this is an iterative dynamical system. Or it may be repeated in continuous time, in the manner of a differential equation; this is a "continuous" dynamical system. In general, continuous dynamical systems are more amenable to mathematical analysis, but discrete dynamical systems are more amenable to computer simulation. For this reason, one often has cause to transform one kind of dynamical system into the other.
A trajectory of a dynamical system is the series of system states that follows from a certain initial (time zero) state. For a deterministic dynamical system, a trajectory will be a simple series, for a stochastic dynamical system, it will be a constantly forward-branching collection of system states. When doing computer simulations of dynamical systems, one computes particular sets of trajectories and takes them as representative.
The key notion for studying dynamical systems is the attractor. An attractor is, quite simply, a characteristic behavior of a system. The striking insight of dynamical systems theory is that, for many mathematical and real-world dynamical systems, the initial state of the system is almost irrelevant. No matter where the system starts from, it will eventually drift into one of a small set of characteristic behaviors, a small number of attractors. The concept of "attractor" is, beyond all doubt, the most important contribution of dynamical systems theory to the general vocabulary of science.
Some systems have fixed point attractors, meaning that they drift into certain "equilibrium" conditions and stay there. Some systems have periodic attractors, meaning that, after an initial transient period, they lock into a cyclic pattern of oscillation between a certain number of fixed states. And finally, some systems have attractors that are neither fixed points nor limit cycles, and are hence called strange attractors. An example of a two-dimensional strange attractor, derived from equation (3) below, may be found in Figure 1. The most complex systems possess all three kinds of attractors, so that different initial conditions lead not only to different behaviors, but to different types of behavior.
The formal definition of "strange attractor" is a matter of some contention. Rather than giving a mathematical definition, I prefer to give a "dictionary definition" that captures the common usage of the word. A strange attractor of a dynamic, as I use the term, is a collection of states which is: 1) invariant under the dynamic, in the sense that if one's initial state is in the attractor, so will be all subsequent states; 2) "attracting" in the sense that states which are near to the attractor but not in it will tend to get nearer to the attractor as time progresses; 3) not a fixed point or limit cycle.
The term "strange attractor" is itself a little strange, and perhaps deserves brief comment. It does not reflect any mathematical strangeness, for after all, fixed points and limit cycles are the exception rather than the rule. Whatever "strangeness" these attractors possess is thus purely psychological. But in fact, the psychological strangeness which "strange attractors" originally presented to their discoverers is a thing of the past. Now that "strange attractors" are well known they seem no stranger than anything else in applied mathematics! Nevertheless, the name sounds appealing, and it has stuck.
Strictly speaking, this classification of attractors applies only to deterministic dynamical systems; to generalize them to stochastic systems, however, one must merely "sprinkle liberally with probabilities." For instance, a fixed point of a stochastic iteration xn+1 = f(xn) might be defined as a point which is fixed with probability one; or a p-fixed point might be defined as one for which P(f(x) = x) > p. In the following, however, we will deal with stochastic systems a different way. In the last section of this chapter, following standard practice, we will think about the IFS random iteration algorithm by transplanting it to an abstract spaces on which it is deterministic. And in Chapter Six I will study the genetic algorithm by approximating it with a certain deterministic dynamical system. While perhaps philosophically unsatisfactory, in practice this approach to stochasticity allows one to obtain results that would become impossibly complicated if translated into the language of true stochastic dynamical systems theory.
Chaos
A great deal of attention has been paid to the fact that some dynamical systems are chaotic, meaning that, despite being at bottom deterministic, they are capable of passing many statistical tests for randomness. They look random. Under some definitions of "strange attractor," dynamics on a strange attractor are necessarily chaotic; under my very general definition, however, this need not be the case. The many specialized definitions of "chaos" are even more various than the different definitions of "strange attractor."
Let us look at one definition in detail. In the context of discrete dynamical systems, the only kind that will be considered here, Devaney defines a dynamical system to be chaotic on a certain set if it displays three properties on that set: sensitive dependence on initial conditions, topological transitivity, and density of repelling periodic points.
I find it hard to accept "density of repelling periodic points" as a necessary aspect of chaos; but Devaney's other two criteria are clearly important. Topological transitivity is a rough topological analogue of the more familiar measure-theoretic concept of ergodicity; essentially what it means is that the dynamics thoroughly mix everything up, that they map each tiny region of the attractor A into the whole attractor. In technical terms an iteration f is topologically transitive on a set A if, given any two neighborhoods U and V in A, the iterates fn(U) will eventually come to have a nonempty intersection with V.
Sensitivity to initial conditions, on the other hand, means that if one takes two nearby points within the attractor and uses them as initial points for the dynamic, the trajectories obtained will rapidly become very different. Eventually the trajectories may become qualitatively quite similar, in that they may have the same basic shape -- in a sense, this is almost guaranteed by topological transitivity. But they will be no more or less qualitatively similar than two trajectories which did not begin from nearby initial points.
Although Devaney did not realize this when he wrote his book, it has since been shown that topological transitivity and density of periodic points, taken together, imply sensitivity to initial conditions. Furthermore, for maps on intervals of the real line, topological transitivity and continuity, taken together, imply density of periodic points. So these criteria are intricately interconnected.
The standard tool for quantifying the degree of chaos of a mapping is the Liapunov exponent. Liapunov exponents measure the severity of sensitive dependence on initial conditions; they tell you how fast nearby trajectories move apart. Consider the case of a discrete-time system whose states are real numbers; then the quantity of interest is the ratio
Rn = |fn(x) - fn(y)|/|x-y| (1)
where fn denotes the n-fold iterate of f. If one lets y approach x then the ratio Rn approaches the derivative of fn at x. The question is: what happens to this derivative as the elapsed time n becomes large? The Liapunov exponent at x is defined as the limiting value for large n of the expression
log[ fn'(x) ]/n (2)
If the difference in destinations increases slowly with respect to n, then the trajectories are all zooming together, the ratio is less than one, and so the Liapunov exponent is negative. If the trajectories neither diverge nor contract, then the ratio is near one, and the Liapunov exponent is the logarithm of one -- zero. Finally, and this is the interesting case, if the difference in destinations is consistently large with respect to n, then something funny is going on. Close-by starting points are giving rise to wildly different trajectories. The Liapunov exponent of the system tells you just how different these trajectories are. The bigger the exponent, the more different.
A system in one dimension has one Liapunov exponent. A system in two dimensions, on the other hand, has two exponents: one for the x direction, computed using partial derivatives with respect to x; and one for the y direction, computed using partial derivatives with respect to y. Similarly, a system in three dimensions has three Liapunov exponents, and so forth.
A positive Liapunov exponent does not guarantee chaos, but it is an excellent practical indicator; and we shall use it for this purpose a little later, in the context of the plane quadratic iteration as given by:
xn+1 = a1 + xn*(a2 + a3*xn + a4*yn) + yn*(a5 + a6*yn)
yn+1 = a7 + xn*(a8 + a9*xn + a10*yn) + yn*(a11 + a12*yn)
(3)
This is an example of a very simple dynamical system which mathematics is at present almost totally unable to understand. Consider, for instance, a very simple question such as: what percentage of "parameter vectors" (a1,...,a12) will give rise to strange attractors? Or in other words, how common is chaos for this iteration? The answer, at present, is: go to the computer and find out!
1.3 THE GENETIC ALGORITHM
Now let us take a great leap up in complexity, from simple polynomial iterations to the dynamic of evolution.
As originally formulated by Darwin and Wallace, the theory of evolution by natural selection applied only to species. As soon as the theory was published, however, theorists perceived that natural selection was in fact a very general dynamic. Perhaps the first to view evolution in this way was Herbert Spencer. Since the time of Darwin, Wallace and Spencer, natural selection has been seen to play a crucial role in all sorts of different processes.
For instance, Burnet's theory of clonal selection, the foundation of modern immunology, states that immune systems continually self-regulate by a process of natural selection. More speculatively, Nobel Prize-winning immunologist Gerald Edelman has proposed a similar explanation of brain dynamics, his theory of "neuronal group selection" or Neural Darwinism. In this view, the modification of connections between neuronal groups is a form of evolution.
The very origin of life is thought to have been a process of molecular evolution. Kenneth Boulding, among many others, has used evolution to explain economic dynamics. Extending the evolution principle to the realm of culture, Richard Dawkins has defined a "meme" as an idea which replicates itself effectively and thus survives over time. Using this language, we may say that natural selection itself has been a very powerful meme. Most recently, the natural selection meme has invaded computing, yielding the idea of evolutionary computation, most commonly referred to by the phrase "genetic algorithms."
These diverse applications inspire a view of evolution as a special kind of dynamic. The evolutionary dynamic is particularly useful for modeling extremely complex systems -- biological, sociological and cultural and psychological systems. Evolutionary dynamics has its own properties, different from other dynamics. All systems which embody adaptive evolution will display some of the characteristic properties of evolutionary dynamics -- along with the characteristics of other dynamics, such as the structure-preserving dynamic of "autopoiesis" or "ecology."