How Does the Brain Create, Change, and Selectively Override Its Rules of Conduct

How Does the Brain Create, Change, and Selectively Override Its Rules of Conduct?

Daniel S. Levine

Department of Psychology

University of Texas at Arlington

Arlington, TX 76019-0528

Abstract. How do we know to talk openly to our friends but be guarded with strangers? How do we switch between different cities, between family and others, between work place, club, and house of worship and still behave in a manner we believe to be appropriate in each setting? How do we develop context-dependent rules about what it is or it is not acceptable to eat or drink? We solve such problems with relatively little effort, much of the time, but are a long way from being able to design an intelligent system that can do what we do. Still, enough knowledge is emerging about the cognitive and behavioral functions of three major parts of the prefrontal cortex, and the various subcortical areas to which they connect, that we can start to construct a neural theory of context-dependent rule formation and learning.

Different rules of conduct, however, can conflict, and both personal growth and change in stress level can lead to alterations in the rules that are followed. Part of this can be explained by the interplay between signals from the hippocampus, signifying task relevance, and the amygdala, signifying emotional salience. Both of these sets of signals influence the behavioral “gate” in the basal ganglia that selectively disinhibits actions in response to the current context. A mathematical implementation of this type of switches between rules is outlined, based on a continuous analog of simulated annealing.

Introduction

A complex high-order cognitive system clearly needs to develop criteria for what actions to perform and what actions to refrain from performing. The more complex the system’s environment, the more flexible and context-sensitive those criteria need to be.

The term “rules” provokes a certain amount of discomfort in the neural network community. To some readers it has the connotations of strict logical IF-THEN rules from a production system as in symbolic artificial intelligence. In fact, Fodor and Pylyshyn (1988) and other symbolic cognitive scientists have argued at times that connectionist neural networks are inherently unsuitable for emulating the human capability of symbolic rule generation.

Yet I mean “rules” not necessarily in this strict sense, although clearly the neural networks in our brain are capable of formal logical operations. I simply mean principles, often consciously describable ones, for organizing one’s behavior and conduct across a range of situations. More often than not these are heuristics or rules of thumb; that is, “do’s” and “don’ts” that are not absolute but tend to apply, in the philosophers’ terminology, ceteris paribus (all other things being equal).

Where do the heuristics we humans employ come from? One of the stock answers is: evolution. That is to say, the answer is that our heuristics are based on behavioral patterns that have been selected for survival and reproductive success. Also, evolutionary psychologists tend to believe that despite some species differences in details, the basics of these behavioral patterns are largely shared with other mammals. Yet as explanations of higher-order cognitive function, the standard evolutionary arguments provide starting points but are incomplete, as I hope to show in this chapter.

There are some behavioral patterns that are based in evolution and present in all of us, such as the patterns of self-interested (or self-protective) behavior and the patterns of social bonding (including altruistic) behavior. Yet we all differ in our criteria for when to engage in which of those behavioral patterns, and in what social and environmental contexts. Moreover, these decision criteria are not exclusively genetic but heavily influenced by learning and by culture (Eisler & Levine, 2002; Levine, 2005).

Yes, the hard-core evolutionist will argue, but all this is dependent on the extensive plasticity of our brains (present in all animals but vastly expanded in humans), and evolution selected us for this trait of plasticity. This is certainly true, but that does not entail evolutionary determinism for each of the behaviors that arise from the plasticity.

Moreover, survival and reproduction do not explain all behaviors. In addition, human beings seek well-being: self-actualization, pleasurable stimulation, aesthetic harmony, mastery, and meaning all can be powerful motivators (see, e.g., Csikszentmihalyi, 1990; Eisler & Levine, 2002; Maslow, 1971; Perlovsky, 2006). While evolutionary fitness can provide plausible functional accounts of most of these motivators, the behaviors they generate have a life of their own apart from their survival or reproductive value.

We now seek to decompose the brain systems involved in the development of rules at many possible levels of complexity.

Key Brain Regions for Rule Development

Hypothalamus, Midbrain, and Brain Stem

The organism’s basic physiological needs are represented in deep subcortical structures that are shared with other animals and that have close connections with visceral and endocrine systems. These regions include several nuclei of the hypothalamus and brain stem.

This part of the brain does not have a neat structure that lends itself readily to modeling; consequently, these areas have been largely neglected by recent modelers despite their functional importance to animals, including humans. Yet these deep subcortical areas played key roles in some of the earliest physiologically based neural networks. Kilmer, McCulloch, and Blum (1969), in perhaps the first computer simulated model of a brain region, placed the organism’s gross modes of behavior (e.g., eating, drinking, sex, exploration, etc.) in the midbrain and brainstem reticular formation. The representations of these behavioral models, whose structure was suggested by the “poker-chip” anatomy of that brain region, were organized in a mutually inhibiting fashion. A similar idea appears in the selective attention network theory of Grossberg (1975), who placed what he called a sensory-drive heterarchy in the hypothalamus. Different drives in the heterarchy compete for activation, influenced both by connections with the viscera (giving advantage in the competition to those drives that most need to be satisfied) and with the cortex (giving advantage to those drives for which related sensory cues are available).

This idea of a competitive-cooperative network of drives or needs has not yet been verified but seems to have functional utility for behavioral and cognitive modeling. Rule making and learning are partly based on computations regarding what actions might lead to the satisfaction of those needs that survive the competition. Yet the competition among needs is not necessarily winner-take-all, and the best decisions in complex situations are those that go some way toward fulfilling a larger number of needs.

But what do we mean by “needs”? There is considerable behavioral evidence that the word should be expanded beyond the purely physiological ones such as hunger, thirst, sex, and protection. The term should also include the needs for social connections, aesthetic and intellectual stimulation, esteem, and self-fulfillment, for example.

The idea of biological drives and purposes beyond those promoting survival and reproduction goes back at least to the psychologist Abraham Maslow’s notion of the hierarchy of needs, a concept that has been widely misunderstood. The word “hierarchy” has been misinterpreted to mean that satisfaction of the lower-level needs must strictly precede any effort to fulfill higher-level needs, an interpretation Maslow explicitly denied (Maslow, 1968, p. 26). But neural network modeling, based on dynamical systems, allows for a more flexible meaning for the word “hierarchy” (or, as McCulloch and Grossberg have preferred, heterarchy). It is a hierarchy in the sense that there is a competitive-cooperative network with biases (see, e.g., Grossberg & Levine, 1975). This means there tends to be more weight toward the lower-level needs if those are unfulfilled, or if there is too much uncertainty about their anticipated fulfillment (see Figure 1 for a schematic diagram). However, the bias toward lower-level need fulfillment is a form of risk aversion, and there are substantial individual personality differences in risk aversion or risk seeking that can either mitigate or accentuate the hierarchical biases.

But this still leaves some wide-open questions for the biologically based neural modeler or the neuroscientist. We know something (though our knowledge is not yet definitive) about deep subcortical loci for the survival and reproductive oriented needs. But are there also deep subcortical loci for the bonding, aesthetic, knowledge, or esteem needs? If there are, it seems likely that these needs are shared with most other mammals. Data such as those of Buijs and Van Eden (2000) suggest that the answer might be yes at least for the bonding needs. These researchers found a hypothalamic site, in a region called the paraventricular nucleus, for the production of oxytocin, a key hormone for social bonding and for pleasurable aspects of interpersonal interactions (including orgasm, grooming, and possibly massage). Also, recent work reviewed by McClure, Gilzenrat, & Cohen (2006) points to a strong role of the locus coeruleus, a midbrain noradrenergic nucleus that is part of the reticular formation, in promoting exploratory behavior.

The deeper subcortical structures of the brain played a strong role in qualitative theories by pioneering behavioral neuroscientists who studied the interplay of instinct, emotion, and reason (e.g., MacLean, 1970; Nauta, 1971; Olds, 1955), and continue to play a strong role in clinical psychiatric observations. Yet these phylogenetically old areas of the subcortex are often neglected by neural modelers, except for some who model classical conditioning (e.g., Brown, Bullock, & Grossberg, 1999; Klopf, 1982). The time is ripe now for a more comprehensive theory of human conduct that will reconnect with some of these pioneering theories from the 1960s and 1970s.

(a)

(b)

Figure 1. (a) Maslow’s hierarchy of needs represented as a pyramid. (From Wikipedia with permission.) (b) A neural network rendition of Maslow’s hierarchy of needs. Arrows represent excitation, filled circles inhibition. All these needs excite themselves and inhibit each other. But the hierarchy is biased toward the physiological and safety needs which have the strongest self-excitation, represented by the darkest self-arrow; the love and esteem needs have the next darkest self-arrow, and the self-actualization needs the lightest.

Amygdala

The amygdala is closely connected with hypothalamic and midbrain motivational areas but “one step up” from these areas in phylogenetic development. The amygdala appears to be the prime region for attaching positive or negative emotional valence to specific sensory events (e.g., Gaffan & Murray, 1990). Hence the amygdala is involved in all emotional responses, from the most primitive to the most cognitively driven. In animals, bilateral amygdalectomy disrupts acquisition and maintenance of conditioned responses, and amygdala neurons learn to fire in response to conditioned stimuli.

The amygdala has particularly well studied in relation to fear conditioning (LeDoux, 1996, 2000). Armony, Servan-Schreiber, Cohen, and LeDoux (1995, 1997) have developed computational models of fear conditioning in which the amygdala is prominent. In these models, there are parallel cortical and subcortical pathways that reach the primary emotional processing areas of the amygdala. The subcortical pathway (from the thalamus) is faster than the cortical, but the cortex performs finer stimulus discrimination than does the thalamus. This suggests that the two pathways perform complementary functions: the subcortical pathway being the primary herald of the presence of potentially dangerous stimuli, and the cortical pathway performing more detailed evaluations of those stimuli.

Through connections between the amygdala and different parts of the prefrontal cortex, emotionally significant stimuli have a selective processing advantage over nonemotional stimuli. Yet that does not mean that emotional processing automatically overrides selective attention to nonemotional stimuli that are relevant for whatever task the organism is currently performing. Pessoa, McKenna, Gutierrez, and Ungerleider (2002) and Pessoa, Kastner, and Ungerleider (2002) found that if subjects were involved in an attentionally demanding cognitive task and emotional faces (e.g., faces that showed a fearful expression) were presented at a task-irrelevant location; amygdalar activation was significantly reduced compared to a situation where the emotional faces were task-relevant. This complex interplay of attention and emotion has been captured in various network model involving amygdala as well as both orbital and dorsolateral prefrontal cortex, such as Grossberg and Seidman (2006) and Taylor and Fragopanagos (2005).

Basal Ganglia and Thalamus

A higher-order rule-encoding system requires a mechanism for translating positive and negative emotional linkages into action tendencies or avoidances. This fits with the popular idea of a gating system: a brain network that selects sensory stimuli for potential processing, and motor actions for potential performance. Most neuroscientists place the gating system in pathways between the prefrontal cortex, basal ganglia, and thalamus (Figure 2a); for a theoretical review see Frank, Loughry, and O’Reilly (2001). The link from basal ganglia to thalamus in Figure 2a plays the role of disinhibition; that is, allowing (based on contextual signals) performance of actions whose representations are usually suppressed.

The most important gating area within the basal ganglia is the nucleus accumbens. Many neuroscientists identify the nucleus accumbens as a link between motivational and motor systems, and therefore a site of action of rewards — whether these rewards come from naturally reinforcing stimuli such as food or sex, learned reinforcers such as money, or addictive drugs (Montague & Berns, 2003).

Clearly, then, influences on the nucleus accumbens from other brain areas are key to choices about which stimulus or action representations are allowed through the gate. Newman and Grace (1999) identify three major influences on that area: the first dealing with context, the second with emotion, and the third with plans.

Figure 2b shows some influences on the nucleus accumbens gating system. The influences from the hippocampus are particularly strong: O’Donnell and Grace (1995) showed that active hippocampal connections can change single accumbens neurons from an inactive to an active state. Since the hippocampus encodes contextual associations for working memory, this can be a vehicle biasing the gates in favor of contextually relevant stimuli.

As Newman and Grace (1999) note, there is also a competing bias in favor of emotionally salient stimuli, regardless of the context. This is mediated by connections to the accumbens from the amygdala (Figure 2b). The hippocampal inputs, associated with “cold” cognition, operate on a slow time scale and promote selective sensitivity to a long-standing task. The amygdalar inputs, associated with “hot” cognition, promote sensitivity to strong and short-duration emotional demands.