CS 539 Project Proposal March 9, 2016

CS 539 Project Proposal March 9, 2016

Jamie Warner, 9068206953

Summary

Sociality is a tendency for animals to associate in cooperative social groups, and it is a response to evolutionary pressures. Communication in a social group is made possible by a shared understanding of messages. Communication is composed of two parts: encoding some information as a signal, and interpreting that signal as information. I will use a broad definition of communication which can be active (spoken word) or passive (facial expressions). The ability of organisms to communicate is the resolve of evolved biological capabilities, which implies that all forms of communication increase an animal's fitness.

Inspired by biology, a recent class of evolutionary algorithms find optima of a success function given an initial genetic state. These work by generating an initial population (P1), evaluating the fitness of all its members, selecting the most fit individuals to reproduce, and breeding using crossover and mutation to generate the next population (Pn+1). If the genetic state of an organism can describe its capabilities to communicate, and if the fitness function resembles the challenges that made social behaviors useful, then the model may learn to simulate social functions. To simulate this evolution of populations, Pn must be a “population of populations,” where each member of Pn is a population of social organisms, and the population's fitness is the collective survival of everyone in it.

This project will first identify some forces that led to the evolution of sociality, namely conservation of energy, better information storage, and adaptability. I will speculate how different aspects of sociality cause these advantages. If a simulation is made with evolutionary forces similar to those during real-life social evolution, then I expect the model to converge to adaptive sociality. If the social functions in the model closely resemble those of living animals, then the “information structures” of the model may also match the physical neurological layout of animals, leading to interesting cross-analysis.

My model will not try to replicate specific information processing centers in the brain, such as visual and auditory processing, but instead will use a general “pattern recognition task” which allows for social learning and communication. An animal will be simulated as an evolving dynamical system under constraints of available caloric energy, a quantity which can be “spent” to perform actions. An Success is defined as correct corresponding “action” output, which is a timed use of energy, in response to various “event” patterns in the input signal. Failure (death) of an individual is defined as running out of energy. If the system fails to identify an input pattern it will undergo an energy penalty, so successful organisms are those that can identify danger consistently and quickly. This model is elaborated later.

The population will consist of many individuals augmented with social behavior so that individuals can send input to each other (in addition to receiving input from the environment). In addition, events that require a collective action response will force adaptive cooperative behavior (i.e., 80 percent of individuals must perform an action, or the population dies). I predict that social communication in the model will lead to a greater average success in a population than in an individual.

I also predict that the introduction of social communication will lead to structural patterns analogous to trust and empathy because these increase a population's collective fitness. Methods of analysis are discussed.

Various options are discussed for choices in animal model. I tentatively conclude that a Jordan-type recurrent neural net satisfies my requirements.

The goal of this research is to make a comparison between the “information structures” found in individuals in the model, and physical neural structures of found in biological organisms. A proposed extension to this model modeled on Marvin Minsky's The Society of Mind, would allow for direct anatomical comparison. An optimistic theoretical application is discussed.

This project will use macro-level (societal) success as a survival constraint to evolve individual patterns. The objective is to study what individual structures, both in the neural nets and in communication, lead to success. I am especially curious to identify those individual traits which create a conflict between individual success and societal success, i.e. social traits which help the individual but harm the society or vice versa, such as freeloading. I will also study social structures that arise from different individuals evolving to take on different roles in the society.

Background

Evolutionary game theory:

Evolutionary game theory is modeling the evolution of social behavior using mathematical principles of natural selection. One application is in identifying the cause of altruistic behavior. If a given behavior increases in individual's reproductive fitness, then the gene that lead to that behavior will propagate. However some altruistic behaviors come at a cost to an individual. Evolutionary game theory resolves the existence of altruistic behaviors because it demonstrates that social behaviors evolve if they benefit a genetically similar population, even if they come at a cost to the individual. Specifically, “Hamilton's rule” states that for any altruistic behavior, the genetic similarity between the donor and recipient must be greater than the ratio between the cost to the donor and the benefit to the recipient. [

Evolution of sociality:

Human brains are capable of deep structural change after birth and into adulthood, an ability called neuroplasticity. For instance, even after the brain is fully formed, it undergoes a restructuring in phonological processing while learning to read. [ All animals are somewhat neuroplastic, or else they would not be able to learn. However fossil evidence shows that our advanced ability to learn is uniquely human [ and that evolution of neuroplasticity was actually sudden and rapid, starting about 200,000 years ago. The evolution was possibly spurred by increased availability of food during this time, which allowed for a more complex brain that consumes more energy [

Professor Blakemore speculates why the human brain did not evolve sooner: “I’m making a speculation that a sudden speciation event happened—a spontaneous mutation, not progressive mutations selected by Darwinian evolution.” Blakemore is referring to the fitness landscape of proto-humans. Prior to “Mitochondrial Eve,” the evolution of larger human brains was limited, either by viability (increasing the size of the brain does gives diminishing returns of speed and information), by feasibility (a larger brain costs more energy), or by a of a survival need for more processing power. “Mitochondrial Eve” was the first human to have a rare genetic mutation that reduced or removed these limitations, leading to the rapid evolution of larger brains. One hypothesis is that the rapid growth of the human brain was driven by a need for complex social groups. Indeed, among primates, more complex social groups are correlated with larger brains. Specifically, evolutionary game theory mathematically demonstrates that altruistic behavior offers a fitness advantage to a species. The evolution of empathy, “perception of the emotional state of another,” allowed organisms to be altruistic in response to another's “pain, need, or distress” [

I propose empathy offers more benefit than just a mechanism for altruism because it allows animals to learn from how others respond to the environment.

Evolutionary advantages of communication

Communication offers several fitness advantages. I expect these to lead to the emergence of social behaviors in my model. The social behaviors developed will be dependent on the rules and parameters of the simulation. I outline three social behaviors here: emotional expression, trust, and teaching.

Emotional expression communicates when a signal is important:

Energy constraints limit the activity of the brain. There is too much environmental input data for the brain to remember it all because there is a limit to amount of encoded information any size-bounded system can store, and because it would cost too much energy for signals near input neurons to always propagate and make changes at deeper layers. Therefore it is necessary for organisms to develop mechanisms to identify what information has value and is worth encoding.

In humans, the reticular activating system, which regulates wakefulness and attention, learns to ignore repetitive, unimportant stimuli while stimulating attention when it needs to. The amygdala is responsible for emotional learning, a mechanism of recording information in response to emotion. (This is the reason many people have “flashbulb memories” of intense moments in their lives.) In this way emotion and attention systems are triggered when information has value.

Sociality provides means for organisms to communicate their attention and emotion, signaling not only the literal state of the environment (e.g. threatening, non-threatening) but also that a given environmental state has value and is worth recording.

In humans, emotion is communicated via facial expressions. If a person notices high levels of emotion in another, so-called “mirror-neurons” are triggered. Mirror neurons respond to the actions of others and actually activate the same patterns of neural activity as though they were performing the action themselves. [ Attention is communicated through the eyes.

Communication of emotion and attention signals when a given moment in time is important. This gives a population two potential advantages:

1. It saves energy because not all organisms need to be constantly alert for danger signals. As long as a “watch guard” is focused on one task, others can spend time on others and redirect their attention if the guard signals emotion.

2. It allows for “social learning.” During an event, the event signal was unknown to an individual, but known to others, then mirror neurons will signal that the event is important, and information should be recorded.

Trust allows for more “information-dense” networks:

Individuals in a population do not know exactly the same information. They respond differently to stimuli due to random variations in their initial states. This random variation allows for redundancy in recognizing important signals in the environment because if one organism fails to identify danger (false negative), but its friends do and communicate the danger, then it will still respond correctly. However these social systems of trust come at a cost of false positives.

Systems of social trust also allow a society to recognize more stimulus patterns than an individual could alone by dividing the task of knowing patterns among individuals. This increased volume of information comes at a cost of redundancy. For this strategy to work, organisms must have “social trust”. If A did not detect a threat, but B did, A will “trust” B and respond anyways.

I predict these forces to balance out, so that organisms' knowledge overlaps but is not identical.

Teaching leads to more adaptable populations:

Neuroplasticity allows for “social evolution,” an advanced ability of an organism to adapt to its environment. Every organism is born as a blank slate, with some neural circuits “hard-coded.” The organism adapts as it gets feedback from its environment so that it learns relevant event signals.

Due to random variations in the initial state of each organism, organisms will encode response patterns differently. Some organisms will be more successful than others at learning these patterns. If animals have the ability to learn from each other, then then they will be able to collectivity emulate the most successful animal. Therefore if organisms can reveal information about the configuration of their internal neural network, they will be more fit.

Social learning offers several advantages:

1. An organism does not need to experience an input signal to learn from it. Instead animals can teach each other correct response patterns, for instance by communicating a pattern and its solution. This way, young organisms can learn danger patterns without dying from them.

2. It saves energy by allowing for more efficient learning. Teaching signals are less noisy than environmental signals, so it costs less energy to learn from a taught signal than an environmental signal.

3. It allows for direct “social learning.” During an event, if a danger signal was unknown to an individual, but known to others, the organism will see the correct response performed by others, and then mirror neurons will teach the organism the correct response to the danger signal.

Model

Individual model (single animal)

Social model (many animals)

Black box animal model:

Inputs and outputs of the animal model will be bit vectors, one bit vector per frame.

The input vector includes the environment, energy level, and (in the social model) communication. The output vector includes an action vector and (in the social model) communication.

The environment vector will only change a little from frame to frame. At certain points, the input signal will contain an “event signal,” a bit pattern in time. Each event signal has a corresponding action mask. If the corresponding action bits of the organism are enabled by the end of the event signal, the animal will receive an energy reward (dependent on the event). Otherwise, the animal incurs an energy penalty which depends on the event it failed.

Activating an action vector costs energy, however, so leaving the action vector enabled will cause the animal to starve (run out of energy).

The environment and communication masks model attention. For every bit of input of the environment and communication vectors, there is an output mask bit. If the mask bit is 0, the corresponding input bit on the next frame is ignored (set to 0). If the mask bit is 1, the animal model receives the corresponding environment and communication input at an energy cost. If all mask bits are 1, then the animal is “alert” (paying close attention to the environment at an energy cost), and if all mask bits are 0, the animal is “resting”. “Freeloading” occurs if the animal ignores the environment and only watches for danger alerts from kin.

The individual animal model will be extended to a social animal model by defining an interface of communication between individuals. Because communication of a message and its interpretation are tightly coupled, responses to social communication will need to be encoded in the initial (genetic) state of animals. That is to say, not all communication can be learned, because without an ability to communicate there is no way to learn it socially! Interpersonal learning, as well as real-time threat communication, will be essential for a population to survive, because no single individual will know all of the event vectors.

Selecting an animal model:

Because I plan to make comparisons between patterns in the neural net and physical patterns of biological neurons, the model I choose must be biologically plausible, at least from the point of view of information storage and communication. A sensible model will have the following properties:

1. Time dependency. Biological networks are dynamical systems, not functions based on a single input. The model must account for neural oscillation.

2. Neuron state. Biological neurons are complex chemical processes, and they have state. Similar to Elman networks, there must be “context units” that provide readable, writable state. Time dependence of state will be discussed. Effective use of state will be necessary to identify event signals in the environment.

3. Vector outputs of neurons. Biological neurons can output a range of neurotransmitters which are interpreted differently. This can be implemented by encoding the output of a neuron as multiple bits instead of a single 0 or 1 value.

While voltage-based models like Izhikevich's “Simple Model of Spiking Neurons” have seen success in emulating voltage spike patterns, they cannot output an arbitrary length vector (requirement 3), and they do not have neuron state beyond voltage.

I rule out physically accurate but computationally intractable models like Hodgkin-Huxley.

Recurrent neural networks as described by Elman[1] satisfy time dependency and vector outputs. However, because the context units on frame n are just a copy of the hidden units from step n-1, I do not believe they will be sufficient for identifying a danger signal across several steps; they can only “remember” one step.

I believe a Jordan-type recurrent neural network (Jordan 1999) will satisfy the requirements. It is my understanding that, since not all state is overwritten every step, Jordan-type RNN's allow for persistent state. I believe this will allow for behavior similar to finite-state machines, where the current state is encoded in context units. This means that, for instance, even if the animal is at rest and not receiving input, fluctuations in its internal state will cause it to eventually wake up.

I intend to use the Stuttgart Neural Network Simulator ( which support Jordan networks.

Potential problems:

Computation time may be a problem.

Ideally the organism's decisions are influenced by its current level of energy, so that can, for instance, go into rest if it is starving. If giving the animal input deltas every frame is not sufficient to generate these behaviors, then it might work to encoding a numerical value of current energy as an input.

Analysis of the model

The model is designed to be modular. Communication, energy, and input masking all can be removed, and the model will still function. I will compare the behavior of the model under each of these configurations to get an understanding of how social behavior adapts to solve problems. The parameters of the model will also be experimented with.

Extension

Instead of modeling an organism as a single neural network, I will model an organism as a collection of “agents,” each a neural net of fixed size. Agents are given a “position” property, and communication between agents is a function of which agents are connected. The latency and throughput of connections are influenced by their physical position. Input agents, those which receive input from the environment, will have fixed positions. If the input agents are positioned in a way that matches the input to biological organisms, i.e. the energy input is by the stomach, and the others are by the head, and the position of the agents is optimized using an evolutionary algorithm, then there may be some resemblance between the position of the agents and configuration of major information pathways in biological bodies. This method of using evolutionary algorithms to determine network structure has been done before in GNARL. [