An article by Larry Yaeger.

These are the words that Larry Yaeger wrote in his web site on the PolyWorld experiment, and they give a good general introduction to his article. In the following pages, I will summarize the things I said in class, give some details on Yaeger’s experiment, and say some of my thoughts on it.

In this seminar we came across many a-life experiments, that use all of the elements mentioned above; genetics, neural networks, behavior, environment etc. PolyWorld is special in that it is the “ultimate” a-life experiment. It is a huge and complex computer-simulator program, which can be used (downloaded) by anyone who wishes to conduct his own a-life experiment. Its’ size and complexity are what is interesting, and they are both its’ strong and weak points.

Yaeger writes in his article “… PolyWorld may be thought of as a sort of electronic primordial soup experiment, in the vein of Urey and Miller’s classic experiment, only commencing at a much higher level of organization.”

The targets Yeager set for his experiment were:

1 – to reach emergence of complex behavior at an ethological level.

2 – to create life as “real” as possible, by using critical life “ingredients”.

3 – to try and reach clues to the development of AI, by using the evolution of neural networks in an ecology.

Yaeger hopes to reach these targets in a different way than other a-life ecologies that we saw (for example – “Tierra”). In PolyWorld (PW), the organisms are “grounded” to their world by a “naturalistic” sense of vision. Other elements in the organisms’ design and development are simulated in a “biological” fashion, such as the design of the neural-network brain. The main goal was to try and refrain from using “rules of the game” and try to reach emergence from basic and “biological-like” building blocks, hence the resemblance to Urey and Miller. 

I will give here a brief review of the specifications and details in PW, to give you a taste of how complex and sophisticated the simulation is. As mentioned before, the point was to use as basic building blocks as possible, and to facilitate as many possible behaviors, in order to reach the ethological level desired.

The landscape of PW is a flat terrain, with perhaps some barriers, food and organisms are scattered randomly and freely. The borders can be like walls, like a table-top (that organisms die if they fall off the edge), or wrapped around. The organisms run their own energy balance, and each one has to solve an energy problem in a different situation, therefor the fitness landscape is different to each one.

Each experiment begins with a predetermined number of organisms, with random (or not) characteristics, and in an “on-line GA” mode, in which an ad-hoc fitness function is used, to determine how the population(s) will develop. Once a population of organisms reaches a state in which it reproduces (instead of having organisms put in by the experimenter) it is left to it’s own, and from now on, the only fitness function is survival. Such a state is called SBS – successful behavior strategy, and an experiment in which an SBS was reached is considered a success. The experiment then may continue, to provide us with different species, which survive using different strategies, relationships among different species and among themselves, interesting behaviors and so on.

The genes determine many characteristics of the organisms:

Size, strength, maximum speed, ID, mutation rate, life span, fraction of energy given to offspring, number of neurons devoted to each component of vision (red green and blue), and eight characteristics of the neural network brain.

The evolution is purely Darwinian, and there is clearly a difference between genotype and phenotype.

To promote the speciation it is possible to limit the ability of organisms of different species to mate and reproduce. 

All activities, including neural activity, consume energy. Energy expendence and storage capacity of an organism are proportional to it’s size and strength. Therefor there is a balance between being big and strong, and energy wasting, and being small and weak but conservative.

There are two types of energies: health energy, and food-value energy. The first being it’s potential in a fight, and the second - how valuable it is to a predator. 

The inputs to the “brain” are: vision, it’s own health energy level, and some random input. The outputs are the suite of seven primitive behaviors: eating, mating, fighting, moving, turning, focusing, and lighting.

Each behavior is expressed by the activation of one neuron. The input neurons are two, plus the three groups for vision (one for each color). The other neurons in the network are in internal groups or clusters, and there are 1 - 5 such groups.

The genetic code of each organism determines the characteristics of these neural groups. In each group the parameters are: the number of excitatory and inhibitory neurons, the synaptic efficacy, CD - connection density (from group i to group j), TD - topological distortion of connection from group i to j (enables the possibility of creating a somatotopic map etc.) and LR - learning rate, a parameter in the Hebbian learning rule.

All these enable a frame to building a network, without forcing a specific architecture.

Yaeger points out that according to some experiments, Hebbian learning can lead to the creation of neural structures, such as the one we find in the primates’ visual cortex. Therefor, in order to initialize the organisms brains, they are exposed, prior to “birth”, to visual stimulation of random noise.

The organisms can “actually see their world”. They see a one dimensional strip of pixels, and they can control their vision by focusing (how wide their field of vision is), and by how many neurons are devoted to perception of each color (determined genetically). Organisms were found to act in accordance with visual input, which is a very important result regarding topics like “perception” or “cognitive ability”.

The basic seven behaviors are listed above (in the neural system section), and aside from the obvious aspects, they can take part in facilitating communication. For example, the behavior “lighting” means that an organism can increase the brightness level of some of the polygons on it’s exterior, or thexpression of “fighting “ behavior is mapped onto the red component of the polygons.http://www.beanblossom.in.us/larryy/Alife.html

 In a typical experiment there are some 300 organisms, with 200 neurons in each brain, each time step in the simulation (synchronous updating) takes about 13 seconds, the average life span is 500 time steps, 100 steps until the first offspring, 500 generations in a week. The code is approximately 15,000 lines (in C++). 

Results: 

The data that is supplied to the experimenter by the program are:

population size, ad-hoc fitness function values, the ratio between the number of organisms born vs organisms created, energy balance of an organism, and an analysis of genetic variation in the population (Hamming distances between genomes).

Good runs were the ones in which an SBS was reached. Further there was found speciation, and species that appeared in more than one run. The interesting species that Yaeger mentions were:

Frenetic Joggers - organisms that run ahead at full speed, eat and mate constantly. A good but not so interesting solution.

Indolent Cannibals – create for themselves a zero dimensional world, they remain around their own kind, mating, fighting and eating each other. Before a change was introduced in which an organism gave some of its’ energy to the offspring, they enjoyed the benefit of having free meals.

Edge Runners - one dimensional world - running along the edges, slowing down or speeding up to find mates or prey. Creates speciation or isolation of their genome.

Special Behaviors - in different species or populations there appeared to be “waves” of behaviors such as predation. It is as if the “tit for tat” strategy emerged in the behavior of whole population.

Other special behaviors were: speeding as a result of visual stimulation - an important result, since it gives evolutionary advantage to good vision, speeding as a result of an attack - defense, fighting as a result of an attack - conservation of energy, foraging and expression of attraction to food, flocking - following other organisms.

 Discussion and Future Directions:

 Some of the goals that Yaeger set were achieved, namely the emergent of complex behaviors out of simple elements. Also, the combination of research in neural networks and artificial evolution can lead to the understanding of the dynamics of natural selection, and the solution of optimization problems. The benefits from PW will appear when further research will be made on the results of such experiments, or when PW will be used to conduct more specific experiments. In other words, there is still work to be done, especially on studying the “brains” of the successful or interesting organisms, and in comparing the results of an experiment in PW with some experiment on a biological ecology.

For future directions or possibilities, other numerous variations can be introduced into PW such as: allowing more primitive behaviors, more variation in the environment, manipulating the populations (interfering by putting in or taking out organisms), or changing the basics - non Hebbian learning, a more realistic neuron, more sophisticated metabolism, and, of course, bigger and longer.

Yaeger, in the article, enters a philosophical discussion about how much PW’s organisms are really “alive”. The basis for the discussion are the criteria thought of by Farmer and Belin as to “what is life”. I will not bring any of it here, but I will mention that the organisms in PW meet all of these criteria (something to think about...).

 Some of My Own Thoughts:

 First of all, Yaeger’s work is very impressive. The computation power and the complexity are amazing, and in this sense it is the ultimate a-life simulation. I have the feeling that someone had to “do the job”.

The philosophy behind this work, that intelligence (or in this case complex behavior) is the result of the evolution of neural networks in an ecology, seems to me to be very true, and this idea can be used to develop NN’s that are very potent, if we let them learn an environment (from all of it’s aspects), for a long time.

PW is, in the end, a tool, and a very potent one. In order to get the most out of it, it should be used in running much more specific experiments. What Yaeger presents in this article is therefor somewhat disappointing. Yaeger succeeds in proving the possibility of “emergence”, and in this sense he is true to Urey and Miller’s legacy. But on the other hand, Yaeger is not the first to prove the ”possibility” of emergence. Others have done the same, using a much simpler tools (such as agents acting according to a list of “world rules”), and the benefit from going down so deep to the genetic level and all the way up through neural network architecture, or making the agent “biological like”, is not very clear. Further more, the results are very complex, and it’s hard to get anything concrete from them. The amount of data is enormous, it is very hard to follow all the 300 organisms or analyze their brains, so aside from emergence, it is technically very hard to prove anything from this. Of course, the other technical problem is that one will need a very powerful computer to run an experiment with PW, and the time it would take is also a factor.

In my opinion, a system like PW can produce a lot of benefits, besides the issues presented in the article, which are a bit problematic, as I explained. I think that if it were possible to simplify the system in the lower levels of organization, then experiments dealing with ecology could be conducted, and on the other hand, if it were possible to “cut off” on the higher level of activities, then perhaps we could try to simulate the emergence of some “lower level” aspect in life, such as the genetic code or something of the sort.