Using Evolution and Deep Learning to Generate Diverse Intelligent Agents
by Marshall JosephSupervisor: Brian Ross
Supervisor: Brian Ross
| Emergent behaviour arises from the interactions between individual components of a system, rather than being explicitly programmed or designed. The evolution of interesting emergent behaviour in intelligent agents is important in some application areas, for example, video game non-playable characters (NPCs). Being able to produce a diversity of high-quality opponents makes human players more engaged in games. In this research, we use genetic programming to evolve intelligent agents in a predator-prey simulation. The task is to evolve prey agents that capture the prey agents in the environment. A main goal, however, is to evolve agents that exhibit interesting and diverse behaviours, rather than the usual ones commonly evolved. First, we train a convolutional neural network to recognize main instances of ``generic'' prey behaviour, as recorded by an image trace of the prey's movement behaviour in the environment. A training set for 6 generic behaviours was used to train the CNN. A training accuracy of 98\% was obtained, as well as a validation performance of 90%. Next, some experiments were performed that merge the CNN with fitness within the genetic programming system. In one experiment, the CNN's classification values are used as a ``diversity score'' which, when weighted with the fitness score, allow both agent quality and diversity to be considered. The result of this is that, with the appropriate weighting, high-quality, diverse solutions are obtained. In another experiment, we use the CNN classification scores to encourage the evolution of one of the known classes of trained behaviours. Results were that this trained behaviour was indeed more frequently evolved, compared to genetic programming runs using fitness alone. One conclusion is that machine learning techniques are a powerful tool to use for assisting the automated generation of diverse, high-quality intelligent agents. |
Copyright (C) 2023 Marshall Joseph
Back up: http://www.cosc.brocku.ca/~bross/