Playing Dance Dance Revolution with a neural network
As a continuation on the research I did on automated gameplay with pong and Doeo, I decided to try to play Dance Dance Revolution using a neural network. The project had two research questions: Is it possible to use a neural network to play the game Dance Dance Revolution? And can we do it without having (full) knowledge of the game?
The first step of the project was to generalize the game into simple rules. These are the six rules that define the Dance Dance Revolution game:
- The game has objects at certain two-dimensional coördinates. These objects can and will move
- It is possible to press four buttons. If an object disappears after pressing a button, it results in a higher final score
- If pressing a button results in an object disappearing, it will cause the same effect if in the future another object is at the same coördinate
- An object can only disappear once during its life span
- Pressing a button when it’s not necessary may result in lower final score
- The goal is to make all objects disappear
The first part of the project was to interface my solver application (written in Java) to Dance Dance Revolution. It was quickly decided that StepMania was a much more suitable game because of its open-source nature. I altered the StepMania source-code so that it would send the X and Y values of all current onscreen objects over a network socket, together with the v-sync signal. The same network socket was also used to inject key-press into the game. This created a very reliable and fast-paced way of interfacing with the game.
The second part was to get the neural network some initial data to work with. For a full session (two to three minutes), a random button is pressed every 110 millisecond, just to see what happens. If one of the objects displayed on the screen disappears within 50 milliseconds of pressing the button, we store the coördinates of that object together with the last pressed button into an array. If it doesn’t disappear, we store the coördinates in the array with the label ‘0’, meaning that nothing happened to this object after pressing the button.
The third part was to get the neural network to take over the wheel. We know what to do in a lot of situations, because of the data gathered in the previous step. However, we have a lot of blanks in this data (where the desired action is unknown) which we need to fill in using a neural network. The problem is that neural networks are not magical. It took a lot of fiddling to get the neural network to behave as requested. The first problem was to fix the network getting lazy. Because there were so many misses in the data gathering session (pressing a button and nothing happened), the network would suggest to never press any button, ever. Because if you never press a button, it would still be 99.37 percent accurate. This was quickly fixed by multiplying the positive-action items in the dataset (where pressing a button resulted in a disappearing object) many, many times. The second problem was duplicate data. Multiple items in the dataset with very close coördinates had different solutions, or the same solution. If two objects in the array were very close by, the one that didn’t result in a button press was removed.
So how did the final solution look like? The final neural network had two inputs, the X and Y value. There were five outputs, one for each button and one for doing nothing. There were two hidden layers in between, each having 25 nodes. All data was normalized (see the first problem in the previous paragraph) and stripped of duplicates (see the second problem in the previous paragraph). During gameplay, the screen was continuously scanned for objects. Every on-screen object’s coördinates were inserted into the neural network. After examining the five outputs, the one with the highest value resulted in the action taken: Press one of the four buttons or do nothing. This was done for each of the objects on-screen.
So how did it perform? In a test-run, it scored 60 flawless notes, 88 perfect notes, 87 great notes, 0 good notes, 0 boo notes, 0 misses and a combo of 235 notes, giving a final score of 913. This is significantly better than my personal high-score, although lower than top-players. Is it possible to use a neural network to play the game Dance Dance revolution? Yes, it most certainly is. Can we do it without having (full) knowledge of the game? Yes, we most certainly can. The algorithm that plays the game only knows of the six very general rules stated above, and is not tailored to this specific game.
A full description of the project can be found in the paper I wrote as part of the project evaluation (Automated game of Dance Dance Revolution using a neural network), the source code can be found in the GitHub repository.
A project that relates to this is a project by Tom Murphy (The First Level of Super Mario Bros. is Easy with Lexicographic Orderings and Time Travel). I, unfortunately, came across this project after presenting mine (it was mentioned during the evaluation of the presentation), otherwise I would have loved to incorporate some of his techniques into this project.