I wrote very hacky python to play chrome://dino/ using Tensorflow in college for a class project.
I didn't know anything about RL so I wracked my brains and came up with an approach using CNNs with the goal of creating a model that could play at least as well as I could. The project was three separate scripts: collect training data, train a model, and then run that model using Selenium. The model would be trained to predict an action (jump or don't) using images of the game (state), and the training data was generated by running a script while playing the game that recorded the screen and adding what keys I was pressing at the time. The CNN was simple: alternating 2D convolutional layers and 2D max pooling layers and two dense classification layers.
First, after a couple hours, I realized my poor GTX 1070 laptop GPU would struggle with even 640x480 captures. Read some docs, did some input processing, and after a week with a lot of Googling, got things running.
However, the accuracy was terrible. It took a while but I realized my data captured where I had lost the game. I started manually deleting the last ~1 second of images from each session and it worked! What a feeling!
Since the game speeds up over time the model hit a limit quickly. I used OpenCV to literally write in numbers on the saved images to provide some kind of info about the length of the game to the game state, and it worked again!
Then I ran into a new problem- the model made it to a part of the game where black and white are inverted (the palette shifts from "day" into "night") and consistently failed. I hadn't made it that far very often so there was too little "nighttime" data. So I learned about data augmentation without knowing the term for it; with a quick script, I copied all my training data with black/white reversed, and the model ended up besting my top score by a solid margin. Never realized I could have just swapped the image colors when running the model.
It was the most fun I'd ever had with programming to that point and kicked off my passion for ML and AI. Ugly manual steps, poorly written code, using the wrong kind of model - but it was true creative problem solving and I loved it.
I didn't know anything about RL so I wracked my brains and came up with an approach using CNNs with the goal of creating a model that could play at least as well as I could. The project was three separate scripts: collect training data, train a model, and then run that model using Selenium. The model would be trained to predict an action (jump or don't) using images of the game (state), and the training data was generated by running a script while playing the game that recorded the screen and adding what keys I was pressing at the time. The CNN was simple: alternating 2D convolutional layers and 2D max pooling layers and two dense classification layers.
First, after a couple hours, I realized my poor GTX 1070 laptop GPU would struggle with even 640x480 captures. Read some docs, did some input processing, and after a week with a lot of Googling, got things running.
However, the accuracy was terrible. It took a while but I realized my data captured where I had lost the game. I started manually deleting the last ~1 second of images from each session and it worked! What a feeling!
Since the game speeds up over time the model hit a limit quickly. I used OpenCV to literally write in numbers on the saved images to provide some kind of info about the length of the game to the game state, and it worked again!
Then I ran into a new problem- the model made it to a part of the game where black and white are inverted (the palette shifts from "day" into "night") and consistently failed. I hadn't made it that far very often so there was too little "nighttime" data. So I learned about data augmentation without knowing the term for it; with a quick script, I copied all my training data with black/white reversed, and the model ended up besting my top score by a solid margin. Never realized I could have just swapped the image colors when running the model.
It was the most fun I'd ever had with programming to that point and kicked off my passion for ML and AI. Ugly manual steps, poorly written code, using the wrong kind of model - but it was true creative problem solving and I loved it.