Summary

The website content provides an overview of eight practical reinforcement learning (RL) project ideas, ranging from beginner-friendly tasks to more complex challenges, along with bonus ideas for further exploration in the field.

Abstract

The article presents a curated list of reinforcement learning project ideas that cater to various skill levels and interests within the RL community. It starts with introductory tasks using OpenAI Gym's toy problems like Cartpole, Lunar Lander, and Taxi, and progresses to more advanced projects such as playing Atari games from pixel input and simulating control tasks with PyBullet. The article also suggests creating custom RL environments with Unity ML-Agents, racing self-driving cars with AWS DeepRacer, mining diamonds in Minecraft using MineRL, and building a Chess bot with OpenSpiel. Additionally, it encourages community engagement through platforms like AIArena for StarCraft II and participating in AI programming competitions. The projects are designed to provide hands-on experience with different RL algorithms, physics engines, and real-world applications, with the aim of inspiring readers to dive deeper into the practical aspects of reinforcement learning.

Opinions

The author believes that solving toy problems from OpenAI Gym is a good starting point for familiarizing oneself with RL frameworks and simple Q-learning algorithms.
It is suggested that tackling Atari environments requires more complex RL algorithms and deep learning libraries, indicating a higher level of difficulty and sophistication in these projects.
The author expresses that PyBullet is a valuable open-source alternative to OpenAI Gym's paid MuJoCo environments for those interested in robotics.
Unity ML-Agents is recommended as a tool for game developers and researchers to train intelligent NPCs and create graphics-rich RL environments, with a note on its user-friendly nature for beginners.
AWS DeepRacer is highlighted as a unique platform that combines a 3D racing simulator with the opportunity to compete in races and even deploy models in a real-world setting with a physical race car.
MineRL's dataset and competitions are presented as significant resources for training agents in an open-world game with sparse rewards and hierarchical tasks.
The article promotes community-driven platforms like AIArena for StarCraft II and AI programming competitions as avenues for collaboration and practical application of RL in mainstream games.
OpenSpiel is mentioned as a valuable tool for exploring RL

8+ Reinforcement Learning Project Ideas

Looking for a way to get practical, hands-on experience with reinforcement learning? Here are some project ideas to inspire you.

This post is a compilation of reinforcement learning (RL) project ideas to check out. I’ve tried to select projects covering a range of different difficulties, concepts, and algorithms in RL.

1. Solve toy problems with OpenAI Gym (beginner-friendly)

Cartpole environment from OpenAI Gym (screenshot taken by Author)

OpenAI Gym has become the de facto standard for reinforcement learning frameworks among researchers and practitioners. Solving toy problems from the gym library will help familiarize you with this popular framework and simple Q-learning algorithms. Good starting points include Cartpole, Lunar Lander and Taxi.

If you’re interested in a step-by-step walkthrough, check out my introductory Q-learning tutorial with Taxi.

2. Play Atari games from pixel input with OpenAI Gym

Atari environment compilation (image by Author)

OpenAI Gym also contains a suite of Atari game environments as part of its Arcade Learning Environment (ALE) framework. Examples include Breakout, Montezuma Revenge, and Space Invaders.

Environment observations are available in the form of screen input or RAM (direct observation of the Atari 2600’s 1024 bits of memory). Solving Atari environments will require the use of more complex RL algorithms and deep learning libraries such as TensorFlow or PyTorch.

Additional resources:

Recommended reading: DeepMind’s original Atari DQN paper
Jupyter notebook tutorial for Space Invaders by Thomas Simonini
Bias-Variance for Deep Reinforcement Learning: How To Build a Bot for Atari with OpenAI Gym

3. Simulate control tasks with PyBullet

PyBullet humanoid example (screenshot taken by Author)

If reinforcement learning applied in robotics is your area of interest, you might have already come across OpenAI Gym’s paid MuJoCo environments. For a free, open-source alternative, I recommend checking out PyBullet.

MuJoCo/PyBullet are physics engines providing real-world-like rigid-body simulations of humanoids as well as other creatures. They can be used to create environments with continuous control tasks (e.g. walking, running, and swimming), making them useful for experimenting with policy gradient methods such as DPG, TRPO and PPO.

UPDATE 19 October: MuJuCo is now free and open-source!

4. Create your own reinforcement learning environment with Unity ML-Agents (beginner-friendly)

Unity ML-Agents PushBlock environment (screenshot taken by Author)

Unity ML-Agents is a relatively new add-on to the Unity game engine. It allows game developers to train intelligent NPCs for games and enables researchers to create graphics- and physics-rich RL environments. Project ideas to explore include:

Experimenting with algorithms like PPO, SAC, GAIL, and Self-Play which have implementations provided out-of-the-box
Training agents in a library of 18+ environments including Dodgeball, Soccer, and classic control problems, within the Unity GUI.
Creating your own custom graphics- and physics-rich 3D RL environment

Additional resources:

5. Race self-driving cars with AWS DeepRacer (beginner-friendly)

AWS DeepRacer League stream (screenshot taken by Author)

AWS DeepRacer is a 3D racing simulator designed to help developers get started with RL using Amazon SageMaker. You’ll need to pay for training and evaluating your model on AWS. It features monthly competitive races as part of the AWS DeepRacer league, which awards prizes and the chance to compete at re:Invent.

Unique to DeepRacer is the option of purchasing a physical 1/18th scale race car for USD399 that will allow you to deploy your model in the real-world.

Some other open-source projects relating to autonomous driving to check out:

6. Mine diamonds in Minecraft with MineRL

MineRL contains an imitation learning dataset of over 60 million frames of recorded human player data in Minecraft. The goal is to train agents that can navigate an open world and overcome inherent challenges such as tasks with lots of hierarchy and sparse rewards.

MineRL provides useful getting started tutorials as well as a helpful community. As an additional incentive, MineRL is currently running two competition tracks with prizes as part of NeurIPS 2021:

Diamond: Obtain a diamond provided a fixed limit of raw pixel sample data and time training.
BASALT: Solve almost-lifelike tasks such as building a house or finding a cave (better for beginners).

7. Join the community at AIArena building agents for StarCraft II

If you’re looking to train agents to play highly complex mainstream games, you should check out AIArena. They run regular streams and ladders for a community of researchers, practitioners, and hobbyists building both scripted and deep learning agents for StarCraft II.

Aside from StarCraft II, here are some other mainstream games with RL frameworks you might be interested in:

8. Build a Chess Bot with OpenSpiel

OpenSpiel by DeepMind is worth taking a look at if you’ve been inspired by programs like StockFish or AlphaGo. It contains a collection of game environments including Chess, Go, Backgammon, and more.

RL in board games is a good space to explore if you’re looking to experiment with tree search/planning (e.g. MCTS), and policy and value networks.

Bonus ideas

Here are some additional project ideas that are also worth checking out:

Predict stock prices with TensorTrade
Train cooperative agents with PettingZoo
Build a Poker bot with RLCard
Join an AI Programming competition or RL competition

Closing remarks

There’s a huge range of exciting projects to explore in reinforcement learning. This list is by no means comprehensive, but I hope it’s given you some inspiration for your own RL project!