A Review on Reinforcement Learning - Methods, Tools, Applications on Games and Beyond 

Yuandong Tian

In this tutorial, we will do an extensive review on recent state-of-the-art RL methods (A3C, APE-X, R2D2, SAC, Self-play learning etc) and their usage on Games and other applications. We also propose our next generation RL tools, ReLA (Reinforcement Learning Assembly). Compared to our previous framework (ELF), ReLA utilizes native Tensor support in PyTorch C++ API, embraces efficient batching and performs parallel network forwarding in multiple simulation threads. We open source ReLA as well as multiple strong baselines on popular game environments (e.g,. Atari, Go). Using ReLA, we establish state-of-the-art performance on the bidding phase of Contract Bridge, a 4-player games with 2 competitive teams, via zero-knowledge self-play and 10x smaller models, trained in several hours. On the playing phase of Bridge, neural-based value function is shown to be better than traditional scoring function (e.g., GIB) relying on sampling, in both prediction accuracy and speed.

 Yuandong Tian is a Research Scientist and Manager in Facebook AI Research, working on deep reinforcement learning and its applications, and theoretical analysis of deep models. He is the lead scientist and engineer for ELF OpenGo and DarkForest Go project. Prior to that, he was a researcher and engineer in Google Self-driving Car team in 2013-2014. He received Ph.D in Robotics Institute, Carnegie Mellon University on 2013, Bachelor and Master degree of Computer Science in Shanghai Jiao Tong University. He is the recipient of 2013 ICCV Marr Prize Honorable Mentions.