Thanks for coming!

Thanks to everyone for coming! I think Reinforcement Learning is a fascinating field, with a lot of potential to both create useful programs and to teach us about how to make better decisions, and I hope I made a convincing case for this!

One reason I’m particularly interested in Reinforcement Learning, and ML more generally, is that I think future AI systems could be extremely transformative, and that one of the most important problems of our time is ensuring that this goes well. And that if you're mathematically inclined, there are a lot of important and fascinating things to research in this area! If you're curious about this issue, I'd highly recommend this introductory article. And if you’re interested in learning more, I’m always happy to chat!

I find feedback extremely motivating, so I'd love to hear any feedback about what went well with the talk, and what I could do better in future talks! You can let me know in this feedback form or just message me. And please reach out if you have any remaining questions, or want pointers for how to learn more!

• Slides

• A great, short introduction to classic RL

• Learning more

  • Classical RL - this tends to be more theoretical and more mathematical, but the ideas are still relevant!

    • Sutton & Barto - the classic textbook, freely available online. Very mathematical, good at covering intuitions.

    • Lecture course by David Silver, creator of AlphaGo and AlphaZero

  • Spinning Up in Deep RL, an excellent guide for getting started in deep reinforcement learning

    • This has a lot of exercises and code. I highly recommend doing them and getting your hands dirty! Reinforcement Learning is a very empirical field, and theory can only take you so far

• References and further reading

  • Deep Reinforcement Learning is Hard

    • Deep Reinforcement Learning Doesn’t Work Yet

    • Lessons Learned Reproducing a Deep Reinforcement Learning Paper

  • Explore/Exploit

    • Introduction to multi-armed bandits

    • See Chapter 2 of Sutton & Barto

    • An excellent podcast episode on applying ideas of explore/exploit to your career

    • Exploration in Deep RL

  • Specifying Rewards

    • Specification Gaming

    • Long list of examples of specification gaming

    • Goodhart’s Law

    • Goodhart’s Law in daily life

    • Series of posts on Value Learning

    • Deep RL from Human Preferences

  • Types of reward

    • From Pongs to Pixels, an introduction to Policy Gradients

  • Q-Learning

    • Introduction to Dynamic Programming

    • Connections between Q-Learning, Dynamic Programming and Dijkstra’s Algorithm

    • Explanation of DeepMind’s Atari solution

  • AlphaZero

    • Simple explanation of the algorithm

    • Why AlphaGo was a big deal

    • The AlphaZero paper (really well written!)

    • OpenAI blog post on Dota2

    • DeepMind blog post on StarCraft

  • Multi-Agent

    • OpenAI Hide & Seek

  • Long list of further resources