Gamasutra is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Gamasutra: The Art & Business of Making Gamesspacer
View All     RSS
July 20, 2019
arrowPress Releases

If you enjoy reading this site, you might also want to check out these UBM Tech sites:

Researchers teach AI to play 'hard' games by showing it YouTube videos

May 30, 2018 | By Alex Wawro

A group of researchers at Google DeepMind have published a paper today outlining how they successfully trained artificial intelligence to play "infamously hard exploration games" with YouTube videos of human playthroughs.

Even if you (like me) aren't well-versed in the art of AI, the paper has some interesting takeaways, including the core premise that it's apparently difficult for deep reinforcement learning algorithms to improve at tasks which, in the words of the paper, take place "where environment rewards are particularly sparse."

"One epitomizing example is Atari’s Montezuma's Revenge, which requires a human-like avatar to navigate a series of platforms and obstacles (the nature of which change substantially room-to-room) to collect point-scoring items," the paper reads. "For example, reaching the first environment reward in Montezuma's Revenge takes approximately 100 environment steps, equivalent to 100^18 possible action sequences."

While researchers say they can feed human demonstrations to the AI to teach the correct way to play a game, this paper seems to suggest that AI learn to play in a more interesting way (i.e. not using artificial demonstrations) by showing it a bunch of (mismatching) YouTube videos of people playing a game, with one earmarked as what the AI should do to receive a reward.

"Specifically, providing a standard RL agent with an imitation reward learnt from a single YouTube video, we are the first to convincingly exceed human-level performance on three of Atari’s hardest exploration games: Montezuma's Revenge, Pitfall! and Private Eye," the paper continues. "Despite the challenges of designing reward functions or learning them using inverse reinforcement learning, we also achieve human-level performance even in the absence of an environment reward signal."

It's fascinating research, and you can read the rest of the details in full in the appropriately titled paper "Playing hard exploration games by watching YouTube" on Arxiv's website. Some demonstration videos (one of which is embedded above) are also available to watch on researcher Tobias Pfaff's YouTube channel.


Related Jobs

DMG Entertainment
DMG Entertainment — Beverly Hills, California, United States

Game Engineer
Hi-Rez Studios
Hi-Rez Studios — Alpharetta, Georgia, United States

Senior Technical Artist
Hi-Rez Studios
Hi-Rez Studios — Alpharetta, Georgia, United States

Unannounced Project - Gameplay Programmer
Build A Rocket Boy Games
Build A Rocket Boy Games — Edinburgh, Scotland, United Kingdom

Senior Animation Programmer

Loading Comments

loader image