Intelligent Machines

Minecraft Shows Robots How to Stop Dithering

A new approach to robot learning was tested in Minecraft, the popular open-ended computer game.

Jul 13, 2015

The computer game Minecraft, which depicts a world made up of retro, pixelated blocks that can be modified and rearranged in endless architectural configurations, has been praised for teaching young players about creativity, problem solving, and survival skills (in certain modes you have to avoid threats including zombies). Well, it turns out even inexperienced robots can learn a thing or two by playing the game.

Minecraft was used to teach a virtual agent what steps to exclude when trying to put a gold block in a furnace.

Stefanie Tellex, a professor at Brown University, is using Minecraft, as well as real-world machines, to explore ways for robots to solve new problems faster and more efficiently. This isn’t something most robots need to do, since they work in a fixed environment, performing work that has been carefully programmed beforehand. But it could be important as robots start to take on more complex, open-ended tasks in less structured settings. A robot designed to help around the home, for instance, would need figure out how to perform different chores.

“You might tell a robot ‘Make me coffee,’ but the next minute you might say ‘Do the laundry,’” says Tellex. “In this context, where you don’t know the goal in advance, there’s this planning problem. Finding the sequence of actions that’s going to work in this particular environment is very challenging. Our approach is about learning that faster.”

The researchers’ algorithm was tested on a real robot programmed to help with cooking.

The challenge for a robot is that it lacks contextual understanding about the world. That means it has to run through all the possible actions that might help it reach a goal. To do the laundry, then, a robot might consider whether using the coffeemaker could help. And for a sophisticated robot working in a complex environment, the number of scenarios that need to be considered may be huge, requiring extensive computational power.

Tellex, together with one of her graduate students, David Abel, and several other colleagues at Brown, developed an algorithm that lets a robot prune away certain possible paths of action by understanding the direction in which a particular task points. Armed with such an algorithm, a robot might understand that doing the laundry does not involve using any kitchen equipment or utensils—knowledge that can be either preprogrammed or learned through experience.

Minecraft was used to test the learning approach. The researchers controlled a virtual character that was tasked with putting a virtual gold block into a virtual furnace, while avoiding a virtual pool of lava. After performing the task in a limited setting, through lengthy trial and error, the algorithm controlling the character learned that certain behaviors, such as placing gold blocks on the ground, could be excluded when trying to achieve the goal.

When asked to perform the same task in a more complex setting, the character could then work through a much smaller set of potential scenarios. And the approach could let robots learn how to perform a task in a virtual setting before taking on real-world scenarios (see “Even Robots Now Have Their Own Virtual World”).

Manuela Veloso, a professor specializing in robotics at Carnegie Mellon University, says virtual learning is an important area for robotics already. “Clearly a robot cannot use trial and error to learn to go down stairs, as it will break at the first trial,” she says. “Simulation, in general, can help train a model, which can then be used by a real robot.”

The Brown researchers also tested the approach with a real machine: a Baxter robot from Rethink Robotics. They assigned the robot to help a person cook, and programmed in an understanding of the actions that could be discounted for certain recipes. This made the machine more efficient when determining its own course of action in helping make brownies.

The approach followed by the Brown researchers is not only more efficient but, in a sense, more human, in that it requires a deeper understanding of a task and its context. “We were inspired by some of the literature on ‘affordances,’ which is how people look at the world when they’re trying to achieve something,” Tellex says.

Using Minecraft to unboggle the robot mind from Brown University on Vimeo.