In the Future, Magnets May Track Your Fingers in Virtual Reality

A research project may hint at how Facebook’s Oculus device will let you explore virtual reality someday: by using a bunch of electromagnets and sensors to track the motions of your individual fingers in three dimensions.

Called Finexus, the project employs four magnetic sensors to track fingernail-sized electromagnets placed on each of the user’s fingertips; it has been found to be accurate to within 1.3 millimeters. The project was created by a group of researchers from the University of Washington and Oculus Research.

Keyu Chen, a graduate student at the University of Washington’s ubiquitous-computing lab, started the project while he was an intern at Oculus Research in Redmond, Washington, last summer. He imagines Finexus being used for games as well as tasks that require a variety of delicate finger motions, like playing a virtual piano, painting, or writing in the air. (A video of Finexus in action shows an example of the latter.)

“Sometimes you want something where you can actually use your fingers for dedicated motion,” Chen says.

Oculus had no comment on the project, which is slated to be presented in a research paper at the ACM CHI 2016 conference on computer-human interaction in San Jose, California, in May.

The interactions Chen imagines could be more detailed than what many of us will experience early on with virtual reality. Oculus’s first consumer headset, Rift, will rely on a plastic controller for input, which may not feel that immersive. (Rift, which is due to be released this winter, will come with a wireless Xbox controller; Oculus will also sell a pair of controllers called Oculus Touch that work with Oculus’s optical-tracking system to give you finer controls, but it’s shipping these a bit later on.)

Unlike tracking systems that rely on cameras to figure out where your head or specific body parts are located, Finexus doesn’t need a direct line of sight between the electromagnet and the sensor. That means, for instance, that if you bend over while playing a virtual-reality game, obscuring the electromagnets on your hands from the sensors, Finexus should still be able to figure out your finger movements.

Chen envisions Finexus eventually being built into a glove and wristband or smartwatch; the glove would house the electromagnets, while the wrist-worn device would hold the sensors. For now, the maximum distance between the sensors and electromagnets is just 12 centimeters, but Chen believes it could be as great as 25 centimeters. Regardless, such a short distance would limit the system to being housed on the body.

Finexus works kind of like a GPS system: the distances between an electromagnet and the four magnetic sensors is calculated, then the intersection of each of those four distances is determined and used to figure out the electromagnet’s position in space. Each electromagnet operates on a different frequency, which Chen says makes it easier to identify them individually.

Chen says electromagnets could be added to the system without requiring more sensors, so you could track all your fingers or render your hands more precisely by placing more than one electromagnet on each finger.

His next step, he adds, will be to refine the Finexus hardware and miniaturize it. After that, he’s not sure what will happen.

“I do not know if [Oculus is] going to put this in a future product, but that is one possibility,” he says.