Projector lights radio tags

Virtual reality puts you in a computer-generated world. Augmented reality allows for a mix of virtual reality and the real world by placing computer-generated objects in a real-world setting.

Researchers from Mitsubishi Electric Research Labs have brought dynamic, computer-generated labels into the physical world with a combination of radio frequency identification (RFID) tags and portable projectors.

Their Radio Frequency Identity and Geometry (RFIG) system consists of a hand-held projector that shines dynamic images onto physical objects of the user's preference, and radio frequency identification tags augmented with photosensors, which identify objects for the projector. Radio frequency identification tags contain tiny, inexpensive chips that are read using radio waves. Photosensors detect light intensity.

The system can be used to find and track inventory, guide robots or precision handling systems on assembly lines, locate small instruments and track movement of items in health care settings, keep track of objects in homes, offices and libraries, and enable games to integrate real and virtual objects, said Ramesh Raskar, a research scientist at Mitsubishi Electric Research Labs.

The radio identification/photosensor tags can be as small as a grain of rice, said Raskar.

To find an object, a user aims a radio frequency reader in the general direction of a collection of tagged objects. Each tag that is in range is activated by the radio frequency signal, prompting its photosensor to take a reading of the existing light. Once this is done, the projector embedded in the reader turns on, and each tag that detects an increase in illumination sends a response indicating that it is in the projector beam and is ready for interaction.

The projector then beams a sequence of about 20 images of horizontal or vertical bars of varying density, which form unique codes indicating horizontal and vertical coordinates. Each tag records the code, then transmits its identity plus the code back to the radio frequency reader. This allows the reader to determine the location of each tag in its range.

The projector then marks the appropriate tags for the user. The projector image is dark where there are no tags, and illuminates areas where there are tags, said Raskar. The tags are accurate to within a millimeter, and can be used to find objects and detect when objects have been moved, said Raskar.

If an object has several radio tags attached to different surfaces, the system can be used to track the object's orientation and shape, for example detecting when an object has shifted or been deformed, said Raskar.

The projector, which the researchers developed previously, includes inertial sensors that track its position, making it possible to project images from a moving projector and compensate for the movements. The projector can also hold one portion of a projection steady while allowing another portion to move, making it possible to project a steady shot of a computer screen and at the same time project a mouse pointer that the user can control by moving the entire projector.

The projector can also capture an image of an object or sets of objects including tag information, then project the image on another surface. This cut-and-paste capability is useful for interacting with tags that are in places that are not suitable for projections, said Raskar.

The radio frequency identity and geometry system solves three interface problems, said Raskar: it locates tags precisely and can give visual feedback, it allows the user to see a subset of tags among many tags, and it allows the user to differentiate among multiple tags responding to a radio-frequency inquiry.

A warehouse worker, for instance, could use the system to view the tag information on a set of objects and digitally mark the tags of selected objects with instructions for coworkers, said Raskar. Or a librarian could use the system to identify which books are upside-down or out of order, he said.

The system is the reverse of the more common idea of putting beacons like light-emitting diodes on the objects themselves so they can be located visually, said Raskar. "We turned this around and said 'what if the tiny indiscernible tag does not have a battery attached to it... can it still be located and visually indicated?'"

The system can also be thought of as the reverse of computer vision, said Raskar. Traditional computer vision uses a centralized, dense, two-dimensional array of pixels to record information about a visual field, and the cameras recording the field are outside the field looking in. The researchers' tags, in contrast, are essentially single-pixel cameras pasted on objects, and these cameras are looking out, toward the projector.

Because radio frequency identification tags can be tiny and require no power source, they could be embedded in packaging to make automated tracking of even inexpensive objects cost-effective.

Most of the researchers' prototype work was done with active, battery-powered radio frequency ID tags because these can be programmed, but the system can be adapted for passive ones as well, said Raskar. In addition, the photosensors could be used to help power passive tags, he said.

The system's radio frequency tags will cost a little more than ordinary radio frequency tags because they contain a photosensor, and it will be two or three years before the passive sensing tags are available at reasonable costs and in quantity, said Raskar.

Initially the tags will be useful in inventory management, said Raskar. The system is really geared towards the first 10 meters and the last 10 meters of the tag's productive life -- initiation of tags at the time of production, and locating objects in a warehouse or human-interaction in cluttered environments, he said.

Within a decade most objects that are bar-coded today will instead carry radio frequency tags, he said. "Adding a geometric notion will be appealing for several applications," including factory automation, health care, entertainment, and even surveillance, said Raskar.

The researchers' next step is to put together a system that would allow more than one reader to work with a set of tags at a time, said Raskar.

Raskar's research colleagues were Paul Beardsley, Jeroen van Baar, Yao Wang, Paul Deitz, Johnny Lee, Darren Leigh and Thomas Willwacher. The researchers are scheduled to present the work at the Association of Computing Machinery (ACM) Siggraph 2004 conference in Los Angeles on August 8 to 12. The research was funded by Mitsubishi Electric Research Labs.

Timeline:   2-3 years, 10 years
Funding:   Corporate
TRN Categories:  Applied Technology; Human-Computer Interaction
Story Type:   News
Related Elements:  Technical paper, "RFID Lamps: Interacting with a Self-Describing World via Photosensing Wireless Tags and Projectors," Association of Computing Machinery (ACM) Siggraph 2004 conference, Los Angeles, August 8-12