"Augmented Reality" in Mobile Devices
You enjoyed Hulk VI so much on your home theater setup that you decided to see it on the big screen. The movie is still playing, but you’re not sure how to find the movie theater where it’s playing. In the old days, you might have printed out directions from MapQuest; but nowadays you don't need to do anything so primitive. Instead, you dock your smartphone on the dashboard as you slip into your car, and instantly it superimposes driving directions to the theater are superimposed on your car's windshield. As you approach your destination, you see a group of tall buildings. Superimposed on the windshield over one of the buildings is the building’s name, the name of the movie theater inside it, the name Hulk VI, and a countdown to show time. "Turn left in 100 yards," the navigator speaks through your stereo as a large turning arrow appears, guiding you into the parking structure.
In Neal Stephenson's book Snow Crash, "gargoyles" are freelance intelligence gatherers who have wired themselves to see (through goggles that annotate all of their experiences) a permanent overlay of data on top of the physical world. In less immersive fashion, we may all become gargoyles as “augmented reality” becomes an everyday experience.
Augmented reality is a catchall term for overlaying what we see with computer-generated contextual data or visual substitutions. The point of the technology is to enhance our ability to interact with things around us by providing us with information immediately relevant to those things.
At work, you might walk around the office and see the name and department of each person you pass painted on them--along with a graphical indicator showing what tasks you owe them or they owe you. Though many case scenarios involve “heads-up” displays embedded in windshields or inside eyeglasses, the augmented reality we have today exists primarily on the “heads-down” screens of smartphones.
Several companies have released programs that overlay position- and context-based data onto a continuous video camera feed. The data comes from various radios and sensors built into modern smartphones, including GPS radios (for identifying position by satellite data), accelerometers (for measuring changes in speed and orientation), and magnetometers (for finding position relative to magnetic north).
In an application called Nearest Places, the names and locations of subway stops, parks, museums, restaurants, and other places of interest are shown on top of an iPhone's video feed. As you walk or turn, the information changes to overlay your surroundings.
"Smartphones and the related apps are the trailblazers for augmented reality," says Babak Parviz, a professor at the University of Washington who specializes in nanotechnology. "In the short to medium term, my guess is that they will dominate the field."
Other prototype applications display information dropped at particular coordinates as 3D models that the user can walk around, or as animations whose details update in 3D relative to the user's position. But the technology for those apps isn't ripe yet; handhelds require a more-precise positioning mechanism in order to handle that kind of data insertion. Fortunately, each smartphone generation seems to include more and better sensors.
In other realms, augmented reality may serve to provide not just additional information, but enhanced vision. One day, infrared cameras mounted on the front of a car will illuminate a far-away object represented as a bright-as-day image on an in-windshield display. Radar signals and wireless receivers will detect and display cars that are out of sight; and one piece of glass will host GPS and traffic reporting.
Leaping past displays, Parviz and his team are working on ways to put the display directly on the eyeball. They’re trying to develop a technology for embedding video circuitry into wearable contact lenses. While wearing such contact lenses, you would see a continuous, context-based data feed overlaid on your field of vision.
Before Parviz's lenses become a reality, augmented reality is likely to become a routine navigation and interaction aid on mobile devices. In addition, game developers may use the technology to overlay complete digital game environments over the reality that gamers see around them.
In Video: Augmented Reality is Coming to a Device Near You