A Realer Virtual World

CAMBRIDGE, MASS. -

For the large majority of Internet users, virtual worlds like Second Life remain a confusing landscape of empty buildings, failed marketing and furry strangers. But Joe Paradiso believes that virtual worlds could be more than an over-hyped gimmick. They just need a dose of reality.

Paradiso, a professor in the Massachusetts Institute of Technology's Media Lab, is working to create what he calls X-Reality or Cross Reality, a system designed to bring virtual and real worlds into a practical sort of alignment. With funding from Second Life parent company, Linden Lab, Paradiso aims to use sensors, displays and software to bring real-world data into virtual worlds and to integrate access to virtual worlds with real-world situations.

Later in November, Paradiso's team of seven Ph.D. students plans to switch on 45 PDA-sized devices mounted on the walls of the Media Lab's building. Each is equipped with an iPhone-like touch screen, a version of Second Life's software, wireless connections, cameras and a variety of audio, motion and infrared sensors.

According to Paradiso's plan, anyone in the building wearing a small electronic badge can walk up to one of the small screens and peer into a landscape in Second Life and communicate with users. Second Life users will likewise use the screens to look into the real world through floating windows in the virtual world, watching passersby or even remotely sitting in on meetings.

"These devices are designed to be like wormholes that let you tunnel through to a second reality," says Paradiso. "Second Life is detached. We're tying it into the real world."

The result, says Paradiso, will be a physical building that users can access from anywhere in the world. And unlike other "virtual meeting" setups where users' digital avatars awkwardly sit around a virtual conference table in Second Life or another constructed world, an X-Reality meeting would take place largely in the real world, with some virtual world users participating via Paradiso's "wormholes."

"We're just extending human perception through these nodes," says Paradiso. "We're funneling bits back and forth to the virtual world."

The potential is for X-Reality go beyond mere meetings. Josh Lifton, a former student of Paradiso's, used his Ph.D. thesis last year to show just how much Second Life can mirror reality. In a project he called "Shadow Lab," Lifton created 35 "smart" power strips and plugged them in throughout the Media Lab. Each was equipped with sensors capturing audio, movement, light, humidity and temperature data, along with the power usage of other appliances drawing electricity from the strip.

Lifton then represented real-time data from those power strip sensors in a Second Life recreation of the Media Lab, with floating tentacles of blue lines representing each node. When a student or professor walked by a sensor in the real world, the tentacle that represented it in the virtual world would wave. Temperature caused the stalks to change color, and they poured out varying degrees of virtual smoke to show how much power was being drawn from the outlet. Users could stroll--or fly--around the map to get a quick sense of goings-on in the real-world Media Lab.

Lifton argues that the "Shadow Lab" setup could be expanded to a more complex scenario like a building's emergency response system. In a fire, for instance, responders could map out the building's temperature and even find inhabitants in the virtual world before risking their lives in the real one. Paradiso offers the more prosaic example of a factory floor outfitted with ubiquitous sensors that lets any executive monitor its manufacturing in the virtual world.

Those ideas for bridging virtual and real worlds may not be as fanciful as they sound. Last February, IBM (nyse: IBM - news - people ) created just this sort of virtual world representation to map the data center of Implenia, a Swiss construction and real estate company.

The virtual reconstruction of Implenia's data center pulled information about the layout of servers and storage, the devices' temperatures, power consumption and data processing over the Internet to IBM's servers, where they were built into a 3-D virtual world model. Implenia's engineers could use that virtual world mock-up to remotely track the data center's power consumption and efficiency in ways that would have otherwise been tough without visiting the location.

"You can remotely enter the data center and actually hear the hum of a power system, to know if that fan is running a little too much. From an auditory clue, you can fix something in the real world," says Colin Paris, an IBM distinguished engineer. "That combination of the real and virtual is something we know will be a very big play, and MIT is really looking at the boundaries."

Putting sensitive real-world data into the virtual world presents a few privacy problems, Paradiso admits. Experiments like the one he'll perform later this month or Lifton's Shadow Lab could let outsiders track the movement of a building's inhabitants and follow their behavior. Like any computer system, Paradiso says X-Reality implementations would require different levels of access based on verifying users' identities. "Ultimately, you have to trust the agent that's doing the tracking," he says.

Lifton, meanwhile, has moved on to less privacy-threatening applications of his X-Reality expertise. As a researcher for the virtual world contractor Electric Sheep, he's working with a major toy company to build a product that would exist in both real and virtual environments. Lifton won't offer details of the toy but compares it loosely to a robot that "remembers" its experiences in Second Life. Win a game in the virtual world, for instance, and the toy gains a new functionality in the real world.

Small-scale entertainment applications like these, says Lifton, may be more realistic in the near term than Paradiso's dreams of large-scale X-Reality. "Entertainment is definitely where this will happen first," he says.

But Paradiso has bigger plans. In fact, he believes virtual worlds as a whole may need an injection of real-world connection before they slip into obscurity. "Everyone's fascinated with virtual worlds, but they're starting to get a bit tired," he says. "How you tie the virtual world into the real--not necessarily one or the other, but a whole continuum, a layered immersion--that's an intriguing story."