'Seems like dark magic': Camera that can see around corners created by UW scientists

Researchers in Wisconsin and Spain have developed a “virtual camera” that can see around corners, an important breakthrough with a wide array of possible applications, from helping searchers locate people lost in caves or rubble to allowing police outside a building to pinpoint an active shooter inside.

Scientists from the University of Wisconsin-Madison and the Universidad de Zaragoza in Spain reported their results Monday in the journal Nature.

The new technology has the promise to help military units engaged in house-to-house fighting in a city such as Mosul in Iraq. Lack of vision in this kind of operation can lead to friendly fire death rates as high as 30 percent, said Andreas Velten, the 39-year-old UW professor who led the research. “That’s why it’s so hard to liberate a city.”

Velten, a professor in the department of Electrical and Computer Engineering and Biostatistics and Medical Informatics, said “virtual camera” equipment also offers the U.S. space program a potential way to peer inside caves on the moon and Mars. NASA is interested in the moon caves as possible locations where astronauts could live. 

The new study is part of a $6 million project paid for largely by the U.S. military and NASA.

“I think this is very important work,” said Jeffrey H. Shapiro, a professor of electrical engineering at the Massachusetts Institute of Technology. “Velten’s team has shown remarkable results in that Nature paper.”

Velten and his team used thousands of pulses of laser light reflected off walls and other surfaces. The individual particles of light, called photons, spread out in a broad array, bounce off surfaces and return to sensors attached to the camera.

Mathematical algorithms then take all of the data that returns to the camera and assemble it into a detailed picture of the area that’s hidden from the human eye. 

So far, the technique described in the new paper has only been used inside the lab. But Velten, who also works for the Morgridge Institute for Research, said, “we want to take it outside the lab, make it smaller, make better computer hardware. We want to be able to look at larger scenes.”

Another innovation building on this work will be a system allowing people to videotape scenes around a corner. “We’ll be able to do that pretty soon,” Velten said, estimating that it might take another year or so to develop.

Other potential uses include helping firefighters detect whether people are inside a burning building and helping the military determine when a hole is actually part of a tunnel. 

Scientists have been working for years to develop this kind of technology, but the effort gained momentum in 2007 with publication of a scientific paper suggesting that it would be possible to use the methods employed by Velten’s team.

Scientific groups at Stanford, Boston University and MIT in the U.S., as well as groups in Britain and Germany have been working on the problem of seeing around corners, said Shapiro at MIT. Shapiro's group has been studying the theoretical side of the problem.

Previous efforts have produced cameras able to see only one or two objects in a hidden area. The new technology allows viewers to reconstruct in detail a room they cannot see from where they are standing.

"What Velten (and his colleagues) have shown is a new approach to the detection,"  said, Daniele Faccio, a professor at the University of Glasgow in Scotland and one of nine chairs in emerging technologies at the Royal Academy of Engineering.

"The result is a very convincing architecture that allows them to image very large scenes in real time in remarkable detail."

Xiaochun Liu, a UW graduate student and the first author on the Nature paper, said his first impression upon witnessing an early demonstration of around-the-corner imaging at MIT was, “(It) seems like dark magic.” The first author of a scientific paper is often the student or postdoctoral researcher who does the majority of the lab work.

Liu said he worked on the theories and algorithms that went into the study. The work was so intense and he felt so fired up by it, Liu said, that he stood for two days straight calibrating the cameras in preparation for the first successful experiment.

“I don’t think there’s a last word yet on how best to do these things," Shapiro said of cameras that peer into hidden areas.

"But there has been substantial progress.”