By 
Remember how mind-blowing it was the first time you booted up MS Paint and scribbled some garbled mess of colors in there? Microsoft Research has managed to recreate the magic in a whole new way by making your living room the canvas, instead of a white patch on your screen.
Named SemanticPaint, tech works by using a depth camera (like the Xbox One's Kinect) to scan in a 3D "picture" of the room, and then using a tablet or heads-up display (like Microsoft's Hololens) to show the painter the recolored version of the room they're in. All you have to do is touch the floor or a chair or a table, and give a voice command telling the software to label it.
But this colorization is more than just running around clicking things with the paint bucket tool for the fun of it; when you label these objects, the system actually takes note of how they're shaped and what they look like. The result is that once you label a chair or two as green, the software is smart enough to label similar chairs correctly with no further help from you, recognizing not only how they look, but where they end and other objects begin. Ditto tables, coffee mugs, bananas, whathaveyou.
The obvious joy to this kind of tech is just slathering virtual paint all over your grimy couch for fun, but it's easy to see the potential for it as a tool for home improvement, or for easily scanning larges spaces into virtual reality and automatically indexing what's inside of them. Microsoft's tech works online and in real time, so someday it could be feasible for Kinect-wielding Hololens-wearers to map entire buildings in just a matter of hours. Just so long as they don't mind the initially gaudy color scheme.
Source: Microsoft Research
