Amazon Wants to Cocoon You With ‘Ambient Intelligence’

The company’s new smart gadget uses radar to track your breathing while you sleep. It’s part of Amazon’s plan to weave its products invisibly into your life.
Amazon Wants to Cocoon You With ‘Ambient Intelligence
Photograph: Halo

It’s an ominous-looking disc that sits on a night table, resembling a tiny satellite dish. It uses radar to monitor your movements while you sleep, combining that data with information about your bedroom—temperature, humidity, and brightness—to measure the quality of your sleep. Around the time you’ve set the alarm—at the instant you enter a lighter stage of sleep—it brightens its semicircle of soft LED light to ease you gently from your slumbers. And this most intimate companion is made by Amazon, one of the world’s biggest—and to some scariest—companies.

Meet Halo Rise, the latest contribution to Amazon’s mission of creating a persistent yet almost undetectable computational cocoon that monitors, listens, and fulfills your every whim and need. It’s one of a bevy of products announced at the company’s annual hardware event today. The event is further confirmation that the ecommerce giant is now a giant hardware provider, only 15 years after its Kindle launch was mocked as an example of a software company veering out of its lane.

While the variety of devices launched or updated are as disparate as a home-roaming robot and a television set, Amazon’s hardware chief, Dave Limp, is emphatic that they all serve the company’s grand mission—constructing a silent infrastructure of ambient intelligence to do just about anything for us.

Think of it as the opposite of the metaverse: Instead of asking people to venture into an artificial virtual world, Amazon wants to weave its computing products, whether it builds them itself or licenses the technology to others, into our homes and cars, to the point where the technology seems invisible, even as it switches on gadgets, alerts us to home intruders, and figures out what we want to watch or read next.

“Our strategy differs from that of the others,” says Limp, slyly referring to Amazon’s big tech competitors. “Our view isn't that you start with the phone and emanate outwards. Instead you start with intelligent devices that are placed throughout the house or the car, that when they interact together, they act better. They're always there.”

Amazon's Halo Rise uses radar to monitor breathing patterns.

Courtesy of Amazon

Limp does admit that this strategy became a necessity when Amazon’s effort to take on Apple and Android with its 2014 Fire Phone flopped. But as he tells it, that failure became liberating. “We weren’t planning failure,” he says. “But sometimes a phoenix does rise—it allows us to accelerate our focus in the home. So after we wound down the phone, our team sat in a room and said, ‘OK, what are we going to do?’ And what came out of that was a mission statement which we have roughly to this day.” The linchpin was the Echo smart speaker, powered by the conversational Alexa interface, evidence of the company’s deep investment in AI for natural language processing.

The next step is moving from language to intent. “One of our long-term objectives is to get customers to talk to Alexa less,” Limp says. In 2020, he says, about 20 percent of Alexa’s tasks were proactive, meaning that the system executed by predicting what the user wanted, not via voice command. By the next year, the percentage rose to a quarter of the system's actions. Now, he says, it’s approaching a third. And when it comes to back-to-the-future-esque daily routines that people grow accustomed to via Alexa—like a morning regimen of waking up to music, having the lights go on, and starting the coffeemaker—those happen automatically nine times out of 10.

Still most of the Amazon ambient universe now depends on voice commands prefixed by the familiar wake word “Alexa.” (Limp says his children think it weird when they visit a neighbor and find that the house doesn’t listen to them.) But, like super-sensitive animals like wolves or spiders, Amazon is increasingly stuffing its devices with an array of nonverbal sensory technologies that pick up changes in light, humidity, temperature, and motion, and improve the previous audio and video capabilities.

The Halo Rise uses all of those to determine whether a boudoir is optimized for sleep. But Amazon is installing improved sensors and accelerometers in new versions of the Echo Dot as well. It’s adding advanced features like 3-D motion detection, color images, and a “bird's-eye” aerial map to its Ring doorbell system. The company’s new products include the Fire TV Omni QLED, which doesn’t have a camera but has sensors built into the screen to adjust to the lighting in a room. When it’s not streaming video, you can use it as a dynamic picture frame to show art, photos, or the widgets you have on your phone. Or it can show you who’s at the door, via your Ring camera. Alexa, of course, is built in.

Photograph: David Ryder/Bloomberg/Getty Images

Also, Amazon’s roaming home robot Astro can now detect pets and send you a video clip of Rover or Tabby if it encounters them on its travels around your home. Amazon is also opening up the Astro operating system to developers, beginning with students at several universities, to devise their own uses for an autonomous robot.

There’s also a lot of sensing in the latest version of the Kindle, which might have been the coolest device Amazon unveiled. On the ebook reader’s 15th anniversary, Amazon is releasing the first version, Kindle Scribe, with a stylus. With a 10.2-inch screen, the $339 device is supersized compared to previous versions of the Kindle, reminiscent of the short-lived Kindle DX. (For $30 more, you get a stylus with a virtual eraser.)

While the Kindle is designed to isolate the owner from the world of devices connected to each other, it was Amazon’s first effort to make a hardware tool that “falls away” so you hardly know it’s there. Amazon founder Jeff Bezos said that the measure of its success would be that you would forget you were using a device and simply be lost in prose. Just as the original Kindle wanted to make you feel like you were reading a plain old book, the Scribe wants to make you feel you’re writing on a piece of paper, even making scratchy sounds for verisimilitude. “It feels just like writing on real paper,” says Kevin Keith, vice president of Amazon devices.

To extend the range of all those ambient devices—as well as laptops, phones, tablets, and other connected appliances—Amazon is adding technology from its Eero Wi-Fi devices to recent models of the Echo Dot smart speaker. Updating past products to perform new tasks is another practice that Limp says gives Amazon an edge over competitors. “I’m a little frustrated that our industry spends a huge amount of money and time trying to convince customers that they always have to upgrade their consumer electronics,” he says. One example of this is Amazon using a software update to upgrade its current high-end smart speaker, Echo Studio, to stream music with spatial audio.

One problem with Amazon’s ambitious push toward an ambient world is that it forces users to choose between competing corporate ecosystems. If you want your Fire TV Omni QLED to show family pics, they must be stored in Amazon’s photo app—the feature doesn’t work with Google or Apple photos. Limp blames this on his competitors, too. “I can't speak for Google or Apple, but I can speak for us,” he says. “We absolutely believe that all these ecosystems do need to interoperate. Sometimes, other companies don't want to get deals done, but over time, I think it'll all work out.”

And then there’s the fear that Amazon itself is so powerful that it’s the wrong company to establish an all-encompassing ambient intelligence in our homes and cars. The FTC wants to sue the company and is demanding that Bezos and current CEO Andy Jassy testify in its investigation. The Senate is considering a bill to constrain Amazon. None of this has discouraged it from trying to weave its technology deeper and deeper into the everyday lives of consumers.

That project is now personified by today's announcement of Halo Rise, the $140 “no-contact bedside sleep tracker, smart alarm and wake-up light” that picks up signals from your bedroom that even the partner sleeping beside you won’t notice. (To get the best use of that data to improve sleep, users must subscribe to the premium Halo app; the first six months are free.) Amazon had already been in the popular sleep tracking-game with its wearable Halo wristband, but a dedicated radar-equipped device is enough to cause stress-breathing anomalies among its critics. I asked Limp whether there might have been any hesitation before developing a product that seems so intimate. Weren’t the Seattle executives wary of releasing a product that causes people to say, “WTF, Amazon is using radar to monitor my sleep?”

The answer was no. Like every other Amazon product, Halo Rise was first proposed via a six-page pitch document that outlined a press release for its future launch. “Amazon is built on customers’ trust—it’s not ‘What would a pundit think?’” Limp says. Amazon had already been working with low-powered sensors, in this case radar, and it figured out that without ever touching anything, you could get a very accurate signal of a person’s breathing and translate that to sleep. “So how do you do it in a safe and private way?” he says. “We chose not to add a camera, and we didn’t need a microphone. Let's make sure that the data is encrypted, in transit and also at rest.”

Amazon’s director of new products, Michael Fisher, says the Halo Rise has other safety features, like a one-button push that turns off the monitoring. All data is wiped from the cloud after 10 days. Amazon has even accounted for the concern that its perceptive little disk might take note of nocturnal boffing. When a breathing pattern of the designated sleeper indicates that a little night music is afoot, it cools its sensors until a sleep pattern emerges—the same silent treatment it employs when the person is reading or watching TV, activities that Amazon says have their own distinctive breathing patterns that Halo Rise doesn’t track. “The device focuses on recognizing you are sleeping,” Fisher says.

Of course, it’s a stretch to think that the Halo Rise will achieve the ubiquity of popular Alexa devices like Echo or Dot. Perhaps it will go the way of the Ring Always Home Cam, the in-home multi-camera security drone announced by Amazon in 2020 that has yet to ship. But regardless of whether people choose to retire and rise with Amazon’s odd new sleep monitor, the company is fixated on producing and enhancing hardware to populate your environment with Amazon sensors, cameras, microphones, and AI, all done in a way that you hardly know it’s there.

And, by the way, Limp says that that Amazon’s home-patrolling drone isn’t dead. Hundreds of test prototypes are already in homes, he says, collecting data that will help the company perfect the device. “I’m still a believer in the product,” he says. At Amazon, it seems, ambient intelligence has no bounds.

Updated 9-28-2022, 5:20 pm EDT: The Astro robot can detect and capture video pets when it comes across them, not actively find them.