Haikubox gives citizen scientists a tool to track birds through the sound of their song

A Sarasota-based electronics company named after a breed of sea turtle has developed an electronic listening device that – when combined with an artificial intelligence database developed by Cornell University – can help transform backyard birders into citizen scientists.

David Mann, a former University South Florida professor, his wife Amy Donner and two other scientists developed and have marketed aquatic listening devices – or hydrophones – that record dolphins, manatees and fish along the Suncoast.

Mann founded Loggerhead Instruments, Inc. in 2004 to make his underwater recorder and more recently wanted to expand into a consumer product.

Previously: Former Sarasota Art Museum director takes new post in St. Petersburg

And: Tiffany windows and lamps to inspire Selby Gardens exhibit

Mann already had a working relationship with Holger Klinck, director of the K. Lisa Yang Center for Conservation Bioacoustics at the Cornell Lab of Ornithology.

Cornell has a library of birdsong dating back to the late 1920s, and the lab has been working on an artificial intelligence algorithm that, through an avian version of Name That Tune, could match songs with the birds that make them.

Mann had been fascinated by birds since his youth, growing up in Syracuse, New York. The two men brainstormed, Klinck brought up Cornell’s BirdNET sound ID app, and Loggerhead’s backyard recorder – the Haikubox – was born.

“First of all it’s a really great collaboration for us; we’re a university, we’re not set up to develop and market products very well. Our expertise is to provide the tools for a project like that,” Klinck said.

What’s a Haikubox?

The Haikubox is a 4-inch by 6-inch by 2½-inch white box with a microphone and a wireless transmitter, with the ability to transmit the sound it hears into the cloud.

Mann chose the name because of nature imagery often present in haiku poems.

Donner noted that haikus often show a careful observation of nature and capture a moment in time.

“Which essentially is what each Haikubox does,” she added..

The homeowner/citizen scientist must plug the box into an outlet and have wireless internet connectivity for the Haikubox to work. It records 24 hours a day.

Once the bird sounds are captured, the Cornell-designed artificial intelligence program listens to it and uses that sound catalog to identify the birdsong.

“The thing is basically sitting outside all the time,” Mann said. “It’s running this neural net that does the identification and when it gets one it sends it out to Cornell.”

Spoiler alert: In a typical Sarasota County backyard, there’s a good chance the sound belongs to a Tufted Titmouse.

The Haikubox owner can learn all about that either through the Haikubox smartphone app or the web site, https://www.Haikubox.com.

The boxes can be purchased on the web site for $399 with a prepaid membership; or the hardware can be bought separately for $190 and a subscription to the service for $59 a year.

A free demonstration account can be downloaded and activated at https://listen.haikubox.com.

After a 50-person Beta test, Haikubox sales started in April.

About 75 boxes have been sold and anyone with an account can listen in on the sound of 129 boxes.

Haikubox owners do not have to share their backyard sound with the ornithology lab, so those units aren’t visible on the app either. But where’s the fun in that?

Wait, does it record everything?

Dogs bark, children squeal and people talk. Yes, all of that can get picked up, but in theory, the AI sifts through it.

“We attempt to filter it out,” Mann said. “We detect cardinals and woodpeckers and things and we also detect humans.

"If we detect humans, we don’t upload anything at all, so we reject it,” he added.

A baseball hitting a bat may sound like a bird and be recorded by the box but the citizen scientists – nicknamed Haikuligans by Mann – can use the computer or web app to tell the AI it's wrong.

“We’re testing some in Europe now, too, and we noticed in Spain, when people are speaking Spanish it’s false detecting,” Mann said. “We don’t want to deal with human speech.”

“Early on kids screaming would sound like a lot of the wading birds – we try to whack-a-mole those,” he added. “You listen to it and then you listen to what the bird sounds like and think, ‘I can see that.’”

Just in case human speech gets by the first algorithm, the lab at Cornell runs the birdsong through a second algorithm that is also designed to filter out human speech, Klinck said, since Cornell follows specific privacy laws that forbid the school from storing human speech recordings.

Those false detects also help Klinck and Cornell fine-tune the AI to hone in on real birds.

The algorithm can now identify 3,000 bird species, including the most common ones in the United States and Europe. Klinck noted that there are about 10,000 bird species worldwide.

A gateway to sights and sounds

The Haikubox app links seamlessly to data gathered from the ornithology lab’s other citizen science project, eBird, available at https://ebird.org.

With eBard, people visit their backyard or go on a walk, note the time and place they spot a particular bird species and then send it to Cornell.

“They send us the data to answer similar kinds of questions about where and when and potentially how many birds are in a specific area,” Klinck said.

But that’s just a snapshot, whereas Haikubox can provide a longer view.

“With a Haikubox you monitor 24/7 basically,” It’s detecting bird species continuously and gives us a really good data set to work with.

“With owls for example, people are not out and about during the night when owls are active but our box would pick them up because owls vocalize,” he added.

Contributing to science

Karen Willey, the retired manager of the Sarasota Audubon Nature Center, was a Haikubox Beta tester and took the box with her when she moved to Asheville, North Carolina, in May.

“I’m learning my birds,” Willey said. “What I love about it is you can have it notify you when there's a new bird."

In downtown Bradenton, Willey said the most common bird her Haikubox picked up was a Northern parula – a small yellow warbler.

“We had an oak in our backyard and a lot of Spanish moss so I believe they were nesting,” she added.

Willey enjoys looking up new bids and likes the idea that the data she’s collecting will help scientists.

All the data collected will help the lab develop a better machine but also track bird migration – where and when they occur in certain parts of the country and how that changes from one year to the next.

“The really neat thing about these boxes is if people keep them going for several years we can actually identify changes in regions,” Klinck said.

The more citizen scientists buy and use the Haikubox, the more data scientists will have for their studies.

“If you look at the regional scale, it doesn’t take that many boxes,” Klinck said. “But obviously we’re always interested in continental scale monitoring, so the more the better.”

Klinck said that with eBird there’s already a clear correlation between submissions and the population density.

“Along the east and west coast, where a lot of population centers are in the U.S. we get a lot of observations but if you go to the Midwest or the central U.S. the density of observations is, by far, not as high,” he added. “Those are valuable areas for us to target to get Haikuboxes out, to supplement the data we get from our eBird users.”

So far, Mann is targeting homeowners for the Haikubox but Klinck can easily see them being valuable in schools.

“How cool would it be? You could install those at schools, for example, and the kids could come in in the morning and see what birds were detected,” he said. “There’s lots of interesting ways that technology could be used in raising awareness beyond just the science I want to do and raise awareness of the challenges those animals face.”

Earle Kimel primarily covers south Sarasota County for the Herald-Tribune and can be reached at earle.kimel@heraldtribune.com. Support local journalism with a digital subscription to the Herald-Tribune.

This article originally appeared on Sarasota Herald-Tribune: Sarasota company helps identify birds through Haikubox