Doppler Labs And The Quest To Build A Computer For Your Ears

I’m trying to listen as Doppler Labs cofounder and CEO Noah Kraft walks me through his company’s new smart earbuds over the din of a crowded restaurant, but the cacophony of chattering diners and clanking silverware is overwhelming. I can’t make out a word. Then, all of a sudden, the background noise disappears. Kraft’s voice comes in loud and clear.

It feels a little magical—even though we’re not actually at a restaurant. I’ve been getting a demonstration at Doppler’s San Francisco headquarters, and all of that background noise was a simulation, pumped into my ears—and then muted—via a cobbled-together tangle of earphones, microphones, circuitry, and other components that are strapped to my head and draped around my neck. (Wearing the contraption, I look a little like Frankenstein, Kraft cheerfully informs me.)

In November, Doppler plans to take this trick out of the lab and ship a pair of sleek, wireless earbuds embedded with the same technology—designed to allow anyone, anywhere to neutralize the hubbub of a real restaurant. Paired with a smartphone, the $300 Here One buds will stream music and answer calls. They will also be capable of remixing your world’s audio in a variety of subtle and sophisticated ways, muting specific sounds—a baby’s cry, your talkative colleagues—and using cues such as your location to suggest appropriate filter settings. The buds will be able to add effects such as reverb to live music, making a neighborhood club sound more like a concert hall. If you are at a concert hall, they’ll be able to overlay the music with streaming commentary, or even let you listen in on a performer’s backstage conversations. You’ll even be able to tune into a ballgame while you’re on a run without blocking the sound of oncoming traffic.

What this all adds up to is a new form of augmented reality—Doppler likes to call it “layered listening”—that has a shot at being both practical and socially acceptable. Google Glass flopped, in part, because wearing a computer display in front of your eyes turned out to be an awkward, off-putting way to interact with the rest of society (a lesson for Magic Leap’s upcoming headset and Microsoft’s Hololens). But people already walk around with buds in their ears—why not exchange them for a pair that’s designed to enhance the world, rather than simply tune it out?

Kraft and Doppler’s cofounder and executive chairman, Fritz Lanman, are leading a 65-person team that’s scrambling to launch Here One in time for the holidays. Their ambitions, however, are lofty enough to keep the company busy for years. “We want to be the future of computing,” Kraft declares. “If Microsoft put a computer on every desk, and Apple put a computer in every pocket, at Doppler Labs, we want to put a computer, speaker, and microphone in everyone’s ears.”

Now Doppler just needs to integrate a complex array of hardware and software technologies into earbuds that are a mere 20.1 millimeters tall—less than the diameter of a nickel. Over the summer of 2016, as the company entered the home stretch of its race to ship Here One, I followed along during numerous visits to its headquarters.

Even if Here One is everything that Doppler wants it to be, it’s not a given that consumers will buy it in droves. Research firm IDC thinks that the overall wearable-computing business will grow at a 20% annual clip over the next four years, but it expects that most of the action will happen on the wrist, in the form of smartwatches and fitness bands, such as the Apple Watch and Fitbit. Its forecast lumps earwear such as Here One and Bragi’s Dash into a catch-all “other” category that it expects to account for a measly 3% of the market in 2020. That hasn’t stopped Doppler from racking up $50 million in funding, from major venture-capital firms as well as music-industry bigwigs such as David Geffen, Quincy Jones, and “Uptown Funk” creator Mark Ronson.

For Doppler investor Kevin Efrusy, the company’s technological aspirations were only part of what made it attractive. “What was more interesting, honestly, was how they defined the problem,” says Efrusy, who, as a partner at Accel, led investments in Facebook and Groupon. “Noah and Fritz looked at the world through a different prism. It wasn’t building a better headphone. If it was, I wouldn’t be excited. They said, ‘Let’s not just think about music or headphones, let’s think about your overall experience in the world around you, and how you make that a whole lot different and better.'”

You only have to spend a little time with the red-bearded, 29-year-old Kraft to understand why he’s been successful at pitching the promise of his company and product. He’s an infectiously enthusiastic salesman, the kind of person who grows only more animated when he’s short on sleep—which, as the CEO of a startup working furiously to hit its product-launch timeline, he often is. “I lose my voice every day by about 5 p.m.,” he says.

San Francisco doesn’t exactly suffer from a shortage of hyper-energetic young tech CEOs, but Kraft, unlike most, is not a born geek. After studying international relations and history at Brown, he stumbled into the movie business. In 2012 and 2013, he was a producer on Bleed for This, a boxing drama executive-produced by Martin Scorsese that will finally hit theaters in November.

That delayed gratification “is one of the reasons I had to leave film,” Kraft confesses. “We shot this film in 2012 and 2013, and then we handed it off to a studio that was like, ‘We’ll release it when it’s ready.’ I can’t do that.” One reminder of his Hollywood past is the ostentatiously decorated championship belt signed by Vinny Paz, the real-life boxer who inspired the film, that’s framed in Kraft’s office. Another is the fact that when he wanted to capture Doppler employees talking about their work for a promotional video, he directed the interviews himself and presented them in lush black and white.

More than Kraft’s film career, it was his lifelong love of music that led him to tech. The founder of several rock bands over the years, Kraft took time during his film-producing stint to work on a friend’s electronic-music tour, helping to pick clubs and promote shows. As the tour went from venue to venue—usually small rooms with subpar acoustics—he was struck that his friend’s music sounded different in every space. “What if we could curate how we hear the world?” he recalls asking himself. “What if we could give ourselves more control?”

By 2013, Kraft had the glimmer of a smart-earware concept in mind, but knew that he couldn’t make it a reality on his own. He floated it by a music executive friend, who put an acquaintance in the tech industry on speakerphone to bat it around.

That tech guy was Fritz Lanman, 35, a former Microsoft executive. Lanman had helped engineer Microsoft’s $240 million investment in Facebook in 2007—a move that prompted plenty of skepticism at the time, but soon looked prescient.

After the Facebook experience, it dawned on Lanman that, rather than channeling Microsoft’s money into promising companies, he might do a lot better investing his own. So he reinvented himself as an angel investor. In the years that followed, he became a walking encyclopedia of startup knowledge by working with more than 70 companies to date, including Pinterest and Square. He also cofounded Livestar, a recommendations engine acquired by Pinterest in 2013.

When Kraft and Lanman met, Lanman had just moved to New York and, along with his investing activities, was serving as executive chairman of two startups based there, ClassPass and Dwnld. A meeting over brunch turned into hours of discussion as the two riffed on Kraft’s concept for a new kind of augmented reality. “Everybody at the time was focused on the eyes, because Google had launched Glass,” Lanman says. “Noah was just so focused on the ears, because he’s a musician.”

As they explored the possibilities of a wearable for the ears, the new acquaintances “kind of mind melded,” recalls Lanman, who, in contrast to Kraft’s vibe of exuberant wonder, has more of a seen-it-all demeanor. He was struck less by Kraft’s lack of tech-industry bona fides than his potential: “I’ve probably met 800 entrepreneurs. You get to pattern-match the ones who won’t take no for an answer, and who will be able to withstand the brutal downtimes in startup lives that are inevitable.” Kraft, meanwhile, saw Lanman as a partner whose technical chops and startup background could help make his brainstorm a reality: “I had no idea how to build anything, let alone a deeply technical hardware product.”

Smitten though he was with Kraft and his concept, Lanman had limited time for another startup. “Originally I was trying to just give him some money as an angel investor,” Lanman says. “He ended up talking me into cofounding the company with him.” Retaining his executive chairmanships at ClassPass and Dwnld, Lanman took on the same title at Doppler. He describes the position as one “that gives me the freedom to be as involved or uninvolved as I need to be, or the company needs me to be.”

As Doppler started to staff up, Lanman’s credibility and connections helped recruit high-powered talent. And the more good people the company found, the easier it was to hire even more of them. Hardware product manager Andrew McIntyre, an early recruit, had worked on Amazon’s Fire tablets. According to Kraft, when Doppler had only around a dozen employees, McIntyre walked into his and Lanman’s office, and declared, “‘My boss’s boss’s boss at Amazon [has been] running Nike Fuelband. He just left. We should hire him.’”

It took six months of badgering by Kraft until McIntyre’s boss’s boss’s boss, a literal graybeard with decades of experience named Kennard Nielsen, agreed to became Doppler’s VP of product. Once he did, he brought even more of his former Amazon colleagues on board and became, as Kraft describes him, “our Obi-Wan.” The company has even printed cards with his five leadership principles—such as, “We live at the leading edge, and we innovate simple, brilliant technology”—for employees to carry in their wallets.

Gints Klimanis, director of audio and digital signal processing engineering, worked at Apple on multiple generations of iPods, iPhones, and iPads before leaving to join Nest Labs, where he toiled on its smart smoke detector. His efforts there involved “a huge amount of audio engineering heard only when something’s really wrong at your house,” he says. Then the Doppler opportunity arose. “When offered the chance to work on something sexy like reinventing hearing and making the world sound more beautiful, how could one resist?”

Other staffers were recruited from companies such as Dolby, Dropbox, Sony, Palantir, Qualcomm, and Sennheiser—a significant feat for a small startup without a particularly high profile. “It wasn’t too hard,” Lanman says. “The good news is our space is exciting and intriguing enough. In-ear computing is at the precipice of becoming a big thing.”

One thing that was hard, originally, was figuring out where Doppler should be headquartered. The company started out in New York, where Kraft and Lanman lived, even though the majority of employees were in San Francisco, where engineers with audio backgrounds are far more plentiful. “It became clear in Q3 of last year that the distributed office was one of the biggest inhibitors for us moving fast,” says Kraft. “Over drinks, I turned to Fritz and said, ‘I think we need to double down on SF.’” Lanman agreed: “It’s hard not to be in San Francisco for HQ if you’re trying to build a transformational tech company, if you want to eventually hire thousands of people.”

When the company decided to shutter the New York office, 14 of 20 staffers relocated to San Francisco, including Kraft, who, pre-Doppler, had spent only a handful of days in the city. Now the vast majority of the company’s employees work in a pair of small, adjacent buildings on a side street in San Francisco’s startup-heavy SOMA neighborhood, with Kraft zipping back and forth, up and down stairs, more or less continuously. He says that this proximity has been a boon: “There are people here who six months ago I barely knew who now are some of my closest confidantes as we move this company forward. You can’t build those types of deep bonds without that face time.”

Still, when I ask him whether he’s fully acclimated himself to his new environs, Kraft doesn’t put on a brave front. “SF is alien to me,” he says. “I’m learning it little by little, but I’m definitely not part of it.” He talks wistfully of Brooklyn music clubs and muses that if Doppler does as well as he hopes, it might need a New York office again in two or three years.

For all the value of Lanman’s tech-industry experience in jumpstarting Doppler, it was Kraft who championed the company’s thoughtful go-to-market strategy, which involved shipping progressively more advanced products and learning about design, manufacturing, and marketing along the way. Doppler’s first offering, 2014’s Dubs Acoustic Filters, weren’t even a piece of consumer electronics: They were just $25 earplugs designed to look stylish and make music sound good, but quieter. Developing and selling them helped the fledgling company begin to think about the challenge of getting people excited about something you stick in your ears.

“Earplugs were always this strange, orange-foam thing,” says Angus Brown, who’s responsible for Doppler’s brand partnerships. “And it was kind of dorky to be putting them on at a concert. We wanted to create something that was a little bit more cool and felt like a lifestyle and fashion statement.”

A year later came the Here Active Listening earbuds, which introduced the first version of Doppler’s audio technology and allowed users to perform tricks such as tweaking how live music sounds and minimizing the engine noise on an airplane. Rather than attempt a full-blown retail rollout, the company offered the earbuds to backers of its Kickstarter campaign, which raised $635,000. Then it partnered with Coachella to sell them to attendees at this year’s music festival—in part to make clear that Doppler’s goals go far beyond competing with Beats and Bose.

In a tent at the event, the Doppler team walked users through the earbuds’ fancy sound filters, showing them how to block out the sound of the crowd—or the entire festival—with a swipe of a finger. “We had 4,000 people wearing a tech product in their ears,” says Kraft. “If it had been a headphone, that wouldn’t have happened.”

Oddly enough, the biggest missing features in Here Active Listening are the ones you might assume even the most mundane earbuds would offer: the ability to make phone calls and stream music and other audio from a smartphone. They’re coming in Here One, along with the next generation of Doppler’s technologies for suppressing unwanted background noise and customizing the sounds you do want to hear. The company is also doing deals with content partners such as museums, sports teams, and music venues, to deliver audio commentary and other enhanced experiences. (It’s keeping the specific providers under wraps until closer to Here One’s release date.)

One day in mid-June, when I arrive at Doppler headquarters for a visit, Kraft greets me by saying that he has a breakthrough to show off. Hunched over a workbench, several staffers are streaming audio wirelessly from one earbud to the other for the first time, using an Office Depot ruler to position the buds 22 cm apart—the width, more or less, of a human head.

The feat matters in part because Doppler chose to accomplish it using near-field magnetic induction technology, a decision that caused more work for the company but provides better battery efficiency than Bluetooth and lowers the risk of latency issues that can leave audio out of sync with video. “Frankly, we always knew we could do [Bluetooth] as a super-backup,” Kraft says. “But my mantra has been that we’re not shipping that.”

The fact that Doppler’s first successful test of bud-to-bud streaming came only around five months before it plans to ship Here One reflects the breakneck pace of its product development. The company is building a complex hardware product that incorporates an array of custom technologies—and trying to move with the speed of an internet company.

In September 2015, key stakeholders spent two days drawing up a list of the features they wanted Here One to offer. “The engineers always want to do, technically, the best job that they can do,” says director of product Matt Jaffe, who formerly worked at Dropbox and brought some of its product-development philosophies with him. “The product managers come in and say, ‘Let’s do what we can, in the time that we have.’”

In the case of Here One, the available time was defined by the need to get the buds into stores before the 2016 holidays. That imperative overrode the exact capabilities that they would ship with.

In fact, what exactly buds will do at launch is still in flux when I visit the Doppler offices in June. “We have a plan A, a plan B, and a plan C,” director of software Nitin Khanna confides. “We have confidence that we’ll have something to ship.” Kraft interjects to put a more upbeat gloss on that plan: “What he’s trying to say is, ‘We have the utmost confidence we will ship a great product—on time.’”

As with most consumer electronics, a sizable chunk of the process of bringing Here One to market—from sourcing components to final manufacturing—is happening in Asia. Unlike most startups its size, however, Doppler has its own feet on the ground in China, in the form of eight employees based in a Shanghai office. The arrangement helps the company wring as much productivity as possible out of every 24-hour period. “Every day, overnight, we have this back and forth, where I’ll work the day shift and they work the night shift,” says Jaffe.

Early this year, Doppler made a big decision that, at first blush, sounds reckless given its timetable for shipping this year: It dumped the contract manufacturer it had been working with and started over with a different partner. The move was particularly momentous given that there weren’t many companies in the world up to the job of cramming advanced components into devices as small as the Here One buds.

According to Nielsen, it would have been more risky to stick with the initial manufacturer. “The first one couldn’t keep up with us,” he says. “I saw things coming that would have prevented us from scaling, and very possibly from shipping this year. It was hard. It was gut-wrenching. Logically, it didn’t make a lot of sense, but it’s starting to pan out that it was a good decision.”

Even Doppler’s more intrepid manufacturing partner has to hustle to keep up with the project’s tight schedule, so the startup is trying to teach some of the things it’s learned about building hardware at an internet pace. “They just aren’t moving fast enough, so we’re helping them to move faster,” Nielsen says.

For all the intricacies of engineering and manufacturing computers small enough to wear in your ears, Here One will be defined at least as much by software as hardware. Crucially, that software will be upgradable. “We are not designing a headset that you buy once, and it doesn’t change after that,” says Khanna. “Almost every aspect of this hardware should be customizable. It should allow us to add features which we hadn’t even thought of when we selected the components.”

To set itself apart from other noise-canceling earware, Doppler wants to get increasingly better at eliminating specific sounds, and if it’s going to turn down the volume on distractions such as sirens and wailing infants, it needs to be able to identify them on the fly, in a multitude of variations. It’s not just babies that have distinct voices; European sirens sound nothing like their U.S. counterparts. That’s a formidable artificial-intelligence challenge that has not been the subject of much research, so a Doppler team is collecting vast quantities of real-world audio for analysis. “Where speech recognition has thousands of published papers, environmental audio classification has dozens,” explains senior software engineer Jacob Meacham as I chat with him in a nook at Doppler’s office dominated by a subwoofer so big it doubles as a couch. “On our second day, we were at the forefront.”

Doppler is using the audio it has collected to train computers to identify particular classes of sounds. When I get a demo in early June, the company hasn’t yet gotten the technology working within its smartphone app, but a PC-based version can identify babies and sirens as well as distinguish between male and female speakers. Some of the time, that is—it’s still very much a work in progress.

Doppler may still be in the early days of its research, but the long-term implications of it go far beyond Kraft’s initial yen to make live music sound better. Here One isn’t a hearing aid; if it were, it would be subject to a torrent of medical-device regulations that the company is happy to avoid for now. “If you’re in real need of a medical device, you need to go buy a hearing aid,” says Lanman. “That’s the fact of the situation, and it’s a regulatory decision about who needs a medical device and who doesn’t.”

Still, for a startup that emphasizes that it isn’t building a product to compete with hearing aids, Doppler spends a lot of time thinking about what its technology could do for people with hearing loss. Two key members of the team—Nielsen and Kristen “K.R.” Liu, director of accessibility and advocacy—are hearing impaired and enthralled by the possibility that Doppler’s technology could help others by amplifying conversations and suppressing background noise. “When I put hearing aids in, I hear,” says Nielsen. “But the sound is so compressed that it’s very flat and tinny. What happens when you put Here in is, we don’t compress sound. We do real-time, full-speed audio processing. And so what I hear is very warm and rich.”

“I’m 38 years old, I’ve worn a hearing aid my entire life, and all of a sudden [with Here One], I’m hearing things I’ve never heard before,” marvels Liu, who, in addition to having 20 years of experience in the consumer electronics field, is an advocate for the hearing impaired and serves on the board of the Hearing Loss Association of America. If Doppler can make in-ear computing fashionable, she hopes it might reduce the stigma associated with admitting that you don’t hear well. “Noah doesn’t have issues hearing,” she says. “I do. But we both have a passion to change that conversation. Let’s change the stigma around what it may mean to have different levels of hearing, and make it a socially acceptable thing to put something in your ear.”

On June 28, Doppler formally announced Here One and began taking preorders. Shortly afterwards, when I ask Kraft how the reveal went, he’s even more ebullient than usual. “The launch went great! It was awesome!” he exults. “We got a lot of press, which is great. We got a lot of traffic, which is great. More importantly, though, people got the core foundational idea.”

“Until this announcement, we were a Kickstarter company,” he continues. “We can now actually talk about our vision, which we haven’t been able to do outside of very closed doors for years. And the joy of it is, we just showed the tip of the iceberg.”

I ask Kraft whether he’s concerned that huge, resource-rich companies such as Amazon and Microsoft will be intrigued by the kind of in-ear computing that Doppler is pioneering and introduce competitive products. “The day after we announced, without exception,” he says, “every single one of those guys wrote us a nice note saying, ‘Hey, wow, you guys are really far along on this thing. You wanna come in and have a talk with us?’ Which was really flattering. We have enough inside information to know that everyone at that level is working on something.”

One shoe drops a little over two months after Doppler makes its announcement, when Apple unveils its much-rumored AirPods wireless earbuds at its iPhone 7 launch in San Francisco. Conceptually, the snow-white gizmos have some obvious similarities with Here One, but they’ve got stick-like appendages that jut out of your ears. (Doppler, by contrast, went to pains to incorporate all of its technology into buds that don’t extrude any more than absolutely necessary.) Jokesters on Twitter quickly compare Apple’s industrial design to everything from electric-toothbrush heads to hair dryers to Snoopy.

By removing the headphone jack on the new iPhone, Apple could help popularize both its own AirPods and devices such as Here One. (Doppler has even added a banner to the top of its website: “No headphone jack? No problem.”) And for all the ways AirPods play in the same category as Here One, they are—for now—just wireless headphones, not a computing platform with the smart noise filtering and other software-based capabilities that Doppler is creating.

By late August, when I drop in at Doppler for another visit, the pieces of that computing platform are tangibly falling into place. Technologies that the company’s engineers had working only in ginned-together demos back in June are running on near-final versions of the Here One buds. Using a feature called Smart Settings, Doppler’s app can do things such as geolocate where you are—at work, for instance, or a restaurant—and automatically apply noise filters designed for that environment. The baby-and-siren-identifying AI that had required a PC now works on the phone, and its accuracy rate is much better. (As Kraft notes, “The magic only works if we’re right most of the time.”)

Even as Doppler puts the finishing touches on Here One, it’s thinking about the long-term future for its platform and tinkering with features that won’t be ready anytime soon. One such project is using the earbuds for real-time language translation. When I get to try an experimental version during one visit, I feel a tad silly once again: I have to don a backpack that contains a pint-size desktop PC and the largest battery you can legally carry on an airplane.

That’s because Doppler wants to handle the translation on a device you can take with you, rather than shuttling it off to the cloud. “You go to an airport in Asia, your phone’s not going to work great,” explains VP of R&D Jeff Baker. “And if it does, it’s going to cost you an arm and a leg. The [translation] engine should be local. Whether it’s in your phone or on a separate piece of hardware, we’re not sure yet.”

I understand why the company is investing in the effort when a Doppler staffer says, “Buenos días, amigo. Cómo estás?” and I hear, “Good day, friend. How are you?” Because Doppler’s technology can control both what you hear and when when you hear it, it’s able to overlap the two halves of a conversation, removing some of the lag normally created by the translation process. “It becomes much more fluid, usable, and enjoyable, instead of being super-frustrating and a gimmick,” says Baker.

As my English/Spanish chat continues, Kraft watches and beams. Afterward, he emphasizes that Doppler isn’t sure how, when—or maybe even if—it will deploy translation to consumers. “We know we want to get there, but we’re not going to compromise our ability to ship this first product,” he says.

If Kraft sounds confident about where Doppler is going, it’s in part because of all he’s learned on the fly to get this far. “I now lead the product meeting,” he tells me. “If you had told me two years ago I’d lead a full-on product meeting with every single team, I’d say, ‘You’re nuts. No chance I’d ever be able to do that.’ But that’s where we are. And Fritz just listens in sometimes.”

When Here One ships, it will bear the imprint of both of Doppler’s cofounders: the music-loving filmmaker and the technologist who has helped dozens of startups get going. On his own, Kraft says, “I would have come up with some gimmicky Beats derivative that probably wouldn’t have worked that well and would have looked cool. And Fritz would have shipped Google Glass for your ears. When you combine those things, hopefully you have a product that is culturally relevant but also really sophisticated.” If Here One pulls that off, it will have a shot at seducing consumers—and Doppler will be that much closer to fulfilling its improbable dream of putting a computer in every ear.

A version of this article appeared in the October 2016 issue of Fast Company magazine.