Why Disney hasn’t shared how many people watched ‘Hamilton’

During Disney’s Tuesday earnings report, amid earnings per share decreasing 94% from a year ago, everyone expected Disney to tout one of the lone bright spots for the company in an effort to buck up investors.

Hamilton.

The filmed version of Lin-Manuel Miranda’s hip-hop Broadway sensation had captured a wave of interest when Disney debuted it on its Disney Plus streaming service for the Fourth of July weekend. Ever since, people have waited for the company to crow about its decision to stream the filmed version of Hamilton rather than wait to release it in theaters, how the company gave everyone the closest we’d get to a summer blockbuster this year, and how it resulted in a huge win for the company.

All we needed were the numbers.

Disney CEO Bob Chapek did mention Hamilton in his remarks to investors on the earnings call, but he would only call it “a huge success” and cite how Disney Plus gave “millions of viewers a whole new way to experience this iconic cultural phenomenon.” And that was that.

With those remarks, Disney joined the game that every streaming service plays in presenting its high-profile bets as winners with a decided lack of detail. In July, Apple declared that Greyhound, the Tom Hanks World War II drama, was Apple TV Plus’s biggest hit to date and that its viewing audience was on par with that of a summer theatrical box-office smash. Hulu feted the Andy Samberg romantic comedy Palm Springs, saying it broke records when it debuted on July 10, netting more hours watched over its first three days than any other film on the service during that period. On Netflix’s last earnings call, the company—which has become a bit more loose-lipped about its data—ticked off a number of viewing brags related to its original titles: 40 million households streamed Mindy Kaling’s series Never Have I Ever, the same number that watched the Steve Carell sitcom Space Force. As for the David Spade romantic comedy The Wrong Missy: 59 million. The Chris Hemsworth thriller Extraction netted a whopping 99 million viewers, the company claims.

All this wonderful news has made headlines, launched tweets, and inspired breathless media posts.

If only anyone knew what it all meant.

Netflix’s self-reported figures, after all, calculate a “view” as anything on the platform that has been watched for at least two minutes. So we’ll never know if those 40 million viewers watched all 10 episodes of Kaling’s series or just the opening credits. As for Greyhound, Apple didn’t break out numbers, period, nor has it revealed how many subscribers Apple TV Plus even has at this point. Who knows how it calculates a view? Palm Springs represents a similar kind of black box.

As more and more companies enter the streaming ecosystem—the last few months alone have seen the launch of Peacock, HBOMax, and Quibi—the need for a universal metric to make heads or tails out of the data that streaming companies do (or don’t) share with consumers, investors, and the creative community has never been more critical. Creators would gain leverage in negotiating contracts (precisely why streamers prefer that they don’t have much information). Consumers could make more informed viewing and subscribing decisions. As for the streamers themselves, they would have access to more contextualized information in terms of how their TV shows and movies stack up against the competition. For those either offering advertising-based streaming (Hulu, Peacock) or planning to do so (HBOMax), agreed-upon metrics would help set the market. Meanwhile, the investment community would no longer be “getting fatigued from everyone claiming, ‘We’re No. 1!,” says Ben Carlson SVP and general manager of streaming and platforms at MarketCast. “Because right now there’s no apples-to-apples comparison.”

Ironically, Carlson points out, “Today, streaming services have more information about their own programing than any entertainment companies in history. But unlike TV ratings, which provide transparency about TV performance across broadcast networks, streaming services don’t have competitive information. Instead, each streamer is very deliberate about the information they share publicly with little transparency or comparability. This leaves filmmakers, agents, talent, and the general public flying blind about the performance of movies and series.”

This lack of transparency has resulted in an arms race among analytics companies, old and new, who are all scrambling to divine quantifiable signals based on whatever tea leaves companies like Netflix sparingly dole out. From established stalwarts such as Nielsen to new upstarts like FlixPatrol, these companies each have their own unique approach to calculating the popularity of a show or movie as well as analyzing the people watching them. The only problem is that because they don’t adhere to a single formula—some put more weight on social media mentions, while others ignore social media entirely—there are simply more numbers and more lists without any overall context.

In other words: confusion piled on top of confusion.

Even the experts aren’t sure where to look for reliable information. The best they can do is cobble together various data points and come up with a decent guesstimate. As Christopher Meir, a film and TV historian who teaches at the Universidad Carlos III de Madrid, says, “At the end of the day, for my research, I just gather all of the opinions and if they all seem to be pointing in the same direction toward a work being popular, that’s good enough for me.”

But should it be?

Popularity patrol

Last January, when Netflix announced that it was going to publish a daily top-10 list of the most popular titles in each of its territories, it marked a a drastic shift for the famously tight-lipped company. The importance of the move was underscored when CEO Reed Hastings said that Netflix was “trying to grow up a little. When we went public in 2002, we were $50 million in revenue and Blockbuster was $5 billion. We had a long fight that we barely came out of and learned to be defensive about hiding information. But the things that made us successful can then trip us up over time.”

But the top-10 lists ultimately only go so far, proving how secrecy nonetheless remains the company’s dominant DNA strand. For example, Netflix doesn’t disclose how many people have viewed the titles, choosing only to rank them in a relative order. Nor is there any demographic information about viewers. We also have no idea how long the shows or movies were watched for—except, of course, that it’s at least 120 seconds.

Nonetheless, this slight pulling back of the curtain provided an opportunity for Tomas Vyskocil, founder and CEO of FlixPatrol, an analytics streaming company based in the Czech Republic. Vyskocil and his colleagues take the top-10 lists and fold them into an algorithm that takes into account Netflix’s subscriber numbers in each of its regions. The formula also allots a “country power” to each region, meaning that a show or movie that’s ranked No. 1 in the U.S. is given more weight than one that’s top-ranked in Iceland. By crunching all of this data—including other metrics provided by an on-staff PhD-level mathematician—FlixPatrol is able to rank Netflix’s top performers across its entire platform. FlixPatrol applies a similar methodology to rank content on nearly a dozen other platforms including Hulu and Disney Plus.

When FlixPatrol debuted its rankings last April, people took note. Within days, Vyskocil says that former Netlix employees were contacting him to share their insights (they told him that the company’s top-10 lists “were not rigged,” though they could be influenced by company marketing and announcements), as well as creators who had worked for Netflix. Michael M. Scott, director of the Netflix original film Dangerous Lies, was one. He told Vyskocil that he loved what FlixPatrol was doing, especially seeing as he received so little data from Netflix. He then asked Vyskocil if he could “reverse engineer” how many viewers had watched Dangerous Lies, using FlixPatrol’s algorithms. Vyskocil said he could and came back to Scott with a number: 35 million households for the film’s first four weeks on the service.

Vyskocil said that it would be nice if he received confirmation on that figure from Netflix, but he says that the data that Netflix releases in earnings reports has been consistent with FlixPatrol’s measurements. But even he admits there is room to expand in order to improve accuracy. The company has started tracking popularity lists on movie and TV databases such as IMDB and Rotten Tomatoes in order to bolster its research, and is venturing into measuring mentions on social media and Google Search. He also says that FlixPatrol is limited by the streaming companies themselves. Not all of them release subscriber numbers and their trending lists can be inconsistent. For example, he said that while Hulu was declaring Palm Springs a hit, the film wasn’t initially showing up on FlixPatrol’s most-popular list.

“We know there are some limitations when you go into the platforms,” he says.

The empire strikes back

If any data company is poised to become the de facto voice of God in the streaming ecosystem, it’s Nielsen. No other company has the brand-name value, track record, and, perhaps most importantly, trust of the entertainment industry than Hollywood’s age-old ratings generator—which, ironically, has often also served as Hollywood’s punching bag by network executives who have faulted the company’s sample-based approached to measuring TV viewers. But in the pitch-black world of streaming data, Nielsen is now a Hollywood ally, one that speaks in its shared language. Nielsen, which debuted its streaming video content measurement system in 2017, calculates streaming views as “ratings.”

Indeed, Nielsen’s approach to tracking streaming is rooted in the same methodology that it uses to track linear TV viewing. It has a panel of 45,000 homes whose TV and streaming viewing Nielsen monitors by audio signals that Nielsen can recognize. Nielsen knows who’s in the household, what content is consumed, and how it’s being consumed (co-viewing, mom and child?) as well as what kind of device is being used (Roku, PlayStation, etc.).

“The key benefit is consistency,” says Brian Fuhrer, SVP and national and cross-platform product leader for Nielsen. “We’re measuring it exactly the same way” as linear TV. “And the numbers are expressed in exactly the same metrics that people have used for years. So you’re able to see what the audience flow is and what the viewer journey is across all of these different services. So what is the Netflix rating across all these programs in prime time compared to other sources? That’s the key.”

But Nielsen’s close relationship with entertainment companies also presents limitations. The company, whose clients include the major studios and networks, has yet to release streaming data for services owned by those companies—such as Peacock and HBOMax. “We want to help them grow their businesses, and [helping] them introduce this in an orderly fashion is what we’re trying to do,” Fuhrer says.

Nielsen’s approach is also dismissive of social media, something that other analytics companies weigh heavily. For instance, MarketCast’s streaming tracker blends social media conversion around TV shows and movies (including whether they are recommended and “evangelized”) with information that it gathers from weekly surveys of 500 individuals between the ages of 18 and 54 in the United States who subscribe to at least one streaming service. “A survey alone wasn’t going to tell the real-time social nature of streaming,” MarketCast’s Carlson says. Parrot Analytics, another upstart company that tracks the popularity of streaming content, measures what it calls “demand expressions”—this metric takes into account factors across the internet like Google search terms of a movie or TV show, Facebook likes, and even Wikipedia traffic.

But Nielsen’s Fuhrer thinks differently. “Certainly, social has an element here. But there are scenarios where programs have a lot of social activity and little or no actual view activity.” The 2013 cultural phenomenon Sharknado, he says, “had a lot of Twitter mentions, but few people actually watched it. So the correlation between social and actual viewing isn’t always as strong.”

Other companies go even further, monitoring not just what consumers are watching, but what they’re buying. This information is especially pertinent to companies like Apple and Amazon whose streaming services are ultimately gateway drugs to larger ecosystems. Streamlytics tracks how someone, say, watches a show on Amazon Prime (as well as what show it is, what talent is involved, and what genre) and then clicks over to Amazon to buy a book. “This makes us vastly different than other people in the space, including Nielsen,” says Streamlytics founder and CEO Angela Benton. “We’re getting an actual data stream to what a consumer is doing across their digital footprint.”

One day our ratings prince will come

People like the historian Meir are optimistic that a universal streaming metric, calculated by an objective third party, is attainable in the not-so-distant future. One of the main hurdles, he says, is the nondisclosure mentality of Silicon Valley companies like Netflix, Amazon, and Apple, which have never been in the business of releasing ratings and box-office figures on a regular basis. “But now that studios are getting into direct-to-consumer streaming in a big way, the idea of an objective measurement will gain some momentum,” he says.

“There might be a point that is you’re not opting into the system, it will look bad for you. That might be the pressure that brings the Silicon Valley companies around. The new entires (to streaming, such as Peacock and HBOMax) are studios first and tech companies second. Whereas the existing streaming companies are the other way around.”

Disney, in fact, might be the one to turn the tide. With more hits like Hamilton coming out on Disney Plus, the company may make a push to create an industry standard to show off its success. Meir adds: “Part of Disney’s culture is to shower the creative with praise. It’s not like you have to sell a studio executive on the importance of reporting your success.”