The real reason we’re not seeing more progress on diversity at work

From reports of bias at Google to ongoing accusations of discrimination at Uber, it seems like we can’t go more than a few weeks without diversity in tech—or a glaring lack thereof—making headlines. Of course, tech isn’t the only culprit. Despite all our talk of equal pay and confronting unconscious bias, industries from fashion to finance remain uncomfortably homogeneous, particularly at senior levels. 

Though we’re making marginal headway, study after study shows that women, people of color, LGBTQ+ employees, and those with disabilities remain vastly underrepresented in leadership roles and executive positions. Only 33 companies on the Fortune 500 are run by women, and just 34% of board seats at those companies are held by women and minorities. 

You’d be hard-pressed to find a company today that hasn’t implemented progressive hiring practices or HR policies designed to promote diversity and inclusion. At a time when there’s more attention on the issue than ever, progress on diversity remains painfully slow. The question is why? 

What diversity programs overlook 

I come from an analytical background. I worked at IBM and was president of Symantec, the cybersecurity giant, before starting my own company that helps professionals ask the right questions and use the right data to see important truths about their business—truths that are often hiding in plain sight. When I talk to leaders about diversity, I hear a familiar refrain: We want to do better, we’re trying to do better, but we just don’t know how to improve. Many point to the fact that they pour resources into recruiting and hiring a diverse workforce, but year after year, the “face” of the company remains largely the same. Nowhere is this more evident than in tech, where despite significant efforts (and massive amounts of PR) to hire diverse recruits, attrition of women and minority employees remains a significant issue. 

Building a diverse talent pipeline is a necessary and positive step, but once inside, the company employees often enter an organizational blindspot. While data on hiring, retention, and attrition may be meticulously tracked, most companies fail to measure all the drivers that influence an individual employee’s experience—and ultimately their choice to either leave or stay. Truly moving the needle on diversity requires putting company culture under the microscope. And that means relying on data—not emotions, intentions, or intuition. 

When it comes to diversity, some patterns are obvious. If women are repeatedly leaving companies at the point where they choose to have children, or failing to return to work or advance after parental leave, that points to a clear problem in how the company supports working mothers. 

Other bias, however, is less obvious. Fairness, for example, is often tracked through looking at access to pay, promotions, and training, but that doesn’t show the whole picture—or even the most important parts of it. Fairness can be quantified in things like time spent with managers, invitations to social events, access to information, even allotment of office space. And it’s these things, along with pay and promotions, that impact individual employees’ experience of being treated equitably or not. 

The cumulative effect of “little” things—daily interactions with managers, casual encounters with colleagues, and talk in the office kitchen—can have a much greater impact on feelings of inclusion than official company policies or compensation schedules. Tracking those encounters is key to uncovering barriers that affect diverse employees where it matters most, in their day-to-day experiences on the job. Yet few companies recognize, let alone track, this kind of information. For those that do, the data can be sobering.  

Quantifying culture from the bottom up

One client of ours faced a familiar problem: they wanted their workforce to reflect the community they operated in. They’d doubled down on their efforts at recruitment, yet year after year they found the dial on diversity had barely budged. By undergoing an audit and tracking employees movements through their first year, we uncovered a pattern of encounters with certain managers that hinted at a deep, and overlooked, problem.

The data showed time and again that employees who were racial minorities were happy to join the company, but as they moved through the onboarding process and came into contact with these individuals, they’d eventually become disengaged and quit. The picture painted by the data was striking and impossible to miss: Race factored heavily into both the length of time employees spent stuck in a particular role and promotions compared to their peers. The company disciplined those problematic managers and has since made great strides in its diversity goals.

This all points to a misconception about culture: We think of it as an intangible set of values that trickle down from the top. In reality, it’s just the opposite. The true culture of an organization is the cumulative product of daily interactions between colleagues or among managers and teams. Gathering data that reveals what’s happening in these interactions—and, more importantly, acting on it—is essential to rooting out the areas where the company’s official stance on D&I diverts from actual practice. 

What better data reveals

This information can’t be obtained through general impressions or even the regular observations of managers. It takes the right tools to measure and analyze these instances so companies can implement solutions that actually work. In the past, gathering this kind of information would have been tedious, if not impossible. But advances in data collection and AI now make it possible to assemble fine-grained information on employee interactions.  

People analytics—the gathering of stats on everything from promotions and time spent in roles to participation rates in training programs—offers a powerful window into how factors like gender, race, and other variables influence workplace culture. More powerful still is pairing this with the burgeoning science of organizational network analysis (or ONA), which seeks to plot and measure actual interactions between employees. By analyzing data from communication and networking tools like email, social media, and calendar apps, in-depth “relationship graphs” emerge—quantifying who’s engaging with whom and in what context. Variables tracked can range from number of emails between colleagues to engagement on social platforms and the amount of meeting time employees get with their manager. ONA might show, for instance, who’s really spending time with managers and how that correlates with promotions. 

This isn’t necessarily rocket science, but it does require a willingness to look at the data, including asking the right questions, then following through and finding the answers. Applied correctly, data and analytics turn the seemingly intractable problem of increasing diversity into something measurable, approachable, and fixable.

So again, the question—one that’s worth billions to companies in reducing turnover and increasing productivity and performance—is why don’t we see more companies taking this approach? 

I’ll be the first to admit the process can be expensive and time-consuming, but more than that it requires courage and resolve that many companies lack. Real data analysis can be a ruthless, uncomfortable process that reveals awkward, often ugly truths. No one wants to acknowledge their managers are biased, call people out for being racist, or look at the hard data that reveals your company’s much-celebrated commitment to inclusivity falls short. It’s not an easy process to undergo, or a pleasant one. But it’s often the only way forward. Until we’re willing to look at the data, and take action on what it reveals, attempts to address diversity will remain inefficient.

John Schwarz is the founder and CEO of Visier, a business intelligence platform.