Science’s 
Diversity Problem

Minority and women researchers have more novel ideas, but they are less likely to be adopted by the scientific mainstream.

• CITE
•  
• SHARE
•  
• COMMENT
•  
• DOWNLOADhttps://ssir.org/pdf/Fall2020-Research-Blei-Science-Diversity.pdf
•  
• PRINT
•  
• ORDER REPRINTS

By Daniela Blei Fall 2020

(Photo by iStock/sanjeri) 

About a decade ago, researchers identified a pernicious phenomenon in corporate America. Despite an abundance of data showing that diversity fosters innovation and drives market growth, historically underrepresented groups—nonwhites and women who diversify labor markets—were not being rewarded for their contributions with the same career advancement and promotional opportunities accorded to white men.

In 2017, when an interdisciplinary team at Stanford University’s Knowledge Creation Lab began to reflect on inequality in science careers, they uncovered a similar paradox. The benefits of diversity for science are well known and even heralded by researchers, and yet many minorities and women struggle to climb the academic ladder to reach tenure and other milestones.

Bas Hofstra, a postdoctoral research fellow at Stanford University’s Graduate School of Education, and Daniel McFarland, a professor of education at Stanford who focuses on data-based sociology, led a team that analyzed the trajectories of about 1.2 million doctoral recipients in the physical sciences, social and behavioral sciences, engineering, biology, earth sciences, and humanities between 1977 and 2015. Drawing on US Census data, Social Security Administration data, the ProQuest Dissertations and Theses database, and Web of Science—a large-scale publication database—the team scrutinized metadata (names, advisors, universities, and degrees) and analyzed text, such as dissertation abstracts, for patterns. By following these “structural and semantic footprints,” they could determine doctoral students’ rates of innovation and whether these contributions to knowledge translated into successful academic careers.

What they found was troubling. Minority and women researchers had more novel ideas, but these ideas were less likely to be adopted by the scientific mainstream, dominated by a white male majority. This reduced the impact of these ideas, resulting in fewer sought-after academic positions for nonwhites and women.

“When someone enters a particular context with an outsider perspective, they tend to look at things in new ways and to make novel connections between old and new ideas,” Hofstra says. “We know that teams that are more diverse are more susceptible to innovation. But then we see that minorities in science often don’t have science careers at the same rate as majority members. If diversity breeds innovation, then why are the people who actually diversify science so underrepresented among professorships and faculty careers?”

Using text analysis methods and, in particular, structural topic modeling, the Stanford team was able to search millions of documents for latent themes to identify substantive, important scientific objects, or concepts, within these texts. This filtering method allowed them to single out doctoral students who linked these concepts in new ways.

“At the heart of our metric are students who combined scientific concepts in ways that hadn’t been done before,” Hofstra says. “That is the moment when scientific novelty is introduced. And this is a pretty basic intuition in the philosophy and sociology of science: defining knowledge as a network of ideas or concepts.”

For Sameer Srivastava, a professor at the University of California, Berkeley’s Haas School of Business, “The introduction of a method that analytically separates novelty, which is measured based on new linkages between concepts that appear in a researcher’s doctoral dissertation, from impact, which is measured based on how often the novel combinations introduced by a researcher are adopted in subsequent work,” marks “a significant advance over typical approaches to measuring innovation such as citation patterns, which can sometimes be influenced by social dynamics that are unrelated to the actual work product.” The importance of this study, Srivastava notes, is that it “helps to explain the persistence of inequality in scientific careers.” 

While addressing disparities in faculty hiring and promotion, the findings challenge basic assumptions about the nature of scientific innovation. “If we look at scientific discovery,” Hofstra says, “it seems that a significant portion of that is guided by attributional biases. What is considered useful innovation is in some way correlated with who introduces that innovation.” Consensus formation is often regarded as one of science’s strengths. But “this also turns out to be a weakness,” Hofstra explains. “The majority has more say over what is a scientifically useful innovation than others. And that runs counter to what good science is.”

Hofstra and his team are pursuing follow-up research to investigate the role of mentors, what gets treated as a good innovation, and when and where biases enter into faculty hiring.