Gender, again?

There has been a resurgence of interest in the last few years regarding gender disparities and bias in science. Recent research has examined, inter alia, lower rates of participation of women in the scientific workforce, the lower rates of female production of scientific work, and lower scientific impact—measured by citations—of female-authored work. Contextual effects—such as effects of funding, environment, and other responsibilities of women—have been used to explain the observed disparities. Bias has been investigated through clever experimental designs. For example, provided with applications identical except for explicitly male/female names, directors (both male and female) of laboratories were more likely to hire the supposed male for laboratory manager positions and offered the male applicants higher salaries (Moss-Racusin et al., 2012). Similarly, an analysis of the contributions of 1.5 million computer programmers publishing on the GitHub platform showed that, when the first name of authors provided an indication of gender, codes produced by women were rejected more by their peers, whereas the opposite was observed when only the initials of the first name of the authors were presented (Terrell et al., 2016).

Encouraging results, but...

Recently, Elsevier—the world's largest commercial publishing house and owner of the Scopus bibliometric database—contributed to the debate with a report entitled “Gender in the Global Research Landscape” (Elsevier, 2017), in which it emphasizes the evolution of the place of women in research over the past two decades. Some results it contains are very encouraging: in each of the countries analyzed, the proportion of women active in research—defined as those, based on Scopus Authors’ Profiles, who contributed to at least one scientific article indexed in the Scopus database—increased between 1996-2000 and 2011-2015, and reached parity in some countries such as Brazil and Portugal. It also presents data on citations, concluding that parity has been achieved: globally, female papers are cited at nearly equal rates to men and exceed male rates in countries such as the United States. While this may sound like good news—and it would be! —this demonstrated parity should be received with a healthy dose of skepticism, given the number of studies showing the opposite. This short analysis aims, on the one hand, to explain the methodological artefact behind the surprising Elsevier results and, in addition, to provide evidence of the gap that persists in terms of citations received.

What is an article "written" by a woman?

The Elsevier findings hinge on a deceptively simple question: What is an article written by a woman? Or by a man? The answer to this question was obvious when authorship was mainly an individual matter, but has been complicated by the ubiquity of collaboration. Figure 1, based on data from the Web of Science for the period 2008-2015 to which we applied our first name-based disambiguation algorithm (N = 8,142,048) (Larivière et al., 2013), illustrates this difficulty. Indeed, we note that nearly 60% of articles have at least one female author and one male author. More than a third of the articles have only male authors, whereas about 6% of the articles have only female authors. Also noteworthy are the differences in terms of citations (ARC = Average of relative citations, a citation indicator that normalizes scores by discipline and year of publication): while articles containing both male and female authors have the highest citation rate (1.11), those written exclusively by women are cited lower than those written exclusively by men (0.86 vs. 1.05). However, when considering articles with at least one man (94% of articles) and those with at least one woman (65%), scientific impact is identical (1.08).

Figure 1. Number of articles and citation rates of the various cases associated with the assignment of an article to a woman or a man, 2008-2015

The mirage

These results demonstrate the problem underlying the Elsevier report. The report relies on "full counting", in which authors are credited for each article on which they are an author. The consequence is that there is high overlap between the sets of “male” and “female” papers, with the overwhelming majority of articles to which women contribute also having men among their authors. In order to overcome this limitation—and considering the dominant role of authors documented in the literature (Larivière et al., 2016)—we compiled, with the dataset used in Figure 1, the differences in citation rates for first and last authors, by gender. The results reinforce previous research on gender disparities: fewer citations are received for articles with women at the beginning of the author list (0.98 vs 1.09) and at the end (1.00 vs 1.08). The gender gap still exists in terms of scientific impact.

A glass ceiling on citations?

Despite its well-known limitations, the journal impact factor is perceived as an indicator of the reputation and visibility of a journal. This is due, among other factors, to the relationship between impact factors and acceptance rates, with high impact factors journals having significantly lower acceptance rates. In light of the citation disparities depicted above, one might expect, therefore, the gender gap in terms of impact factor and citations to be similar. Yet our results on the differences in male and female citations and impact factors show that, for both first and last authored papers (Figure 2), the average impact factor for men and women are much closer to parity than citations and that, systematically, the gap in citations is much greater than the gap in impact factors, and always in favor of men. Moreover, in some disciplines—such as Earth and Space, Biology, Engineering, and Social Sciences—the two indicators show opposite trends: while women publish in journals with higher impact factors than men, they obtain lower citation rates. The inset of Figure 2 shows the pattern observed at the level of all disciplines combined: while in both author positions women’s papers normalized citation scores are lower than those of their impact factors, the opposite is observed for men’s papers.

Figure 2. Male / female differences in normalized impact factor and average of the relative citations for articles written as first and last author, by discipline, and mean values obtained all disciplines combined (inset), 2008-2015

To better understand this phenomenon—similar to the Matilda effect documented by Margaret Rossiter nearly 25 years ago, where women are less recognized than their male counterparts (Rossiter, 1993)—we have grouped articles by women and men according to nine classes of standardized impact factors, and compared the citation rates of the articles in each of these classes. The results demonstrate that the largest citation disparity occurs in those journals of the highest impact factor (Figure 3). This shows that, even when women publish their works in high impact factor journals, this work is cited lower, both in absolute and relative terms.

Figure 3. Average of the relative citations of the articles, according to the impact factor class, for articles written as first and last author, 2008-2015

The prevailing winds

Despite the progress towards equality depicted in the Elsevier report, there remains a bias in the reception of female work, particularly in some of the most prestigious scientific journals. Even when subject to the same process of peer review—seen as the gold standard for evaluating quality—the work of women is cited at a lower rate, both in the position of first and last author. It may be argued that this is the result of differences in topics or other properties of the paper. However, studies have shown that, when controlling for all the properties of a paper, women’s work is grossly undercited (Caplar et al., 2016). The mounting evidence reinforces the relative and absolute neglect of women’s work, despite progress towards equity in science.