Open Access News

James A. Evans and Jacob Reimer, Open Access and Global Participation in Science, Science Magazine, February 20, 2009.  (The DOI-based URL is broken, at least for now.)  Only this abstract is free online, at least so far:

Previous investigations into the impact of open-access journals on subsequent citations confounded open and electronic access and failed to track availability over time. With new data, we separated these effects. We demonstrate that a journal receives a modest increase in citations when it comes online freely, but the jump is larger when it first comes online through commercial sources. This effect reverses for poor countries where free-access articles are much more likely to be cited. Together, findings suggest that free Internet access widens the circle of those who read and make use of scientists' investigations.

Also see Stevan Harnad's comments.  Read his full post or this summary:

Evans & Reimer (2009) show that a large portion of the increased citations generated by making articles freely accessible online ("Open Access," OA) come from Developing-World authors citing OA articles more. It is very likely that a within-US comparison based on the same data would show much the same effect: making articles OA should increase citations from authors at the Have-Not universities (with the smaller journal subscription budgets) more than from Harvard authors. Articles by Developing World (and US Have-Not) authors should also be cited more if they are made OA, but the main beneficiaries of OA will be the best articles, wherever they are published. This raises the question of how many citations – and how much corresponding research uptake, usage and progress – are lost when publishers embargo their authors for 6-12 months from making their articles OA....

For other comments, see Elie Dolgin, Online access = more citations, The Scientist, February 19, 2009 (free registration required).  Excerpt:

...In the most extensive study to date -- covering around 26 million articles from more than 8,000 journals published from 1998 to 2005 -- University of Chicago sociologist James Evans, together with neurobiology grad student Jacob Reimer, found that making an article freely available on the internet increased the number of citations, but only by about 8%, which was far less than some previous claims.

When the authors looked just at poorer countries, however, they found that the influence of open access was more than twice as strong. For example, in Bulgaria and Chile, researchers cited nearly 20% more open access articles, and in Turkey and Brazil, the number of citations rose by more than 25%. Free online availability "is not a huge driver of science in the first world, but it shapes parts of science in the rest of world," Evans told The Scientist. "Scientists and scholars in poorer countries are disproportionately citing articles that are freely available to them."

"The results make a lot of sense," Gunther Eysenbach, a health policy and e-health researcher at the University of Toronto who was not involved in the research, told The Scientist. "In countries with lower income, the [open access] effect is bigger than in countries where researchers have access to the literature anyway -- that's quite intuitive."

Stevan Harnad, an open access advocate and cognitive neuroscientist at the University of Quebec at Montreal who was not involved with the study, said that the authors should have compared public and private institutions closer to home to test whether the same effect was true in the developed world. "[Evans] looked at the big picture, but he could have cut the cake a bit finer and found the same effects if he compared the Harvards and the have nots," Harnad told The Scientist. "It's a shame that with such rich data he didn't look at other such important and pressing questions."

Free online access conferred the greatest citation advantage in the life sciences, and no significant influence in three areas -- chemistry, physics and the social sciences -- which the authors chalked up to a culture of pre-print databases and personal archiving....

Philip Davis, a Cornell University grad student in science communications who also studies the effects of open access on citation records, praised the article's size and breadth, but noted that Evans "can't measure the article-level details, he can only look at the journal level." The study lumped together articles from entire journal volumes, which overlooks author-pay models that make some but not all articles in a journal freely available, Davis noted....

Evans dismissed Davis' criticism....The study compared, for example, articles published in Science in 2003 and cited in 2004 -- when the journal was still under its one-year embargo period -- to the same articles cited in 2006, at which point the papers were freely available. When he didn't use fixed effects and compared journals to each other, Evans noted, the open access effect was much larger -- by at least an order of magnitude -- and the numbers of authors and pages significantly influenced the results. "In my reported estimation this is not the case."

Also see Yun Xie, Open, electronic access to research crucial for global reach, Ars Technica, February 19, 2009.  Excerpt:

...It may come as no surprise that gross national income greatly influences the importance of open access. In countries with a gross national income of $2,300 per capita, open access to articles can increase their citation by up to 30 percent. Geographically, the effects were most apparent for developing nations in the Southern Hemisphere. Although open access benefits poorer countries the most, there is a limit to what it can achieve. Since it requires electronic access, some of the poorest countries cannot enjoy its full benefits. Electronic access is an essential factor in the success of a journal article; in well-funded Northern and Western nations, online availability alone was about 40 percent more important than open access.

The influence of open access varies greatly depending on the discipline of research, as well. Fields like chemistry, physics, and social sciences show no observable change, while economics and business get over a 20 percent boost in citations....

Regarding the overall importance of open access, Evans and Reimer state that their “work provides clear support for its ability to widen the global circle of those who can participate in science and benefit from it.” ...

Also see Philip Davis, Open Access and Global Participation in Science, Scholarly Kitchen, February 19, 2009.  Excerpt:

...Advocates for open access will see this article as supporting their cause.  But those who spend time reading the methodology will notice that message is not as clear as the article implies.

The researchers are not comparing open access journals with subscription-access journals, as reported in the recent article by Tove Faber Frandsen.  Evans and Reimer are comparing the effect of freely available articles to subscription-access articles.  But this is still an oversimplification.

Due to the size of the study (26 million articles published between 1998 and 2005 in over 8,000 journals), the researchers were unable to code individual articles as being OA or not, so they coded entire volumes.  For example, articles from the journal Science are OA when they are older than one year.  Articles from PNAS are all subscription-access in the first six months (in spite of the fact that about one-third are author-pays OA), after which they are all coded OA.  Because of the macro-level of the study, no attempt was made to find other sources of free copies.  In other words, this study focuses entirely on open access publishing.  Some freely available articles will be coded as subscription-access articles, and the result is an overly conservative estimate of the open access effect.

The important detail that may be missed is that the source of the vast majority of OA articles in this study were published by non-profit scientific societies who use the subscription model in tandem with a delayed-access model.  If anyone should be claiming victory, it should be them....

Also see the interview with James Evans from the National Science Foundation, February 19, 2009.

Update (2/24/09). Also see Mike Eisen's comments:

...This paper - and the response to it - has many flaws. A few of the most egregious:

1) The analysis shows that there are a lot of scientists out there benefiting from free access 

First, the authors and a lot of people responding to this paper seem to assume that the modest increase in the number of citations arising from the transition to free access is somehow an argument against free access. This is silly. Even if free access didn’t change citation numbers at all, it would still be an unambiguously good thing for a wealth of other reasons that I won’t rehash here.

The reason that open access opponents are so excited by the supposed conclusions of this paper are that a small citation increase associated with free access bolsters one of their favorite tropes - that all the important people (i.e. people who might eventually cite your paper) already have access to it because they are affiliated with a major research university in the developed world. If this were true journals making their contents freely available would have very little effect on which articles they read and cite.

But even this paper - now being cited as evidence that free access is unimportant - reports at least an 8% increase in the number of citations associated with free access - suggesting that there are a significant number of active researchers out there who are getting access to articles only because they are available freely online. This may sound like a small number, but collectively across the global scientific community we’re talking about tens or hundreds of thousands of scientists.

Do the publishers really want to argue that even this modest increase in citations is unimportant? If so, I’ll remind them of it next time they issue a press release touting the 5% increase in their impact factor…

2) The 8% number comes from analysis of a lot of old articles. If you look only at articles that become freely online within two years of publication, the increase in citations is 20%....

So, for articles that are less than 2 years old, the effect is close to 20%. And the curve is clearly rising as you get to shorter time frames. Doing a little extrapolation it looks like the effect of immediate free access should be at least 50%.

3.) The raw data for the paper are not available to confirm and/or reanalyze the authors’ claims

Where the hell is the data for this paper? I’d love to look at the validity of their analyses and to do some of my own. But, whoops, I can’t. Because the data are private, and not provided anywhere by the authors or Science. This is even though Science’s own publication policy makes it clear that:

After publication, all data necessary to understand, assess, and extend the conclusions of the manuscript must be available to any reader of Science....

Update (2/25/09).  Also see Stevan Harnad's two elaborations on his first comments (1, 2), which I've blogged separately here.