Open Access
How to translate text using browser tools
1 July 2006 Open, Fair, and Free Journal Ranking for Researchers
GUILLAUME CHAPRON, AURÉLIE HUSTÉ
Author Affiliations +

Decisions in research management, such as hiring individuals or funding departments, are often driven by metrics ranking research quality. Toward the end of every June, researchers wait anxiously for Thomson Scientific to release the latest ISI impact factors so they can update the ranking of their academic achievements. Impact factors are almost universally accepted as the standard measure of journal quality, and hence of researcher quality too. Whether the journals in which one has recently published have seen their impact factor rising or falling, or whether those journals have been included or excluded from the ISI impact factor list, can dramatically affect one's career.

Recently, alternative indicators for ranking research have been put forward. Hirsch (2005) proposed an indicator called the h index to evaluate the productivity of scientists (Ball 2005). A scientist's h index is the highest number of his or her papers that have each received at least that number of citations. Because individual researcher and journal rankings are intrinsically related, it has been suggested the h index could be fairly used to rank journals as well (Braun et al. 2005). Doing so would avoid the pitfalls of other commonly used ranking methods (Kokko and Sutherland 1999, Cockerill 2004), and would have the benefit of measuring both significance and sustainability in scientific production; an added benefit is that the h index is difficult to manipulate. Any journal that publishes papers with a seminal, long influence would be rewarded by a higher h index, whereas its ISI impact factor (if it has one) would not be affected by citations more than two years after an article's publication. If journals were ranked according to their h index, the hierarchy would better reflect journal status, as shown in the box for journals with a focus on biology. For example, BioScience has a 2004 ISI impact factor of 3.041, but its h index of 35 would put it among the leading journals for the biological disciplines.

The h index, however, is hardly intended to be the ultimate ranking indicator. Other indices have been suggested since the publication of Hirsch's paper. Taber (2005) commented that it may be better to use a “c index,” which would be the total number of papers from a researcher (or a journal) cited more than once by other research groups (journals) in the most recent calendar year. Bollen and colleagues (2006) followed a different approach, one that distinguishes journal popularity from prestige: Popularity reflects the crude number of citations, whereas prestige reflects the quality of the publications citing a journal's articles (Ball 2006). Bollen and colleagues' ranking algorithm is similar to the complex one used by Google to rank Web pages in search results, called PageRank. Although results from the PageRank-based approach differ somewhat from ISI impact factors, there is much overlap.

Every indicator will have its own strengths and weaknesses, but we believe that the main advantage of the h index has not been stressed enough. Given the importance of ranking in today's competitive scientific environment, it is critical to remember that researchers are the constituency most in need of a ranking indicator that is both fair and easy to compute. The algorithm behind the h index should make it the favorite for the research community. Because the h index is based solely on the overall number of citations, not on citations during a given time period or on the quality of the journals in which the citations appear, it can very easily be computed from most literature databases. One has only to query all papers published in a journal during a given time period and rank the results by the numbers of times each paper has been cited. Suppose that a journal published 200 papers during the past five years. These papers are ranked by frequency of citation, with the paper at the top of the list being the one that was cited most often. Suppose that the 30th paper on the list had been cited 31 times, and the 31st one 30 times. This journal would have an h index of 30, because 30 is the number of papers that received at least as many citations as their ranked position.

Anyone could perform such an evaluation of a journal's h index if free databases such as Google Scholar (which already gives citation results) improved their advanced search pages so as to sort results by citations. Using the h index would make it possible to easily and quickly score and rank any journal covered by a freely accessible database, including those journals without an ISI impact factor. Because scientists today are required to publish more and better work, and because of the ever growing number of journals, we believe scientists' careers will benefit from having their evaluating and funding bodies use the h index for ranking.

References cited

1.

P. Ball 2005. Index aims for fair ranking of scientists. Nature 436:900. Google Scholar

2.

P. Ball 2006. Prestige is factored into journal ratings. Nature 439:770. Google Scholar

3.

J. Bollen, M. A. Rodriguez, and H. Van de Sompel . 2006. Journal status. (9 May 2006;  www.arxiv.org/abs/cs.GL/0601030). Google Scholar

4.

T. Braun, W. Glänzel, and A. Schubert . 2005. A Hirsch-type index for journals. The Scientist 19:8. Google Scholar

5.

M. J. Cockerill 2004. Delayed impact: ISI's citation tracking choices are keeping scientists in the dark. BMC Bioinformatics 5:93. Google Scholar

6.

J. E. Hirsch 2005. An index to quantify an individual's scientific research output. Proceedings of the National Academy of Sciences 102:16569–16572. Google Scholar

7.

H. Kokko and J. Sutherland . 1999. What do impact factors tell us?. Trends in Ecology and Evolution 14:382–384. Google Scholar

8.

D. Taber 2005. Quantifying publication impact. Science 309:2166. Google Scholar

Appendices

Appendix

Index ranking of the top 10 journals focusing on biology, covering papers published in the period 2000–2004.

  1. FASEB Journal: 71 Bioessays: 60

  2. Proceedings of the Royal Society B, Biological Sciences: 47

  3. Philosophical Transactions of the Royal Society B, Biological Sciences: 42

  4. Journal of Experimental Biology: 36

  5. BioScience: 35

  6. Microscopy Research and Technique: 35

  7. Radiation Research: 34

  8. Journal of Theoretical Biology: 27

  9. Biological Reviews: 25

i0006-3568-56-7-558-f01.gif
GUILLAUME CHAPRON and AURÉLIE HUSTÉ "Open, Fair, and Free Journal Ranking for Researchers," BioScience 56(7), 558-559, (1 July 2006). https://doi.org/10.1641/0006-3568(2006)56[558:OFAFJR]2.0.CO;2
Published: 1 July 2006
Back to Top