Publication Metrics

#limjunlong Original Series

Metric

In research, a metric is a quantifiable measurement used to assess various aspects of research output, such as its quality, impact, and productivity. Metrics are used to evaluate the performance of individual researchers, research institutions, and even entire fields of research. It is important to keep in mind that there are limitations to consider when evaluating competencies using metrics indicator and metric tool.

Google Scholar

Note: This is not a paid advertisement.

Google Scholar (https://scholar.google.com/) is one of the most common tools for calculating publication metrics. The Google account can associate with Android smartphone to build any researcher research custom profile.

After adding in all the long list of publications, the Google user can share the profile to the public domain to let other people see their research impact, experiences and history. One of the most prominent item on  the Google Scholar is the:

• Citation value, 

• h-index value, and 

• i10-index values. 

On a smartphone, we have to click on CITED BY words to see the metrics due to the small size of the screen.

Total Citation

The most obvious indicator for any human who understand the number system. Any manuscript published will take a few years to several years to obtain a sizeable citation value per year before taping off or oscillate over the years. In most cases, we can generalised that a new postdoc will have the higher likelihood of having a lower total citation than another decade old postdoc. 

Time is necessary

Even in today's cloud technology, hyper fast internet connectivity and ever fast processors, it will take time to 'do marketing' for any published work. 

Take for example, a conference paper can be further improve with more simulation and/or experiments which can be rewritten into a journal paper, vice versa; all the tasks will take time to do and complete. Not to mention that even online-only journals do not publish on daily basis and reputable journals may take any time ranging from 2 months to 1 years from the time upon receiving the manuscript to being published. 

Which is to say, the person who just read your paper right now and decided to cite it (as it is useful) may not have his/her paper capture by index to count into your total citation right away, not today or even tomorrow. Other reasons may be it takes time for search engine to index and figure out how to link related papers together, generating DOI link, etc. before a potential cite-reader  find your published paper.

Citation is important

Certain researcher managers will emphasis their researchers to publish in their curated list of top journals / conferences. Taking a closer look at prestigious well known journals, they have plenty of journal papers that are never ever cited even after so many years. Enough said about 'quality of paper' in a big impact factor journal.

h-index - Overview

It is a common misrepresentation that this single-number metrics can represent a researcher's scientific lifetime achievement. But before we go deeper, we will go through some of the basic of this metric. The h-index is an author-level metric that calculates the citation impact of one's publication emphasising mostly on the most cited papers and citation power count through the author publication lifetime.

To calculate this h-index number, we first list all the researcher's publication sorted according to the total number of citations. Next from the most cited paper, we start with h-index value of 1. Then we go down one ranked of the most cited paper and increase the numerical value of h-index by 1. We stop going down the ranks when the cited paper count is smaller than then the numerical value of h-index at that rank. (Click here to see how Wikipedia explains it).

Advantages

We are living creatures living in a world where time is limited and we do not have the technology to go time travel yet. Most researchers working in top research organisations do not have the leisure of time to dig out or verify the history of a new postdoc or experienced research fellow CV hence a quick first cut by the h-index value. 

As like many older proposed system or solutions, the h-index was proposed in 2005 by Jorge E. Hirsch, a physicist, to address the overreliance on:

1. Total number of papers, or 

2. Total number of citations

when gauging a researcher's past performance and capability. 

Don't judge a book by its cover

The h-index is also commonly used by non-experience and non-technical hiring managers too. One can choose to perceive thoughts that a person capability in relation to their h-index, hence the h-index is how they label a person by perceived 'book cover'.

Well established researchers and well trained hiring managers often look in more than just the past impact of those potential hires. In a successful organisation, an important criteria is the character of the person. In this modern world of social media like Facebook, Instagram and LinkedIn, large corporations have began to request hiring staff to screen potential employee's trait, habit and characteristic by examining their post/comments and whether they can possibility 'sync' with their current employees. 

Weakness in the metrics

Clearly we see that that maximum value of h-index is the maximum number of publication that one researcher has ever published. It is worth noting that the total citations and number of papers are counted regardless whether the author is the first author, the middle author or the last author of a long list of co-authors. Newer international publishers may have extra section in their published papers to include the author contributions such as funding acquisition, project management, project administration or supervision all of which is indirect research work and may or may not influence the success rate of the research or experiments. In other words, if one has ample resources, one can publish as many paper as they like by allocating funding.

The h-index has higher sensitivity to highly cited papers and low sensitivity to the least cited papers. In other words, highly cited papers are the determination of the h-index. Once the top cited papers surpass the h top class, it is totally unimportant whether these paper get 50, 100 or 1000 more citations. Researchers with extremely cited paper will have similar or equal h-index as other nonextremely cited paper researchers.

Manipulation to increase h-index

Self-citation is often done and acceptable throughout the research community. It is important to note that not all self-citation is coercive, or indeed improper. When done correctly, it will give a proportion of credits to the author or point curious readers for further reading in relation between the newly published work and cited author's past work. 

The problem comes when an author(s), in a regular paper, knowingly and intentionally cite numerous of their paper in one go by designing the manuscript in a particular miraculous way. On the other side, journal can also become on the dark side by siding with the unethical authors to inflate the journal's impact factor for the reason of artificially boosting the journal's scientific reputation by coercive or other targeted methodologies.

Even in the context of a review paper or invited paper, the publisher's editor / associate editors / reviewers have to carefully examine the manuscript's reference list and invoke the manuscript's authors to make appropriate amendments to the manuscript before accepting and publishing it. Failure to do so may cause the many decades old ecosystem of the research community to collapsed and lose trust from funding sources. Big influential index like Thomson Reuters' Journal Citation Reports is known to temporary exclusion journals for such miraculous  behavior. 

i10-index

Right under the row of h-index in Google Scholar is the i10-index.

The i10-index is a metric that is used to measure an individual researcher's productivity and impact based on their publication record. It was developed by Google Scholar and is calculated by counting the number of publications that an author has written that have been cited at least ten times. For example, an author with an i10-index of 20 has written 20 papers that have each been cited at least 10 times. 

Highly Cited Researchers in Singapore

Selected Singapore List in Year 2020

1. Prof Michael Meaney, Singapore Institute for Clinical Sciences (SICS)

   • Ian CG Weaver, Nadia Cervoni, Frances A Champagne, Ana C D'Alessio, Shakti Sharma, Jonathan R Seckl, Sergiy Dymov, Moshe Szyf, Michael J Meaney, " Epigenetic programming by maternal behavior," Nature neuroscience, vol. 7, no. 8, pp. 847-854, 2004. DOI: 10.1038/nn1276 [link]

     • Abstract: Here we report that increased pup licking and grooming (LG) and arched-back nursing (ABN) by rat mothers altered the offspring epigenome at a glucocorticoid receptor (GR) gene promoter in the hippocampus. Offspring of mothers that showed high levels of LG and ABN were found to have differences in DNA methylation, as compared to offspring of 'low-LG-ABN' mothers. These differences emerged over the first week of life, were reversed with cross-fostering, persisted into adulthood and were associated with altered histone acetylation and transcription factor (NGFI-A) binding to the GR promoter. Central infusion of a histone deacetylase inhibitor removed the group differences in histone acetylation, DNA methylation, NGFI-A binding, GR expression and hypothalamic-pituitary-adrenal (HPA) responses to stress, suggesting a causal relation among epigenomic state, GR expression and the maternal effect on stress responses in the offspring. Thus we show that an epigenomic state of a gene can be established through behavioral programming, and it is potentially reversible.

     • Google Citation = 6305 / Web of Science = 3153

2. Prof Nick Barker, Institute of Molecular and Cell Biology (IMCB) 

   • Nick Barker, Johan H Van Es, Jeroen Kuipers, Pekka Kujala, Maaike Van Den Born, Miranda Cozijnsen, Andrea Haegebarth, Jeroen Korving, Harry Begthel, Peter J Peters, Hans Clevers, " Identification of stem cells in small intestine and colon by marker gene Lgr5," Nature, vol. 449, 7165, pp. 1003-1007, 2007. DOI: 10.1038/nature06196 [link]

     • Abstract: The intestinal epithelium is the most rapidly self-renewing tissue in adult mammals. It is currently believed that four to six crypt stem cells reside at the +4 position immediately above the Paneth cells in the small intestine; colon stem cells remain undefined. Lgr5 (leucine-rich-repeat-containing G-protein-coupled receptor 5, also known as Gpr49) was selected from a panel of intestinal Wnt target genes for its restricted crypt expression. Here, using two knock-in alleles, we reveal exclusive expression of Lgr5 in cycling columnar cells at the crypt base. In addition, Lgr5 was expressed in rare cells in several other tissues. Using an inducible Cre knock-in allele and the Rosa26-lacZ reporter strain, lineage-tracing experiments were performed in adult mice. The Lgr5-positive crypt base columnar cell generated all epithelial lineages over a 60-day period, suggesting that it represents the stem cell of the small intestine and colon. The expression pattern of Lgr5 suggests that it marks stem cells in multiple adult tissues and cancers.

     • Google Citation = 4762 / Web of Science = 3153

3. Prof Zhang Yong-Wei, Institute of High Performance Computing (IHPC) 

4. Professor Dusit (Tao) Niyato, School of Computer Science and Engineering (SCSE) Nanyang Technological University

5. Professor Xiaodong CHEN, School of Materials Science and Engineering, Nanyang Technological University

6. Professor Dingyuan TANG, School of Electrical and Electronic Engineering, Nanyang Technological University

7. Assistant Professor Guoqing CHANG, Division of Physics and Applied Physics, Nanyang Technological University

8. Professor Wong Tien Yin, Duke-NUS Medical School & Department of Ophthalmology, Yong Loo Lin School of Medicine, National University of Singapore

9. Associate Professor Qiu Cheng Wei, Department of Electrical and Computer Engineering, National University of Singapore

End Note

• This is a viewpoint article that is highly systematic and highly-opinionated. 

• Any feedback, please email to research@limjunlong.com .

• Any new content suggestions, please email to research@limjunlong.com .

Reference with Abstract

• Lutz Bornmann, and Hans‐Dieter Daniel, " What do we know about the h index?," Journal of the American Society for Information Science and Technology, vol. 58, no. 9, pp. 1381-1385, Jul 2007. DOI: 10.1002/asi.20609

  • Abstract (Modified): Jorge Hirsch recently proposed the h index to quantify the research output of individual scientists. The new index has attracted a lot of attention in the scientific community. The claim that the h index in a single number provides a good representation of the scientific lifetime achievement of a scientist as well as the simple calculation of the h index using common literature databases lead to the danger of improper use of the index. We describe the advantages and disadvantages of the h index and summarize the studies on the convergent validity of this index. We also introduce corrections and complements as well as single‐number alternatives to the h index.

• L Egghe, "An improvement of the h-index: The g-index," ISSI newsletter, 2006.

  • Brief: For a set of papers, ranked in decreasing order of the number of citations that they received, the h-index is the (unique) highest number of papers that received h or more citations.

• J. E. Hirsch, " Does the h index have predictive power?," In Proc. of National Academy of Sciences, vol. 104, no. 49, pp. 19193-19198, Oct. 2007. DOI: 10.1073/pnas.0707962104

  • Abstract: Bibliometric measures of individual scientific achievement are of particular interest if they can be used to predict future achievement. Here we report results of an empirical study of the predictive power of the h index compared with other indicators. Our findings indicate that the h index is better than other indicators considered (total citation count, citations per paper, and total paper count) in predicting future scientific achievement. We discuss reasons for the superiority of the h index.

• S. Alonso, F. J. Cabrerizo, E. Herrera-Viedma, and F. Herrera, " h-Index: A review focused in its variants, computation and standardization for different scientific fields," Journal of Informetrics, vol. 3, no. 4, pp. 273-289, Oct. 2009. DOI: 10.1016/j.joi.2009.04.001

  • Abstract: The h-index and some related bibliometric indices have received a lot of attention from the scientific community in the last few years due to some of their good properties (easiness of computation, balance between quantity of publications and their impact and so on). Many different indicators have been developed in order to extend and overcome the drawbacks of the original Hirsch proposal. In this contribution we present a comprehensive review on the h-index and related indicators field. From the initial h-index proposal we study their main advantages, drawbacks and the main applications that we can find in the literature. A description of many of the h-related indices that have been developed along with their main characteristics and some of the works that analyze and compare them are presented. We also review the most up to date standardization studies that allow a fair comparison by means of the h-index among scientists from different research areas and finally, some works that analyze the computation of the h-index and related indices by using different citation databases (ISI Citation Indexes, Google Scholar and Scopus) are introduced.

• Judit Bar-IIan, " Which h-index? — A comparison of WoS, Scopus and Google Scholar," Scientometrics, vol. 74, no. 2, pp. 257-271, Feb. 2008. DOI: 10.1007/s11192-008-0216-y

  • Abstract: This paper compares the h-indices of a list of highly-cited Israeli researchers based on citations counts retrieved from the Web of Science, Scopus and Google Scholar respectively. In several case the results obtained through Google Scholar are considerably different from the results based on the Web of Science and Scopus. Data cleansing is discussed extensively.

• Bihui Jin, Li Ming Liang, Ronald Rousseau, and Leo Egghe, " The R- and AR-indices: Complementing the h-index," Chinese Science Bulletin, vol. 52, no. 6, pp. 855-863, Mar. 2007. DOI: 10.1007/s11434-007-0145-9 

  • Abstract: Based on the foundation laid by the h-index we introduce and study the R- and AR-indices. These new indices eliminate some of the disadvantages of the h-index, especially when they are used in combination with the h-index. The R-index measures the h-core’s citation intensity, while AR goes one step further and takes the age of publications into account. This allows for an index that can actually increase and decrease over time. We propose the pair (h, AR) as a meaningful indicator for research evaluation. We further prove a relation characterizing the h-index in the power law model.

• Lutz Bornmann, and Hans-Dieter Daniel, " Does the h-index for ranking of scientists really work?," Scientometrics,  vol. 65, no. 3, pp. 391-392, Dec. 2005. DOI: 10.1007/s11192-005-0281-4

  • Abstract (Modified): Hirsch has proposed the h-index as a single-number criterion to evaluate the scientific output of a researcher: A scientist has index h if h of his/her N_p papers have at least h citations each, and the other (N_p − h) papers have fewer than h citations each. In a study on committee peer review we found that on average the h-index for successful applicants for post-doctoral research fellowships was consistently higher than for non-successful applicants.

• n.d. (n.d.). A*STAR researchers among the world’s highly cited. Retrieved Apr. 9, 2021 from A*STAR

• n.d. (n.d.). 25 NUS researchers rank among the world’s most cited scientists. Retrieved Apr. 9, 2021 from NUS

• Sierra Williams (2014, 31st March). Four reasons to stop caring so much about the h-index. Retrieved Apr. 9, 2021 from The London School of Economics and Political Science

  • The h-index is dumb when it comes to authorship.

  • The h-index ignores science that isn’t shaped like an article.

  • Comparing h-indices is comparing apples and oranges.

• This reference list is to be populated.

Image courtesy

By Philip Uglow from Pixabay  

Keywords

opinion, statistics, index, citation, impact factor, g-index, citation index, advantages, drawbacks, astar, i2r, h index, hindex, research, science, ntu, nus, sutd, sim, Singapore

Tags

#impact factor #index #i2r #astar #science #singapore #limjunlong