Publication Metrics

Research Impact 

By 3rd Year of Post-doc in 2021, Jun Long has:

#limjunlong Original Series

Please read the disclaimer first before continuing reading if its your first time reading on this site.

Introduction

One of the most common metric used by managerial roles nowadays for 'judging' a person research capability is the h-index. A possible reason for them to used it is the stereotype behaviour and protective nature for more experienced and elderly researchers. While its true that prestigious awards and fellowship awards correlate their awardees' success with the h-index but what many readers are missing out there are that there are in deed many additional criteria. All those success researchers are highly established in their respectively fields and have very long duration lifetime achievement and history (~30 to even 50 years!). Most are current or have held top research positions in their institute or organisation with countless presentations as keynote and invited speakers.

#doyouknow

Being well-known researchers, if you Google their name, their names should appears top ranked if not on the first page.

Metric

In research, a metric is a quantifiable measurement used to assess various aspects of research output, such as its quality, impact, and productivity. Metrics are used to evaluate the performance of individual researchers, research institutions, and even entire fields of research. It is important to keep in mind that there are limitations to consider when evaluating competencies using metrics indicator and metric tool.

Metric Indicators

Some commonly used metrics, for informative purpose, in research include:

Metric Tools

Several tools are available to measure and track research metrics:

Google Scholar

Note: This is not a paid advertisement.

Google Scholar (https://scholar.google.com/) is one of the most common tools for calculating publication metrics. The Google account can associate with Android smartphone to build any researcher research custom profile.

After adding in all the long list of publications, the Google user can share the profile to the public domain to let other people see their research impact, experiences and history. One of the most prominent item on  the Google Scholar is the:

On a smartphone, we have to click on CITED BY words to see the metrics due to the small size of the screen.

Total Citation

The most obvious indicator for any human who understand the number system. Any manuscript published will take a few years to several years to obtain a sizeable citation value per year before taping off or oscillate over the years. In most cases, we can generalised that a new postdoc will have the higher likelihood of having a lower total citation than another decade old postdoc. 

Time is necessary

Even in today's cloud technology, hyper fast internet connectivity and ever fast processors, it will take time to 'do marketing' for any published work. 

Take for example, a conference paper can be further improve with more simulation and/or experiments which can be rewritten into a journal paper, vice versa; all the tasks will take time to do and complete. Not to mention that even online-only journals do not publish on daily basis and reputable journals may take any time ranging from 2 months to 1 years from the time upon receiving the manuscript to being published. 

Which is to say, the person who just read your paper right now and decided to cite it (as it is useful) may not have his/her paper capture by index to count into your total citation right away, not today or even tomorrow. Other reasons may be it takes time for search engine to index and figure out how to link related papers together, generating DOI link, etc. before a potential cite-reader  find your published paper.

Citation is important

Certain researcher managers will emphasis their researchers to publish in their curated list of top journals / conferences. Taking a closer look at prestigious well known journals, they have plenty of journal papers that are never ever cited even after so many years. Enough said about 'quality of paper' in a big impact factor journal.

h-index - Overview

It is a common misrepresentation that this single-number metrics can represent a researcher's scientific lifetime achievement. But before we go deeper, we will go through some of the basic of this metric. The h-index is an author-level metric that calculates the citation impact of one's publication emphasising mostly on the most cited papers and citation power count through the author publication lifetime.

To calculate this h-index number, we first list all the researcher's publication sorted according to the total number of citations. Next from the most cited paper, we start with h-index value of 1. Then we go down one ranked of the most cited paper and increase the numerical value of h-index by 1. We stop going down the ranks when the cited paper count is smaller than then the numerical value of h-index at that rank. (Click here to see how Wikipedia explains it).

Advantages

We are living creatures living in a world where time is limited and we do not have the technology to go time travel yet. Most researchers working in top research organisations do not have the leisure of time to dig out or verify the history of a new postdoc or experienced research fellow CV hence a quick first cut by the h-index value. 

As like many older proposed system or solutions, the h-index was proposed in 2005 by Jorge E. Hirsch, a physicist, to address the overreliance on:

when gauging a researcher's past performance and capability. 

Don't judge a book by its cover

The h-index is also commonly used by non-experience and non-technical hiring managers too. One can choose to perceive thoughts that a person capability in relation to their h-index, hence the h-index is how they label a person by perceived 'book cover'.

Well established researchers and well trained hiring managers often look in more than just the past impact of those potential hires. In a successful organisation, an important criteria is the character of the person. In this modern world of social media like Facebook, Instagram and LinkedIn, large corporations have began to request hiring staff to screen potential employee's trait, habit and characteristic by examining their post/comments and whether they can possibility 'sync' with their current employees. 

Weakness in the metrics

Clearly we see that that maximum value of h-index is the maximum number of publication that one researcher has ever published. It is worth noting that the total citations and number of papers are counted regardless whether the author is the first author, the middle author or the last author of a long list of co-authors. Newer international publishers may have extra section in their published papers to include the author contributions such as funding acquisition, project management, project administration or supervision all of which is indirect research work and may or may not influence the success rate of the research or experiments. In other words, if one has ample resources, one can publish as many paper as they like by allocating funding.


The h-index has higher sensitivity to highly cited papers and low sensitivity to the least cited papers. In other words, highly cited papers are the determination of the h-index. Once the top cited papers surpass the h top class, it is totally unimportant whether these paper get 50, 100 or 1000 more citations. Researchers with extremely cited paper will have similar or equal h-index as other nonextremely cited paper researchers.

Manipulation to increase h-index

Self-citation is often done and acceptable throughout the research community. It is important to note that not all self-citation is coercive, or indeed improper. When done correctly, it will give a proportion of credits to the author or point curious readers for further reading in relation between the newly published work and cited author's past work. 

The problem comes when an author(s), in a regular paper, knowingly and intentionally cite numerous of their paper in one go by designing the manuscript in a particular miraculous way. On the other side, journal can also become on the dark side by siding with the unethical authors to inflate the journal's impact factor for the reason of artificially boosting the journal's scientific reputation by coercive or other targeted methodologies.

Even in the context of a review paper or invited paper, the publisher's editor / associate editors / reviewers have to carefully examine the manuscript's reference list and invoke the manuscript's authors to make appropriate amendments to the manuscript before accepting and publishing it. Failure to do so may cause the many decades old ecosystem of the research community to collapsed and lose trust from funding sources. Big influential index like Thomson Reuters' Journal Citation Reports is known to temporary exclusion journals for such miraculous  behavior. 

i10-index

Right under the row of h-index in Google Scholar is the i10-index.

The i10-index is a metric that is used to measure an individual researcher's productivity and impact based on their publication record. It was developed by Google Scholar and is calculated by counting the number of publications that an author has written that have been cited at least ten times. For example, an author with an i10-index of 20 has written 20 papers that have each been cited at least 10 times. 

"The i10-index is intended to provide a simple and objective way of comparing the research impact of different researchers, regardless of their field or the number of years they have been active in research."

Highly Cited Researchers in Singapore

Selected Singapore List in Year 2020

Selected highest cited paper for your information

End Note

Reference with Abstract

Image courtesy

By Philip Uglow from Pixabay  

Keywords

opinion, statistics, index, citation, impact factor, g-index, citation index, advantages, drawbacks, astar, i2r, h index, hindex, research, science, ntu, nus, sutd, sim, Singapore

Tags

#impact factor #index #i2r #astar #science #singapore #limjunlong