Impact Indicators

Number and percentage of publication in the top 1% and 10% based on citations by category, year, and document type.

The % Documents in Top 1% indicator is the top one percent most cited documents (as defined in the description of Average Percentile) in a given subject category, year and publication type divided by the total number of documents in a given set of documents, displayed as a percentage. A higher value is considered to be higher performance. A value of “1” for a set of documents represents that one percent of the publications in that set are in the top one percent of the world regardless of subject, year and document type and would therefore be considered to be performing at the same level as world average. A value above “1” represents that more than one percent of papers in the set are in the top one percent of the world and a value of less than “1” would represent that less than one percent of the papers in the set are in the top one percent of the world.

The % Documents in Top 1% indicator is considered to be an indicator of research excellence as only the most highly cited papers would make the top one percent in their respective field, year and document type. The indicator can be used in conjunction with other indicators to provide a more complete picture of performance. The % Documents in Top 1% indicator can be applied to any level of aggregation (author, institution, national/international, field).

Although the top one percent is a relevant measure of excellence, by its nature it is typically only a small percentage of any document set and therefore the statistical relevance of small sample sizes is a significant concern.

The % Documents in Top 1% is best used with large datasets such as the accumulated publications of an institution, country or region and for a publication window of several years. The % Documents in Top 10% is very similar to the % Documents in Top 1% simply with a threshold of 10 percent instead of one percent. Therefore, typical performance will be around a value of 10 and values of higher than 10 would be considered above average performance. The two indicators complement each other very well to give a broader picture of highly performing research (10 percent) and excellence (one percent).

The % Documents in Top 10% is also more appropriate than the % Documents in Top 1% when the size of the data set is smaller. However, it is still only appropriate for large to medium size data sets and should be used with a great deal of caution when looking at small datasets such as the output of an individual author.

Complementary indicators include:

• Normalized Citation Impact
• Journal Normalized Citation Impact
• Average Percentile

The following is a sample citation distribution. InCites also includes pre-calculated indicators of collaboration which can be used for complementary performance indicators.

Publications in the Top 1%

Percentage of publications in the top 1% based on citations by category, year, and document type.

When to Use It

• Research excellence: % Documents in Top 1% is an indicator of research excellence, as only the most highly cited papers would make the top 1% in their respective field, year, and document type.
• Large datasets: % Documents in Top 1% is best used with large datasets such as the accumulated publications of an institution, country or region, and for a publication window of several years.

Limitations and Precautions

Small datasets: Although the top 1% is a relevant measure of excellence, by its nature it is typically only a small percentage of any document set and therefore the statistical relevance of small sample sizes is a significant concern. This, along with the % Documents in Top 10% indicator, should be used with a great deal of caution when looking at small datasets such as the output of an individual researcher.

Top 10%: % Documents in Top 1% can be used in conjunction with other indicators to provide a more complete picture of performance. It is complemented very well by the % Documents in Top 10% indicator, giving a broader picture of high performance (10%) and excellence (1%). The % Documents in Top 10% is also more appropriate than the % Documents in Top 1% when the size of the dataset is smaller, but is still more appropriate for large to medium size datasets.

How It's Calculated

Top 1% most cited documents (as defined in the description of the Average Percentile indicator) in a given subject category, year, and publication type ÷ total number of documents in a given set of documents = % Documents in Top 1%

How to Interpret It

A higher value is considered to be higher performance. A value of “1” for a set of documents represents that 1% of the publications in that set are in the top 1% of the world regardless of subject, year, and document type and would therefore be considered to be performing at the same level as world average. A value above “1” represents that more than one percent of papers in the set are in the top 1% of the world and a value of less than “1” would represent that less than 1% of the papers in the set are in the top 1% of the world.

Complementary Indicators

• % Documents in Top 10%
• Category Normalized Citation Impact
• Journal Normalized Citation Impact
• Average Percentile

The percentage of publications in a set that have received at least one citation. It shows the extent to which other researchers in the scientific community utilize the research output produced by an entity. You can also think of this indicator as the inverse of the number of papers that didn’t get cited.

The %Documents Cited indicator will vary depending on the selected time period and publication types included in the analysis. The % Documents Cited is not a normalized indicator. For example, if the analysis includes documents that have been published during the current or recent years, some of these documents may not have had time to accrue citations.

Complementary indicators include:

• Citation Impact
• Impact Relative to World

Calculate the citation impact of a set of documents by dividing the total number of citations by the total number of publications. Citation impact shows the average number of citations a document received.

Citation Impact has been extensively used as a bibliometric indicator in research performance evaluation. You can also apply it at all organization levels. However, there are limitations. For example, it ignores the total volume of research outputs.

The following table hows an example of the Citation Impact for two researchers. Researcher A has only one publication that received 50 citations while Researcher B has published 10 documents that received 200 citations. Researcher A has a higher Citation Impact (50) than Researcher B (20), even though Researcher B published more documents and received more citations overall.

	Total Publications	Total Citations	Citation Impact
Researcher A	1	50	50
Researcher B	10	200	20

At the field level, the Citation Impact of certain disciplines is often higher than in other scientific fields due to several factors, like the degree to which references from other fields are cited.

This is a different version of Times Cited per year depicting the number of citations received in each year. In other words, the number of documents from each year that cite (reference) a paper published by a selected institution in same year or prior. For example, for any source document, count citing documents, and group by publication year of citing document. Brunet Institute published 322 documents in 2016, and they were cited 6736 times in 2016, and 7880 times in 2017.

In 2002 Burnet Institute published 19 publication and from that only 1 citation received in that year. Now, in 2003 published 40 publications and from that year it received 66 citations, and this include current and previous year. i.e., Burnet Institute published 59 publication (2002+2003) and from that 66 citations are received in 2003, and it goes on...

Sample download

The Hirsh index (h-index) was introduced by J. Hirsch in 2005. A researcher has an h-index if they have at least h publications for which they received at least h citations. For example, Researcher A has an h-index of 13 if they published at least 13 documents for which they received at least 13 citations. Its popularity as a bibliometric indicator is because it combines productivity (number of documents) and impact (number of citations) in one index.

You can apply the h-index to any level of aggregation (author, institution, journal, etc.), and it can reveal information about how citations are distributed over a set of documents. At the author level, the h-index indicates a researcher’s lifetime scientific achievements. Some clear advantages of the h-index are:

• It's a mathematically simple index
• It encourages large amounts of impactful research work while at the same time discourages publishing unimportant output
• A single highly cited publication do not influence the h-index (unlike the Citation Impact).

However, the h-index is a time-dependent measure: it is proportional to the length of a researcher’s career and how many articles they published. For example, early career researchers would be at a disadvantage when compared to more senior researchers because the latter would have had more time to produce more work and receive more citations for their output.

	Total Publications	Total Citations	Citation Impact	H-Index
Researcher A	1	50	50	1
Researcher B	10	200	20	10
Researcher C	10	200	20	5

This table shows an example of how to apply h-index at the author level:

• Researcher A received 50 citations with only one publication.
• Researcher B published 10 documents with 20 citations for each document.
• Researcher C shares the same number of publications and citations as Researcher B.

According to the definition of the h-index, Researcher A, who has only one publication and 5ra0 citations, will have an h-index of 1. Researcher B, who has 10 publications and 20 citations for each publication, will have an h-index of 10. Although Researcher C has the same number of published documents and citations as Researcher B, their citations are more concentrated in five documents than in the rest of their work. This is why Researcher C has an h-index of 5 and not 10.

In this example we don't account for the researchers’ ages (the time interval between when the first and last document were published) and the disciplines in which the researchers are active. The h-index can be very different across disciplines due to differences in the average citation rates, and therefore sensitive to the disciplinary background of research output, as research entities publish in different subject mixes.

Assessing the productivity of a large set of publications is the first step in a series of bibliometric analysis that we can apply in order to obtain a deeper understanding of the performance of our research output.

WoS Author Records

These two new indicators are added to the WoS Author Record analysis in the Researchers explorer to help with the assessment of the author self-citations’ impact. Self-citations occur when a document from a researcher is citing a prior document where the researcher was an author.

The calculation is based on the uniqueness of WoS Author Records, where the person name, the affiliations and the publications associated to the WoS Author Record of the researcher are taken into account.

This is calculated based on all the publications associated to the WoS Author Record of the researcher. InCites search provides the documents authored by the researcher according a number of filters, and these indicators are calculated according the sum of the citation count type in each publication.

Organizations and Publication Sources

All documents belonging to an organization have been grouped as an entity, then the entity received citations have been tracked to detect self-citations. In this case a self-citation is a citation received from the same entity that provided the citation. The same process has been applied for publication sources.

This indicator displays the citation impact of the set of publications as a ratio of world average. For this, the Citation Impact is divided against the Citation Impact of Global baseline