Performance Figures in Research Assessment
Based on the institutional repository OceanRep the GEOMAR Library provides performance figures for internal and external research assessments.
- Citation Analysis
- Tools
- When working with figures from citation analysis these points are worth to consider
- Hirsch-index (h-Index)
- Impact Factor of Journals
- altmetrics
"Citations in the actual sense are common practice in scientific literature since the middle of the 19th century. Metaphorically speaking, the citation network of scientific articles is the glue that links publications with related content. The number of citations is seen as a direct measure for the (documented) resonance or the impact an article, a journal or a scientist has generated so far."
From: The impact of Solid State Communications in view of the ISI Citation data, DOI: dx.doi.org/10.1016/S0038-1098(03)00442-3
Science Citation Index:
The concept of citation analysis was developed in the late 1950's. The central tool is the database Science Citation Index (SCI), established by the Institute for Scientific Information (ISI), Philadelphia, USA, now part of the Thomson-Group. Its fundamental idea is to use publications of scientists and include references as an assessment criterion to evaluate the relevance of their scientific work.
Over the years the SCI was complemented by the SSCI (Social Sciences Citation Index) and the AHCI (Arts and Humanities Citation Index). During the last years CPCI (Conference Proceedings Citation Index) and BCI (Book Citation Index) were added. The joint platform is known under the name "Web of Science". Under the same platform there are more databases which can be licensed (Web of Knowledge).
SCI only indexes a defined selection of journals. Find here more on the selection process.
Scopus
2004 Elsevier launched a competitive product named Scopus. Scopus covers more journals than SCI and is also based on a defined list.
Google Scholar Citation
Contrary to this Google Scholar (GS) aggregates metadata via text mining. The idea is to identify as many scientific publications on the net as possible. Consequently different journal articles, book contributions and more is indexed.
Microsoft Academic Search
A similiar approach is made by Micorsoft Academic Search.
Different sources generate different numbers.
Depending on the sources of the base of course results like the h-index come out with different values.
| unique author | incompletenesses and errors in reference lists |
| negativ citations | temporal delays in the citation process |
| self citations | new research processes are recognized disproportionately late |
| citation cartels | age of the cited person |
| who is cited (prestige...) | amount of average citations heavily depends on the subject (differs by factor 7) |
| undocumented influences (acknowledgements, etc.) | majority of all publications is not cited at all, or only a few times |
Hirsch-index (h-Index)
A newer way to calculate comparable numbers of citations from different authors: "The h-index is indicated by a green horizontal line in Web of Science. The number of items above this line, which is h, have at least h citations. For example, an h-index of 20 means that there are 20 items that have 20 citations or more."
Developed by J.E. Hirsch and published in Proceedings of the National Academy of Sciences of the United States of America 102 (46): 16569-16572 November 15 2005.
Impact Factor of Journals
Another metric is the calculation of impact factors of journals.
Thomson-Reuters offers such a opportunity with the Journal Citation Report (JCR) (GFZ access only). Here inter alia the Journal Impact Factor (JIF) of individual journals is calculated.
Calculation for 2000 S = number of articles and reviews, which were published 1998 and 1999 in specific journals (Source Items) R= number of citations in 2000 of all publications, which call the in S evaluated journals year's issues 1998 and 1999 as source (References) R/S = Journal Impact Factor |
SCImago Journal & Country Rank (SJR) is based on Scopus and also offers journal based evaluation.
Such metrics always refer to the entire magazine and can not be used as personal data. They are an indicator of "relative importance" within a subject area, a comparison across different disciplines doesn't make sense. Factors to consider: discipline, share of methodologically articles, review articles proportion, original language, period of data analyzed. All said things about quote analysis has an effect. The "San Francisco Declaration on Research Assessment (DORA)" formulated clear criticism of the excesses of the use of the impact factor in research evaluation. |
Altmetrics
Next to established metrics such as SCI and Scopus, the possibilities of "Altmetrics" are been studied for years.
"altmetrics" is the creation and study of new metrics based on the Social Web for analyzing, and informing scholarship.
J. Priem, D. Taraborelli, P. Groth, C. Neylon (2010), Altmetrics: A manifesto, (v.1.0), 26 October 2010. altmetrics.org/manifesto
Several publishers like Copernicus Publishing now establish a programme called ALM (Article-level metrics) to quantify the usage, impact, saves, and discussion of scholarly work at the article level.
In detail the usage is collected from:
- Individual accesses to the Copernicus library servers (download statistics)
- Impact is counted from CrossRef and Google Scholar citations
- Saves are counted from reference management tools CiteULike and Mendeley
- Discussions are represented by Research Blogging, Facebook, ScienceSeeker, Nature Blogs, Wikipedia, Wordpress.com, Reddit, and Google Blogs.
What's the use?
In comparison to the traditional way of measuring impact at the journal level, ALMs offer a more informative way of assessing the overall influence and reach of the articles themselves.
Article-level metrics are available for all articles of journals published by Copernicus Publications. Authors can stay up to date with their published articles and share the information about the impact of their published work with peers, funding institutions, research bodies, and the overall scientific community.
It will be interesting to see how quickly ALM becomes a standard in the majority of publishing houses (it is no coincidence that Open Access publishers lead the way) and how quickly authors appreciate the additional information to their publications.
[GFZ Potsdam]
