Recent from talks
Leiden Manifesto
Knowledge base stats:
Talk channels stats:
Members stats:
Leiden Manifesto
The Leiden Manifesto for research metrics (LM) is a list of "ten principles to guide research evaluation", published as a comment in Volume 520, Issue 7548 of Nature, on 22 April 2015. It was formulated by public policy professor Diana Hicks, scientometrics professor Paul Wouters, and their colleagues at the 19th International Conference on Science and Technology Indicators, held between 3–5 September 2014 in Leiden, The Netherlands.
The LM was proposed as a guide to combat misuse of bibliometrics when evaluating scientific research literature. Examples of commonly used bibliometrics for science, or scientometrics, are the h-index, impact factor, and websites displaying indicators such as Altmetrics. According to Hicks et al., these metrics often pervasively misguide evaluations of scientific material.
Motivations for codification of the Leiden Manifesto arose from a growing worry that "impact-factor obsession" was leading to inadequate judgement of scientific material that should be worthy of fair evaluation. The lead author of the LM, Diana Hicks, hoped that publishing in Nature would expand the ideas already commonplace in the scientometrics sphere to the broader scientific community. Although the principles of the LM are not new to scientometricians, Hicks et al. desired to create a unification of ideas to guide future editors and reviewers of scientific literature.
Interest in the reform of how research assessment is conducted is an ongoing debate in the scientific community. A well known declaration intended to quell the use of the impact factor, the San Francisco Declaration on Research Assessment was published around two years prior to the manifesto. The LM broadened the ideas presented in the DORA, which has now since been signed by over 2000 organizations and over 15000 individuals.
One of the main concerns about overuse of citation-based performance indicators came from the observation that smaller research organizations and institutions may be negatively affected by their metric indices. In one public debate at the Centre for Science and Technology Studies at Leiden University, it was acknowledged that indicators which measure citations may give "more weight to publications from fields with a high expected number of citations than to publications from fields with a low expected number of citations".
Although the main focus of the LM is based on the use of scientometrics for research evaluation, in its background, Hicks et al. also explain why overuse of metrics can adversely affect the wider scholarly community, such as the position of universities in global rankings. According to Hicks et al., scientific metrics such as citation rate are used far too much for ranking the quality of universities (and thus the quality of their research output).
The background for the LM describes why misusing metrics is becoming a larger problem in the scientific community. The journal impact factor, originally created by Eugene Garfield as a method for librarians to collect data to facilitate selecting journals to purchase, is now mainly used as a method of judging journal quality. This is seen by Hicks et al. as an abuse of data in order to examine research too hastily. For example, an impact factor, while a good metric to measure the size and experience of a journal, may or may not be sufficient to accurately describe the quality of its papers, and even less so for a single paper.
Consisting of ten concise principles, along with a description for each, the Leiden Manifesto aims to reconstruct the way that research evaluations by academic publishers and scientific institutions are done. Its emphasis lies in detailed and close evaluation of research, rather than the excessive use of quantitative data in evaluations. It aims to promote academic excellence and fairness with thorough scrutiny, as well as remove possible perverse incentives of using scientometrics, such as judgement of academic capability and university quality.
Hub AI
Leiden Manifesto AI simulator
(@Leiden Manifesto_simulator)
Leiden Manifesto
The Leiden Manifesto for research metrics (LM) is a list of "ten principles to guide research evaluation", published as a comment in Volume 520, Issue 7548 of Nature, on 22 April 2015. It was formulated by public policy professor Diana Hicks, scientometrics professor Paul Wouters, and their colleagues at the 19th International Conference on Science and Technology Indicators, held between 3–5 September 2014 in Leiden, The Netherlands.
The LM was proposed as a guide to combat misuse of bibliometrics when evaluating scientific research literature. Examples of commonly used bibliometrics for science, or scientometrics, are the h-index, impact factor, and websites displaying indicators such as Altmetrics. According to Hicks et al., these metrics often pervasively misguide evaluations of scientific material.
Motivations for codification of the Leiden Manifesto arose from a growing worry that "impact-factor obsession" was leading to inadequate judgement of scientific material that should be worthy of fair evaluation. The lead author of the LM, Diana Hicks, hoped that publishing in Nature would expand the ideas already commonplace in the scientometrics sphere to the broader scientific community. Although the principles of the LM are not new to scientometricians, Hicks et al. desired to create a unification of ideas to guide future editors and reviewers of scientific literature.
Interest in the reform of how research assessment is conducted is an ongoing debate in the scientific community. A well known declaration intended to quell the use of the impact factor, the San Francisco Declaration on Research Assessment was published around two years prior to the manifesto. The LM broadened the ideas presented in the DORA, which has now since been signed by over 2000 organizations and over 15000 individuals.
One of the main concerns about overuse of citation-based performance indicators came from the observation that smaller research organizations and institutions may be negatively affected by their metric indices. In one public debate at the Centre for Science and Technology Studies at Leiden University, it was acknowledged that indicators which measure citations may give "more weight to publications from fields with a high expected number of citations than to publications from fields with a low expected number of citations".
Although the main focus of the LM is based on the use of scientometrics for research evaluation, in its background, Hicks et al. also explain why overuse of metrics can adversely affect the wider scholarly community, such as the position of universities in global rankings. According to Hicks et al., scientific metrics such as citation rate are used far too much for ranking the quality of universities (and thus the quality of their research output).
The background for the LM describes why misusing metrics is becoming a larger problem in the scientific community. The journal impact factor, originally created by Eugene Garfield as a method for librarians to collect data to facilitate selecting journals to purchase, is now mainly used as a method of judging journal quality. This is seen by Hicks et al. as an abuse of data in order to examine research too hastily. For example, an impact factor, while a good metric to measure the size and experience of a journal, may or may not be sufficient to accurately describe the quality of its papers, and even less so for a single paper.
Consisting of ten concise principles, along with a description for each, the Leiden Manifesto aims to reconstruct the way that research evaluations by academic publishers and scientific institutions are done. Its emphasis lies in detailed and close evaluation of research, rather than the excessive use of quantitative data in evaluations. It aims to promote academic excellence and fairness with thorough scrutiny, as well as remove possible perverse incentives of using scientometrics, such as judgement of academic capability and university quality.