Hubbry Logo
AltmetricsAltmetricsMain
Open search
Altmetrics
Community hub
Altmetrics
logo
8 pages, 0 posts
0 subscribers
Be the first to start a discussion here.
Be the first to start a discussion here.
Altmetrics
Altmetrics
from Wikipedia
The original logotype from the Altmetrics Manifesto[1]
A ranking of five scholarly papers based on their Altmetric Attention Scores.
OOIR.org shows which scholarly papers are "trending" based on Altmetric Attention Scores.

In scholarly and scientific publishing, altmetrics (stands for "alternative metrics") are non-traditional bibliometrics[2] proposed as an alternative[3] or complement[4] to more traditional citation impact metrics, such as impact factor and h-index.[5] The term altmetrics was proposed in 2010,[1] as a generalization of article level metrics,[6] and has its roots in the #altmetrics hashtag. Although altmetrics are often thought of as metrics about articles, they can be applied to people, journals, books, data sets, presentations, videos, source code repositories, web pages, etc.

Altmetrics use public APIs across platforms to gather data with open scripts and algorithms. Altmetrics did not originally cover citation counts,[7] but calculate scholar impact based on diverse online research output, such as social media, online news media, online reference managers and so on.[8][9] It demonstrates both the impact and the detailed composition of the impact.[1] Altmetrics could be applied to research filter,[1] promotion and tenure dossiers, grant applications[10][11] and for ranking newly-published articles in academic search engines.[12]

Over time, the diversity of sources mentioning, citing, or archiving articles has gone down. This happened because services ceased to exist, like Connotea, or because changes in API availability. For example, PlumX removed Twitter metrics in August 2023.[13]

Adoption

[edit]

The development of web 2.0 has changed the research publication seeking and sharing within or outside the academy, but also provides new innovative constructs to measure the broad scientific impact of scholar work. Although the traditional metrics are useful, they might be insufficient to measure immediate and uncited impacts, especially outside the peer-review realm.[1]

Projects such as ImpactStory,[14][15] and various companies, including Altmetric,[14][16] Plum Analytics[14][17][18][19] and Overton[20] are calculating altmetrics. Several publishers have started providing such information to readers, including BioMed Central, Public Library of Science (PLOS),[21][22] Frontiers,[23] Nature Publishing Group,[24] and Elsevier.[25][26] The NIHR Journals Library also includes altmetric data alongside its publications.[27]

In 2008, the Journal of Medical Internet Research started to systematically collect tweets about its articles.[28] Starting in March 2009, the Public Library of Science also introduced article-level metrics for all articles.[21][22][29] Funders have started showing interest in alternative metrics,[30] including the UK Medical Research Council.[31] Altmetrics have been used in applications for promotion review by researchers.[32] Furthermore, several universities, including the University of Pittsburgh are experimenting with altmetrics at an institute level.[32]

However, it is also observed that an article needs little attention to jump to the upper quartile of ranked papers,[33] suggesting that not enough sources of altmetrics are currently available to give a balanced picture of impact for the majority of papers.

Important in determining the relative impact of a paper, a service that calculates altmetrics statistics needs a considerably sized knowledge base. The following table shows the number of artefacts, including papers, covered by services:

Website Number of artefacts
Plum Analytics ~ 52.6 million[34]
Altmetric.com ~ 28 million[35]
ImpactStory ~ 1 million[36]
Overton ~ 11 million[37]

Categories

[edit]

Altmetrics are a very broad group of metrics, capturing various parts of impact a paper or work can have. A classification of altmetrics was proposed by ImpactStory in September 2012,[38] and a very similar classification is used by the Public Library of Science:[39]

  • Viewed – HTML views and PDF downloads
  • Discussed – journal comments, science blogs, Wikipedia, Twitter, Facebook and other social media
  • Saved – Mendeley, CiteULike and other social bookmarks
  • Cited – citations in the scholarly literature, tracked by Web of Science, Scopus, CrossRef and others
  • Recommended – for example used by F1000Prime[40]

Viewed

[edit]

One of the first alternative metrics to be used was the number of views of a paper. Traditionally, an author would wish to publish in a journal with a high subscription rate, so many people would have access to the research. With the introduction of web technologies it became possible to actually count how often a single paper was looked at. Typically, publishers count the number of HTML views and PDF views. As early as 2004, the BMJ published the number of views for its articles, which was found to be somewhat correlated to citations.[41]

Discussed

[edit]

The discussion of a paper can be seen as a metric that captures the potential impact of a paper. Typical sources of data to calculate this metric include Facebook, Google+, Twitter, Science Blogs, and Wikipedia pages. Some researchers regard the mentions on social media as citations. For example, citations on a social media platform could be divided into two categories: internal and external. For instance, the former includes retweets, the latter refers to tweets containing links to outside documents.[42] The correlation between the mentions and likes and citation by primary scientific literature has been studied, and a slight correlation at best was found, e.g. for articles in PubMed.[4]

In 2008 the Journal of Medical Internet Research began publishing views and tweets. These "tweetations" proved to be a good indicator of highly cited articles, leading the author to propose a "Twimpact factor", which is the number of Tweets it receives in the first seven days of publication, as well as a Twindex, which is the rank percentile of an article's Twimpact factor.[28] However, if implementing use of the Twimpact factor, research shows scores to be highly subject specific, and as a result, comparisons of Twimpact factors should be made between papers of the same subject area.[28] While past research in the literature has demonstrated a correlation between tweetations and citations, it is not a causative relationship. At this point in time, it is unclear whether higher citations occur as a result of greater media attention via Twitter and other platforms, or is simply reflective of the quality of the article itself.[28]

Recent research conducted at the individual level, rather than the article level, supports the use of Twitter and social media platforms as a mechanism for increasing impact value.[43] Results indicate that researchers whose work is mentioned on Twitter have significantly higher h-indices than those of researchers whose work was not mentioned on Twitter. The study highlights the role of using discussion based platforms, such as Twitter, in order to increase the value of traditional impact metrics.

Besides Twitter and other streams, blogging has shown to be a powerful platform to discuss literature. Various platforms exist that keep track of which papers are being blogged about. Altmetric.com uses this information for calculating metrics, while other tools just report where discussion is happening, such as ResearchBlogging and Chemical blogspace.

[edit]

Platforms may even provide a formal way of ranking papers or recommending papers otherwise, such as Faculty of 1000.[44]

Saved

[edit]

It is also informative to quantify the number of times a page has been saved, or bookmarked. It is thought that individuals typically choose to bookmark pages that have a high relevance to their own work, and as a result, bookmarks may be an additional indicator of impact for a specific study. Providers of such information include science specific social bookmarking services such as CiteULike and Mendeley.

Cited

[edit]

The cited category is a narrowed definition, different from the discussion. Besides the traditional metrics based on citations in scientific literature, such as those obtained from Google Scholar, CrossRef, PubMed Central, and Scopus, altmetrics also adopt citations in secondary knowledge sources. For example, ImpactStory counts the number of times a paper has been referenced by Wikipedia.[45] Plum Analytics also provides metrics for various academic publications,[46] seeking to track research productivity. PLOS is also a tool that may be used to utilize information on engagement.[46]

Numerous studies have shown that scientific articles disseminated through social media channels (i.e. Twitter, Reddit, Facebook, YouTube, etc) have substantially higher biblometric scores (downlodas, reads and citations) than articles not advertised through social media. In the fields of plastic surgery,[47] hand surgery[48] and more, higher Altmetric scores are associated with better short-term bibliometrics.

Interpretation

[edit]

While there is less consensus on the validity and consistency of altmetrics,[49] the interpretation of altmetrics in particular is discussed. Proponents of altmetrics make clear that many of the metrics show attention or engagement, rather than the quality of impacts on the progress of science.[39] Even citation-based metrics do not indicate if a high score implies a positive impact on science; that is, papers are also cited in papers that disagree with the cited paper, an issue for example addressed by the Citation Typing Ontology project.[50]

Altmetrics could be more appropriately interpreted by providing detailed context and qualitative data. For example, in order to evaluate the scientific contribution of a scholar work to policy making by altmetrics, qualitative data, such as who's citing online[12] and to what extent the online citation is relevant to the policymaking, should be provided as evidence.[51]

Regarding the relatively low correlation between traditional metrics and altmetrics, altmetrics might measure complementary perspectives of the scholar impact. It is reasonable to combine and compare the two types of metrics in interpreting the societal and scientific impacts. Researchers built a 2*2 framework based on the interactions between altmetrics and traditional citations.[4] Further explanations should be provided for the two groups with high altmetrics/low citations and low altmetrics/high citations.[28][4] Thus, altmetrics provide convenient approaches for researchers and institutions to monitor the impact of their work and avoid inappropriate interpretations.

Controversy

[edit]

The usefulness of metrics for estimating scientific impact is controversial.[52][53][54][55] Research has found that online buzz could amplify the effect of other forms of outreach on researchers' scientific impact. For the nano-scientists that are mentioned on Twitter, their interactions with reporters and non-scientists positively and significantly predicted higher h-index, whereas the non-mentioned group failed.[43] Altmetrics expands the measurement of scholar impact for containing a rapid uptake, a broader range of audiences and diverse research outputs. In addition, the community shows a clear need: funders demand measurables on the impact of their spending, such as public engagement.

However, there are limitations that affect the usefulness due to technique problems and systematic bias of construct, such as data quality, heterogeneity and particular dependencies.[53] In terms of technique problems, the data might be incomplete, because it is difficult to collect those online research outputs without direct links to their mentions (i.e. videos) and identify different versions of one research work. Additionally, whether the API leads to any missing data is unsolved.[4]

As for systematic bias, like other metrics, altmetrics are prone to self-citation, gaming, and other mechanisms to boost one's apparent impact such as performing citation spam in Wikipedia. Altmetrics can be gamed: for example, likes and mentions can be bought.[56] Altmetrics can be more difficult to standardize than citations. One example is the number of tweets linking to a paper where the number can vary widely depending on how the tweets are collected.[57] Besides, online popularity may not equal to scientific values. Some popular online citations might be far from the value of generating further research discoveries, while some theoretical-driven or minority-targeted research of great science-related importance might be marginalized online.[28] For example, the top tweeted articles in biomedicine in 2011 were relevant to curious or funny content, potential health applications, and catastrophe.[4] Altmetric state that they have systems in place to detect, identify and correct gaming.[58] Finally, recent research has shown Altmetrics reproduce gendered biases found in disciplinary publication and citation practices: for example, journal articles authored exclusively by female scholars score 27% lower on average than exclusively male-authored outputs. At once, this same research shows 0 attention scores are more likely for male-authored articles.[59]

Altmetrics for more recent articles may be higher because of the increasing uptake of the social web and because articles may be mentioned mainly when they are published.[60] As a result, it might not be fair to compare the altmetrics scores of articles unless they have been published at a similar time. Researchers has developed a sign test to avoid the usage uptake bias by comparing the metrics of an article with the two articles published immediately before and after it.[60]

It should be kept in mind that the metrics are only one of the outcomes of tracking how research is disseminated and used. Altmetrics should be carefully interpreted to overcome the bias. Even more informative than knowing how often a paper is cited, is which papers are citing it. That information allows researchers to see how their work is impacting the field (or not). Providers of metrics also typically provide access to the information from which the metrics were calculated. For example, Web of Science shows which are the citing papers, ImpactStory shows which Wikipedia pages are referencing the paper, and CitedIn shows which databases extracted data from the paper.[61]

Another concern of altmetrics, or any metrics, is how universities or institutions are using metrics to rank their employees make promotion or funding decisions,[62] and the aim should be limited to measure engagement.[63]

The overall online research output is very little and varied among different disciplines.[28][4] The phenomenon might be consistent with the social media use among scientists. Surveys has shown that nearly half of their respondents held ambivalent attitudes of social media's influence on academic impact and never announced their research work on social media.[64] With the changing shift in open science and social media use, the consistent altmetrics across disciplines and institutions will more likely be adopted.

Ongoing research

[edit]

The specific use cases and characteristics is an active research field in bibliometrics, providing much needed data to measure the impact of altmetrics itself. Public Library of Science has an Altmetrics Collection[65] and both the Information Standards Quarterly and the Aslib Journal of Information Management recently published special issues on altmetrics.[66][67] A series of articles that extensively reviews altmetrics was published in late 2015.[68][69][70]

There is other research examining the validity of one altmetrics[4][28] or make comparisons across different platforms.[60] Researchers examine the correlation between altmetrics and traditional citations as the validity test. They assume that the positive and significant correlation reveals the accuracy of altmetrics to measure scientific impact as citations.[60] The low correlation (less than 0.30[4]) leads to the conclusion that altmetrics serves a complementary role in scholar impact measurement such as the study by Lamba (2020) [71] who examined 2343 articles having both altmetric attention scores and citations published by 22 core health care policy faculty members at Harvard Medical School and a significant strong positive correlation (r>0.4) was observed between the aggregated ranked altmetric attention scores and ranked citation/increased citation values for all the faculty members in the study. However, it remains unsolved that what altmetrics are most valuable and what degree of correlation between two metrics generates a stronger impact on the measurement. Additionally, the validity test itself faces some technical problems as well. For example, replication of the data collection is impossible because of the instant changing algorithms of data providers.[72]

See also

[edit]

References

[edit]
[edit]
Revisions and contributorsEdit on WikipediaRead on Wikipedia
from Grokipedia
Altmetrics, short for "alternative metrics," are quantitative indicators used to assess the broader societal and online with scholarly outputs—such as journal articles, datasets, and books—by aggregating data from sources including mentions, news coverage, blog discussions, references, and documents, thereby complementing or contrasting traditional bibliometric measures centered on peer citations. The term was coined in September 2010 by Jason Priem, then a doctoral student in , who proposed it via as a means to harness emergent web data for evaluating research influence more rapidly and diversely than citation delays allow. This innovation stemmed from recognition that digital platforms enable real-time tracking of attention, potentially revealing impacts on public discourse, , and interdisciplinary audiences overlooked by citation networks. Following Priem's introduction, altmetrics gained traction through early manifestos and tools like ImpactStory (co-founded by Priem) and Altmetric.com, established in 2011 by Euan Adie to compute aggregated "Attention Scores" weighting source volume, influence, and recency. These scores, visualized via "donuts" categorizing attention sources (e.g., Twitter, Reddit, mainstream news), have been integrated into publisher platforms and institutional repositories to signal potential reach, with empirical studies showing moderate positive correlations to eventual citation counts in fields like medicine and ecology, though often lagging in predictive power for long-term influence. Adoption has accelerated amid open access movements and funder demands for demonstrable societal return on research investment, yet altmetrics remain adjunctive rather than substitutive, as they prioritize visibility over validated quality or causal effects. Criticisms highlight altmetrics' vulnerability to noise, such as inflated scores from controversial or sensational content rather than rigorous , and susceptibility to manipulation via bots or coordinated sharing campaigns, undermining claims of measuring true impact. Peer-reviewed analyses, including evaluations against UK quality scores, reveal weak or inconsistent links to expert-assessed excellence across disciplines, suggesting altmetrics better proxy attention than substantive value and risk incentivizing performative over depth. Despite these limitations, ongoing refinements—such as and source weighting—aim to enhance reliability, positioning altmetrics as a partial tool in multifaceted amid evolving landscapes.

Origins and History

Coining of the Term and Initial Manifesto

The term altmetrics was coined by Jason Priem, then a doctoral at the at Chapel Hill's School of Information and Library Science, in a post on September 29, 2010. In the post, Priem wrote: "I like the term #articlelevelmetrics, but it fails to imply diversity of measures. Lately, I'm liking #altmetrics," positioning the concept as an alternative to traditional article-level metrics focused primarily on citations, emphasizing instead a broader array of online indicators. Shortly thereafter, Priem co-authored the Altmetrics Manifesto with Dario Taraborelli, Paul Groth, and Cameron Neylon, published online in October 2010. The manifesto critiqued conventional citation-based evaluation as "slow" and lagging "years behind real impact," arguing that it incentivizes "conventionality" and overlooks accountability in while most papers eventually garner some citations regardless of influence. It proposed altmetrics as complementary measures drawing from interactions (such as tweets and shares), mentions, references, coverage, downloads, and other web-based signals to quantify the "diverse, heterogeneous impacts" of scholarship that citations miss, including rapid dissemination in the "" of online scholarly discourse. This rationale stemmed from empirical observations of growing online scholarly activity, where web data provided near-real-time proxies for influence unavailable through delayed citation counts. The manifesto's reception was polarized: it garnered enthusiasm among and advocates for highlighting timely, non-elite forms of impact beyond journal gatekeeping, but elicited from bibliometricians, who questioned altmetrics' validity in reliably measuring substantive scholarly influence amid potential from transient online attention.

Early Developments and Key Milestones (2010–2020)

In , the launch of dedicated altmetrics aggregators marked a pivotal step in operationalizing the concept beyond theoretical manifestos. , founded by Euan Adie, began tracking scholarly mentions across , news outlets, and policy documents, providing researchers with aggregated attention data via embeddable widgets. Concurrently, Plum Analytics emerged, offering services to institutions for monitoring diverse impact indicators such as downloads, shares, and discussions. These tools capitalized on the expanding availability of APIs from platforms like and , facilitating automated collection of online interactions tied to research outputs via DOIs. By 2012, further milestones included the public launch of Impactstory, an open-source platform developed by Jason Priem and others to generate narrative profiles of research influence, integrating metrics from sources like blogs, , and academic networks. PLOS ONE advanced practical adoption that year by announcing its Altmetrics Collection, extending its 2009 Article-Level Metrics system—which already captured views, citations, and early social signals—to highlight articles with notable online buzz. Partnerships proliferated, with journals embedding provider badges (e.g., Altmetric's colorful donuts) to display real-time scores, enabling authors and readers to gauge broader societal engagement without relying solely on delayed citation counts. From 2016 to 2020, altmetrics saw maturation through exploratory integrations into evaluation practices, driven partly by empirical critiques of traditional metrics' vulnerabilities, such as citation cartels where coordinated groups artificially boost journal rankings via reciprocal citing. Publishers and funders piloted altmetrics in assessments to capture faster, multifaceted impacts, though challenges in data consistency persisted due to varying source weights and platform algorithm changes. This period's growth was underpinned by widespread DOI usage—reaching over 100 million assignments by 2018—and enhanced access, which lowered barriers to harvesting signals from expanding digital ecosystems, fostering aggregator scalability and cross-platform .

Definition and Core Concepts

Precise Definition and Scope

Altmetrics encompass quantitative indicators derived from online activities surrounding scholarly outputs, such as mentions, shares, and discussions on platforms (e.g., X, formerly ), blogs, news sites, and reference managers, capturing post-publication attention traces that are systematically trackable via public APIs. These metrics focus exclusively on digital footprints of engagement occurring after a work's release, excluding pre-publication processes like or traditional database citations, and are limited to verifiable, platform-specific counts rather than qualitative assessments of influence. For example, a research paper might accrue altmetric data from 50 X posts linking to it within days of publication, reflecting immediate visibility among online audiences. The core scope delineates measurable impacts as aggregated counts of interactions from predefined sources, such as policy citations in documents or saves in tools like , while excluding unquantifiable or offline influences like classroom adoptions or private discussions. Altmetrics do not extend to causal evaluations of real-world application, such as changes or technological implementations stemming from research, as these require longitudinal tracing beyond API-accessible data. Empirical studies from the , analyzing datasets from platforms like and blogs, demonstrated that altmetrics capture visibility in non-academic spheres but exhibit low to moderate with citation counts (Spearman rho often 0.1–0.3 across disciplines), indicating they delineate a separate dimension of attention with minimal overlap in identifying high-impact works. A key construct within this scope is the Altmetrics Attention Score, defined as a weighted summation of attention sources tailored to individual outputs, prioritizing rapid, diverse signals over normalized benchmarks. This score quantifies exposure breadth—for instance, weighting a outlet mention higher than a single —but inherently measures raw visibility without verifying comprehension, endorsement, or quality, as sources may include automated shares, bots, or dissenting commentary. Thus, while altmetrics empirically bound scholarly impact to observable online metrics, they necessitate interpretation cautious of gaming vulnerabilities, such as coordinated amplification campaigns observed in early platform data.

Distinction from Traditional Bibliometrics

Traditional , exemplified by metrics such as the —which quantifies a researcher's productivity and citation impact—and journal impact factors—which average citations received by articles in a journal over a two-year window—rely exclusively on formal peer citations within scholarly . These measures accumulate slowly, often requiring years for citations to accrue, as they depend on deliberate evaluation and incorporation by domain experts, ensuring a degree of rigor through vetted validation. In divergence, altmetrics aggregate signals from non-academic online activities, such as mentions and webpage views, enabling near-immediate feedback (frequently within days of publication) but introducing susceptibility to transient noise, including bots, spam, or coordinated campaigns that do not necessitate substantive engagement. Causal differences arise from their foundational mechanisms: bibliometric citations represent causal chains of expert acknowledgment and knowledge building, filtered by and relevance to advancing fields, whereas altmetrics track broader attention diffusion, often amplified by platform algorithms that prioritize virality and over evidentiary depth, potentially fostering chambers where ideological alignment or drives metrics independently of scholarly merit. Empirical analyses underscore this disconnect, with multiple studies reporting weak positive correlations between altmetric attention scores and citation counts—typically Pearson's r values under 0.3 across disciplines—suggesting altmetrics more reliably indicate early or than enduring intellectual influence. For instance, a 2014 comprehensive comparison of top articles found altmetrics captured distinct high-visibility outliers but failed to align with citation-based rankings, highlighting their non-equivalence in assessing impact. While bibliometrics exhibit shortcomings, such as field-specific biases (e.g., faster citation rates in biomedicine versus mathematics) and insensitivity to non-citation societal effects, they mitigate superficial inflation through their lag and exclusivity to expert networks. Altmetrics, conversely, risk overemphasizing manipulable or ephemeral signals—evident in cases where retracted papers garnered sustained online buzz post-retraction—without inherent safeguards against low-quality amplification, though they can reveal impacts invisible to citation logs, like policy uptake. This empirical divergence implies neither fully supplants the other; correlations remain too modest for altmetrics to predict bibliometric success reliably, positioning them as complementary diagnostics rather than proxies.

Data Sources and Measurement Categories

Social Media Mentions and Discussions

Social media mentions and discussions form a core component of altmetrics by tracking conversational engagement with scholarly outputs on platforms including (rebranded as X in 2022), , and blogs. These sources aggregate metrics such as shares, likes, retweets, comments, and direct mentions, which reflect public and peer discourse rather than passive consumption. , a primary provider, monitors over 14,000 blogs alongside social platforms for links and references to research, emphasizing discussion volume as an indicator of broader societal attention. Despite generating high volumes of data—often in the thousands of mentions for high-profile papers—these metrics exhibit a low due to factors like automated bot activity, self-promotional posts, and ephemeral trends that dilute meaningful scholarly dialogue. A 2012 analysis of altmetrics highlighted persistent noise from irrelevant or superficial interactions, even after normalization efforts, underscoring that raw counts rarely equate to substantive . Platforms like contribute niche discussions in subreddits, but these are prone to echo chambers and non-expert commentary, further complicating interpretability. Empirical studies from the , such as a 2013 examination of and other services, revealed statistically significant but weak correlations between tweet volumes and traditional metrics like citations (e.g., positive scores associated with higher citations across disciplines), with even looser ties to article downloads or sustained engagement. This suggests discussions serve more as early buzz indicators than reliable proxies for deeper influence, as short-lived conversations often fail to predict long-term scholarly uptake. In the 2020s, platform dynamics have intensified challenges; following the Twitter-to-X rebrand, overall activity declined yet remained above pre-2020 levels, while academic users reported reduced visibility and engagement due to algorithmic shifts favoring paid or high-follower accounts. Many scholars migrated to alternatives like Bluesky, eroding X's dominance in tracking research discussions and prompting altmetrics providers to adapt data collection amid fragmented audiences. Providers continue monitoring X for its residual role in academic sharing, but the shift highlights the fragility of reliance on single-platform discussions for robust altmetrics.

Views, Downloads, and Saves

Views in altmetrics encompass page views on publisher websites, captured through server logs or integrated , providing an early indicator of article accessibility shortly after . Downloads, similarly, track full-text retrievals such as PDF files, with often standardized under the COUNTER to enable consistent reporting and comparability across publishers and platforms. These usage metrics accumulate more rapidly than traditional citations, frequently registering within days of release, as they reflect immediate online consumption rather than delayed peer validation. Saves, or bookmarking actions, are derived from reference management tools like and , where users archive articles for personal libraries, signaling perceived future relevance. Open access publication enhances these metrics, with studies showing open access articles in medical journals achieving significantly higher page views and PDF downloads than subscription-based equivalents, attributed to barrier-free access. This boost underscores how publication models influence visibility, though raw counts from diverse sources may vary in reliability due to differences in tracking methodologies. Despite their , views, downloads, and saves primarily trace passive exposure and do not reliably correlate with comprehension or substantive impact, as metrics capture access events without verifying content or retention. For example, automated bots or cursory scans can inflate views, while saves may reflect topical interest over thorough review, limiting their interpretive value without contextual analysis. Thus, these indicators serve best as proxies for reach in altmetrics frameworks, complemented by more active measures elsewhere.

Recommendations, Citations, and Policy Documents

Altmetrics encompass references to scholarly outputs in policy documents, which include government guidelines, white papers, reports from policy institutes, and international organization publications such as those from the (WHO). These mentions typically involve explicit citations or discussions of research findings informing policy decisions, signaling potential real-world application beyond academia. Empirical studies using data reveal such policy citations are infrequent, affecting a very small proportion of scientific papers—often less than 1% in large samples from databases like —yet they carry substantial interpretive weight as indicators of societal influence due to the authoritative nature of the citing entities. Recommendations in altmetrics capture user-endorsed highlights from platforms like , where readers rate and suggest books, providing insight into public engagement with scholarly monographs or accessible texts. Similarly, Facebook recommendations track algorithmic or user-promoted shares that endorse content, though these are distinguished from mere mentions by their affirmative intent. These sources bridge informal public validation with structured feedback, offering a hybrid metric that complements traditional but remains susceptible to popularity biases rather than rigorous evaluation. Citations within altmetrics extend to non-journal contexts, such as early or alternative references in repositories like (PMC), clinical guidelines, and syllabi, which reflect practical adoption in , healthcare, or preliminary scholarly discourse. Clinical guidelines, for instance, document research integration into evidence-based medical protocols, with tracking citations from sources providing healthcare decision guidance. Government citations in policy overlap here, forming a hybrid category that approximates traditional in formality while emphasizing applied impact; however, these constitute a minor fraction of overall altmetric signals, typically under 5% of attention scores in domain-specific analyses, underscoring their rarity amid dominant volume.

Calculation Methods and Scoring Systems

Components of the Altmetric Attention Score

The Altmetric Attention Score (AAS) aggregates attention events from tracked online sources into a single metric through an automated, proprietary that emphasizes volume, source type, and contextual modifiers. Each unique mention—defined as one per individual per source to prevent duplication from repeated shares—is weighted according to the perceived reach of the platform or outlet, then adjusted for factors such as the poster's influence (e.g., follower count or account authority) and, in some cases, recency thresholds for historical data inclusion. The core computation follows a summation structure: AAS is the rounded sum across sources of (mention count from source * source weight * applicable modifiers). Source weights assign higher values to outlets with broader audiences; for instance, mainstream news articles receive a base weight of 8, blogs 5, policy documents or edits 3, and posts 1 (with at 0.25). Modifiers refine this: news sources are tiered by prominence (e.g., major outlets like amplified over niche sites), while mentions incorporate "reach" (audience size), "promiscuity" (tendency to share indiscriminately), and bias adjustments to discount low-effort or automated activity. Certain sources, like reads or citation databases, contribute zero weight by design. Illustrative examples highlight the aggregation without implying equivalence to scholarly quality. A 2023 research article garnering 100 unique mentions and 5 outlet citations might yield a base of 100 (tweets at weight 1) plus 40 ( at weight 8), totaling around 140 before modifiers potentially elevating it to approximately 150 if influential accounts or high-tier are involved. In practice, final scores diverge from raw tallies due to these adjustments; for example, 84 tweets combined with 2 news mentions produced an AAS of 85 rather than a simple 100, reflecting deductions for retweet discounts (0.85 factor) or lower-influence posters. Scores vary substantially by , with biomedical outputs often registering higher due to denser and ecosystems compared to fields like , where attention clusters in peer reviews or syllabi (both weight 1).

Algorithmic Weighting and Potential Biases in Computation

The Altmetric Attention Score assigns differential weights to mentions from various sources based on an that factors in source type, reach, and perceived influence, with documents and outlets typically receiving higher multipliers than individual posts such as tweets. For instance, a single mention in a document may contribute substantially more to the score than a tweet, reflecting assumptions about the former's authoritative impact, though exact multipliers remain proprietary and vary by context. These weights derive from heuristic judgments rather than rigorous empirical testing against outcomes like long-term citation accrual or policy adoption, leading to critiques that they embed unverified assumptions about source quality. Algorithm designers prioritize factors like outlet prestige and dissemination potential, yet analyses indicate limited validation, with weights potentially overemphasizing transient visibility over substantive engagement. A key bias arises from the algorithm's sensitivity to volume-driven attention, which amplifies sensational or timely topics prone to hype, as higher-weight sources like often prioritize novelty over depth. This can distort scores towards or spikes, deviating from neutral impact measurement. For example, in 2020, COVID-19-related research outputs saw altmetric scores surge due to intensive media coverage, with top papers achieving scores far exceeding norms in comparable fields, driven by aggregated mentions rather than balanced influence assessment. Such computational biases risk conflating raw with meaningful impact, as unweighted or low-weight sources (e.g., expert ) may signal deeper but contribute less, while hype cycles inflate scores without corresponding of causal influence. Empirical reveals that these choices, while intending to approximate societal , often lack transparency and , potentially perpetuating imbalances favoring accessible, populist channels over specialized ones.

Adoption and Practical Implementation

Integration in Academic Institutions and Publishers

Major publishers have incorporated altmetrics into their platforms to provide authors and readers with indicators of online attention beyond traditional citations. Elsevier began displaying Attention Scores and badges on article pages across hundreds of journals starting in 2015, integrating data into tools like to track mentions, news coverage, and policy citations. Similarly, implemented article-level metrics, including altmetrics components such as social shares and views, as early as 2013-2014 to highlight broader dissemination of content. These integrations allow publishers to embed altmetric widgets directly in digital object identifiers (DOIs) and journal interfaces, facilitating real-time visibility tracking for over 250 Elsevier titles by the mid-2010s. Academic institutions have adopted altmetrics for internal evaluations and reporting, often through institutional licenses to aggregators. Universities in the UK, for instance, leveraged altmetrics during preparations for the 2021 (REF) to quantify publication trends and societal engagement, with analyses showing higher attention scores for openly accessible outputs. Adoption rates vary, but by 2020, numerous higher education institutions subscribed to altmetrics services to benchmark departmental outputs against peers, driven by mandates from funders like emphasizing demonstrable public impact. Key motivations include capturing timely societal reach, particularly amplified by open access policies; studies indicate open access articles garner 20-50% higher altmetric scores due to increased online accessibility and sharing. This complements citation lags in fast-moving fields, enabling institutions to report on policy citations and media uptake in funding bids. While traditional bibliometrics remain dominant, altmetrics integration has grown empirically, with publisher dashboards reporting sustained use for author services and journal benchmarking since the 2010s.

Available Tools, Platforms, and Vendor Examples

, developed by , is a leading proprietary platform that aggregates attention data from sources including , news outlets, blogs, and policy documents, presenting it via embeddable badges and detailed visualizations such as the "donut" graphic representing the Attention Score. It offers free tools like the for browser-based querying of article metrics and access for non-commercial , enabling institutions and individuals to integrate real-time attention indicators into repositories and personal profiles. However, its proprietary aggregation methods limit full transparency in source selection and scoring. PlumX Metrics, provided by following its acquisition of Plum Analytics, categorizes interactions into five groups—captures (e.g., saves in ), citations, mentions, social shares, and usage (views/downloads)—and displays them through icons on publisher platforms without a single weighted score. This approach allows granular analysis of types but relies on 's for comprehensive coverage, with harvesting that may prioritize affiliated content. Dimensions, also from , supplies free badges that combine altmetrics with citation data in interactive formats, facilitating quick assessments of publication reach across academic and online channels. For transparency-focused alternatives, Impactstory stands out as an open-source platform enabling researchers to create profiles tracking diverse impacts via public APIs, integrating with for verifiable outputs and emphasizing accessible, shareable metrics without commercial gatekeeping. While proprietary vendors like and PlumX offer broader, automated tracking suitable for institutional integration, open tools such as Impactstory promote verifiability through code availability, though they may cover fewer sources due to reliance on volunteer-maintained data feeds.

Applications and Interpretive Frameworks

Valid Use Cases for Broader Impact Assessment

Altmetrics enable the tracking of dissemination into spheres by aggregating citations in official documents, offering quantifiable evidence of influence on processes that traditional citation metrics overlook. A study examining .com data found that mentions effectively identify articles shaping governmental and organizational agendas, with and document counts correlating to documented broader impacts in areas like and environmental regulation. In climate , for example, papers achieving high Altmetric Attention Scores through coverage—such as a 2023 of ExxonMobil's warming projections with a score of 8,686 from 823 outlets—have fueled debates on corporate responsibility, demonstrating how media uptake signals potential pathways to adoption. These metrics complement citation-based assessments by revealing interdisciplinary and societal engagement, particularly for publications that amplify reach beyond academic silos. Data from Pfizer-sponsored works (2014–2019) showed articles receiving markedly higher mentions and contributions to policy discussions compared to closed access counterparts, with 18 of the top 20 highest-scoring outputs being openly available. Similarly, in the environment and domain, altmetrics analysis of over 12,000 mentions and 1,500 news items for publications (2012–2021) provided dashboards of audience engagement—distinguishing researchers from public actors—and supported evaluations for and by evidencing non-scholarly . Such applications succeed when altmetrics are integrated as supplementary indicators, capturing rapid-response impacts like public discourse amplification in urgent fields. Empirical cases confirm their role in validating efficacy, where elevated scores from diverse sources predict sustained external attention without supplanting peer-reviewed rigor.

Methodological Limitations and Misinterpretation Risks

Altmetrics scores can mislead users by conflating raw attention volume with substantive validation or comprehension, as mentions and shares frequently prioritize over rigorous evaluation of content quality. For example, interactions often capture transient interest without indicating whether the audience has read, understood, or endorsed the underlying , leading to overinterpretation of virality as merit. A prominent illustration involves retracted publications, which retain elevated altmetric attention long after withdrawal; analysis of cross-platform data from 2010 to 2020 revealed that social media and news mentions for such papers declined minimally post-retraction, perpetuating potential misinformation through uncorrected visibility. In one dataset of over 3,000 retracted biomedical articles, 279 retracted within a year still accrued ongoing media and social engagement, underscoring how altmetrics fail to dynamically adjust for validity flags. Methodologically, altmetrics suffer from low inter-provider consistency, with scores for identical outputs varying due to disparate times, source coverage, and algorithmic thresholds; a 2015 comparison of 1,000 articles across providers like Altmetric.com and ImpactStory found discrepancies in counts for the same metrics, such as Mendeley readers, attributable to incomplete API syncing and selective indexing. This variability precludes reproducible results, as re-queries at different intervals or via alternative platforms yield divergent figures, complicating longitudinal or cross-study assessments. Disciplinary imbalances exacerbate under-measurement risks, particularly in humanities fields where online dissemination norms favor traditional outlets over social platforms; a 2015 study of Swedish humanities publications from 2012 documented sparse altmetric coverage, with fewer than 10% registering notable scores compared to STEM equivalents, reflecting lower baseline engagement rather than diminished impact. Such gaps arise from humanities scholars' reliance on non-digital networks, rendering altmetrics ill-suited for equitable evaluation across domains without field-normalized adjustments.

Empirical Evidence on Effectiveness

Correlations with Citation-Based Metrics

Studies examining the relationship between altmetrics, particularly the Attention Score (AAS), and traditional citation counts have generally reported weak to moderate positive correlations, with Pearson's values typically ranging from 0.19 to 0.40 across various fields. A 2021 of health sciences publications found a pooled correlation of =0.19 (95% CI: 0.12–0.25), indicating limited overlap between attention and scholarly citations. Similarly, analyses in and literature for articles published around 2016 yielded =0.40 for citations but only =0.25 with journal impact factors, highlighting discipline-specific variability and overall modest associations. Correlations tend to be stronger for recently published and articles, where altmetrics capture initial visibility that aligns more closely with early citation accrual. publications exhibit an "altmetrics advantage," with higher AAS driven by broader online dissemination, though this does not consistently translate to sustained citation gains. However, these links weaken over time; altmetrics modestly forecast short-term citations (within 1–2 years) but show negligible predictive power for long-term citation trajectories, suggesting they reflect transient buzz rather than enduring scholarly influence. Inconsistencies across studies underscore that while positive correlations exist, they are insufficient to equate altmetric attention with citation-based impact, with r values rarely exceeding 0.3 in cross-disciplinary samples and often lower in mature fields like (r≈0.25). This pattern implies altmetrics may proxy early diffusion but fail to capture deeper quality signals embedded in citations accumulated over 5+ years.

Longitudinal Studies and Predictive Validity Assessments

A longitudinal analysis of altmetric indicators from PlumX across publications from 2012 to 2015 revealed distinct life cycles, with social media mentions (e.g., ) peaking within the first year and declining sharply thereafter, while readership metrics like showed more sustained accumulation over multiple years but still plateaued after 3–5 years. Policy document citations exhibited the slowest growth, often requiring 5–10 years to reach meaningful levels, contrasting with the rapid but ephemeral nature of news and blog coverage. These patterns persisted across fields, underscoring altmetrics' sensitivity to source-specific temporal dynamics rather than uniform predictors of ongoing attention. Over a decade (2008–2018 cohorts), coverage and trends in five altmetric sources evolved variably: Twitter coverage surged from under 30% for older papers to over 90% for recent ones, reflecting platform maturity, while blogs declined in relative prominence and policy mentions remained sparse even after 10 years. readership correlated with later citations in some analyses, yet overall altmetric-citation associations weakened when adjusting for publication age, as early social buzz often decoupled from cumulative scholarly uptake. Assessments of indicate limited forecasting of enduring impact, with altmetrics explaining modest variance in long-term citations (typically R² < 0.20 across fields), as immediate attention from sources like fails to reliably anticipate sustained citation trajectories beyond initial visibility. During the period (2020–2022), altmetric scores inflated dramatically for related outputs due to heightened public and media engagement, yet this attention yielded only short-term citation premiums (e.g., 4.8-fold initial boost fading by 66–77% within a year), highlighting a decoupling from persistent scholarly influence. Longitudinal tracking post-2020 confirms that such spikes do not enhance journals' or papers' long-term metrics proportionally, attributing stagnation in to altmetrics' toward transient virality over rigorous validation.

Controversies and Criticisms

Vulnerability to Manipulation and Gaming

Altmetrics' dependence on unverified signals exposes them to manipulation via automated bot networks that simulate engagement through repeated mentions, likes, or shares. These bots, often deployed in coordinated campaigns, can generate illusory spikes in attention scores by mimicking organic activity without contributing meaningful . For instance, actors may employ scripts to amplify a paper's visibility on platforms like , exploiting APIs that altmetrics providers track, thereby inflating scores by factors observed in empirical analyses of suspicious bursts. Coordinated human-driven efforts further exacerbate vulnerabilities, such as organized groups or journal-affiliated accounts systematically sharing content to boost metrics absent genuine scholarly interest. Documented cases from onward reveal self-promotional tactics by publishers, where repetitive cross-posting across networks elevated attention without proportional evidence of broader impact. This gaming circumvents traditional gates, as altmetrics aggregate raw counts rather than vetted interactions, enabling rapid, low-cost inflation that traditional citations resist due to their slower, expert-mediated accrual. Empirical scrutiny has detected anomalous patterns in 10-20% of tracked social signals attributable to non-organic sources, including bot-like repetition and echo-chamber amplification, which erode the metrics' verifiability. Providers like acknowledge these risks but rely on post-hoc filtering, such as excluding obvious spam, yet causal loopholes persist in open ecosystems where manipulators adapt faster than detection algorithms. Consequently, unmitigated gaming undermines altmetrics' role in , as inflated scores may prioritize visibility over substantive influence, distorting incentives in open-access environments.

Prioritization of Virality Over Scholarly Rigor

Altmetrics, by aggregating mentions from , outlets, and documents, often elevate research outputs that achieve rapid public dissemination through or controversy rather than enduring scholarly validation. This mechanism inherently favors virality, as platforms like (now X) and amplify emotionally charged or timely topics, rewarding attention over methodological soundness. For instance, in 2021, 98 of the 100 scientific articles with the highest scores focused on , including studies on and —treatments that generated intense debate but were subsequently criticized for lacking robust evidence in randomized trials. Such patterns demonstrate how altmetrics can prioritize immediate buzz, incentivizing researchers to frame findings in ways that provoke outrage or hope, diverging from the slower, peer-driven scrutiny that underpins scientific merit. Empirical analyses reveal limited alignment between high altmetric attention and indicators of rigor, such as low retraction rates or prestigious awards. Retracted COVID-19 papers, particularly those on vaccines, continued to accrue altmetric mentions post-retraction, with some sustaining elevated scores due to persistent online sharing of misinformation. Studies examining broader correlations find weak or field-specific links between altmetric scores and peer-assessed quality, with no consistent for scholarly excellence; for example, altmetrics showed the lowest association with quality in arts and , and only modest ties in sciences. This disconnect underscores an incentive structure where hype—often tied to public fears or trends like 2020s vaccine skepticism—outpaces validation, normalizing media-fueled metrics as proxies for impact despite their divergence from causal evidence of scientific advancement. The resultant distortion encourages a feedback loop wherein scholars pursue viral appeal, potentially at the expense of rigorous, incremental work that may not garner immediate but contributes to foundational knowledge. High-profile cases, such as debated studies amassing amid safety concerns, illustrate how altmetrics can validate dynamics over empirical falsification, eroding the first-principles emphasis on replicability and in favor of performative dissemination. Without adjustments for these biases, altmetrics risk entrenching a system where scholarly rigor yields to the transient rewards of controversy.

Influence of Media and Ideological Biases

Altmetrics' reliance on mentions from outlets and embeds ideological biases prevalent in those platforms, as providers like .com curate lists of tracked sources that favor , which empirical analyses have shown to exhibit a systemic left-leaning tilt in coverage. For instance, a study of U.S. outlets found that all except ' Special Report and scored to the left of the political center on a guest measure derived from transcripts. This curation process limits altmetrics to a narrow selection of predominantly progressive-leaning sources, potentially skewing scores toward aligning with dominant narratives in academia and , where left-wing perspectives prevail due to institutional homogeneity. Critics argue that such weighting disadvantages heterodox or conservative-leaning research, as mainstream outlets often underreport or critically frame dissenting findings on topics like biological sex differences or policy critiques of progressive interventions, resulting in lower visibility and scores. In , for example, altmetrics reveal a highly hierarchical distribution of that correlates with patterns but likely extends to ideological , suppressing minority viewpoints in a field where conservative scholars report systemic marginalization. Defenders counter that altmetrics' inclusion of diversifies signals beyond elite media gatekeepers, capturing engagement that can elevate contrarian work, though empirical data indicate social platforms historically amplify left-leaning until recent platform changes. Platform data from 2023 onward suggest partisan topics dominating altmetrics often reflect media priorities, with progressive-aligned issues like climate policy or equity garnering disproportionate mentions, while conservative-leaning analyses on economic or receive muted coverage due to source selection biases. This dynamic risks conflating virality with validity, as ideological echo chambers in curated outlets prioritize narrative fit over empirical rigor, underscoring altmetrics' vulnerability to societal skews rather than neutral impact .

Recent Developments and Future Directions

Advancements in AI-Driven Analysis (2023–2025)

In September 2025, Altmetric launched an AI-powered sentiment analysis feature in its Explorer platform, utilizing large language models to assess the emotional tone of online mentions tracking research outputs. This tool processes mentions from sources such as social media and news outlets, assigning scores from -3 (strongly negative) to +3 (strongly positive) based on expressed opinions toward the research itself, thereby enabling differentiation between mere attention volume and qualitative reception. The feature supports sentiment classification by generating breakdowns, visualizations, and filters within the platform, allowing users to identify trends like endorsements from key opinion leaders or emerging reputation risks. Integration of AI-driven enhances contextual understanding, moving beyond raw counts in the Altmetric Attention Score to incorporate nuance, such as distinguishing advocacy from criticism. Early vendor evaluations indicate improved interpretability for research impact assessment, though independent empirical validation of accuracy enhancements, including potential reductions in interpretive noise from ambiguous mentions, awaits peer-reviewed confirmation. AI advancements have also enabled real-time processing of expansive, heterogeneous data streams in altmetrics, facilitating more responsive tracking of attention dynamics across platforms. While specific implementations for anomaly detection—such as flagging irregular spikes in mentions indicative of coordinated activity—remain undetailed in public disclosures, the underlying machine learning capabilities position altmetrics for refined filtering of non-substantive noise, pending demonstration through controlled studies. These developments, concentrated in proprietary tools like Altmetric's, reflect a shift toward hybrid quantitative-qualitative metrics, with ongoing refinements reported to prioritize research-specific sentiment over general virality.

Emerging Research and Potential Reforms

Recent analyses of altmetrics data from November 2021 to November 2024 indicate a temporal decline in overall Attention Scores and mentions on X (formerly Twitter), coinciding with the platform's acquisition and rebranding, as well as external events like the 2024 U.S. elections. Despite this, X remains a dominant source for research-related posts, with weekly activity exceeding pre-2019 levels and showing increased virality through higher repost ratios, though total posts have fallen from pandemic peaks of over 3 million per month to around 2 million by early 2024. In response to platform fragmentation, altmetrics providers have integrated emerging networks like Bluesky in December 2024, enabling tracking across its growing user base of over 23 million accounts to capture scholarly discussions migrating from X. Emerging research advocates broadening altmetrics scopes to enhance reliability, including systematic coverage of decentralized platforms such as Mastodon and Bluesky, alongside non-English content from regions like China and India to mitigate Anglo-centric biases. Proposals emphasize adding timestamps to scores to account for content ephemerality, such as post deletions on X, and leveraging large language models for contextual sentiment analysis to discern substantive engagement from transient noise. These reforms aim to improve metric stability and interoperability via protocols like ActivityPub, fostering a more resilient framework less dependent on single platforms. Potential directions include hybrid indices combining altmetrics with traditional citations, where recent studies demonstrate altmetrics' utility in early of long-term citation counts, suggesting integrated models could better validate impact. Reforms prioritizing —such as enhanced weighting for expert-driven outlets over general public mentions—seek to emphasize causal connections to verifiable outcomes, like adoption or clinical application, rather than raw volume, though empirical validation of such adjustments remains preliminary. Ongoing work calls for greater transparency in weighting algorithms to align metrics with scholarly rigor amid evolving digital landscapes.

References

Add your contribution
Related Hubs
User Avatar
No comments yet.