Hubbry Logo
RankingRankingMain
Open search
Ranking
Community hub
Ranking
logo
7 pages, 0 posts
0 subscribers
Be the first to start a discussion here.
Be the first to start a discussion here.
Ranking
Ranking
Ranking
View on Wikipedia
from Wikipedia

A ranking is a relationship between a set of items, often recorded in a list, such that, for any two items, the first is either "ranked higher than", "ranked lower than", or "ranked equal to" the second.[1] In mathematics, this is known as a weak order or total preorder of objects. It is not necessarily a total order of objects because two different objects can have the same ranking. The rankings themselves are totally ordered. For example, materials are totally preordered by hardness, while degrees of hardness are totally ordered. If two items are the same in rank it is considered a tie.

By reducing detailed measures to a sequence of ordinal numbers, rankings make it possible to evaluate complex information according to certain criteria. Thus, for example, an Internet search engine may rank the pages it finds according to an estimation of their relevance, making it possible for the user quickly to select the pages they are likely to want to see.

Analysis of data obtained by ranking commonly requires non-parametric statistics.

Strategies for handling ties

[edit]

It is not always possible to assign rankings uniquely. For example, in a race or competition two (or more) entrants might tie for a place in the ranking.[2] When computing an ordinal measurement, two (or more) of the quantities being ranked might measure equal. In these cases, one of the strategies below for assigning the rankings may be adopted.

A common shorthand way to distinguish these ranking strategies is by the ranking numbers that would be produced for four items, with the first item ranked ahead of the second and third (which compare equal) which are both ranked ahead of the fourth.[3] These names are also shown below.

Standard competition ranking ("1224" ranking)

[edit]

In competition ranking, items that compare equal receive the same ranking number, and then a gap is left in the ranking numbers. The number of ranking numbers that are left out in this gap is one less than the number of items that compared equal. Equivalently, each item's ranking number is 1 plus the number of items ranked above it. This ranking strategy is frequently adopted for competitions, as it means that if two (or more) competitors tie for a position in the ranking, the position of all those ranked below them is unaffected (i.e., a competitor only comes second if exactly one person scores better than them, third if exactly two people score better than them, fourth if exactly three people score better than them, etc.).

Thus if A ranks ahead of B and C (which compare equal) which are both ranked ahead of D, then A gets ranking number 1 ("first"), B gets ranking number 2 ("joint second"), C also gets ranking number 2 ("joint second") and D gets ranking number 4 ("fourth").

This method is called "Low" by IBM SPSS[4] and "min" by the R programming language[5] in their methods to handle ties.

Modified competition ranking ("1334" ranking)

[edit]

Sometimes, competition ranking is done by leaving the gaps in the ranking numbers before the sets of equal-ranking items (rather than after them as in standard competition ranking). The number of ranking numbers that are left out in this gap remains one less than the number of items that compared equal. Equivalently, each item's ranking number is equal to the number of items ranked equal to it or above it. This ranking ensures that a competitor only comes second if they score higher than all but one of their opponents, third if they score higher than all but two of their opponents, etc.

Thus if A ranks ahead of B and C (which compare equal) which are both ranked ahead of D, then A gets ranking number 1 ("first"), B gets ranking number 3 ("joint third"), C also gets ranking number 3 ("joint third") and D gets ranking number 4 ("fourth"). In this case, nobody would get ranking number 2 ("second") and that would be left as a gap.

This method is called "High" by IBM SPSS[4] and "max" by the R programming language[5] in their methods to handle ties.

Dense ranking ("1223" ranking)

[edit]

In dense ranking, items that compare equally receive the same ranking number, and the next items receive the immediately following ranking number. Equivalently, each item's ranking number is 1 plus the number of items ranked above it that are distinct with respect to the ranking order.

Thus if A ranks ahead of B and C (which compare equal) which are both ranked ahead of D, then A gets ranking number 1 ("first"), B gets ranking number 2 ("joint second"), C also gets ranking number 2 ("joint second") and D gets ranking number 3 ("Third").

This method is called "Sequential" by IBM SPSS[4] and "dense" by the R programming language[6] in their methods to handle ties.

Ordinal ranking ("1234" ranking)

[edit]

In ordinal ranking, all items receive distinct ordinal numbers, including items that compare equal. The assignment of distinct ordinal numbers to items that compare equal can be done at random, or arbitrarily, but it is generally preferable to use a system that is arbitrary but consistent, as this gives stable results if the ranking is done multiple times. An example of an arbitrary but consistent system would be to incorporate other attributes into the ranking order (such as alphabetical ordering of the competitor's name) to ensure that no two items exactly match.

With this strategy, if A ranks ahead of B and C (which compare equal) which are both ranked ahead of D, then A gets ranking number 1 ("first") and D gets ranking number 4 ("fourth"), and either B gets ranking number 2 ("second") and C gets ranking number 3 ("third") or C gets ranking number 2 ("second") and B gets ranking number 3 ("third").

In computer data processing, ordinal ranking is also referred to as "row numbering".

This method corresponds to the "first", "last", and "random" methods in the R programming language[5] to handle ties.

Fractional ranking ("1 2.5 2.5 4" ranking)

[edit]

Items that compare equal receive the same ranking number, which is the mean of what they would have under ordinal rankings; equivalently, the ranking number of 1 plus the number of items ranked above it plus half the number of items equal to it. This strategy has the property that the sum of the ranking numbers is the same as under ordinal ranking. For this reason, it is used in computing Borda counts and in statistical tests (see below).

Thus if A ranks ahead of B and C (which compare equal) which are both ranked ahead of D, then A gets ranking number 1 ("first"), B and C each get ranking number 2.5 (average of "joint second/third") and D gets ranking number 4 ("fourth").

Here is an example: Suppose you have the data set 1.0, 1.0, 2.0, 3.0, 3.0, 4.0, 5.0, 5.0, 5.0.

The ordinal ranks are 1, 2, 3, 4, 5, 6, 7, 8, 9.

For v = 1.0, the fractional rank is the average of the ordinal ranks: (1 + 2) / 2 = 1.5. In a similar manner, for v = 5.0, the fractional rank is (7 + 8 + 9) / 3 = 8.0.

Thus the fractional ranks are: 1.5, 1.5, 3.0, 4.5, 4.5, 6.0, 8.0, 8.0, 8.0

This method is called "Mean" by IBM SPSS[4] and "average" by the R programming language[5] in their methods to handle ties.

Statistics

[edit]
This section is an excerpt from Ranking (statistics).[edit]

In statistics, ranking is the data transformation in which numerical or ordinal values are replaced by their rank when the data are sorted.

For example, the ranks of the numerical data 3.4, 5.1, 2.6, 7.3 are 2, 3, 1, 4.

As another example, the ordinal data hot, cold, warm would be replaced by 3, 1, 2. In these examples, the ranks are assigned to values in ascending order, although descending ranks can also be used.

Ranks are related to the indexed list of order statistics, which consists of the original dataset rearranged into ascending order.

Sports

[edit]
This section is an excerpt from Standings (sports).[edit]
A partial view of the Green Monster at Fenway Park, with standings for the American League East division at the end of the 2007 Major League Baseball season
In sports, standings, rankings, or league tables group teams of a particular league, conference, or division in a chart based on how well each is doing in a particular season of a sports league or competition. These lists are generally published in newspapers and other media, as well as the official web sites of the sports leagues and competitions.

Education

[edit]

League tables are used to compare the academic achievements of different institutions. College and university rankings order institutions in higher education by combinations of factors. In addition to entire institutions, specific programs, departments, and schools are ranked. These rankings usually are conducted by magazines, newspapers, governments and academics. For example, league tables of British universities are published annually by The Independent, The Sunday Times, and The Times.[7] The primary aim of these rankings is to inform potential applicants about British universities based on a range of criteria. Similarly, in countries like India, league tables are being developed and a popular magazine, Education World, published them based on data from TheLearningPoint.net. [citation needed]

It is complained that the ranking of England's schools to rigid guidelines that fail to take into account wider social conditions actually makes failing schools even worse. This is because the most involved parents will then avoid such schools, leaving only the children of non-ambitious parents to attend.[8]

Business

[edit]

In business, league tables list the leaders in the business activity within a specific industry, ranking companies based on different criteria including revenue, earnings, and other relevant key performance indicators (such as market share and meeting customer expectations) enabling people to quickly analyze significant data.[9]

Applications

[edit]

The rank methodology based on some specific indices is one of the most common systems used by policy makers and international organizations in order to assess the socio-economic context of the countries. Some notable examples include the Human Development Index (United Nations), Doing Business Index (World Bank), Corruption Perceptions Index (Transparency International), and Index of Economic Freedom (the Heritage Foundation). For instance, the Doing Business Indicator of the World Bank measures business regulations and their enforcement in 190 countries. Countries are ranked according to ten indicators that are synthesized to produce the final rank. Each indicator is composed of sub-indicators; for instance, the Registering Property Indicator is composed of four sub-indicators measuring time, procedures, costs, and quality of the land registration system. These kinds of ranks are based on subjective criteria for assigning the score. Sometimes, the adopted parameters may produce discrepancies with the empirical observations, therefore potential biases and paradox may emerge from the application of these criteria.[10]

Other examples

[edit]
  • In politics, rankings may focus on the comparison of economic, social, environmental and governance performance of countries. Politicians themselves have also been ranked, based on the extent of their activities.[11]
  • In relation to credit standing, the ranking of a security refers to where that particular security would stand in a wind up of the issuing company, i.e., its seniority in the company's capital structure. For instance, capital notes are subordinated securities; they would rank behind senior debt in a wind up. In other words, the holders of senior debt would be paid out before subordinated debt holders received any funds.
  • Search engines rank web pages by their expected relevance to a user's query using a combination of query-dependent and query-independent methods. Query-independent methods attempt to measure the estimated importance of a page, independent of any consideration of how well it matches the specific query. Query-independent ranking is usually based on link analysis; examples include the HITS algorithm, PageRank and TrustRank. Query-dependent methods attempt to measure the degree to which a page matches a specific query, independent of the importance of the page. Query-dependent ranking is usually based on heuristics that consider the number and locations of matches of the various query words on the page itself, in the URL or in any anchor text referring to the page.
  • In webometrics, it is possible to rank institutions according to their presence in the web (number of webpages) and the impact of these contents, such as the Webometrics Ranking of World Universities.
  • In video gaming, players may be given a ranking. To "rank up" is to achieve a higher ranking relative to other players, especially with strategies that do not depend on the player's skill.
  • The TrueSkill ranking system is a skill based ranking system for Xbox Live developed at Microsoft Research.
  • A bibliogram ranks common noun phrases in a piece of text.
  • In language, the status of an item (usually through what is known as "downranking" or "rank-shifting") in relation to the uppermost rank in a clause; for example, in the sentence "I want to eat the cake you made today", "eat" is on the uppermost rank, but "made" is downranked as part of the nominal group "the cake you made today"; this nominal group behaves as though it were a single noun (i.e., I want to eat it), and thus the verb within it ("made") is ranked differently from "eat".
  • Academic journal[broken anchor]s are sometimes ranked according to impact factor; the number of later articles that cite articles in a given journal.

See also

[edit]

References

[edit]
[edit]
Revisions and contributorsEdit on WikipediaRead on Wikipedia

Ranking

View on Grokipedia
from Grokipedia
Ranking is a relational structure imposed on a set of entities, whereby each pair is compared to determine a total or partial order of , performance, or value, often formalized in as a that may allow ties. This process underpins across domains such as elections, where voter preferences aggregate to rank candidates; , where teams or athletes are sequenced by metrics like win-loss records; and , where algorithms sort results by scores. Mathematical foundations draw from , employing methods like pairwise comparisons or scoring aggregation to mitigate inconsistencies in , though real-world applications frequently encounter challenges like in preferences (e.g., Condorcet paradoxes). Notable variants include competition ranking, which skips numbers after ties to reflect gaps in performance; ordinal ranking, assigning sequential positions without gaps; and fractional ranking, averaging positions for equals to preserve continuity. While rankings facilitate efficient prioritization and , they spark controversies in contexts like corporate performance appraisals—where forced or stack ranking systems, popularized in the 1980s, incentivize cutthroat competition and suppress collaboration, leading to their abandonment by firms like —or educational evaluations, where metrics distort institutional behaviors toward superficial gains in selectivity over substantive quality.

Fundamentals

Definition and Basic Principles

In and related fields, a refers to the assignment of positions to elements of a set based on a comparative relation, typically yielding a where every pair of distinct elements is strictly comparable, ensuring a complete linear arrangement without ambiguities in relative positioning./07%3A_Relations/7.04%3A_Partial_and_Total_Ordering) This structure contrasts with partial orders, which permit incomparabilities between some elements, as seen in applications like aggregation where not all items can be directly ranked against each other./07%3A_Relations/7.04%3A_Partial_and_Total_Ordering) The foundational principle derives from , where the relation must satisfy totality (for any two elements aa and bb, either a≺ba \prec b or b≺ab \prec a), transitivity (if a≺ba \prec b and b≺cb \prec c, then a≺ca \prec c), and irreflexivity (no a≺aa \prec a) for strict rankings. Basic principles of ranking emphasize the ordinal nature of the output, focusing on relative positions rather than cardinal differences in magnitude, which distinguishes rankings from interval or scales in measurement . In statistical contexts, rankings transform by sorting observations and assigning integers corresponding to their order, with ties often resolved via methods like average ranking (e.g., for tied values at positions 3 and 4 in a list of 5, both receive rank 3.5) to maintain consistency. This process preserves the underlying order while mitigating the influence of outliers or non-normal distributions, enabling non-parametric analyses such as the Wilcoxon rank-sum test, which relies on the ranks themselves rather than original values for hypothesis testing. Rankings underpin applications across domains, from electoral systems—where voter preferences form individual total orders aggregated into a collective ranking—to algorithms that prioritize results based on scores converted to ranks. However, real-world rankings frequently encounter challenges like intransitivities (e.g., Condorcet cycles in voting, where A>BA > B, B>CB > C, C>AC > A) or incomplete information, necessitating extensions beyond pure total orders, such as weak orders that incorporate equivalence classes for ties. These principles ensure rankings reflect causal priorities or empirical comparisons faithfully, prioritizing comparability and stability over absolute quantification.

Types of Rankings

Rankings in and related fields are broadly classified by their completeness, allowing for distinctions between total rankings, where every pair of elements is comparable, and partial rankings, where some elements may remain incomparable. Total rankings correspond to linear orders or total preorders, ensuring a complete of elements, as seen in applications like outcomes or aggregation where all items must be ordered relative to one another. Partial rankings, modeled as partial orders, permit incomparabilities, which arise in scenarios such as hierarchical structures or incomplete data, and can be extended to total rankings via theorems like Szpilrajn's extension theorem. Another key classification distinguishes ordinal from cardinal rankings based on the informational content encoded. Ordinal rankings capture only relative orderings without quantifying the magnitude of differences, as in ranking candidates by pairwise preferences in voting systems. Cardinal rankings incorporate numerical values to represent intensities or utilities, enabling computations like weighted averages, as utilized in methods such as the for decision-making under multiple criteria. This distinction is central in , where ordinal approaches avoid interpersonal utility comparisons but may lose efficiency, while cardinal methods can optimize aggregate welfare but risk strategic manipulation. Rankings may further vary by handling of equivalences or ties, leading to strict rankings (antisymmetric, no ties) versus weak rankings (preorders allowing indifference classes). Strict rankings enforce unique positions, suitable for competitive zero-sum settings like sports leagues, whereas weak rankings group tied elements, preserving transitivity in broader models. These types intersect; for instance, a total ordinal ranking might use dense numbering to assign consecutive integers to tied groups, while cardinal variants could assign identical scores to equivalents.

Historical Development

Origins in Mathematics and Early Applications

The mathematical foundations of ranking emerged in the late 18th century amid debates over fair methods for aggregating individual preferences into collective orders, particularly within the French Academy of Sciences. Jean-Charles de Borda, a mathematician and naval engineer, proposed an early positional ranking system in his 1784 Mémoire sur les élections au scrutin, motivated by flaws in plurality voting observed in academy elections. Borda's method assigned points to candidates based on their ranked positions across voter ballots: in an election with m candidates, the highest-ranked receives m-1 points, the next m-2, down to 0 for the lowest, with the aggregate score determining the overall ranking. This approach aimed to account for the intensity of preferences rather than mere first-place votes, providing a quantitative basis for ordinal comparisons. The critiqued Borda's system shortly thereafter, publishing his Essai sur l'application de l'analyse à la probabilité des décisions rendues à la pluralité des voix in 1785, which introduced pairwise comparison as a foundational ranking technique. Condorcet's method evaluated by conducting hypothetical head-to-head contests between each pair, declaring a the Condorcet winner if it defeated every opponent by vote; rankings followed from the of these pairwise where possible. He also identified the , where cyclic preferences (e.g., A beats B, B beats C, C beats A by ) prevent a stable ranking, highlighting inherent challenges in deriving total orders from partial voter inputs. These innovations drew on to assess decision reliability, framing ranking as a problem of probabilistic consensus rather than deterministic tallying. Early applications centered on internal academy proceedings, where the French Academy served as a testing ground for refining electoral processes during the Enlightenment and . Borda's method gained traction post-Revolution, influencing the Academy's adoption of for membership selections by 1795, as endorsed by Pierre Daunou, and was used in electing figures like to the National Institute between 1795 and 1815. Condorcet's pairwise approach, though less immediately implemented due to paradox risks, informed probabilistic analyses of decisions and legislative voting, extending ranking principles to broader under . These developments marked the shift from ordering to axiomatic, preference-based ranking, laying groundwork for later extensions in social choice without reliance on cardinal utilities.

Evolution in Statistics and Social Choice

In , the formal study of aggregating individual rankings into collective orderings originated in the late amid French Enlightenment debates on . Jean-Charles de Borda introduced the in 1781, a method assigning points to candidates proportional to the number of alternatives ranked below them by voters, aiming to reflect the intensity of preferences through positional scoring. Shortly thereafter, in 1785, the advanced pairwise majority comparisons, defining a Condorcet winner as the alternative that defeats every rival in head-to-head contests, while also demonstrating the wherein cyclic preferences prevent transitive social rankings despite individual transitivity. These foundational approaches highlighted tensions between and coherent aggregation, influencing subsequent voting systems like approval and range voting. The 20th century brought rigorous impossibilities to social choice rankings. Kenneth Arrow's 1951 theorem proved that no non-dictatorial method can aggregate three or more ordinal rankings into a satisfying , , and , underscoring inherent limitations in deriving transitive group orderings from diverse individual preferences. This result spurred developments in Condorcet extensions and scoring rules, such as the Kemeny-Young method minimizing inversions relative to an ideal ranking, though computational intractability limited practical adoption for large electorates. Parallel evolution occurred in statistics, where rankings provided robust, distribution-free tools for inference amid growing data from non-normal sources. formulated the rank correlation coefficient in 1904, measuring monotonic associations between paired observations by correlating their ranks, thus avoiding parametric assumptions like linearity required by Pearson's correlation and proving effective for in and . Building on this, Frank Wilcoxon developed the rank-sum test in 1945 for two-sample comparisons, ranking combined observations and summing ranks within groups to test location shifts without normality, offering greater power against heavy-tailed distributions than t-tests. Mid-century extensions generalized ranking to multi-group settings. William Kruskal and W. Allen Wallis introduced the Kruskal-Wallis test in 1952, an analog to one-way ANOVA that ranks all observations across k groups and computes between-group variance in average ranks, detecting differences in medians under minimal assumptions and applicable to heterogeneous variances. These non-parametric advances, rooted in permutation invariance, proliferated in experimental sciences by the , as evidenced by their integration into standard texts like Hollander and Wolfe's, emphasizing empirical superiority in small samples or outliers over parametric rivals. The interplay between social choice and statistical ranking intensified post-1950s, with shared concerns over robustness to heterogeneity; statistical ranks informed social choice simulations of voting paradoxes, while aggregation challenges inspired rank-based robust estimation in , such as median-based orderings over means. This convergence yielded hybrid methods, like probabilistic rankings via bootstrap resampling of preferences, prioritizing causal interpretability over idealized axioms.

Core Methods

Strategies for Handling Ties

In ranking systems, ties arise when two or more entities receive identical scores or evaluations, complicating the assignment of distinct ordinal positions. This issue is prevalent in statistical analysis, competitions, and processes, where unresolved ties can distort measures like rank correlations or aggregate standings. Standard approaches aim to maintain consistency with the underlying data distribution while minimizing in downstream computations, such as variance estimates in non-parametric tests. The most common statistical strategy is the mid-rank or average rank method, where tied values are assigned the mean of the ranks they would occupy if ordered distinctly. For instance, if two observations tie for positions 3 and 4 in a list of five, both receive rank 3.5, preserving the overall mean rank while reducing variance compared to untied data. This approach is widely adopted in rank-order statistics and non-parametric methods like , as it avoids inflating or deflating the sum of ranks and facilitates unbiased estimation of parameters. Empirical studies show it performs robustly for tied datasets, though it requires corrections for correlation coefficients to account for reduced variability. Alternative methods include minimum rank assignment, which grants tied entities the lowest possible rank (e.g., both receive rank 3 in the above example, with subsequent items starting at 5), or maximum rank, assigning the highest (e.g., both get 4, skipping none). These are less common in pure statistical contexts due to their asymmetry, which can bias order statistics toward over- or under-ranking, but they appear in software implementations for specific applications like competition scoring. Dense ranking assigns identical ranks without skipping (e.g., 3,3,4), preserving sequential order and minimizing gaps, while standard competition ranking skips ranks (e.g., 3,3,5) to reflect the "lost" positions. The choice impacts aggregation; dense ranking suits dense datasets, whereas competition ranking aligns with ordinal scarcity in events like athletics. In domains requiring decisive outcomes, such as matching algorithms or , ties are often resolved via hierarchical tie-breakers—secondary criteria like additional metrics, random lotteries, or predefined priorities—or methods like single or multiple tie-breaking lotteries. For example, in mechanisms, single tie-breaking uses a uniform random order across all ties, while multiple applies independent lotteries per group, with hybrid variants shown to dominate in under certain dominance criteria. These prevent indeterminacy but introduce variability, necessitating evaluation against stability metrics. Random tie-breaking, while equitable in expectation, can amplify in small samples and is critiqued for lacking unless seeded deterministically. Selection of a depends on the ranking's purpose: statistical neutrality favors mid-ranks for preserving distributional properties, while operational contexts prioritize resolvability via tie-breakers to enable actions like winner selection. No universal method eliminates all distortions, as ties inherently compress information, and empirical validation—such as comparing coefficients pre- and post-correction—is recommended for quantitative rankings.

Statistical and Non-Parametric Techniques

Statistical techniques for ranking often involve parametric models that assign underlying probabilities or strengths to items, enabling inference on rankings from observed preferences or comparisons. The Bradley-Terry model, introduced in 1952, posits that in pairwise comparisons, the probability that item ii outranks item jj equals πiπi+πj\frac{\pi_i}{\pi_i + \pi_j}, where πk\pi_k represents the latent strength of item kk. Parameter estimates are obtained via maximum likelihood, allowing derivation of overall rankings by ordering the π^i\hat{\pi}_i. This model underpins applications in sports ratings and paired preference studies, assuming independence of comparisons. The Plackett-Luce model extends this framework to full or partial rankings by modeling the probability of a specific ordering as the product, over successive positions, of the strength of the chosen item divided by the sum of strengths of remaining items: P(ρ)=∏j=1mπρ(j)∑k=jmπρ(k)P(\rho) = \prod_{j=1}^m \frac{\pi_{\rho(j)}}{\sum_{k=j}^m \pi_{\rho(k)}}, where ρ\rho is the ranking. Developed independently by Plackett in 1975 and generalizing Luce's 1959 choice axiom, it accommodates ties and incomplete data through extensions. typically uses iterative methods like minorization-maximization, with applications in recommender systems and sensory evaluation. Non-parametric techniques, by contrast, eschew distributional assumptions, relying instead on ranks or permutations for robustness against outliers and non-normality. , ρ=1−6∑di2n(n2−1)\rho = 1 - \frac{6 \sum d_i^2}{n(n^2 - 1)} where did_i are rank differences, quantifies monotonic association between two rankings, serving as the Pearson correlation on ranked data. It tests concordance without assuming linearity, with significance via permutation or t-approximation for large nn. Kendall's tau, another rank-based measure, counts concordant and discordant pairs: τ=2n(n−1)(C−D)\tau = \frac{2}{n(n-1)} (C - D), where CC and DD are agreeing and disagreeing pairs across rankings. Developed by Kendall in and refined in 1945, τb\tau_b adjusts for ties, offering sensitivity to order inversions over Spearman's distance-based approach. Both are distribution-free, with exact p-values computable via hypergeometric probabilities for small samples, and find use in validating aggregated rankings or detecting preference shifts. For hypothesis testing on rankings, non-parametric procedures like the Friedman test extend to multiple related samples, ranking items within blocks and comparing average ranks via chi-squared approximation: χ2=12nk(k+1)∑Rj2−3n(k+1)\chi^2 = \frac{12}{nk(k+1)} \sum R_j^2 - 3n(k+1), where kk items are ranked nn times. Post-hoc pairwise Wilcoxon signed-rank tests assess specific differences, preserving ordinal structure without parametric forms. These methods, detailed in texts on ranking statistics, prioritize empirical rank distributions over modeled probabilities.

Computational Approaches

Algorithmic Ranking Models

Algorithmic ranking models refer to deterministic computational frameworks that generate total or partial orderings from input data, such as numerical scores, pairwise preferences, or graph structures, by applying fixed rules or optimization procedures rather than data-driven parameter learning. These models prioritize efficiency and interpretability, often solving ranking as a sorting, aggregation, or minimization problem. Common implementations include score aggregation, pairwise tournament resolution, and iterative propagation methods, with roots in and . Unlike statistical models, they emphasize exact or solutions to well-defined objectives, though many face challenges from computational intractability for large inputs. Score-based ranking algorithms assign a scalar utility or relevance value to each item and sort them in descending order, providing a straightforward mechanism for ordinal output. For instance, in information retrieval, the Vector Space Model computes cosine similarity between query and document term vectors, weighting terms by frequency and inverse document frequency (TF-IDF), to rank documents by projected relevance; this approach, formalized in 1975, scales linearly with vector dimensions after indexing. Similarly, BM25, an enhancement of the binary independence model introduced in the 1990s, refines probabilistic scoring with term saturation functions to mitigate length bias, achieving state-of-the-art performance on benchmarks like TREC without iterative training. These methods assume additivity of features, enabling O(n log n) sorting via standard algorithms like quicksort, but they falter when scores lack cardinal meaning or interdependencies exist. Optimization-based models, such as the Kemeny-Young method, aggregate multiple partial rankings by selecting the that minimizes the sum of pairwise disagreements (equivalent to Kendall-tau distance) across inputs, formulated as a minimum problem on a weighted graph. Proposed in 1959, it yields a Condorcet-efficient solution when cycles are absent but is NP-hard in general, with exact branch-and-bound solvers achieving feasibility for up to 20-30 items via symmetry reduction and bounding techniques as of 2023. Approximations, including local search or relaxations, trade optimality for scalability, often within 5-10% of the global minimum on real-world datasets like elections or sports outcomes. This approach excels in consensus-seeking scenarios but requires complete pairwise data, exposing vulnerabilities to strategic manipulation via implications. Graph-based iterative algorithms propagate rankings through network structures, computing stationary scores via matrix operations or fixed-point convergence. , deployed by since 1998, models web pages as nodes in a , assigning authority proportional to incoming links damped by a teleportation factor (typically 0.15), solved via in O(E log n) time for sparse graphs with E edges. Variants like (Hyperlink-Induced Topic Search), introduced in 1998, alternately optimize hub and authority eigenvectors, converging in 10-20 iterations for typical corpora but susceptible to spam through link farms. These models capture transitive influences causally, outperforming flat scoring in directed acyclic graphs, yet demand damping to ensure and handle dangling nodes explicitly. Empirical evaluations on citation networks confirm their robustness to noise when link quality is high.

Learning to Rank in Machine Learning

Learning to rank (LTR) constitutes a supervised framework designed to train models that generate -based orderings of candidate items, such as documents or products, in response to a query. The core objective is to learn a scoring function f(q,xi)f(q, x_i) from training data comprising queries qq, feature vectors xix_i for items ii, and associated labels yiy_i (typically ordinal grades like 0 for irrelevant to 4 for perfect match), such that sorting items by predicted scores y^i=f(q,xi)\hat{y}_i = f(q, x_i) minimizes discrepancies with the ground-truth ranking implied by yiy_i. This approach addresses the ordinal of rankings, where absolute scores matter less than relative positions, and has been empirically validated in tasks through datasets like LETOR, which provide thousands of query-document pairs for benchmarking. LTR methods diverge into three paradigms based on how they formulate the loss during training: , pairwise, and listwise. methods regress or classify individual items' scores independently, akin to standard regression tasks, using losses like squared error or on y^i\hat{y}_i versus yiy_i; subsequent ranking follows from sorting these scores. This simplifies computation but overlooks inter-item dependencies, often yielding suboptimal performance on ranking metrics, as evidenced by comparisons in benchmark evaluations where pointwise models lag behind relational approaches by 5-10% in normalized (NDCG). Pairwise approaches, by contrast, focus on relative orders by considering document pairs (i,j)(i, j) where yi>yjy_i > y_j, optimizing a loss that penalizes violations of y^i>y^j\hat{y}_i > \hat{y}_j, such as pairwise logistic loss or in support vector machines for ranking (RankSVM). Pioneered in models like RankNet, which employs neural networks with gradient-based updates approximating probabilistic pairwise preferences via , these methods directly enforce ordinal constraints and have demonstrated superior empirical results in applications, reducing ranking errors by capturing pairwise swaps more effectively than pointwise alternatives. Listwise methods extend this by treating the entire permutation of items as the instance, directly approximating ranking metrics like NDCG or mean average precision (MAP) through surrogate losses, such as soft-ranking formulations or permutation probability distributions (e.g., ListNet's between predicted and ideal list probabilities). These capture global list structure and , often outperforming pairwise methods in large-scale evaluations—for instance, LambdaMART, a gradient-boosted tree variant incorporating listwise lambda gradients, achieved state-of-the-art NDCG scores on Yahoo! datasets in 2009 competitions. Modern implementations, including those in libraries like and , support listwise objectives with NDCG approximations, enabling scalable training on millions of examples while maintaining statistical consistency with evaluation metrics. Evaluation in LTR emphasizes position-aware metrics over accuracy, with NDCG prioritizing higher ranks for highly relevant items via discounted gains (e.g., NDCG@10 weights top positions exponentially) and averaging precision across levels; empirical studies confirm these better correlate with user satisfaction in retrieval tasks than . Despite computational demands—listwise methods scaling quadratically or worse without approximations—advances in estimation and sampling have rendered LTR viable for production systems, as deployed in search engines since the mid-2000s.

Applications in Specific Domains

Sports and Competitive Events

In sports and competitive events, rankings serve to order participants or teams by performance, influencing seeding, playoff qualification, and . These systems aggregate outcomes from matches or competitions, often incorporating factors like opponent strength to mitigate schedule variability. Objective methods, such as rating systems and statistical models, predominate in professional leagues, while subjective polls persist in contexts like collegiate athletics where human judgment accounts for qualitative elements. Elo rating systems, originally developed for chess in the 1960s by , adapt probabilistic models to predict match outcomes based on rating differentials, adjusting ratings post-game to reflect actual results against expectations. In soccer, employs a modified Elo variant known as the "SUM" method since March 2018, calculating points exchanged per match via the formula incorporating actual result minus expected result, scaled by match importance (e.g., friendlies yield 5-10 points, finals 60) and opponent ranking. This yields monthly global rankings for 210+ national teams, with holding the top spot as of October 2025 after their 2022 victory. Chess federations like update Elo ratings after rated games, with top players exceeding 2800 points, emphasizing pairwise comparisons over cumulative points. Tennis rankings, managed by the ATP for men, accumulate points from up to 19 tournaments over 52 weeks, with Grand Slam winners earning 2000 points and ATP Masters 1000 champions 1000, decaying older results to prioritize recency. This "race" system ensures dynamic shifts, as seen in Djokovic's record 428 weeks at No. 1 through 2024. In baseball, uses win-loss records for divisional standings, supplemented by to forecast "true" talent: expected win percentage equals (runs scored)^1.83 divided by [(runs scored)^1.83 + (runs allowed)^1.83], revealing teams like the 2007 Boston Red Sox outperforming their record en route to a title. American college football rankings blend human and computational inputs; the Associated Press Poll, since 1936, aggregates votes from journalists for a top-25 list, while the committee, introduced in 2014, selects four semifinalists considering , head-to-head results, and conference championships alongside computer models like the , which solves a minimizing bias in win-loss adjustments. Ties in standings often resolve via head-to-head records, conference winning percentage, or multi-team tiebreakers prioritizing comparative victories. These methods, while predictive, face critiques for undermeasuring intangibles like home-field advantage, prompting ongoing refinements toward hybrid objective-subjective frameworks.
SportPrimary Ranking MethodKey FeaturesExample Citation
Soccer (FIFA)Modified Elo (SUM)Points exchange based on expected vs. actual outcome, match importance multiplier
Tennis (ATP)Accumulated points (52-week rolling)Tournament-specific awards, best-of-19 events
Baseball (MLB)Win-loss + Runs scored/allowed ratio for expected wins
Human polls + computer modelsVotes adjusted for schedule strength

Education and Academic Evaluation

In academic evaluation, serves as a primary ranking method for high school students, calculated by comparing a student's cumulative grade point average (GPA) against peers in the same graduating class to determine positions, such as top 10% or . This ordinal ranking provides context for admissions by normalizing performance within a school's competitive environment, with empirical studies indicating that high school outperforms scores as a predictor of GPA, as lower-ranked students with high scores still underperform relative to their rank peers. However, class rank's utility is constrained by its school-specific nature, which fails to account for variations in grading rigor or cohort quality across institutions, leading the National Association of Principals to recommend against its routine publication for admissions due to diminished comparative value. Norm-referenced grading systems, which explicitly rank students against each other rather than absolute standards, have been shown to enhance overall in controlled settings by fostering , though they may demotivate lower-ranked students and exacerbate inequality in heterogeneous classrooms. In higher education, student evaluation often employs ranks derived from assessments like standardized exams or course grades, with methods such as the Combined Compromise Solution (CoCoSo) applied in some contexts to aggregate multi-criteria learning outcomes into holistic rankings, prioritizing criteria like knowledge retention and application skills. These approaches align with non-parametric ranking techniques but face criticism for overemphasizing relative positioning over absolute mastery, potentially discouraging . University rankings, such as those by U.S. News & World Report or QS World University Rankings, aggregate institutional performance through weighted composites of metrics including research output (e.g., citation counts), academic reputation surveys, faculty-to-student ratios, and international diversity, often employing normalized scores and z-standardization to produce ordinal lists. For instance, QS assigns 40% weight to academic reputation based on global surveys, 20% to employer reputation, and the remainder to bibliometrics and staff-student ratios, aiming to reflect research impact and employability. Critiques highlight methodological flaws, including opaque weighting schemes, overreliance on subjective surveys prone to response biases, and incentives for institutions to game metrics—such as inflating publication counts or hiring adjuncts to improve ratios—without necessarily enhancing educational quality. These rankings influence and , with suggesting they promote in research but undervalue and equity, as metrics favor resource-rich, research-intensive institutions over those emphasizing undergraduate instruction. A methodological review notes frequent changes in indicators and lack of , undermining reliability, while structural biases toward English-language publications disadvantage non-Western universities. Despite benefits like for improvement, rankings' emphasis on quantifiable proxies correlates weakly with graduate outcomes in some analyses, prompting calls for multidimensional evaluations incorporating peer-reviewed assessments over aggregated scores.

Business and Economic Analysis

Ranking methods in business and economic analysis facilitate , , and comparative evaluation of entities ranging from sovereign states to corporations. agencies (CRAs), including the "Big Three" of Standard & Poor's, Moody's, and Fitch, assign ordinal scales—such as AAA to D for long-term ratings—to gauge the likelihood of debt repayment, thereby reducing between issuers and investors. These ratings directly influence borrowing costs, with empirical studies indicating that a one-notch downgrade can elevate sovereign bond yields by 20-60 basis points, amplifying economic vulnerabilities during downturns. However, CRAs' issuer-pays model has drawn criticism for potential conflicts of interest, as evidenced by their delayed downgrades of subprime mortgage-backed securities prior to the , contributing to procyclical effects that exacerbate market instability. At the macroeconomic level, international organizations produce composite rankings to benchmark economic performance and policy environments. The International Monetary Fund's World Economic Outlook, updated biannually, ranks countries by nominal GDP, with the leading at approximately $28.78 trillion in 2025 projections, followed by at $19.53 trillion, reflecting disparities in aggregate output driven by , , and dynamics. Similarly, the World Bank's discontinued Doing Business report (2004-2020) ranked 190 economies on regulatory across 10 topics, such as starting a and enforcing contracts, with consistently topping the list due to streamlined procedures that correlate with higher inflows. These indices inform policy reforms but face scrutiny for overemphasizing quantifiable metrics at the expense of qualitative factors like institutional or environmental sustainability, potentially biasing toward developed economies. In corporate contexts, ranking models evaluate and strategic positioning, often integrating multi-criteria . Techniques like MULTIMOORA, which combines ratio and reference-point approaches weighted by entropy-based measures, rank performance appraisal methods or suppliers by aggregating financial, operational, and sustainability indicators. Forced ranking systems, historically applied by firms like under "rank and yank" policies, categorize employees into quintiles (e.g., top 20% rewarded, bottom 10% terminated) to enforce relative performance differentiation, though evidence suggests they foster short-termism and demotivation without sustained productivity gains. Sustainable development-oriented rankings, such as those evaluating firms on environmental, social, and governance criteria, prioritize long-term viability; for instance, methodologies incorporating life-cycle assessments rank companies by , revealing that top performers achieve 15-20% lower operational costs through optimized supply chains. Overall, while these tools enhance decision-making, their effectiveness hinges on robust data inputs and resistance to gaming, as manipulated rankings can distort market signals and investment flows.

Controversies and Critiques

Methodological Limitations and Manipulation

Ranking methodologies encounter profound theoretical constraints, as demonstrated by Arrow's impossibility theorem, which states that no non-dictatorial social choice function exists to aggregate individual ordinal preferences into a collective ranking satisfying unrestricted domain, Pareto efficiency, and independence of irrelevant alternatives for three or more alternatives. This result underscores the impossibility of devising a universally fair aggregation rule without imposing restrictive assumptions on preferences or outcomes. Complementing this, the Condorcet paradox reveals that even with transitive individual rankings, majority pairwise comparisons can produce cyclic preferences—such as a scenario where option A beats B, B beats C, and C beats A—rendering transitive social rankings infeasible without arbitrary resolution mechanisms. Empirical implementations amplify these issues through and design flaws. Ordinal ranking methods, common in evaluations from sports to academia, preserve only relative order while discarding preference intensities, precluding meaningful arithmetic operations like averaging and often yielding misleading aggregates. Rankings frequently depend on arbitrary metric weights or proxies—such as citation counts or reputation surveys—that introduce , volatility (e.g., rank shifts from minor tweaks), and domain-specific biases, like overemphasizing output at the expense of efficacy. errors persist due to incomplete disclosure of methodologies, further eroding reliability across systems like or employee rankings. Manipulation exploits these vulnerabilities, as formalized by the , which proves that any onto, non-dictatorial rule aggregating ordinal preferences over three or more alternatives admits strategic misreporting by at least one voter to achieve a preferred outcome. In practice, agents adjust reported features or behaviors—such as fabricating pairwise comparisons or gaming input signals—to skew aggregated results, particularly in sequential or algorithmic settings where incomplete information allows predictive exploitation of rankers. This susceptibility manifests in domains like search algorithms, where entities inflate visibility through coordinated signals, or evaluation systems prone to insider strategic reporting, undermining the integrity of final orderings despite safeguards.

Fairness, Bias, and Social Impacts

Ranking systems, particularly algorithmic ones, can embed originating from training data that reflect historical disparities or proxy variables correlated with protected attributes like race or . For instance, in learning-to-rank models used in search and recommendation, data biases lead to disparate exposure for underrepresented groups, where algorithms prioritize items based on past interactions that disadvantage minorities. These biases arise causally from feedback loops: initial inequalities in data amplify over iterations, reducing utility for affected groups without explicit fairness constraints. In search engines, ranking algorithms exert via the search engine manipulation (SEME), where subtle shifts in result order—up to 20% in demonstrated studies—can sway undecided voters' opinions by 20% or more on political issues. This occurs because users perceive higher-ranked results as more authoritative, fostering chambers or polarization when algorithms favor ideologically aligned content. from controlled experiments shows such manipulations persist even when users are aware of potential , highlighting causal impacts on public discourse. Sports rankings often suffer from subjective human biases, such as conference favoritism in polls, where Big Ten and SEC teams receive undue elevation despite on-field metrics; analysis of 2014-2023 data revealed overperformance penalties for non-power conference teams relative to recruiting talent-to-result ratios. Computer-based systems like Elo ratings mitigate this by relying on objective win-loss data, avoiding reputational or regional prejudices evident in human polls. In academic evaluation, ranking under disparate uncertainty disadvantages groups with higher prediction errors, as shown in models where equal-opportunity criteria fail to account for varying data quality across demographics, leading to unfair resource allocation in admissions or funding. Assessment processes are further impaired by forces disrupting interactivity and adaptation, such as rigid metrics that overlook contextual factors, per qualitative studies of moderation practices. Business ranking models, exemplified by credit scoring, perpetuate disparities through noisy historical data; a 2021 analysis of mortgage approvals found algorithms less accurate for and applicants, denying loans at rates 40% higher than for comparable applicants due to unrepresentative training sets encoding past lending biases. These systems causally exacerbate by limiting access to capital for low-income or minority groups, with AI variants risking amplification if not debiased via alternative data sources. Socially, biased rankings erode trust in institutions and widen divides: in domains like hiring or lending, they reinforce cycles of , while in information retrieval, they shape collective beliefs, potentially undermining democratic processes through polarized information flows. Reforms like merit-based exposure in dynamic ranking have shown promise in simulations, balancing utility with equity, though real-world deployment requires verifying against ground-truth outcomes rather than proxy fairness metrics.

Evidence on Effectiveness and Reforms

Empirical assessments of ranking systems reveal moderate in predictive and comparative roles across domains, though often limited by methodological inconsistencies and narrow metrics. In higher education, global university rankings correlate with research outputs such as publications and citations, which form up to 76% of scores in systems like and , yet show no consistent link to improvements in or overall institutional . Inconsistencies are evident, with individual universities fluctuating widely across rankings—for instance, positions ranging from 24th to 125th—due to varying weights on surveys (averaging 39.8% of scores) and biases toward English-language publications or elite resources. Surveys indicate rankings influence 58% of college-bound high school seniors' application decisions, enhancing selectivity for top institutions, but they more often perpetuate loops than drive substantive self-improvement, as past rankings condition future perceptions without causal evidence of gains. In sports, algorithmic models demonstrate predictive accuracies of 58-65% for outcomes, with dynamic systems incorporating historical outperforming static polls; for example, network-based rankings achieve 63.7% accuracy in ATP events, improving on traditional win-loss scores by factoring opponent strength. Advanced in slightly exceed subjective rankings in forecasting wins, though overall accuracies hover below 70%, highlighting rankings' utility for bracketing but vulnerability to schedule variance and upsets. Business firm rankings, such as "Great Place to Work's 100 Best," correlate with superior financial and stability, with listed companies showing persistent advantages in attitudes and metrics like revenue growth, though remains debated amid selection effects. In search engines, effectiveness metrics like Normalized Discounted Cumulative Gain (NDCG) evaluate relevance, with systems achieving high scores on benchmark datasets, but real-world utility depends on query diversity and user satisfaction proxies like click-through rates, which overlook long-tail or adversarial content. Critiques underscore systemic flaws, including gaming through data manipulation and overemphasis on quantifiable proxies that neglect causal factors like innovation ecosystems. Reforms propose empirical validation via psychometric testing, audited data standards, and transparency frameworks like the Federal Committee on Statistical Methodology's quality guidelines to reduce nonresponse biases in surveys. Shifting to ordinal ratings over strict hierarchies, personalizing via stakeholder weights (e.g., student outcomes over Nobel counts), and incorporating context-specific indicators—such as or digital infrastructure—could enhance reliability, as demonstrated in localized systems outperforming global ones in performance alignment. Broader proposals advocate collaborative development with universities to prioritize societal impact over competition, mitigating short-term metric-chasing.

References

  1. https://aimath.org/pastworkshops/mathofrankingrep.pdf
  2. https://asaibab.math.ncsu.edu/Module1Ranking.pdf
  3. https://study.com/learn/lesson/ordering-ranking-data-process-examples.html
  4. https://www.i4cp.com/productivity-blog/2012/07/16/four-major-flaws-of-force-ranking
  5. https://deloitte.wsj.com/cio/its-official-forced-ranking-is-dead-1402372957
  6. https://www.washingtonpost.com/education/2023/09/26/big-problems-with-ranking-schools/
  7. https://eli.thegreenplace.net/2018/partial-and-total-orders/
  8. https://www.sciencedirect.com/science/article/abs/pii/S002224960400029X
  9. https://projecteuclid.org/journals/annals-of-mathematical-statistics/volume-31/issue-2/Contributions-to-the-Theory-of-Rank-Order-Statistics--The/10.1214/aoms/1177705904.full
  10. https://yadda.icm.edu.pl/baztech/element/bwmeta1.element.baztech-fa1d1555-ca71-4107-86dd-37a322c2ac07/c/Ameljanczyk_1_CSMM_2_2015a.pdf
  11. http://www.columbia.edu/~md3405/DT_Order_15.pdf
  12. https://economics.osu.edu/sites/economics.osu.edu/files/JMP_Semin_Kim.pdf
  13. https://old.maa.org/press/periodicals/convergence/the-french-connection-borda-condorcet-and-the-mathematics-of-voting-theory-voting-in-and-after-the
  14. https://digitalcommons.ursinus.edu/cgi/viewcontent.cgi?article=1000&context=triumphs_liberal
  15. https://old.maa.org/press/periodicals/convergence/the-french-connection-borda-condorcet-and-the-mathematics-of-voting-theory-condorcet-s-use-of
  16. https://www.lse.ac.uk/cpnss/assets/documents/voting-power-and-procedures/workshops/2010/duBaffy2010-Felsenthal.pdf
  17. https://www.pnas.org/doi/10.1073/pnas.0702634104
  18. https://methods.sagepub.com/ency/edvol/encyclopedia-of-measurement-and-statistics/chpt/spearmans-rho
  19. https://significancemagazine.com/how-frank-wilcoxon-helped-statisticians-walk-the-non-parametric-path-2/
  20. https://methods.sagepub.com/ency/edvol/encyc-of-research-design/chpt/kruskal-wallis-test
  21. https://apps.dtic.mil/sti/tr/pdf/ADA096093.pdf
  22. https://www.microsoft.com/en-us/research/wp-content/uploads/2016/11/social_choice_theory.pdf
  23. https://online.stat.psu.edu/stat415/lesson/20/20.3
  24. https://blogs.sas.com/content/iml/2021/04/19/tied-ranks.html
  25. https://www.tandfonline.com/doi/abs/10.1080/02664763.2015.1043870
  26. https://www.statology.org/excel-rank-with-ties/
  27. https://web.stanford.edu/~iashlagi/papers/rank-dominance.pdf
  28. https://onlinelibrary.wiley.com/doi/full/10.3982/TE4762
  29. https://arxiv.org/html/2406.07121v1
  30. https://real-statistics.com/reliability/bradley-terry-model/
  31. https://www.dratings.com/creating-a-ratings-model-with-bradley-terry/
  32. https://statisticaloddsandends.wordpress.com/2024/04/24/what-is-the-plackett-luce-model/
  33. https://cran.r-project.org/web/packages/PlackettLuce/PlackettLuce.pdf
  34. https://arxiv.org/abs/1909.06722
  35. https://statistics.laerd.com/statistical-guides/spearmans-rank-order-correlation-statistical-guide.php
  36. https://stats.libretexts.org/Bookshelves/Applied_Statistics/Biological_Statistics_%28McDonald%29/05%253A_Tests_for_Multiple_Measurement_Variables/5.02%253A_Spearman_Rank_Correlation
  37. https://www.utdallas.edu/~herve/Abdi-KendallCorrelation2007-pretty.pdf
  38. https://statistics.laerd.com/spss-tutorials/kendalls-tau-b-using-spss-statistics.php
  39. https://link.springer.com/book/10.1007/978-1-4939-1471-5
  40. http://ndl.ethernet.edu.et/bitstream/123456789/39417/1/Mayer%2520Alvo.pdf
  41. https://www.sciencedirect.com/topics/computer-science/ranking-algorithm
  42. https://www.ibm.com/docs/en/filenet-p8-platform/5.7.0?topic=domain-document-retrieval-ranking
  43. https://spotintelligence.com/2024/07/26/ranking-algorithms/
  44. https://www.sciencedirect.com/science/article/pii/S0377221722005847
  45. https://ieeexplore.ieee.org/document/4548226/
  46. https://asu.elsevierpure.com/en/publications/x-rank-explainable-ranking-in-complex-multi-layered-networks
  47. https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/tr-2007-40.pdf
  48. https://link.springer.com/book/10.1007/978-3-642-14267-3
  49. https://netman.aiops.org/~peidan/ANM2022/2.MachineLearningBasics/LectureCoverage/31.LearningToRank.pdf
  50. https://lucidworks.com/blog/abcs-learning-to-rank
  51. https://xgboost.readthedocs.io/en/latest/tutorials/learning_to_rank.html
  52. https://masseyratings.com/theory/massey.htm
  53. https://www.ncaa.com/news/football/article/2025-01-22/college-football-rankings-every-poll-explained-how-they-work
  54. https://www.chess.com/terms/elo-rating-chess
  55. https://inside.fifa.com/fifa-world-ranking/procedure-men
  56. https://inside.fifa.com/fifa-world-ranking/men
  57. https://www.atptour.com/en/rankings/rankings-faq
  58. https://www.mlb.com/glossary/advanced-stats/pythagorean-winning-percentage
  59. https://www.colleyrankings.com/matrate.pdf
  60. https://www.nassp.org/class-rank-gpa-and-grading/
  61. https://pmc.ncbi.nlm.nih.gov/articles/PMC3685191/
  62. https://www.sciencedirect.com/science/article/pii/S1477388015301407
  63. https://www.researchgate.net/publication/379061325_Student_Ranking_Based_on_Learning_Assessment_Using_the_Simplified_PIPRECIA_Method_and_CoCoSo_Method
  64. https://www.norc.org/content/dam/norc-org/pdf2024/college-rankings-review.pdf
  65. https://www.frontiersin.org/journals/education/articles/10.3389/feduc.2025.1619897/full
  66. https://www.sciencedirect.com/science/article/pii/S2405872616300570
  67. https://pmc.ncbi.nlm.nih.gov/articles/PMC11830122/
  68. https://files.eric.ed.gov/fulltext/EJ1055590.pdf
  69. https://direct.mit.edu/qss/article/5/3/533/123261/University-rankings-in-the-context-of-research
  70. https://pmc.ncbi.nlm.nih.gov/articles/PMC10597825/
  71. https://unctad.org/system/files/official-document/osgdp20081_en.pdf
  72. https://www.tandfonline.com/doi/full/10.1080/07907184.2025.2487785
  73. https://capmktsreg.org/report-on-credit-rating-agencies-during-covid-19/
  74. https://archive.doingbusiness.org/en/rankings
  75. https://www.sciencedirect.com/science/article/pii/S0148296319307428
  76. https://fbr.springeropen.com/articles/10.1186/s11782-017-0022-6
  77. https://www.business.com/articles/the-end-of-rank-and-yank-management-practices-revisited/
  78. https://www.researchgate.net/publication/278324560_A_Comprehensive_performance_evaluation_and_ranking_methodology_under_a_sustainable_development_perspective
  79. https://publications.iadb.org/en/do-credit-rating-agencies-add-value-evidence-sovereign-rating-business-institutions
  80. https://www.investopedia.com/terms/a/arrows-impossibility-theorem.asp
  81. https://peterchng.com/blog/2024/04/06/condorcet-paradox-and-its-implications/
  82. https://maskin.scholars.harvard.edu/sites/g/files/omnuum10606/files/condorcet_voting_e.maskin_e.foley_march_2025_0.pdf
  83. https://www.rudderstack.com/learn/Data/what-is-ordinal-data/
  84. https://foodsafety.institute/research-methodology/applying-ordinal-scale-social-science-research/
  85. https://pmc.ncbi.nlm.nih.gov/articles/PMC2174504/
  86. https://www.researchgate.net/publication/373923751_Critiques_and_Limitations_of_University_Rankings
  87. https://www.sciencedirect.com/science/article/pii/S0165176500003128
  88. https://www.stat.berkeley.edu/~mossel/publications/ManipFocs10.pdf
  89. https://arxiv.org/html/2407.01916
  90. https://ieeexplore.ieee.org/document/10564181/
  91. https://link.springer.com/article/10.1007/s10458-024-09676-3
  92. https://arxiv.org/pdf/1908.09635
  93. https://dl.acm.org/doi/10.1145/3533380
  94. https://www.cs.cornell.edu/~tj/publications/singh_joachims_19a.pdf
  95. https://www.pnas.org/doi/10.1073/pnas.1419828112
  96. https://subba-lakshmi.medium.com/social-and-political-impact-of-web-search-techniques-an-overview-7821677498a0
  97. https://lawlessrepublic.com/college-football-news-rumors-opinion-big-10-sec-bias-college-football-ranking-system-ap-poll-top-25
  98. https://espnthefan.com/featured/homepage-feature-3/10-year-on-field-results-reveal-trends-biases-in-recruiting-rankings/
  99. https://dubstat.com/elo-ratings-the-ultimate-sports-ranking-system/
  100. https://arxiv.org/html/2309.01610v3
  101. https://pmc.ncbi.nlm.nih.gov/articles/PMC10453954/
  102. https://themarkup.org/denied/2021/08/25/the-secret-bias-hidden-in-mortgage-approval-algorithms
  103. https://www.technologyreview.com/2021/06/17/1026519/racial-bias-noisy-data-credit-scores-mortgage-loans-fairness-machine-learning/
  104. https://hai.stanford.edu/news/how-flawed-data-aggravates-inequality-credit
  105. https://www.cs.cornell.edu/~tj/publications/morik_etal_20a.pdf
  106. https://pmc.ncbi.nlm.nih.gov/articles/PMC5841788/
  107. https://digitalcommons.montclair.edu/context/president_pubs/article/1011/viewcontent/University_rankings_evidence_and_a_conceptual_framework.pdf
  108. https://pmc.ncbi.nlm.nih.gov/articles/PMC3514705/
  109. https://harvardsportsanalysis.org/2013/02/grading-the-accuracy-of-espn-and-football-outsiders-power-rankings/
  110. https://www.researchgate.net/publication/227516890_Are_the_100_Best_better_An_empirical_investigation_of_the_relationship_between_being_a_Great_Place_to_Work_and_firm_performance
  111. https://towardsdatascience.com/metrics-that-matter-a-simple-guide-to-search-ranking-evaluation-4030084c35b4/
  112. https://www.cesaer.org/content/5-operations/2024/20241125-rankings-workshop-report/20241125-rankings-workshop-report.pdf
Add your contribution
Related Hubs
User Avatar
No comments yet.